Ok so after 6 months I have solved this problem!
It was several things, the other server (A Webmethods Integration Server) is a little picky about what kinds of protocols it accepts. So after some fiddling around here are the two things that fixed it:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
and
req.ProtocolVersion = HttpVersion.Version10;
So the whole code is:
ServicePointManager.Expect100Continue = true;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://websiteURL:9000");
//Header Settings
req.Method = "POST"; // Post method
req.ContentType = "text/xml";// content type
req.KeepAlive = false;
req.ProtocolVersion = HttpVersion.Version10;
//Certificate with private key
X509Certificate2 cert = new X509Certificate2("Cert.der","Password");
req.ClientCertificates.Add(cert);
req.PreAuthenticate = true;
String XML = "Test Message"//reader.ReadToEnd();
byte[] buffer = Encoding.ASCII.GetBytes(XML);
req.ContentLength = buffer.Length;
// Wrap the request stream with a text-based writer
Stream writer = req.GetRequestStream();
// Write the XML text into the stream
writer.Write(buffer, 0, buffer.Length);
writer.Close();
WebResponse rsp = req.GetResponse();
StreamReader responseStream = new StreamReader(rsp.GetResponseStream());
Hope this helps someone else in the future.