Multipart forms from C# client

Question

I am trying to fill a form in a php application from a C# client (Outlook addin). I used Fiddler to see the original request from within the php application and the form is transmitted as a multipart/form. Unfortunately .Net does not come with native support for this type of forms (WebClient has only a method for uploading a file). Does anybody know a library or has some code to achieve this? I want to post different values and additionally (but only sometimes) a file.

Thanks for your help, Sebastian

Solution 1

This is cut and pasted from some sample code I wrote, hopefully it should give the basics. It only supports File data and form-data at the moment.

public class PostData
{

    private List<PostDataParam> m_Params;

    public List<PostDataParam> Params
    {
        get { return m_Params; }
        set { m_Params = value; }
    }

    public PostData()
    {
        m_Params = new List<PostDataParam>();

        // Add sample param
        m_Params.Add(new PostDataParam("email", "MyEmail", PostDataParamType.Field));
    }


    /// <summary>
    /// Returns the parameters array formatted for multi-part/form data
    /// </summary>
    /// <returns></returns>
    public string GetPostData()
    {
        // Get boundary, default is --AaB03x
        string boundary = ConfigurationManager.AppSettings["ContentBoundary"].ToString();

        StringBuilder sb = new StringBuilder();
        foreach (PostDataParam p in m_Params)
        {
            sb.AppendLine(boundary);

            if (p.Type == PostDataParamType.File)
            {
                sb.AppendLine(string.Format("Content-Disposition: file; name=\"{0}\"; filename=\"{1}\"", p.Name, p.FileName));
                sb.AppendLine("Content-Type: text/plain");
                sb.AppendLine();
                sb.AppendLine(p.Value);                 
            }
            else
            {
                sb.AppendLine(string.Format("Content-Disposition: form-data; name=\"{0}\"", p.Name));
                sb.AppendLine();
                sb.AppendLine(p.Value);
            }
        }

        sb.AppendLine(boundary);

        return sb.ToString();           
    }
}

public enum PostDataParamType
{
    Field,
    File
}

public class PostDataParam
{


    public PostDataParam(string name, string value, PostDataParamType type)
    {
        Name = name;
        Value = value;
        Type = type;
    }

    public string Name;
    public string FileName;
    public string Value;
    public PostDataParamType Type;
}

To send the data you then need to:

HttpWebRequest oRequest = null;
oRequest = (HttpWebRequest)HttpWebRequest.Create(oURL.URL);
oRequest.ContentType = "multipart/form-data";                       
oRequest.Method = "POST";
PostData pData = new PostData();

byte[] buffer = encoding.GetBytes(pData.GetPostData());

// Set content length of our data
oRequest.ContentLength = buffer.Length;

// Dump our buffered postdata to the stream, booyah
oStream = oRequest.GetRequestStream();
oStream.Write(buffer, 0, buffer.Length);
oStream.Close();

// get the response
oResponse = (HttpWebResponse)oRequest.GetResponse();

Hope thats clear, i've cut and pasted from a few sources to get that tidier.

Solution 2

Thanks for the answers, everybody! I recently had to get this to work, and used your suggestions heavily. However, there were a couple of tricky parts that did not work as expected, mostly having to do with actually including the file (which was an important part of the question). There are a lot of answers here already, but I think this may be useful to someone in the future (I could not find many clear examples of this online). I wrote a blog post that explains it a little more.

Basically, I first tried to pass in the file data as a UTF8 encoded string, but I was having problems with encoding files (it worked fine for a plain text file, but when uploading a Word Document, for example, if I tried to save the file that was passed through to the posted form using Request.Files[0].SaveAs(), opening the file in Word did not work properly. I found that if you write the file data directly using a Stream (rather than a StringBuilder), it worked as expected. Also, I made a couple of modifications that made it easier for me to understand.

By the way, the Multipart Forms Request for Comments and the W3C Recommendation for mulitpart/form-data are a couple of useful resources in case anyone needs a reference for the specification.

I changed the WebHelpers class to be a bit smaller and have simpler interfaces, it is now called FormUpload. If you pass a FormUpload.FileParameter you can pass the byte[] contents along with a file name and content type, and if you pass a string, it will treat it as a standard name/value combination.

Here is the FormUpload class:

// Implements multipart/form-data POST in C# http://www.ietf.org/rfc/rfc2388.txt
// http://www.briangrinstead.com/blog/multipart-form-post-in-c
public static class FormUpload
{
    private static readonly Encoding encoding = Encoding.UTF8;
    public static HttpWebResponse MultipartFormDataPost(string postUrl, string userAgent, Dictionary<string, object> postParameters)
    {
        string formDataBoundary = String.Format("----------{0:N}", Guid.NewGuid());
        string contentType = "multipart/form-data; boundary=" + formDataBoundary;

        byte[] formData = GetMultipartFormData(postParameters, formDataBoundary);

        return PostForm(postUrl, userAgent, contentType, formData);
    }
    private static HttpWebResponse PostForm(string postUrl, string userAgent, string contentType, byte[] formData)
    {
        HttpWebRequest request = WebRequest.Create(postUrl) as HttpWebRequest;

        if (request == null)
        {
            throw new NullReferenceException("request is not a http request");
        }

        // Set up the request properties.
        request.Method = "POST";
        request.ContentType = contentType;
        request.UserAgent = userAgent;
        request.CookieContainer = new CookieContainer();
        request.ContentLength = formData.Length;

        // You could add authentication here as well if needed:
        // request.PreAuthenticate = true;
        // request.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
        // request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(System.Text.Encoding.Default.GetBytes("username" + ":" + "password")));

        // Send the form data to the request.
        using (Stream requestStream = request.GetRequestStream())
        {
            requestStream.Write(formData, 0, formData.Length);
            requestStream.Close();
        }

        return request.GetResponse() as HttpWebResponse;
    }

    private static byte[] GetMultipartFormData(Dictionary<string, object> postParameters, string boundary)
    {
        Stream formDataStream = new System.IO.MemoryStream();
        bool needsCLRF = false;

        foreach (var param in postParameters)
        {
            // Thanks to feedback from commenters, add a CRLF to allow multiple parameters to be added.
            // Skip it on the first parameter, add it to subsequent parameters.
            if (needsCLRF)
                formDataStream.Write(encoding.GetBytes("\r\n"), 0, encoding.GetByteCount("\r\n"));

            needsCLRF = true;

            if (param.Value is FileParameter)
            {
                FileParameter fileToUpload = (FileParameter)param.Value;

                // Add just the first part of this param, since we will write the file data directly to the Stream
                string header = string.Format("--{0}\r\nContent-Disposition: form-data; name=\"{1}\"; filename=\"{2}\";\r\nContent-Type: {3}\r\n\r\n",
                    boundary,
                    param.Key,
                    fileToUpload.FileName ?? param.Key,
                    fileToUpload.ContentType ?? "application/octet-stream");

                formDataStream.Write(encoding.GetBytes(header), 0, encoding.GetByteCount(header));

                // Write the file data directly to the Stream, rather than serializing it to a string.
                formDataStream.Write(fileToUpload.File, 0, fileToUpload.File.Length);
            }
            else
            {
                string postData = string.Format("--{0}\r\nContent-Disposition: form-data; name=\"{1}\"\r\n\r\n{2}",
                    boundary,
                    param.Key,
                    param.Value);
                formDataStream.Write(encoding.GetBytes(postData), 0, encoding.GetByteCount(postData));
            }
        }

        // Add the end of the request.  Start with a newline
        string footer = "\r\n--" + boundary + "--\r\n";
        formDataStream.Write(encoding.GetBytes(footer), 0, encoding.GetByteCount(footer));

        // Dump the Stream into a byte[]
        formDataStream.Position = 0;
        byte[] formData = new byte[formDataStream.Length];
        formDataStream.Read(formData, 0, formData.Length);
        formDataStream.Close();

        return formData;
    }

    public class FileParameter
    {
        public byte[] File { get; set; }
        public string FileName { get; set; }
        public string ContentType { get; set; }
        public FileParameter(byte[] file) : this(file, null) { }
        public FileParameter(byte[] file, string filename) : this(file, filename, null) { }
        public FileParameter(byte[] file, string filename, string contenttype)
        {
            File = file;
            FileName = filename;
            ContentType = contenttype;
        }
    }
}

Here is the calling code, which uploads a file and a few normal post parameters:

// Read file data
FileStream fs = new FileStream("c:\\people.doc", FileMode.Open, FileAccess.Read);
byte[] data = new byte[fs.Length];
fs.Read(data, 0, data.Length);
fs.Close();

// Generate post objects
Dictionary<string, object> postParameters = new Dictionary<string, object>();
postParameters.Add("filename", "People.doc");
postParameters.Add("fileformat", "doc");
postParameters.Add("file", new FormUpload.FileParameter(data, "People.doc", "application/msword"));

// Create request and receive response
string postURL = "http://localhost";
string userAgent = "Someone";
HttpWebResponse webResponse = FormUpload.MultipartFormDataPost(postURL, userAgent, postParameters);

// Process response
StreamReader responseReader = new StreamReader(webResponse.GetResponseStream());
string fullResponse = responseReader.ReadToEnd();
webResponse.Close();
Response.Write(fullResponse);

Solution 3

With .NET 4.5 you currently could use System.Net.Http namespace. Below the example for uploading single file using multipart form data.

using System;
using System.IO;
using System.Net.Http;

namespace HttpClientTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var client = new HttpClient();
            var content = new MultipartFormDataContent();
            content.Add(new StreamContent(File.Open("../../Image1.png", FileMode.Open)), "Image", "Image.png");
            content.Add(new StringContent("Place string content here"), "Content-Id in the HTTP"); 
            var result = client.PostAsync("https://hostname/api/Account/UploadAvatar", content);
            Console.WriteLine(result.Result.ToString());
        }
    }
}

Solution 4

Building on dnolans example, this is the version I could actually get to work (there were some errors with the boundary, encoding wasn't set) :-)

To send the data:

HttpWebRequest oRequest = null;
oRequest = (HttpWebRequest)HttpWebRequest.Create("http://you.url.here");
oRequest.ContentType = "multipart/form-data; boundary=" + PostData.boundary;
oRequest.Method = "POST";
PostData pData = new PostData();
Encoding encoding = Encoding.UTF8;
Stream oStream = null;

/* ... set the parameters, read files, etc. IE:
   pData.Params.Add(new PostDataParam("email", "[email protected]", PostDataParamType.Field));
   pData.Params.Add(new PostDataParam("fileupload", "filename.txt", "filecontents" PostDataParamType.File));
*/

byte[] buffer = encoding.GetBytes(pData.GetPostData());

oRequest.ContentLength = buffer.Length;

oStream = oRequest.GetRequestStream();
oStream.Write(buffer, 0, buffer.Length);
oStream.Close();

HttpWebResponse oResponse = (HttpWebResponse)oRequest.GetResponse();

The PostData class should look like:

public class PostData
{
    // Change this if you need to, not necessary
    public static string boundary = "AaB03x";

    private List<PostDataParam> m_Params;

    public List<PostDataParam> Params
    {
        get { return m_Params; }
        set { m_Params = value; }
    }

    public PostData()
    {
        m_Params = new List<PostDataParam>();
    }

    /// <summary>
    /// Returns the parameters array formatted for multi-part/form data
    /// </summary>
    /// <returns></returns>
    public string GetPostData()
    {
        StringBuilder sb = new StringBuilder();
        foreach (PostDataParam p in m_Params)
        {
            sb.AppendLine("--" + boundary);

            if (p.Type == PostDataParamType.File)
            {
                sb.AppendLine(string.Format("Content-Disposition: file; name=\"{0}\"; filename=\"{1}\"", p.Name, p.FileName));
                sb.AppendLine("Content-Type: application/octet-stream");
                sb.AppendLine();
                sb.AppendLine(p.Value);
            }
            else
            {
                sb.AppendLine(string.Format("Content-Disposition: form-data; name=\"{0}\"", p.Name));
                sb.AppendLine();
                sb.AppendLine(p.Value);
            }
        }

        sb.AppendLine("--" + boundary + "--");

        return sb.ToString();
    }
}

public enum PostDataParamType
{
    Field,
    File
}

public class PostDataParam
{
    public PostDataParam(string name, string value, PostDataParamType type)
    {
        Name = name;
        Value = value;
        Type = type;
    }

    public PostDataParam(string name, string filename, string value, PostDataParamType type)
    {
        Name = name;
        Value = value;
        FileName = filename;
        Type = type;
    }

    public string Name;
    public string FileName;
    public string Value;
    public PostDataParamType Type;
}

Solution 5

In the version of .NET I am using you also have to do this:

System.Net.ServicePointManager.Expect100Continue = false;

If you don't, the HttpWebRequest class will automatically add the Expect:100-continue request header which fouls everything up.

Also I learned the hard way that you have to have the right number of dashes. whatever you say is the "boundary" in the Content-Type header has to be preceded by two dashes

--THEBOUNDARY

and at the end

--THEBOUNDARY--

exactly as it does in the example code. If your boundary is a lot of dashes followed by a number then this mistake won't be obvious by looking at the http request in a proxy server

View more solutions

Comments

[Chathuranga Wijeratna]

This works like a charm www.briangrinstead.com/blog

[Todd Menier]

If you don't mind a small library dependency, Flurl makes this about as simple as it gets. [Disclaimer: I'm the author]

[user2276101]

THIS IS IMPORTANT! Thank you so much for mentioning this, I never would have considered it otherwise and it was the only thing messing me up!

[Jeff Hellman]

Spectacular. I modified to show progress, support PUT requests and use my own cookiecontainer but this was a ton of help to get me on my way.

[rizwan]

encoding.GetByteCount(postData) may be a better option if you only need to determine byte count

[DRapp]

Excellent baseline. I made minor adjustments to accommodate for our needs, but definitely a HUGE help in multi-part boundary posting. Significantly simplified my work. Thanks

[Sunil Kumar S C]

#Brian I exactly followed your code But its not Working :(

[Daniel Lobo]

I don't think this works, the boundary is not defined in the Content-Type and there must be added to the left "--" of the boundary for each one that is written and "--" to the right of the last one. Don't ask me how this magical "--" appears, they are implied in the official documentation: w3.org/TR/html401/interact/forms.html#h-17.13.4.2 If you don't like magical requirements as me, you are going to have a hard time making this work as I did.

[dnolan]

This code definitely works for scenarios I have used it in. However I see that while I'm defining the full boundary to be send in the post data, I have not specified adding this onto the overall content header as the boundary delimiter. As for the magic number, this is determined by the client to ensure it is unique and will not appear inside the content anywhere, see here for more detail: w3.org/Protocols/rfc1341/7_2_Multipart.html

[Lutaaya Huzaifah Idris]

what if am getting details of a multipart form in the URL , here am asking about files like a pdf

[Daniël Tulp]

you can add more form-data content (like an Id) with content.Add(new StringContent("1"),"MyId");

[Benny]

so the file is read to the memory before sending, what if it's a big file