How to set large string inside HttpContent when using HttpClient?

Question

So, I created a HttpClient and am posting data using HttpClient.PostAsync().

I set the HttpContent using

HttpContent content = new FormUrlEncodedContent(post_parameters); where post_parameters is a list of Key value pairs List<KeyValuePair<string, string>>

Problem is, when the HttpContent has a large value (an image converted to base64 to be transmitted) I get a URL is too long error. That makes sense - cause the url cant go beyond 32,000 characters. But how do I add the data into the HttpContent if not this way?

Please help.

Solution 1

I figured it out with the help of my friend. What you would want to do is avoid using FormUrlEncodedContent(), because it has restrictions on the size of the uri. Instead, you can do the following :

    var jsonString = JsonConvert.SerializeObject(post_parameters);
    var content = new StringContent(jsonString, Encoding.UTF8, "application/json");

Here, we don't need to use HttpContent to post to the server, StringContent gets the job done !

Solution 2

FormUrlEncodedContent internally uses Uri.EscapeDataString : from reflection, I can see that this method has constants limiting the size of request length.

A possible solution is to create a new implementation of FormUrlEncodedContent by using System.Net.WebUtility.UrlEncode (.net 4.5) to bypass this limitation.

public class MyFormUrlEncodedContent : ByteArrayContent
{
    public MyFormUrlEncodedContent(IEnumerable<KeyValuePair<string, string>> nameValueCollection)
        : base(MyFormUrlEncodedContent.GetContentByteArray(nameValueCollection))
    {
        base.Headers.ContentType = new MediaTypeHeaderValue("application/x-www-form-urlencoded");
    }
    private static byte[] GetContentByteArray(IEnumerable<KeyValuePair<string, string>> nameValueCollection)
    {
        if (nameValueCollection == null)
        {
            throw new ArgumentNullException("nameValueCollection");
        }
        StringBuilder stringBuilder = new StringBuilder();
        foreach (KeyValuePair<string, string> current in nameValueCollection)
        {
            if (stringBuilder.Length > 0)
            {
                stringBuilder.Append('&');
            }

            stringBuilder.Append(MyFormUrlEncodedContent.Encode(current.Key));
            stringBuilder.Append('=');
            stringBuilder.Append(MyFormUrlEncodedContent.Encode(current.Value));
        }
        return Encoding.Default.GetBytes(stringBuilder.ToString());
    }
    private static string Encode(string data)
    {
        if (string.IsNullOrEmpty(data))
        {
            return string.Empty;
        }
        return System.Net.WebUtility.UrlEncode(data).Replace("%20", "+");
    }
}

To send large content, it's better to use StreamContent.

Solution 3

This code works for me, basically you send post data "application/x-www-form-urlencoded" within string content over http client, hope this can help anyone with the same problem like me

void sendDocument()
    {
        string url = "www.mysite.com/page.php";
        StringBuilder postData = new StringBuilder();
        postData.Append(String.Format("{0}={1}&", HttpUtility.HtmlEncode("prop"), HttpUtility.HtmlEncode("value")));
        postData.Append(String.Format("{0}={1}", HttpUtility.HtmlEncode("prop2"), HttpUtility.HtmlEncode("value2")));
        StringContent myStringContent = new StringContent(postData.ToString(), Encoding.UTF8, "application/x-www-form-urlencoded");
        HttpClient client = new HttpClient();
        HttpResponseMessage message = client.PostAsync(url, myStringContent).GetAwaiter().GetResult();
        string responseContent = message.Content.ReadAsStringAsync().GetAwaiter().GetResult();
    }

Comments

[Muraad]

I cannot use this solution because i am using .Net 4.0 and i do not have the access to the method UrlEncode(). I cannot upgrade due to work restrictions, any other thoughts?

[Cybermaxs]

by adding a System.Web dependency, you could find a similar method HttpUtility.UrlEncode

[Muraad]

the implementation you gave me has the same restrictions as FormUrlEcondedContent() , but now instead of throwing an exception it skips the call to post to the server.

[jvrdelafuente]

This solution has worked for me. I have some restriction in the api that I am consuming and I had to consume that using "application/x-www-form-urlencoded". Thanks.

[eXavier]

> Here, we don't need to use HttpContent to post to the server, StringContent gets the job done ! StringContent actually is HttpContent (it is derived from abstract HttpContent similarly to FormUrlEncodedContent)

[GregoryBrad]

Just out of interest, whats happening here? .Replace("%20", "+")

[mafu]

This is by far the easiest way. Anything else I've seen on here up to early 2016 is a PITA.

[Mr. B]

Never said this to anyone on stack overflow. You're a total badass!

[HankCa]

But the OPs problem was posting large amounts of data, which you have not demonstrated can be done in this way. However I see what you are doing - using application/x-www-form-urlencoded but bypassing using class FormUrlEncodedContent which seems to have the content length restriction.

[Hallmanac]

This basically worked for me. The thing I changed was to use the WebUtility.UrlEncode("prop2") instead of HtmlEncode. I had a weird API end point that required a JSON array inside a form post body. Sometimes that array could be massive which is why I needed something like this. Good stuff!

[Parveen]

Worked for me, Thanks for the help.

[Sicae]

Was facing this issue do to an out of control viewstate on a site i dont control but need to interface with... this custom class saved my ass. :P thanks