HttpClient vs HttpWebRequest

Question

I have a large file which I have to send to a web api client...The data is multi part. The issue is , if the file is sent over http web request, then it is uploaded quickly on the webapi. For this request, file contents are written over the request stream directly.

Where as if the same file is sent over Httpclient (.net 4.5), the upload is slow when compared to http web request. I am using multipartformdatacontent in Httpclient post async.

So, for large files, do we have to use only web request? or is there any settings on Httpclient that makes the upload faster?

Solution 1

HttpClient is more like a head-less browser. It a powerfull and ideal tool if you are going to be creating many http request. For example you can set default headers and stuff. Here are the top 5 ways it differs from an HttpWebRequest which is taken from here

1. An HttpClient instance is the place to configure extensions, set default headers, cancel outstanding requests and more.
2. You can issue as many requests as you like through a single HttpClient instance.
3. HttpClients are not tied to particular HTTP server or host; you can submit any HTTP request using the same HttpClient instance.
4. You can derive from HttpClient to create specialized clients for particular sites or patterns
5. HttpClient uses the new Task-oriented pattern for handling asynchronous requests making it dramatically easier to manage and coordinate multiple outstanding requests.

Solution 2

I was using FileStreamContent with httpclient...But when I used ByteArrayContent, it worked fine.

I am not sure how and why this made the difference, but sending bytes over the stream is a better way rather than sending the stream

Solution 3

Perhaps you were instantiating HttpClient in a using block which could explain performance issues. E.g.

  using (var httpClient = new HttpClient() )
  {
      var result = await httpClient.GetAsync("http://example.com");
      Console.WriteLine(result.StatusCode);
  }

Here the instance of HttpClient is being disposed immediately after the request whereas it should arguably be a long lived object (E.g. the lifetime of the application).

[edit - added context]

Disposing the instance also closes the connection but leaves the socket in a waiting state for a set duration. For each execution of this code, the os will attempt to create a new socket connection, and since there is a limit to how quickly this can be completed, performance/reliability issues can arise.

Reusing the same HttpClient instance means better reuse of open sockets and more efficient use of system resources.

More info here.

Comments

[Peter Ritchie]

HttpWebREquest models a single request. HttpClient models, well, a client--something that makes multiple requests. HttpClient is much more recent, so more likely to have up-to-date knowledge. Although, I don't know if it's specifically faster in certain areas; but certainly more recommended if you're doing anything in the WebAPI space or REST.

[Darrel Miller]

HttpClient uses HttpWebRequest under the covers to actually make the HTTP request, so you should be able to get the same performance.

[user2325247]

yes..but again the performance depends on how the content is passed through the client.. I used FileStreamContent and now I had changed to ByteArrayContent...

[user2325247]

Thanks. This doesnt answer my question. I have an await on postAsync method and even that doesnt help. Am wondering how webrequest is faster when compared with httpclient?

[Darrel Miller]

Where did you get FileStreamContent? That isn't a default in-the-box HttpContent implementation.

[Darrel Miller]

Nope. I'm pretty sure it is not in either of those.

[user2325247]

my bad..it is the StreamContent I was talking about. msdn.microsoft.com/en-us/library/…

[Paddy]

Attribution is normally recommended: blogs.msdn.com/b/henrikn/archive/2012/02/11/…

[Giuseppe]

Attribution twice in the post and another time in comments is quite funny... :-)

[pim]

Great answer. The first 7 words alone could solve many of the "why can't I scrape xyz.com using HttpClient?"

[KansaiRobot]

The resource in the link uses JsonArray which does not seem to be available

[ToolmakerSteve]

Showing code snippets in the question, and in this answer, would be helpful. I speculate that your byte array is significantly larger than the default stream buffer, and this helped httpclient send it with less overhead.