Are there any issues with my multithreaded HttpClient?

Question

So after dabbling with Java and HttpClient, I've decided to transition to C# to try and lower the memory usage while increasing speed. I've been reading tons of articles provided by members on here regarding async vs multithreading. It appears multithreading would be the better direction.

My program will be accessing a server and sending the same requests over and over until a 200 code is retrieved. Reason being is because during peak hours the traffic is very high. It makes the server is VERY hard to reach and throws 5xx errors. The code is very bare, I didn't want to add the loop yet to help with simplicity for the viewers.

I feel that I am heading towards the right direction but I would like to reach out to the community to break out of any bad habits. Thanks again for reading.

Note to moderators: I am asking to check my code for discrepancies, I am well aware that it's okay to multithread HttpClient.

using System;
using System.Net.Http;
using System.Collections.Generic;
using System.Net.Http.Headers;
using System.Threading;

namespace MF
{
    class MainClass
    {
        public static HttpClient client = new HttpClient();
        static string url = "http://www.website.com";

        public static void getSession() {
            StringContent queryString = new StringContent("{json:here}");

            // Send a request asynchronously continue when complete
            var result = client.PostAsync(new Uri(url), queryString).Result;

            // Check for success or throw exception
            string resultContent = result.Content.ReadAsStringAsync().Result;

            Console.WriteLine(resultContent);
        }
        public static void Run() 
        {
            getSession ();
          // Not yet implemented yet..
          //doMore ();
        }

        public static void Main (string[] args)
        {
            Console.WriteLine ("Welcome!");

            // Create the threads
            ThreadStart threadref1 = new ThreadStart(Run);
            ThreadStart threadref2 = new ThreadStart(Run);
            Console.WriteLine("In the main: Creating the threads...");
            Thread Thread1 = new Thread(threadref1);
            Thread Thread2 = new Thread(threadref1);

            // Start the thread
            Thread1.Start();
            Thread2.Start();
        }
    }
}

Also, I am not sure if it matters but I am running this on my MacBook Pro and plan to run it on my BeagleBoard.

Solution 1

Here is how I would do it if I were you. I would leverage async as much as possible as it's much more efficient than using Threads (you most likely won't have to context-switch all the time which is expensive).

class MainClass
{
    public static HttpClient client = new HttpClient();
    static string url = "http://www.website.com";

    public static async Task getSessionAsync()
    {
        StringContent queryString = new StringContent("{json:here}");

        // Send a request asynchronously continue when complete
        using (HttpResponseMessage result = await client.PostAsync(url, queryString))
        {
            // Check for success or throw exception
            string resultContent = await result.Content.ReadAsStringAsync();
            Console.WriteLine(resultContent);
        }
    }

    public static async Task RunAsync()
    {
        await getSessionAsync();
        // Not yet implemented yet..
        //doMore ();
    }

    public static void Main(string[] args)
    {
        Console.WriteLine("Welcome!");

        const int parallelRequests = 5;
        // Send the request X times in parallel
        Task.WhenAll(Enumerable.Range(1, parallelRequests).Select(i => RunAsync())).GetAwaiter().GetResult();

        // It would be better to do Task.WhenAny() in a while loop until one of the task succeeds
        // We could add cancellation of other tasks once we get a successful response
    }
}

Note that I do agree with @Damien_The_Unbeliever: if the server has issues under heavy load, you shouldn't be adding unnecessary load (doing X times the same request) and contribute to the server's issues. Ideally you'd fix the server code but I can understand that it's not yours.

Solution 2

The following methods are thread-safe:

CancelPendingRequests
DeleteAsync
GetAsync
GetByteArrayAsync
GetStreamAsync
GetStringAsync
PostAsync
PutAsync
SendAsync

More details:

• Is HttpClient safe to use concurrently?
• http://blogs.msdn.com/b/henrikn/archive/2012/08/07/httpclient-httpclienthandler-and-httpwebrequesthandler.aspx

Solution 3

Messing with the headers is not thread safe.

For example, swapping out an OAuth access token with a new one is not thread safe. Facing this right now with a Xamarin mobile app. I have a crash report where one thread was modifying headers while another thread was attempting a request.

Solution 4

Reading the docummentation of HttpClient:

Any public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe.

Don't take risks. Have a separate HTTP client per thread.

It seems that you are blocking the thread waiting for the reply and you are doing it from a thread that does not do anything extra. Then why bother with async/await? You can use plain simple blocking calls.

Also - your program now finishes immediately after starting the threads. You might want to wait for the threads to finish before returning from main. You can do that by this code at the end of your program:

Thread1.Join();
Thread2.Join();

Update based on comments:

1. Decide how many parallel requests you want to make - this will be your number of threads.
2. Make main thread wait for signal using ManualResetEvent.
3. In each thread keep submitting your requests. As soon as you get answer that you are waiting for - signal the ManualResetEvent and allow your main function to return.

View more solutions