When to use threading and how many threads to use

Question

I have a project for work. We had written a module and there as a #TODO to implement threading to improve the module. I'm a fairly new python programmer and decided to take a whack at it. While learning and implementing the threading, I had the question similar to How many threads is too many? because we have a queue of about maybe 6 objects that need to be processed, so why make 6 threads (or any threads at all) to process objects in a list or queue when the processing time is negligible anyway? (Each object takes at most about 2 seconds to process)

So I ran a little experiment. I wanted to know if there were performance gains from using threading. See my python code below:

import threading
import queue
import math
import time

results_total = []
results_calculation = []
results_threads = []

class MyThread(threading.Thread):
    def __init__(self, thread_id, q):
        threading.Thread.__init__(self)
        self.threadID = thread_id
        self.q = q

    def run(self):
        # print("Starting " + self.name)
        process_data(self.q)
        # print("Exiting " + self.name)


def process_data(q):
    while not exitFlag:
        queueLock.acquire()
        if not workQueue.empty():
            potentially_prime = True
            data = q.get()
            queueLock.release()
            # check if the data is a prime number
            # print("Testing {0} for primality.".format(data))
            for i in range(2, int(math.sqrt(data)+1)):
                if data % i == 0:
                    potentially_prime = False
                    break
            if potentially_prime is True:
                prime_numbers.append(data)
        else:
            queueLock.release()

for j in [1, 2, 3, 4, 5, 10, 15, 20, 25, 30, 40, 50, 75, 100, 150, 250, 500,
          750, 1000, 2500, 5000, 10000]:
    threads = []
    numberList = list(range(1, 10001))
    queueLock = threading.Lock()
    workQueue = queue.Queue()
    numberThreads = j
    prime_numbers = list()
    exitFlag = 0

    start_time_total = time.time()
    # Create new threads
    for threadID in range(0, numberThreads):
        thread = MyThread(threadID, workQueue)
        thread.start()
        threads.append(thread)

    # Fill the queue
    queueLock.acquire()
    # print("Filling the queue...")
    for number in numberList:
        workQueue.put(number)
    queueLock.release()
    # print("Queue filled...")
    start_time_calculation = time.time()
    # Wait for queue to empty
    while not workQueue.empty():
        pass

    # Notify threads it's time to exit
    exitFlag = 1

    # Wait for all threads to complete
    for t in threads:
        t.join()
    # print("Exiting Main Thread")
    # print(prime_numbers)
    end_time = time.time()
    results_total.append(
            "The test took {0} seconds for {1} threads.".format(
                end_time - start_time_total, j)
            )
    results_calculation.append(
            "The calculation took {0} seconds for {1} threads.".format(
                    end_time - start_time_calculation, j)
            )
    results_threads.append(
            "The thread setup time took {0} seconds for {1} threads.".format(
                    start_time_calculation - start_time_total, j)
            )
for result in results_total:
    print(result)
for result in results_calculation:
    print(result)
for result in results_threads:
    print(result)

This test finds the prime numbers between 1 and 10000. This set up is pretty much taken right from https://www.tutorialspoint.com/python3/python_multithreading.htm but instead of printing a simple string I ask the threads to find prime numbers. This is not actually what my real world application is but I can't currently test the code I've written for the module. I thought this was a good test to measure the effect of additional threads. My real world application deals with talking to multiple serial devices. I ran the test 5 times and averaged the times. Here are the results in a graph:

My questions regarding threading and this test are as follows:

1. Is this test even a good representation of how threads should be used? This is not a server/client situation. In terms of efficiency, is it better to avoid parallelism when you aren't serving clients or dealing with assignments/work being added to a queue?

2. If the answer to 1 is "No, this test isn't a place where one should use threads." then when is? Generally speaking.

3. If the answer to 1 is "Yes, this is ok to use threads in that case.", why does adding threads end up taking longer and quickly reaches a plateau? Rather, why would one want to use threads as it takes many times longer than calculating it in a loop.

I notice that as the work to threads ratio gets closer to 1:1, the time taken to set up the threads becomes longer. So is threading only useful where you create threads once and keep them alive as long as possible to handle requests that might enqueue faster than they can be calculated?

Solution 1

No, this is not a good place to use threads.

Generally, you want to use threads where your code is IO-bound; that is, it spends a significant amount of time waiting on input or output. An example might be downloading data from a list of URLs in parallel; the code can start requesting the data from the next URL while still waiting for the previous one to return.

That's not the case here; calculating primes is cpu-bound.

Solution 2

You're right to think that multithreading is a questionable move here for good reason. Multithreading, as it stands, is great and in the right applications can make a worlds difference in running times.

However, on the other hand, it also adds additional complexity to any program that implements it (especially in python). There are also time penalties to consider when using multithreading, such as those that occur when doing context switches or the time it takes to actually create a thread.

These time penalties are negligent when your program has to process thousands upon thousands of resource intense tasks because the time you would save from having multithreading far outweighs the little bit of time it takes to get the threads ready. For your case though, I'm not sure your needs meet those requirements. I didn't look to deep into what type of objects you were processing but you stated they only took about 2 seconds, which isn't awful and you also said that you only have 6 items at a time to process. So on average we can expect the main part of your scrip to run for 12 seconds. In my opinion, that is not necessary for multithreading because it will take a second or two to get the threads ready and then pass the instructions to them, whereas in one thread your python script would already be well into processing its second object in that time.

In short, I would save multithreading unless you need it. For example, huge datasets like those used for gene sequencing (big thing in Python) benefit greatly from it because multiple threads can help process these massive files concurrently. In your case, it doesn't look like the ends justify the means. Hope this helps

Solution 3

Threading in python is used to run multiple threads (tasks, function calls) at the same time. Note that this does not mean that they are executed on different CPUs. Python threads will NOT make your program faster if it already uses 100 % CPU time. In that case, you probably want to look into parallel programming.

from: https://en.wikibooks.org/wiki/Python_Programming/Threading

This is due to the mechanism called GIL. As Daniel pointed out, threads in python are only useful when you have IO-bound code. But then again, for IO-bound code it may be better to use lighter threads running on top of some event loop (using gevent, eventlet, asyncio or similar) as then you can easily run 100s (and more) of parallel operations with very little per thread overhead.

If what you want is to use more than 1 core of CPU to speed up execution, take a look at multiprocessing module.