Thread Safety in Python's dictionary
Solution 1
Python's built-in structures are thread-safe for single operations, but it can sometimes be hard to see where a statement really becomes multiple operations.
Your code should be safe. Keep in mind: a lock here will add almost no overhead, and will give you peace of mind.
https://web.archive.org/web/20201108091210/http://effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm has more details.
Solution 2
Google's style guide advises against relying on dict atomicity
This is explained in further detail at: Is Python variable assignment atomic?
Do not rely on the atomicity of built-in types.
While Python’s built-in data types such as dictionaries appear to have atomic operations, there are corner cases where they aren’t atomic (e.g. if
__hash__
or__eq__
are implemented as Python methods) and their atomicity should not be relied upon. Neither should you rely on atomic variable assignment (since this in turn depends on dictionaries).Use the
Queue
module's Queue data type as the preferred way to communicate data between threads. Otherwise, use the threading module and its locking primitives. Learn about the proper use of condition variables so you can usethreading.Condition
instead of using lower-level locks.
And I agree with this one: there is already the GIL in CPython, so the performance hit of using a Lock will be negligible. Much more costly will be the hours spent bug hunting in a complex codebase when those CPython implementation details change one day.
nmat
Updated on September 20, 2021Comments
-
nmat over 2 years
I have a class which holds a dictionary
class OrderBook: orders = {'Restaurant1': None, 'Restaurant2': None, 'Restaurant3': None, 'Restaurant4': None} @staticmethod def addOrder(restaurant_name, orders): OrderBook.orders[restaurant_name] = orders
And I am running 4 threads (one for each restaurant) that call the method
OrderBook.addOrder
. Here is the function ran by each thread:def addOrders(restaurant_name): #creates orders ... OrderBook.addOrder(restaurant_name, orders)
Is this safe, or do I have to use a lock before calling
addOrder
? -
hobs almost 10 yearsHere's the effbot.org why-to/how-to on implementing a lock
-
andy over 9 yearsShould consider single operation vs. composite operations,such as get-add-set.
-
Shihab Shahriar Khan about 7 yearsproblem is, when I'm frequently reading/writing that dict, that peace of mind will cost me a lot.
-
max over 4 years"a lock here will add almost no overhead": why is that?
-
Ruli over 3 yearsThe link is dead, see it e.g. here: titanwolf.org/Network/Articles/…
-
lionbigcat over 2 yearsdo you mind adding sources? would like to read up on, esp. your
Note
. -
Zack Light almost 2 yearsWhat about many add operations to something like a list. When a list resizes automatically, would the change of the reference to underlying containers cause any problems?