When working with multithreaded applications in Python, handling shared state is one of the most common hurdles. A frequent scenario involves processing a parent dictionary where individual worker threads are assigned distinct nested dictionaries to modify. But is mutating different inner dictionaries in separate threads actually thread-safe?

The Short Answer: Yes, with Caveats

Yes, it is thread-safe provided that each thread strictly operates on its own distinct inner dictionary object and does not mutate the outer (parent) dictionary concurrently.

In Python, dictionaries are reference types. When you pass data['a'] to Thread 1 and data['b'] to Thread 2, each thread is mutating a completely independent object in memory. Because the threads are not reading or writing to the same dictionary keys or the same memory addresses, no data races or race conditions will occur on those objects.

Understanding the Code Pattern

Consider the following implementation using concurrent.futures.ThreadPoolExecutor:

from concurrent.futures import ThreadPoolExecutor

data = {
    'a': {'c': 5, 'd': 6},
    'b': {'I': 8}
}

def process(sub_dict):
    # Safe: Modifying a nested dictionary exclusive to this thread
    sub_dict['processed'] = True
    sub_dict['value'] = sub_dict.get('c', 0) * 10

with ThreadPoolExecutor(max_workers=2) as executor:
    # Pass distinct inner dictionaries directly to worker threads
    futures = [executor.submit(process, sub_dict) for sub_dict in data.values()]
    
    # Wait for all tasks to complete
    for future in futures:
        future.result()

print(data)
# Output:
# {'a': {'c': 5, 'd': 6, 'processed': True, 'value': 50},
#  'b': {'I': 8, 'processed': True, 'value': 0}}

In this example, the parent dictionary (data) is only read during the list comprehension setup before the tasks begin running in earnest. Once inside the process function, worker threads operate strictly on their allocated nested dictionary (sub_dict).

Important Caveats to Watch Out For

While the pattern above is secure, subtle bugs can arise if you violate any of the following boundaries:

1. Modifying the Parent Dictionary Concurrently

If one thread adds, deletes, or reassigns keys in the top-level data dictionary while other threads are iterating over it, Python will raise a RuntimeError: dictionary changed size during iteration.

2. Shared References (Aliasing)

Ensure that your nested dictionaries are truly unique objects. If multiple keys in the parent dictionary point to the exact same inner dictionary in memory, concurrent writes will create a race condition:

# DANGEROUS: Both keys reference the exact same dictionary in memory
shared_dict = {'status': 'pending'}
data = {
    'task1': shared_dict,
    'task2': shared_dict
}

3. Read-Modify-Write Operations

Even inside an isolated dictionary, operations that are not atomic (such as incrementing counters: sub_dict['count'] += 1) require synchronization if multiple threads ever touch the same sub-dictionary.

Best Practices for Thread-Safe Data Manipulation

  • Pass Disjoint Objects: Explicitly pass only the specific sub-object each thread needs, rather than the entire global or parent container.
  • Use Thread Locks for Shared State: If threads must occasionally write to the parent dictionary or cross-reference other nested keys, guard those operations with a threading.Lock.
  • Consider Free-Threaded Python (3.13+): With Python 3.13's experimental support for disabling the Global Interpreter Lock (GIL via PEP 703), relying on object isolation rather than GIL-dependent atomicity is more critical than ever.

Conclusion

Dividing workload by assigning distinct nested dictionaries to independent threads is an effective and safe concurrency pattern in Python. As long as the parent dictionary remains static during execution and inner dictionaries are not shared between workers, your implementation will remain thread-safe without needing explicit locks.