Python Inter-thread Communication Tutorial with Examples

Inter-thread communication in Python refers to the ability for multiple threads to exchange information while working concurrently.

Proper communication between threads is essential for synchronizing tasks, sharing data, and coordinating execution.

Python provides several synchronization primitives and data structures, such as Locks, Queues, Events, Conditions, and Semaphores, that make it easier to manage communication between threads.

In this tutorial, we will cover:

1. What is Inter-thread Communication?
2. Using queue.Queue for Thread Communication
3. Using threading.Event for Signaling
4. Using threading.Condition for Coordinated Communication
5. Using threading.Lock for Synchronization
6. Using threading.Semaphore for Managing Resource Access
7. Examples and Use Cases

Let’s explore each method with examples!

1. What is Inter-thread Communication?

In Python, threads often need to share data or synchronize their actions to complete tasks efficiently.

Inter-thread communication allows threads to exchange information and ensure that resources are accessed in a thread-safe manner.

This is important to avoid race conditions, deadlocks, and other concurrency issues.

Python's Synchronization Primitives:

• queue.Queue: A thread-safe queue used to exchange data between threads.
• threading.Event: Used for signaling between threads.
• threading.Condition: Provides a mechanism for threads to wait for some condition before proceeding.
• threading.Lock: A simple lock for preventing multiple threads from accessing shared resources simultaneously.
• threading.Semaphore: Limits access to a resource by allowing a fixed number of threads to access it.

2. Using queue.Queue for Thread Communication

queue.Queue is a thread-safe data structure used to facilitate communication between threads. It ensures that multiple threads can safely add and retrieve items from the queue without any risk of data corruption.

Example 1: Using queue.Queue for Producer-Consumer Communication

import threading
import queue
import time

# Create a shared queue
task_queue = queue.Queue()

def producer():
    for i in range(5):
        item = f"Task-{i+1}"
        print(f"Producer: Adding {item}")
        task_queue.put(item)  # Add task to the queue
        time.sleep(1)

def consumer():
    while True:
        item = task_queue.get()  # Retrieve task from the queue
        if item is None:
            break  # Exit the loop when the producer is done
        print(f"Consumer: Processing {item}")
        time.sleep(2)  # Simulate task processing
        task_queue.task_done()

# Create and start producer and consumer threads
producer_thread = threading.Thread(target=producer)
consumer_thread = threading.Thread(target=consumer)

producer_thread.start()
consumer_thread.start()

# Wait for the producer to finish
producer_thread.join()

# Send signal to stop the consumer
task_queue.put(None)

# Wait for the consumer to finish
consumer_thread.join()

print("Main thread has finished.")

Explanation:

• A producer thread adds tasks to the queue.Queue().
• A consumer thread retrieves and processes tasks from the queue.
• The queue is thread-safe, so both the producer and consumer can operate on the queue concurrently without issues.

3. Using threading.Event for Signaling

threading.Event is a synchronization primitive used to signal threads. It allows one or more threads to wait for an event to be set before they can continue their execution.

Example 2: Using threading.Event for Signaling

import threading
import time

# Create an event object
event = threading.Event()

def worker():
    print("Worker: Waiting for event to be set")
    event.wait()  # Wait until the event is set
    print("Worker: Event has been set, proceeding...")

def controller():
    print("Controller: Doing some work")
    time.sleep(3)  # Simulate some work
    print("Controller: Setting the event")
    event.set()  # Signal the worker to continue

# Create and start the worker thread
worker_thread = threading.Thread(target=worker)
worker_thread.start()

# Create and start the controller thread
controller_thread = threading.Thread(target=controller)
controller_thread.start()

# Wait for both threads to complete
worker_thread.join()
controller_thread.join()

print("Main thread has finished.")

Explanation:

• The worker thread waits for the event to be set before it continues execution.
• The controller thread sets the event after some work is completed, signaling the worker to continue.

4. Using threading.Condition for Coordinated Communication

threading.Condition is used to coordinate communication between threads by allowing one or more threads to wait for a certain condition to be met.

Example 3: Using threading.Condition for Coordinated Communication

import threading
import time

# Create a condition object
condition = threading.Condition()

# Shared resource
shared_data = []

def producer():
    global shared_data
    with condition:
        for i in range(1, 6):
            shared_data.append(f"Data-{i}")
            print(f"Producer: Added Data-{i}")
            time.sleep(1)  # Simulate work
            condition.notify()  # Notify the consumer that data is available

def consumer():
    global shared_data
    with condition:
        while True:
            if not shared_data:
                print("Consumer: Waiting for data...")
                condition.wait()  # Wait until notified by the producer
            else:
                item = shared_data.pop(0)
                print(f"Consumer: Processed {item}")
                time.sleep(2)  # Simulate task processing
                if item == "Data-5":
                    break

# Create and start producer and consumer threads
producer_thread = threading.Thread(target=producer)
consumer_thread = threading.Thread(target=consumer)

producer_thread.start()
consumer_thread.start()

# Wait for both threads to finish
producer_thread.join()
consumer_thread.join()

print("Main thread has finished.")

Explanation:

• The producer adds data to a shared list and notifies the consumer when data is available.
• The consumer waits for the producer to notify it, processes the data, and then waits again for the next notification.

5. Using threading.Lock for Synchronization

threading.Lock is used to synchronize access to a shared resource. It ensures that only one thread can access the resource at a time, preventing race conditions.

Example 4: Using threading.Lock for Synchronization

import threading
import time

# Create a lock object
lock = threading.Lock()

# Shared resource
counter = 0

def increment_counter():
    global counter
    for _ in range(5):
        with lock:
            counter += 1
            print(f"Thread {threading.current_thread().name}: Counter = {counter}")
        time.sleep(1)

# Create and start two threads
thread1 = threading.Thread(target=increment_counter, name="T1")
thread2 = threading.Thread(target=increment_counter, name="T2")

thread1.start()
thread2.start()

# Wait for both threads to finish
thread1.join()
thread2.join()

print("Final Counter Value:", counter)

Explanation:

• The lock ensures that only one thread can modify the counter at a time.
• Both threads increment the counter, and the lock prevents race conditions.

6. Using threading.Semaphore for Managing Resource Access

A semaphore controls access to a resource by allowing a fixed number of threads to access the resource concurrently. It is useful when you have limited resources, such as a pool of connections.

Example 5: Using threading.Semaphore for Resource Management

import threading
import time

# Create a semaphore allowing 2 threads to access the resource at a time
semaphore = threading.Semaphore(2)

def worker(name):
    print(f"{name}: Waiting to acquire the semaphore")
    with semaphore:
        print(f"{name}: Acquired the semaphore")
        time.sleep(2)  # Simulate task
    print(f"{name}: Released the semaphore")

# Create and start multiple worker threads
threads = []
for i in range(5):
    thread = threading.Thread(target=worker, args=(f"Worker-{i+1}",))
    threads.append(thread)
    thread.start()

# Wait for all threads to finish
for thread in threads:
    thread.join()

print("Main thread has finished.")

Explanation:

• The semaphore allows only 2 threads to hold the resource at a time.
• Other threads wait until a thread releases the semaphore before acquiring it.

7. Examples and Use Cases

Example 6: Combining Queue and Lock for Thread-safe Data Sharing

You can combine multiple synchronization mechanisms for more complex thread communication.

import threading
import queue
import time

# Create a shared queue and lock
task_queue = queue.Queue()
lock = threading.Lock()

def producer():
    for i in range(5):
        with lock:
            task = f"Task-{i+1}"
            print(f"Producer: Adding {task}")
            task_queue.put(task)  # Add task to the queue
        time.sleep(1)

def consumer():
    while True:
        task = task_queue.get()
        if task is None:
            break  # Exit loop

 when the producer is done
        with lock:
            print(f"Consumer: Processing {task}")
        time.sleep(2)
        task_queue.task_done()

# Create and start producer and consumer threads
producer_thread = threading.Thread(target=producer)
consumer_thread = threading.Thread(target=consumer)

producer_thread.start()
consumer_thread.start()

# Wait for producer to finish and signal the consumer to stop
producer_thread.join()
task_queue.put(None)
consumer_thread.join()

print("Main thread has finished.")

Explanation:

• The lock ensures that both the producer and consumer safely interact with the queue.
• The queue allows for asynchronous communication between the producer and consumer.

Summary of Key Concepts for Python Inter-thread Communication

Conclusion

In Python, inter-thread communication is critical for ensuring that threads can safely share data, signal each other, and work together efficiently. In this tutorial, we covered:

• Using queue.Queue for producer-consumer communication.
• Using threading.Event for signaling between threads.
• Using threading.Condition for more coordinated thread communication.
• Using threading.Lock to synchronize access to shared resources.
• Using threading.Semaphore to manage access to a limited resource.