Python threading module
Python’sthreadingmodule is one of the standard libraries used to implement multithreaded programming. Multithreading allows a program to execute multiple tasks at the same time, thereby improving program efficiency and responsiveness.
threadingThe module provides tools for creating and managing threads, allowing developers to easily write concurrent programs.
What is a thread?
You can think of threads as employees in an office:
-
A single-threaded program is like having only one employee, who must sequentially complete all tasks such as printing documents, replying to emails, and making coffee.
-
A multithreaded program is like having multiple employees, who cansimultaneouslyperform different tasks, greatly improving work efficiency.
In computer science:
- Process: A running program with independent memory space (for example, the browser and music player you have open at the same time are two processes).
- Thread: An independent execution flow within a process, and the basic unit of CPU scheduling. All threads in the same processshare the process’s memory space(such as global variables).
Why use multithreading?
In a single-threaded program, tasks are executed one after another. If a task needs to wait (e.g., waiting for a network response or file reading), the entire program is blocked until the task completes. Multithreading allows the program to continue executing other tasks while waiting for one task, thus improving overall program performance.
Python threads and the Global Interpreter Lock (GIL)
Python has a mechanism called the Global Interpreter Lock (GIL), which ensures that only one thread can execute Python bytecode at any given time.
What does this mean?For CPU-intensive tasks (such as scientific computing, image processing), due to the GIL, multithreading usually cannot take advantage of multiple cores to speed up computation, and may even become slower due to thread switching overhead.
So, where is Python multithreading useful?For I/O-intensive tasks (such as network requests, file reading/writing, waiting for user input), threads release the GIL while waiting for I/O operations to complete, allowing other threads to run. This can significantly improve the overall responsiveness and efficiency of the program, because while you wait for a webpage response, the program can handle another task.
How to use the threading module?
To usethreadingthe module, the first step is to import it:
import threading import time # 用于模拟耗时操作
The most basic way to create a thread is to usethreading.Threadthe class.
Syntax explanation:
thread_obj = threading.Thread(target=函数名, args=(参数元组,))
- target: Specifies the function to be executed after the thread starts.
- args: The argument passed to the target function must be a tuple. If there is only one argument, it must be written in
(parameter,)the form `(arg,)`.
1. Creating threads
In Python, you can, by inheriting thethreading.Threadclass or by directly using thethreading.Threadconstructor, create a thread.
Method 1: Inheritancethreading.ThreadClass
Example
class MyThread(threading.Thread):
def run(self):
print("Thread starts executing")
# Write the code to be executed by the thread here
print("Thread execution finished")
# Create a thread instance
thread = MyThread()
# Start the thread
thread.start()
# Wait for the thread to complete
thread.join()
print("Main thread finished")
Method 2: Using thethreading.Threadconstructor.
Example
def my_function():
print("Thread starts executing")
# Write the code to be executed by the thread here
print("Thread execution finished")
# Create a thread instance
thread = threading.Thread(target=my_function)
# Start the thread
thread.start()
# Wait for the thread to complete
thread.join()
print("Main thread finished")
2. Thread synchronization
In multithreaded programming, multiple threads may access shared resources at the same time, which can lead to data inconsistency problems. To avoid this, thread synchronization mechanisms such as locks (Lock）。
Example
# Create a lock object
lock = threading.Lock()
def my_function():
with lock:
print("Thread starts executing")
# Write the code to be executed by the thread here
print("Thread execution finished")
# Create a thread instance
thread1 = threading.Thread(target=my_function)
thread2 = threading.Thread(target=my_function)
# Start the thread
thread1.start()
thread2.start()
# Wait for the thread to complete
thread1.join()
thread2.join()
print("Main thread finished")
3. Inter-thread communication
Inter-thread communication can be implemented through a queue (Queue).Queueis thread-safe and can safely pass data between multiple threads.
Example
import queue
def worker(q):
while not q.empty():
item = q.get()
print(f"Processing item: {item}")
q.task_done()
# Create a queue and populate it with data
q = queue.Queue()
for i in range(10):
q.put(i)
# Create thread instances
thread1 = threading.Thread(target=worker, args=(q,))
thread2 = threading.Thread(target=worker, args=(q,))
# Start the threads
thread1.start()
thread2.start()
# Wait for all items in the queue to be processed
q.join()
print("All items processed")
Common classes, methods, and attributes
1. Core classes
| Class/Method/Attribute | Description | Example |
|---|---|---|
threading.Thread | Thread class, used to create and manage threads | t = Thread(target=func, args=(1,)) |
threading.Lock | Mutex lock (primitive lock) | lock = Lock() |
threading.RLock | Reentrant lock (can be acquired multiple times by the same thread) | rlock = RLock() |
threading.Event | Event object, used for thread synchronization | event = Event() |
threading.Condition | Condition variable, used for complex thread coordination | cond = Condition() |
threading.Semaphore | Semaphore, controls the number of concurrent threads | sem = Semaphore(3) |
threading.BoundedSemaphore | Bounded semaphore (prevents the count from exceeding the initial value) | b_sem = BoundedSemaphore(2) |
threading.Timer | Timer thread, delayed execution | timer = Timer(5.0, func) |
threading.local | Thread-local data (stored independently for each thread) | local_data = threading.local() |
2. Common methods/attributes of Thread objects
| Method/Attribute | Description | Example |
|---|---|---|
start() | Start the thread | t.start() |
run() | Method executed by the thread (can be overridden) | Override this method when defining a custom class |
join(timeout=None) | Blocks the current thread until the target thread finishes | t.join() |
is_alive() | Check whether the thread is running | if t.is_alive(): |
name | Thread name (can be modified) | t.name = "Worker-1" |
daemon | Daemon thread flag (automatically ends when the main thread exits) | t.daemon = True |
ident | Thread identifier (`None` before the thread is started)None） | print(t.ident) |
3. Common methods of Lock/RLock
| Method | Description | Example |
|---|---|---|
acquire(blocking=True, timeout=-1) | Acquire the lock (blocking or non-blocking) | lock.acquire() |
release() | Release the lock | lock.release() |
locked() | Check whether the lock is held | if not lock.locked(): |
4. Common methods of Event
| Method | Description | Example |
|---|---|---|
set() | Set the event to true and wake up all waiting threads | event.set() |
clear() | Reset the event to false | event.clear() |
wait(timeout=None) | Block until the event is true or a timeout occurs | event.wait(2.0) |
is_set() | Check the event status | if event.is_set(): |
5. Common methods of Condition
| Method | Description | Example |
|---|---|---|
wait(timeout=None) | Release the lock and block until notified or a timeout occurs | cond.wait() |
notify(n=1) | Wake up at mostnwaiting threads | cond.notify(2) |
notify_all() | Wake up all waiting threads | cond.notify_all() |
6. Module-level functions/attributes
| Function/Attribute | Description | Example |
|---|---|---|
threading.active_count() | Return the number of currently active threads | print(threading.active_count()) |
threading.current_thread() | Return the current thread object | print(threading.current_thread().name) |
threading.enumerate() | Return a list of all active threads | for t in threading.enumerate(): |
threading.main_thread() | Return the main thread object | if threading.current_thread() is threading.main_thread(): |
threading.get_ident() | Return the current thread’s identifier (Python 3.3+) | print(threading.get_ident()) |
Example
1. Basic Thread Creation
Example
def worker(num):
print(f"Worker {num} started")
threads = []
for i in range(3):
t = threading.Thread(target=worker, args=(i,))
threads.append(t)
t.start()
for t in threads:
t.join()
2. Using Locks to Protect Shared Resources
Example
count = 0
def increment():
global count
with lock: # Automatically acquire and release the lock
count += 1
threads = [threading.Thread(target=increment) for _ in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
print(count) # Output: 10
3. Event Synchronization
Example
def waiter():
print("Waiting for event...")
event.wait()
print("Event triggered!")
t = threading.Thread(target=waiter)
t.start()
# The main thread triggers the event
threading.Event().wait(2.0) # Simulated delay
event.set()
t.join()
4. Producer-Consumer Model (Condition)
Example
from threading import Condition
queue = []
cond = Condition()
MAX_ITEMS = 5
def producer():
for _ in range(10):
with cond:
while len(queue) >= MAX_ITEMS:
cond.wait()
item = random.randint(1, 100)
queue.append(item)
print(f"Produced {item}")
cond.notify()
def consumer():
for _ in range(10):
with cond:
while not queue:
cond.wait()
item = queue.pop(0)
print(f"Consumed {item}")
cond.notify()
threading.Thread(target=producer).start()
threading.Thread(target=consumer).start()
Notes
Global Interpreter Lock (GIL): Python's GIL restricts only one thread from executing Python bytecode at a time. Therefore, in CPU-intensive tasks, multithreading may not provide performance improvements. For I/O-intensive tasks, multithreading is still beneficial.
Thread Safety: In a multithreaded environment, ensure that access to shared resources is thread-safe, avoiding data races and deadlocks.
Thread Count: Creating too many threads may exhaust system resources and affect program performance. Control the number of threads reasonably, or use a thread pool (
ThreadPoolExecutor) to manage threads.