Python Threading: Divide Tasks Across Multiple Threads

You will learn how to create and start threads in Python, assign independent work to each thread, wait for completion with join(), and safely collect results. You will also learn when threads are useful and when another concurrency approach is a better fit.

A thread is a separate path of execution inside one Python process. Threads let a program begin one task without having to wait for another task to finish first.

Threading is especially useful when work spends time waiting, such as:

• Downloading data from several URLs
• Reading several files
• Calling external APIs
• Waiting for database or network responses

Python's threading module provides the Thread class. You create a thread with a function to run, call start() to schedule it, and call join() when the main program must wait for it to finish.

For standard CPython, the Global Interpreter Lock (GIL) means threads usually do not make CPU-heavy pure-Python calculations run in parallel across CPU cores. For CPU-bound work such as large numerical calculations or image processing, multiprocessing is often more appropriate. Threads are still an excellent fit for I/O-bound work because a waiting thread can allow another thread to run.

Imagine a restaurant manager assigning orders to several cooks.

• The main thread is the manager.
• Each worker thread is a cook.
• A task is an order.
• start() tells a cook to begin working.
• join() means the manager waits until every cook has completed their assigned order.

This helps only when the orders can be prepared independently. If one cook needs the result of another cook's work first, the manager must coordinate that dependency.

Create a Thread by passing a callable through target and its arguments through args.

from threading import Thread
import time


def process_task(task_number):
    print(f"Task {task_number} started")
    time.sleep(1)  # Simulates waiting for I/O
    print(f"Task {task_number} finished")


threads = []

for task_number in range(1, 4):
    thread = Thread(target=process_task, args=(task_number,))
    threads.append(thread)
    thread.start()

for thread in threads:
    thread.join()

print("All tasks are complete.")

args=(task_number,) is a one-item tuple. The trailing comma is required; (task_number) is only the value in parentheses, not a tuple.

The three tasks start near the same time. Because each one waits for one second, the overall elapsed time is roughly one second rather than roughly three seconds. The exact order of printed lines is not guaranteed.

Consider this smaller example:

from threading import Thread
import time


def load(name):
    print(f"Loading {name}")
    time.sleep(0.5)
    print(f"Loaded {name}")


first = Thread(target=load, args=("profile",))
second = Thread(target=load, args=("messages",))

first.start()
second.start()

first.join()
second.join()

print("Page data is ready")

1. Python defines load; it does not run it yet.
2. first and second are thread objects, each configured to call load with a different argument.
3. first.start() begins load("profile") in a separate thread.
4. second.start() begins load("messages") in another separate thread.
5. Both threads may reach sleep() around the same time. While one is waiting, the other can execute.
6. pauses the main thread until the first task is complete.

Threads are commonly used for independent I/O-bound operations:

• API aggregation: request a user's profile, notifications, and recommendations from different services at once.
• File tools: read, upload, or copy several independent files concurrently.
• Web scraping: download pages concurrently while respecting the target site's rate limits and terms.
• Monitoring: poll multiple servers or devices without waiting for each request one at a time.
• Desktop applications: perform slow file or network work in a worker thread so the user interface remains responsive.

Threads do not automatically make tasks correct. Tasks should be independent or carefully synchronized when they share data.

In production code, concurrent.futures.ThreadPoolExecutor is often clearer than manually creating one Thread per task. A pool limits the number of active workers and provides Future objects for results and errors.

from concurrent.futures import ThreadPoolExecutor
import time


def fetch_record(record_id):
    time.sleep(0.2)  # Represents a network request
    return {"id": record_id, "status": "ok"}


record_ids = [101, 102, 103, 104]

with ThreadPoolExecutor(max_workers=4) as executor:
    records = list(executor.map(fetch_record, record_ids))

print(records)

Useful real-project patterns include:

• Bounded concurrency: choose max_workers rather than launching thousands of threads.
• Validation before work: reject invalid inputs before submitting tasks.
• Error handling: call future.result() so exceptions from a worker are surfaced.

Calling the function instead of passing it

Broken code:

thread = Thread(target=process_task(1))

This runs process_task(1) immediately in the main thread. Pass the function itself and its arguments separately:

thread = Thread(target=process_task, args=(1,))

Forgetting start()

Creating a thread does not run it:

thread = Thread(target=process_task, args=(1,))
# Nothing happens yet.
thread.start()

Forgetting join() when results are needed

Without join(), the main thread can continue before workers complete. Join threads, or use futures, before using their final results.

Assuming print order is predictable

Thread scheduling varies. Do not write program logic that depends on one thread printing or finishing before another unless you explicitly coordinate them.

Updating shared data unsafely

This can lose updates:

counter = 0


def ():
     counter
    counter += 

Choose threads when tasks mostly wait on I/O and can proceed independently. Choose sequential code when concurrency adds no practical benefit.

from threading import Thread

thread = Thread(target=function_name, args=(argument_1,))
thread.start()  # Begin work in another thread
thread.join()   # Wait for that work to finish

• target receives a function, without parentheses.
• args must be a tuple; use (value,) for one argument.
• Start all threads before joining them to allow overlap.
• Output order is nondeterministic.
• Use threads mainly for I/O-bound tasks.
• Use ThreadPoolExecutor for a controlled number of similar tasks.
• Protect shared mutable data with threading.Lock, or avoid sharing it.

What is threading in Python?

Threading lets one Python process make progress on multiple tasks at overlapping times. It is most useful for tasks that spend time waiting on I/O.

Does start() wait for a thread to finish?

No. start() begins the thread and returns quickly. Use join() when the current thread must wait for completion.

Why is args=(value,) written with a comma?

The comma creates a one-item tuple. Without it, Python treats (value) as simply value.

Can two Python threads run at exactly the same time?

For I/O-bound tasks, threads can overlap effectively. In standard CPython, CPU-bound pure-Python code is limited by the GIL, so threads usually do not provide multi-core CPU parallelism.

How do I get values back from a thread?

Use a Queue, store results with appropriate synchronization, or use ThreadPoolExecutor and retrieve each future's result.

Is it safe for threads to modify the same list or dictionary?

Do not assume shared updates are safe as a complete operation. Use a lock or a queue, or structure the program so each worker returns an independent result.

Should I create one thread for every item in a large list?

Usually no. Use ThreadPoolExecutor(max_workers=...) to limit concurrency and prevent excessive resource use.

• ThreadPoolExecutor — a higher-level way to run many I/O tasks with a limited worker pool.
• join() — waits for a thread to finish before continuing.
• Thread synchronization — coordinates threads that share resources or depend on timing.
• Locks (threading.Lock) — protect shared mutable state from conflicting updates.
• Queue (queue.Queue) — safely passes work or results between threads.
• Multiprocessing — a better fit for CPU-bound Python workloads.
• Asyncio — an alternative concurrency model for many I/O operations.
• The Global Interpreter Lock (GIL) — explains an important limit of CPU-bound threading in CPython.