Python Tutorial / Choose Threads, Processes or asyncio

Threads, processes and asyncio offer different ways to make progress on multiple tasks. Choose according to CPU work, external waiting and library support, then measure the actual workload rather than assuming concurrency makes it faster.

Distinguish concurrency from parallel execution

In ordinary GIL-enabled CPython, adding threads does not directly parallelize arbitrary pure-Python CPU work. Extensions releasing the GIL and free-threaded builds change the conditions, so consider the interpreter build and libraries together.

For the relevant APIs and platform requirements, see Choose Threads, Processes or asyncio documentation.

Compare the interfaces with one complete program

Save and run this as a .py file rather than assuming identical process behavior in a notebook or interactive session.

Save the complete code below as concurrency.py in a separate practice folder, using UTF-8. Run python concurrency.py in a terminal with your project virtual environment selected. Outside a virtual environment on macOS or Ubuntu, use python3 when that is your interpreter command.

import asyncio
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def square(value):
    return value * value

async def async_square(value):
    await asyncio.sleep(0.01)
    return square(value)

async def gather_values():
    return await asyncio.gather(*(async_square(x) for x in [1, 2, 3]))

def main():
    with ThreadPoolExecutor(max_workers=2) as pool:
        print(list(pool.map(square, [1, 2, 3])))
    with ProcessPoolExecutor(max_workers=2) as pool:
        print(list(pool.map(square, [1, 2, 3])))
    print(asyncio.run(gather_values()))

if __name__ == "__main__":
    main()

Expected output:

[1, 4, 9]
[1, 4, 9]
[1, 4, 9]

Respect entry points and cooperative waiting

Process startup, including on Windows, can import the main module. The main guard prevents recursive startup. Defining square at module level makes it accessible to worker processes.

An async function performing long CPU work or synchronous network calls without yielding can block the event loop. asyncio.sleep demonstrates cooperative waiting; it does not make a real HTTP request.

Measure realistic work and design failure handling

This tiny square example illustrates APIs and is not a performance benchmark. Process startup can cost more than the work itself. Measure representative data while recording environment, worker counts and memory.

map returns results in input order, although tasks may finish in another order. Bound concurrency, synchronize shared state and design error propagation, cancellation and shutdown. Do not create unlimited tasks or rely on unsynchronized changes to a shared list. asyncio.run cannot be nested inside an already running event loop.

The API collection project gives a concrete workflow whose network and storage phases should be understood before introducing parallel requests.