Question
How to Profile a Python Script: Timing and Profiling in Python
Question
Project Euler and similar coding challenges often impose time limits, and people frequently compare how fast their solutions run. In Python, a common beginner approach is to add manual timing code inside __main__, but that can feel clumsy.
What is a good way to measure and profile how long a Python program takes to run?
Short Answer
By the end of this page, you will understand the difference between timing a Python program and profiling it, when to use tools like time, timeit, and cProfile, and how to identify which parts of your script are actually slow instead of only measuring total runtime.
Concept
When people say they want to know how long a Python script takes, they may mean two different things:
- Timing: measuring the total runtime of the program.
- Profiling: finding out where the program spends its time.
These are related, but they solve different problems.
- If you only want to know, "Did my script finish in 0.2 seconds or 2 seconds?" then simple timing is enough.
- If you want to know, "Which function is making it slow?" then you need profiling.
In Python, there are several standard tools for this:
timeortime.perf_counter()for quick wall-clock timingtimeitfor benchmarking small code snippets reliablycProfilefor profiling full programs and functionspstatsfor reading and sorting profiler results
This matters in real programming because optimization should be based on evidence. Without profiling, developers often guess incorrectly about what is slow. A profiler shows real bottlenecks so you can improve the right part of the code.
Mental Model
Think of your program like a road trip.
- Timing tells you the total trip took 3 hours.
- Profiling tells you where the time went: 2 hours in traffic, 30 minutes at fuel stops, and 30 minutes driving normally.
If you only know the total time, you know the trip was slow, but not why. Profiling is the map that shows the delays.
Syntax and Examples
If you only want total runtime, a simple pattern is to use time.perf_counter().
import time
start = time.perf_counter()
# Code to measure
result = sum(i * i for i in range(1_000_000))
end = time.perf_counter()
print(f"Result: {result}")
print(f"Elapsed time: {end - start:.6f} seconds")
time.perf_counter() is preferred for timing because it provides a high-resolution clock suitable for measuring durations.
If you want to benchmark a small statement multiple times, use timeit.
import timeit
execution_time = timeit.timeit("sum(i * i for i in range(1000))", number=1000)
print(f"Total time: {execution_time:.6f} seconds")
This runs the code many times and gives a more stable result than timing a single execution.
If you want to profile a script and see which functions consume time, use cProfile.
From the command line:
Step by Step Execution
Consider this example:
import cProfile
def slow_square_sum(n):
total = 0
for i in range(n):
total += i * i
return total
cProfile.run("slow_square_sum(100000)")
Here is what happens step by step:
- Python imports the
cProfilemodule. - The function
slow_square_sum(n)is defined. cProfile.run("slow_square_sum(100000)")starts the profiler.- The profiler executes the string as Python code.
- The function runs its loop from
0to99999. - On each iteration,
i * iis computed and added tototal. - When the function finishes, the profiler stops collecting data.
- Python prints a profiling report.
A typical report includes columns like these:
ncalls: how many times the function was calledtottime: time spent in the function body itselfcumtime: time spent in the function including other function calls
Real World Use Cases
Profiling and timing are useful in many practical situations:
- Coding competitions: check whether your solution fits a time limit.
- Data processing scripts: find which parsing, filtering, or aggregation step is slow.
- Web APIs: identify whether time is spent in database calls, JSON serialization, or business logic.
- Automation scripts: compare two approaches for reading files, making requests, or transforming data.
- Scientific computing: locate slow numerical loops before rewriting them with NumPy or C extensions.
- Batch jobs: measure full runtime and monitor regressions after code changes.
Example: a CSV processing script may seem slow overall, but profiling could reveal that string conversion—not file reading—is the real bottleneck.
Real Codebase Usage
In real projects, developers usually combine simple timing with structured profiling.
Common patterns include:
- Quick local measurement with
time.perf_counter()around a specific block. - Function-level profiling with
cProfilewhen a script or endpoint feels slow. - Benchmarking alternatives with
timeitbefore choosing between two small implementations. - Guarded profiling code so it only runs in development, not production.
- Profiling a main entry point instead of scattering print statements everywhere.
Example of a guarded timing block:
import time
DEBUG_TIMING = True
if DEBUG_TIMING:
start = time.perf_counter()
# main work here
values = [x * 2 for x in range(1_000_000)]
if DEBUG_TIMING:
end = time.perf_counter()
print(f"Main work took {end - start:.6f} seconds")
Example of profiling a function cleanly:
import cProfile
import pstats
def load_and_process_data():
data = [(i) i ()]
[(x) x data]
profiler = cProfile.Profile()
profiler.enable()
load_and_process_data()
profiler.disable()
stats = pstats.Stats(profiler)
stats.sort_stats().print_stats()
Common Mistakes
A common beginner mistake is using the wrong tool for the job.
1. Using manual timing when you actually need profiling
This only tells you the total runtime:
import time
start = time.perf_counter()
run_program()
print(time.perf_counter() - start)
That is useful, but it does not tell you which function is slow.
2. Using time.time() for fine-grained benchmarking
import time
start = time.time()
result = sum(range(1000))
end = time.time()
print(end - start)
This works, but time.perf_counter() is usually better for measuring short durations.
3. Timing code only once
Single runs can be noisy because of:
- OS scheduling
- background processes
- cache warm-up
- interpreter startup
Use timeit for small snippets that need repeated measurement.
4. Profiling tiny one-line expressions with cProfile
cProfile is great for whole functions and scripts, but for micro-benchmarks, is often a better choice.
Comparisons
| Tool | Best for | Measures | Good choice when |
|---|---|---|---|
time.perf_counter() | Quick timing | Elapsed wall-clock duration | You want total runtime for a block of code |
timeit | Micro-benchmarking | Repeated execution timing | You want to compare small code snippets fairly |
cProfile | Full profiling | Function call stats and time distribution | You want to know where the program spends time |
pstats | Profile analysis | Sorted profiler output | You want to inspect cProfile results more clearly |
Another useful comparison:
Cheat Sheet
# Quick timing
import time
start = time.perf_counter()
# code here
end = time.perf_counter()
print(end - start)
# Benchmark a small snippet repeatedly
import timeit
timeit.timeit("sum(range(1000))", number=1000)
# Profile a full script
python -m cProfile my_script.py
python -m cProfile -s tottime my_script.py
python -m cProfile -s cumtime my_script.py
# Profile a function in code
import cProfile
cProfile.run("my_function()")
# Use pstats for better control
import cProfile
import pstats
profiler = cProfile.Profile()
profiler.enable()
my_function()
profiler.disable()
stats = pstats.Stats(profiler)
stats.sort_stats("cumtime").print_stats(10)
Key rules:
time.perf_counter()is good for elapsed timing.timeitis best for small repeated benchmarks.cProfileis best for finding slow functions.
FAQ
What is the best way to measure how long a Python script takes?
For total runtime, use time.perf_counter() around the code you want to measure. If you want function-by-function breakdowns, use cProfile.
How do I profile a Python script from the command line?
Run:
python -m cProfile my_script.py
You can sort the output with:
python -m cProfile -s tottime my_script.py
What is the difference between timeit and cProfile?
timeit measures how long a small snippet takes when run many times. cProfile shows which functions consume time in a full program.
Should I use time.time() or time.perf_counter()?
For measuring elapsed duration, prefer time.perf_counter() because it is designed for accurate timing.
Why is my benchmark inconsistent between runs?
Results can vary because of system load, caching, interpreter startup, and other background activity. Repeated measurements with timeit help reduce noise.
Mini Project
Description
Build a small Python script that processes a list of numbers, then measure both its total runtime and which function is responsible for most of the work. This project demonstrates the practical difference between simple timing and real profiling.
Goal
Create a script that times a computation with time.perf_counter() and profiles it with cProfile to identify the slowest function.
Requirements
- Create at least two Python functions that do separate pieces of work.
- Measure the total runtime of the main workflow using
time.perf_counter(). - Profile the workflow using
cProfile. - Print the profiling results sorted by total or cumulative time.
- Make the script runnable as a standalone Python file.
Keep learning
Related questions
@staticmethod vs @classmethod in Python Explained
Learn the difference between @staticmethod and @classmethod in Python with clear examples, use cases, mistakes, and a mini project.
Call a Function by Name in a Python Module
Learn how to call a function by name in a Python module using strings, getattr, and safe patterns for dynamic function dispatch.
Catch Multiple Exceptions in One except Block in Python
Learn how to catch multiple exceptions in one Python except block using tuples, with examples, mistakes, and real-world usage.