Back to Details

optimizing-python-performance

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Python Performance Optimization

Python性能优化

Profiling Quick Start

性能分析快速入门

bash
undefined
bash
undefined

PyInstrument (statistical, readable output)

PyInstrument（统计型，输出易读）

python -m pyinstrument script.py
python -m pyinstrument script.py

cProfile (detailed, built-in)

cProfile（详细型，内置工具）

python -m cProfile -s cumulative script.py
python -m cProfile -s cumulative script.py

Memory profiling

内存分析

pip install memray memray run script.py memray flamegraph memray-*.bin
undefined
pip install memray memray run script.py memray flamegraph memray-*.bin
undefined

PyInstrument Usage

PyInstrument 使用方法

python
from pyinstrument import Profiler

profiler = Profiler()
profiler.start()
result = my_function()
profiler.stop()
print(profiler.output_text(unicode=True, color=True))
python
from pyinstrument import Profiler

profiler = Profiler()
profiler.start()
result = my_function()
profiler.stop()
print(profiler.output_text(unicode=True, color=True))

Memory Analysis

内存分析

python
import tracemalloc

tracemalloc.start()
python
import tracemalloc

tracemalloc.start()

... code ...

... 代码 ...

snapshot = tracemalloc.take_snapshot() for stat in snapshot.statistics('lineno')[:10]: print(stat)
undefined
snapshot = tracemalloc.take_snapshot() for stat in snapshot.statistics('lineno')[:10]: print(stat)
undefined

Benchmarking (pytest-benchmark)

基准测试（pytest-benchmark）

python
def test_encode_benchmark(benchmark):
    result = benchmark(encode, 37.7749, -122.4194)
    assert len(result) == 12
bash
pytest tests/ --benchmark-only
pytest tests/ --benchmark-compare
python
def test_encode_benchmark(benchmark):
    result = benchmark(encode, 37.7749, -122.4194)
    assert len(result) == 12
bash
pytest tests/ --benchmark-only
pytest tests/ --benchmark-compare

Common Optimizations

常见优化技巧

python
undefined
python
undefined

Use set for membership (O(1) vs O(n))

使用集合进行成员判断（O(1) 对比 O(n)）

valid = set(items) if item in valid: ...
valid = set(items) if item in valid: ...

Use deque for queue operations

使用deque实现队列操作

from collections import deque queue = deque() queue.popleft() # O(1) vs list.pop(0) O(n)
from collections import deque queue = deque() queue.popleft() # O(1) 对比 list.pop(0) 的 O(n)

Use generators for large data

针对大数据使用生成器

def process(items): for item in items: yield transform(item)
def process(items): for item in items: yield transform(item)

Cache expensive computations

缓存耗时计算

from functools import lru_cache
@lru_cache(maxsize=1000) def expensive(x): return compute(x)
from functools import lru_cache
@lru_cache(maxsize=1000) def expensive(x): return compute(x)

String building

字符串拼接

result = "".join(str(x) for x in items) # Not += in loop
undefined
result = "".join(str(x) for x in items) # 不要在循环中使用 +=
undefined

Algorithm Complexity

算法复杂度

Operationlistsetdict
LookupO(n)O(1)O(1)
InsertO(1)O(1)O(1)
DeleteO(n)O(1)O(1)
For detailed strategies, see:
  • PROFILING.md - Advanced profiling techniques
  • BENCHMARKS.md - CI benchmark regression testing
操作listsetdict
查找O(n)O(1)O(1)
插入O(1)O(1)O(1)
删除O(n)O(1)O(1)
如需详细策略，请查看：
  • PROFILING.md - 高级性能分析技巧
  • BENCHMARKS.md - CI基准测试回归检测

Optimization Checklist

优化检查清单

Before Optimizing:
- [ ] Confirm there's a real problem
- [ ] Profile to find actual bottleneck
- [ ] Establish baseline measurements

Process:
- [ ] Algorithm improvements first
- [ ] Then data structures
- [ ] Then implementation details
- [ ] Measure after each change

After:
- [ ] Add benchmarks to prevent regression
- [ ] Verify correctness unchanged
- [ ] Document why optimization needed
优化前：
- [ ] 确认存在真实的性能问题
- [ ] 通过性能分析找到实际瓶颈
- [ ] 建立基准测量值

优化过程：
- [ ] 优先优化算法
- [ ] 其次优化数据结构
- [ ] 最后优化实现细节
- [ ] 每次修改后进行测量

优化后：
- [ ] 添加基准测试以防止性能回归
- [ ] 验证功能正确性未改变
- [ ] 记录优化的必要性

Learn More

了解更多

This skill is based on the Performance section of the Guide to Developing High-Quality Python Libraries by Will McGinnis.
本技能基于Will McGinnis撰写的《高质量Python库开发指南》中的性能章节，该指南链接为Guide to Developing High-Quality Python Libraries。