Short Description
The story of taking a CPU-heavy pricing API from 9 seconds to under 2. We profiled before optimizing (with a small Python decorator), refactored to clean architecture so the fixes were even possible, escaped the GIL with ProcessPoolExecutor, collapsed sequential micro-service calls with asyncio.gather, added HTTP connection pooling and Redis caching, and hardened retries with backoff and jitter. All numbers from a real production codebase, measured under load with Locust, tradeoffs included. A practical walkthrough of why measuring first beats fixing what you happen to notice.
When I joined my first company, we were at a stage where the MVP was successful and we were trying to aggressively scale. The existing codebase was messy and highly coupled, but it had served its function of verifying product-market fit. Now the system had to handle scale, and that’s where I started my work at. The part I owned was the latency of a CPU-intensive API that took 9 long seconds to calculate and respond, measured with Locust. Cutting that down was necessary if we wanted to serve more customers with ease.