How I Halved My Backend Python App’s Memory Usage and saved us Money

We’ve all been there. Your monitoring dashboard turns red. Memory is spiking. The immediate temptation is to double the instance size—bump that t3.medium to a t3.large and move on. But today, I want to show you why that is the lazy way out.

We have a standard startup stack: FastAPI for the backend and Celery for background tasks, all running on a cost-effective EC2 instance with 4GB of RAM. The vital metrics for the server looked bizarre: Memory was screaming at 75% utilization, yet the CPU was barely waking up, sitting idle at 2-3%.

This specific signature—High RAM, Low CPU—is the hallmark of an architecture that is over-provisioned for concurrency but under-optimized for memory. Let me show you the three specific knobs I turned to cut our memory footprint in half, without spending a single extra cent.

---

Lesson 1: The Math of Processes vs. Threads

First, let’s look at the “Before” snapshot of the system. We were running:

• API: 4 Gunicorn workers

• Celery: 8 workers across 3 queues (2 normal queues and a priority queue)

• Total: 12 Python processes.

Now, here is the expensive part. Each Python worker process consumes about 250MB of memory on average.

On a 4GB instance, that leaves almost no room for the OS. We needed to consolidate.

The Solution: Threads over Processes

In Python, a Process has its own memory space (it loads the whole interpreter, imports, and app code). A Thread shares memory with other threads in the same process.

I switched our FastAPI configuration from workers=2 (processes) to workers=1 with threads=4.

Why does this work?

Since our app is I/O bound (waiting on Database queries, BigQuery, and OpenAI calls), we don’t need multiple CPU cores fighting for the GIL (Global Interpreter Lock). Threads can handle concurrent requests efficiently because they release the lock while waiting for I/O.

• Old Config (Processes): ~500MB usage for equivalent concurrency.

• New Config (Threads): ~250MB usage. We just saved 50% memory on the API layer.

---

Lesson 2: The “Leaking Bucket” Theory (Handling Memory Leaks)

Even if your code is perfect, Python processes (especially those using libraries like Pandas or Numpy) tend to hold onto memory over time. It’s called memory fragmentation, or sometimes, just a plain old leak.

If you process a large Dataframe in a Celery task, that memory might not be fully released back to the OS immediately.

The Fix : tuning the --max-tasks-per-child parameter

This setting tells the Celery worker: “After you finish X number of tasks, kill yourself and let a fresh process take your place.”

We experimented a bit with this parameter and finally settled on a value of 100.

Here is a Trade-off Matrix for the different values:

• Too Low (e.g., 10): Great for memory, but you waste CPU restarting workers constantly (~15% overhead).

• Too High (e.g., 1000): Low overhead, but memory leaks accumulate.

• The Sweet Spot (100): With our autoscaling enabled, workers naturally shut down when idle. This setting is just a safety net to ensure no single worker stays alive long enough to become bloated.

---

Lesson 3: The “Thundering Herd” (Gunicorn Jitter)

Just like Celery, we want our Gunicorn (API) workers to restart occasionally to clear out memory “gunk.” We set --max-requests 1000 to restart the worker after 1000 HTTP requests.

But there is a danger here.

Imagine you have 2 workers. If they both start at the same time, and both hit their 1000th request at the same time, they will both restart at the same time.

Result: Your API goes down for 2 seconds.

The Fix: tuning the --max-requests-jitter parameter

We added --max-requests-jitter 50.

This adds randomness. One worker might restart at 980 requests, the other at 1040. This staggering ensures that while one worker is recycling, the other is still serving traffic.

---

The Final Report Card

By understanding the difference between I/O bound and CPU bound tasks, and acknowledging that Python processes are memory-heavy, we drastically changed our footprint.

Before:

• 12 Total Workers

• ~3GB Memory Usage (75% Load)

After:

• 6 Total Workers (API threads + reduced Celery concurrency)

• ~1.5GB Memory Usage (40% Load)

So the next time your server metrics are turning red, before vertically scaling your servers try tuning your app and worker configurations. Go check your supervisor.conf or Dockerfile and answer these questions -

1. Are you running multiple Gunicorn workers for an I/O bound app? Try threads.

2. Do your Celery workers live forever? Add max-tasks-per-child.

3. Are you monitoring RSS memory? watch -n 5 ‘ps aux | grep celery’ is your friend.

STOP THROWING MONEY at your cloud provider.