AI agents break connection pooling by holding the slot while they think

For roughly two decades, backend infrastructure has been tuned for one kind of client: a human. A click on a dashboard checks a connection out of the pool, runs a few well-indexed queries, returns the connection, and ships back JSON. The whole interaction might cost 50 ms of database time.

That speed is what makes connection pooling viable at all. Middleware like PgBouncer and HikariCP is built on an implicit contract: transactions stay microscopic. AI agents wired straight into data stores do not honour that contract, and the ways they break it are worth walking through slowly, because the symptoms look nothing like the cause.

What a database connection costs

In PostgreSQL, each connection maps to an OS-level process that consumes on the order of 10 MB of RAM. Five thousand direct connections can eat 50 GB in pure process overhead, which is memory the buffer cache no longer gets. So we cap live backends with a pool, commonly around 100 connections, and rely on turnover to make the math work: if a request holds a slot for 50 ms, 100 slots support thousands of requests per second.

The contract, then, is not really about connection count. It is about occupancy time. Everything downstream of this point is what happens when occupancy quietly grows by two orders of magnitude.

The agentic loop that breaks it

Agents commonly follow a ReAct-style loop: receive a prompt, have the LLM propose SQL or a plan, run the query, and feed the raw rows back to the model to summarise or decide the next step.

The failure mode hides in step four. Implementations that open a transaction, run the query, and then keep the connection checked out while the model processes the result are holding a pooled slot across an inference call. A round trip to a hosted model sits in the 3 to 5 second range on a good day. The slot that was budgeted for 50 ms of query work now spends 5,000 ms idle, waiting on a GPU somewhere else entirely.

With a pool of 100, it takes exactly 100 concurrently thinking agents to occupy every slot. Request 101, perhaps a human loading the homepage, queues behind them. Latency jumps, timeouts fire, and the dashboard shows the strangest signature of this whole failure: the database at 2 percent CPU while the application falls over. Nothing is slow. Everything is waiting.

sequenceDiagram
  participant U as User / caller
  participant B as Backend
  participant P as Pool
  participant D as PostgreSQL
  participant L as LLM API

  U->>B: prompt / task
  B->>P: acquire connection
  P->>D: query + transaction scope
  D-->>B: rows
  Note over B,D: Naive: connection stays checked out
  B->>L: send context for reasoning (seconds)
  L-->>B: next SQL or answer
  B->>P: release connection

The second-order effect: buffer cache eviction

Pool exhaustion is the loud failure. The quiet one is what agent-generated SQL does to memory.

Human traffic hits predictable, indexed paths, so the database keeps the hot working set in RAM and most reads never touch disk. Agents generating ad hoc SQL are not predictable. An agent asked about seasonal merchandise trends can innocently emit a query that scans tens of millions of log rows. That scan drags cold pages in from SSD at volume, and limited RAM means the hot pages that ordinary traffic depends on get evicted to make room.

When normal traffic returns, its warm pages are gone and everyday queries fall back to disk I/O. The agent has not just borrowed capacity; it has degraded the cache locality of every other client, and the recovery takes as long as it takes the working set to warm back up.

The decoupled pattern

The fix is a rule, and the rule is absolute: the reasoning layer never holds a live database handle while it waits on inference.

Fetch the data, commit or end the session, and return the connection to the pool immediately. Then call the model asynchronously with a payload that stands alone. If the model proposes a follow-up query, acquire a fresh connection for it, another 50 ms of occupancy, and release again. Pool occupancy stays proportional to query work instead of inference latency, which is the contract pooling was built on.

Two supporting moves matter at scale. Route agent-generated SQL to isolated read replicas, so an agent that writes a bad scan thrashes a secondary’s cache while the primary stays warm for production traffic. And put a boundary in front of core tables, restricted APIs or pre-aggregated views, rather than letting agents aim raw SQL at the schema; the blast radius of a generated query should be a design decision, not an accident.

Scaling is mostly the discipline of knowing where time is spent. Inference latency is external and you do not control it. Letting it dictate your internal database throughput is a choice, and it is one you can decline.