Connection pool sizing is a queueing theory problem, not a tuning knob

A service starts timing out under load. The pool is exhausted, the logs are full of Timeout waiting for connection from pool, and the obvious fix is to raise the pool size from 20 to 50.

Latency gets worse.

Someone raises it to 100 on the theory that 50 was not enough. Now the database is at 90 percent CPU doing the same amount of useful work it was doing before, p99 has tripled, and the team is discussing a bigger instance.

The pool was never the constraint. It was the only thing keeping the load bounded, and raising it removed the protection while leaving the actual problem exactly where it was.

The number comes out of one equation

Little’s law says that for a stable system, the average number of items in the system equals the arrival rate multiplied by the average time each item spends there.

L = λ × W

L = connections needed
λ = requests per second
W = seconds each request holds a connection

A service handling 500 requests per second, each holding a connection for 20ms:

L = 500 × 0.020 = 10 connections

Ten. Not twenty, not fifty. Add headroom because arrival is bursty rather than smooth and holding time has a distribution rather than a single value, and you land somewhere around 15.

That result surprises people, and the reason it surprises them is that the holding time is much smaller than they imagine. A query that executes in 2ms does not hold a connection for 2ms. It holds it for checkout, network round trip, execution, result transfer and return, which might be 20ms. But it is still 20ms, not 200ms, unless something in the request is doing something it should not.

Which points at the far more common problem. When the arithmetic says you need 10 connections and you are exhausting 50, the interesting number is not the pool size. It is W.

Holding time is where the bug usually is

Rearranged, the equation says something more useful: for a fixed pool, your maximum throughput is L / W. Every millisecond a connection stays checked out costs you capacity.

Things that inflate W without anyone deciding to:

An N+1 query pattern turns one 20ms checkout into 400 sequential round trips, and holds the connection for all of them while the database sits mostly idle.

A transaction opened at the start of a request handler and committed at the end holds a connection across everything in between, including template rendering, serialisation and any HTTP call to another service.

An external API call inside a transaction is the extreme version, and it has become dramatically more common now that request handlers call model inference. A three second inference call inside a transaction turns W from 0.02 into 3.0, and the same pool that supported 500 requests per second now supports three. That is the shape that makes a database sit at 2 percent CPU while every connection is checked out.

flowchart LR
    subgraph good["W = 20ms"]
        A1["checkout"] --> A2["query 2ms"] --> A3["return"]
        A3 --> AT["~50 req/s per connection"]
    end

    subgraph bad["W = 3000ms"]
        B1["checkout"] --> B2["query 2ms"] --> B3["external call 3s"] --> B4["return"]
        B4 --> BT["~0.33 req/s per connection"]
    end

    style AT stroke:#4ade80,stroke-width:3px,color:#fff
    style BT stroke:#ef4444,stroke-width:3px,color:#fff

Same pool. A hundred and fifty times the capacity difference. No amount of pool sizing recovers that, because the pool is not what is broken.

The ceiling on the other side

Little’s law tells you what your application needs. It says nothing about what the database can usefully absorb, and that is the second constraint.

A database executing CPU bound queries can genuinely run about as many at once as it has cores. The formula on the PostgreSQL wiki, which HikariCP also recommends, is roughly cores × 2 + effective_spindles, where the doubling accounts for queries that are briefly waiting on I/O rather than computing. On an 8 core instance that lands around 16 to 20 active connections.

Past that, connections do not execute sooner. They time-slice. The database context switches between them, each switch evicting the previous query’s working set from L1 and L2, which is precisely the pattern that makes oversized thread pools reduce throughput rather than increase it. Lock contention grows too, because more concurrent transactions means more chances to collide on the same rows.

In Postgres there is an additional cost that catches people, because a connection is a separate operating system process rather than a thread. Several megabytes of memory each, plus process setup, plus a shared snapshot structure that every backend contends on. Five hundred connections is gigabytes of overhead before a single query runs.

throughput
    ^
    |            ___________________
    |          /
    |        /
    |      /
    |    /
    |  /
    |/
    +--------------------------------> connections
              ^
              knee: database is saturated.
              Past here, latency rises and
              throughput does not.

The pool being smaller than the database’s capacity is a feature. It is admission control. Requests queue in your application, where you can observe the queue, apply a timeout and shed load, rather than queueing inside the database where you can do none of those things.

The multiplication nobody performs

Every calculation above is per instance. The database sees the sum.

20 pods × pool of 20 = 400 connections

Nobody chose 400. Somebody chose 20, twice, in two different config files, at two different times, and autoscaling chose the rest. When traffic spikes and the HPA doubles the pod count, the connection count doubles with it, which means the database gets hit hardest at exactly the moment it is already under pressure.

This is the number worth putting on a dashboard, and almost nobody does. SELECT count(*) FROM pg_stat_activity against max_connections is a two second check that frequently produces a surprise.

Once instance counts are dynamic, a pooler in front of the database stops being optional. PgBouncer in transaction mode multiplexes many client connections onto few server connections, so 400 application connections become 25 actual backends. Transaction mode has real constraints, since session state such as prepared statements and advisory locks does not survive the multiplexing, and finding that out during an incident is unpleasant. It is still the standard answer, because the alternative is coordinating pool sizes across every deployment by hand.

Timeouts are part of the sizing decision

A pool has two numbers and people only tune one.

connectionTimeout is how long a caller waits for a connection before failing. The instinct is to set it high so requests do not fail. That instinct is wrong, and it is the same reasoning that makes unbounded retries turn a slow dependency into an outage.

A 30 second connection timeout means that under saturation, requests pile up for 30 seconds holding threads, memory and upstream connections, and the client that made the request gave up 25 seconds ago. You have converted a fast failure into a slow one and consumed resources producing an answer nobody is waiting for.

Set it to something close to your actual latency budget, a second or two. Under saturation you fail fast, the circuit breaker upstream opens, and load sheds in a controlled way rather than by exhaustion.

maxLifetime matters for a duller reason. Connections that live forever accumulate server side state and eventually meet a firewall or load balancer that silently drops idle TCP sessions, producing errors that look random. Setting it below the shortest infrastructure timeout in the path makes the pool recycle connections before something else kills them.

Written out, with the reasoning attached to each number rather than left to be rediscovered later:

HikariConfig config = new HikariConfig();

// Little's law: 500 req/s x 0.020s holding time = 10, plus headroom
// for burstiness. Capped below what an 8 core instance executes in
// parallel, and sized so pods x pool stays under max_connections.
config.setMaximumPoolSize(15);
config.setMinimumIdle(15);          // fixed size; avoids churn under bursty load

// Fail inside the latency budget rather than after the caller gave up.
config.setConnectionTimeout(2_000);

// Recycle before the network path drops an idle session.
config.setMaxLifetime(600_000);
config.setIdleTimeout(0);           // irrelevant with a fixed size pool

// Surfaces acquisition wait and usage time, which is where W is measured.
config.setMetricRegistry(meterRegistry);

Setting minimumIdle equal to maximumPoolSize is the part worth calling out. A pool that shrinks when idle has to open new connections at the start of a traffic spike, and connection establishment against Postgres means process creation plus TLS negotiation at precisely the wrong moment. A fixed size pool costs a few idle connections and removes that cliff.

The order I would work through it

Measure holding time before touching pool size, because if W is inflated then every other number is derived from a broken input. Connection acquisition wait time and connection usage time are both exposed by HikariCP metrics and both are worth graphing.

Then compute the pool from Little’s law using measured numbers rather than a default, and cap it at what the database can execute in parallel.

Then multiply by instance count and compare against max_connections, including whatever the autoscaler is allowed to scale to rather than the current replica count.

Then set the connection timeout to your latency budget instead of a comfortable looking round number.

What makes this worth doing carefully is that pool exhaustion presents as a capacity problem and almost never is one. The pool is a queue with a fixed number of servers, it obeys arithmetic that has been well understood since the 1950s, and the arithmetic will tell you which of the three possible problems you have. Raising the number until the errors stop is the one approach that answers nothing, and it usually works just well enough to hide the real cause until the next traffic peak.