Instances behind an AWS NAT gateway start failing to connect to one busy third-party API while every other destination works fine, and the gateway's ErrorPortAllocation metric is non-zero. What is happening, and what are your options?

## Why the failure is destination-specific The give-away in the symptom is that only *one* destination is failing. That rules out most network causes — a broken route, a missing gateway, or a saturated link would degrade everything — and points straight at how address translation allocates ports. When a NAT gateway forwards an outbound flow, it replaces the instance's private source address with its own and picks a source port so the reply can be matched back to the right originator. A flow is identified by the tuple of protocol, source address, source port, destination address, and destination port. Because the gateway's address and the destination are fixed for a given API, the only field left to vary is the **source port**, and a port range is finite. Two flows to *different* destinations may reuse the same source port harmlessly; two flows to the *same* destination cannot. So the capacity ceiling is per unique destination endpoint, not per gateway. Talking to a thousand different hosts is cheap; hammering one host is what runs out. ## The number, and how to use it AWS documents a NAT gateway as supporting up to about **55,000 simultaneous connections to each unique destination**, per IP address associated with the gateway. Quote it with the caveat that limits move, and then use it the way an engineer should — as an order of magnitude that tells you whether your connection *rate* is plausible, not as a number to recite. The more useful framing is the arithmetic behind the exhaustion. Ports are not released the instant a request finishes; a NAT gateway holds a translation entry for a while after a flow goes idle, and it drops idle TCP flows after a documented **350 seconds**. So the ports in use at any moment are roughly the connection establishment rate multiplied by how long each entry is held — not the number of requests in flight. A service issuing hundreds of new connections per second to one endpoint, each entry held for even a minute, is well into five figures of simultaneous entries while appearing to have only a handful of concurrent requests. That gap between apparent concurrency and actual port consumption is the whole trap. ## Confirming it Three signals, in order of directness: - **`ErrorPortAllocation`** in the `AWS/NATGateway` namespace — the count of times the gateway could not allocate a source port. Any non-zero value is the diagnosis, not a hint. - **`ActiveConnectionCount`** — the concurrent connection count through the gateway, useful for seeing the climb that precedes the failures. - **`IdleTimeoutCount`** — flows dropped for idleness, which tells you entries are being held rather than closed cleanly. ``` aws cloudwatch get-metric-statistics \ --namespace AWS/NATGateway \ --metric-name ErrorPortAllocation \ --dimensions Name=NatGatewayId,Value=nat-0123456789abcdef0 \ --start-time 2026-08-21T00:00:00Z --end-time 2026-08-21T06:00:00Z \ --period 300 --statistics Sum ``` On the application side the failure looks like connection timeouts or resets to one host, often intermittent at first and correlated with peak traffic — which is why teams chase the third party's status page for hours before looking at the gateway. ## The fixes, best first **Reduce the number of flows.** This is the real fix, and it is on the client. An HTTP client that reuses connections — keep-alive with a properly sized connection pool, or HTTP/2 multiplexing where the endpoint supports it — turns tens of thousands of short flows into a handful of long ones. A service that constructs a new client object per request, which is a very common defect, defeats pooling entirely and is usually the root cause. Fixing this also removes the handshake latency you were paying on every call. **Give the gateway more source ports.** You can associate additional Elastic IP addresses with a public NAT gateway; each address carries its own port range toward the same destination, multiplying capacity. It is a genuine fix and cheap to apply, but it is a multiplier on a design that is still burning ports, so treat it as a way to buy time or as headroom for a legitimately high-volume path. **Spread the load.** With per-AZ NAT gateways in place, callers in different zones already use different gateways with different addresses. If a single zone's callers are the problem, splitting them across additional subnets and gateways divides the port pressure. **Take the destination off the NAT path.** Where the destination is reachable by a private route rather than over the internet, the flows never touch the NAT gateway and the ceiling stops applying. That mechanism belongs to another discussion, but it is worth naming as the structural answer when the destination allows it. ## What a strong answer sounds like Start from the symptom's shape — one destination failing, others healthy — and derive the cause rather than reciting a limit. Name the metric that proves it. Then order the remedies with client-side connection reuse first, because adding addresses to absorb a bug the application will keep committing is treating the symptom. Mentioning that idle-entry hold time makes port consumption much higher than the concurrent request count is the detail that shows you have actually debugged this.