HTTP 502, 504, and 500 Errors: SRE Troubleshooting Guide for Server and Gateway Failures

Few things induce panic in engineering teams quite like a surge of 5xx HTTP errors during peak traffic. While 4xx client errors (such as 400 Bad Request or 404 Not Found) indicate an issue with how the client formatted their request, 5xx server errors announce that the server infrastructure encountered an internal failure and could not deliver a valid response.

Among 5xx status codes, the most frequent culprits in modern cloud environments are 500 Internal Server Error, 502 Bad Gateway, and 504 Gateway Timeout.

Because modern web stacks decouple frontend routing (reverse proxies and CDNs) from backend compute (microservices, containers, and serverless workers), identifying which layer of the stack failed requires a systematic understanding of proxy-to-upstream communication.

This guide explores the architectural boundary between reverse proxies and application servers, details the exact triggers for 500, 502, 503, and 504 errors, and outlines an SRE-tested playbook for diagnosing issues using an API speed test and server diagnostic tools.

The multi-tier architecture: Where 5xx errors live

In legacy monoliths, a single Apache or IIS web server received the TCP handshake and executed application code directly. Today, almost every web application routes requests through an intermediate proxy layer:

[ Client Browser / App ]
          │
          ▼
 [ Edge CDN (Cloudflare, Fastly, AWS CloudFront) ]
          │
          ▼
 [ Reverse Proxy / Load Balancer (NGINX, Envoy, HAProxy, AWS ALB) ]
          │  (Gateway Boundary)
          ▼
 [ Upstream Application Server (Node.js, Go, Python Gunicorn, PHP-FPM) ]
          │
          ▼
 [ Database / Cache / Third-Party Microservices ]

When diagnosing 5xx errors, the term "Gateway" or "Proxy" refers to the load balancer or reverse proxy, while "Upstream" or "Origin" refers to the backend application server listening behind it.

502 Bad Gateway: The upstream connection collapse

An HTTP 502 Bad Gateway response indicates that the reverse proxy (the gateway) contacted the upstream application server to fulfill a request, but received an invalid, corrupt, or abrupt termination in response.

In simpler terms: The proxy knocked on the door, but the upstream application either slammed the door shut or collapsed mid-sentence.

Common causes of 502 errors

1. Upstream application process crash (OOM or unhandled exception)

If your Node.js, Python, or Go worker encounters an unhandled exception or runs out of RAM, the operating system kernel or process manager terminates the process immediately.

If this happens while the reverse proxy is waiting for an HTTP header response, the TCP connection drops with a TCP RST (reset) packet. NGINX records this in /var/log/nginx/error.log as:

[error] 14202#0: *1492 recv() failed (104: Connection reset by peer) while reading response header from upstream

On Linux systems, check the kernel ring buffer to verify if the Out of Memory (OOM) killer terminated the process:

dmesg -T | grep -i -E 'killed process|oom_reaper'

2. Upstream process not running or listening on the wrong port

If your system rebooted or the backend service failed to restart, the proxy attempts to initiate a TCP handshake to 127.0.0.1:3000 or a Unix domain socket (/var/run/app.sock) and gets connection refused:

[error] 14202#0: *1495 connect() failed (111: Connection refused) while connecting to upstream, client: 198.51.100.10, server: example.com, upstream: "http://127.0.0.1:3000/api"

Verify service status and active listening ports using systemd and network inspection commands:

systemctl status my-backend-service
ss -tulpn | grep 3000

3. Socket permission mismatch

When connecting via Unix sockets (common with PHP-FPM, uWSGI, and Gunicorn), the user running the web server (e.g., www-data or nginx) must have read and write permissions on the .sock file:

[crit] 14202#0: *1501 connect() to unix:/var/run/php/php8.2-fpm.sock failed (13: Permission denied) while connecting to upstream

4. Upstream response header buffer overflow

If your backend application returns huge cookies, excessive JWT session tokens, or numerous custom headers, NGINX's default upstream buffer (4KB or 8KB) may overflow:

[error] 14202#0: *1505 upstream sent too big header while reading response header from upstream

Fix: Increase the proxy buffer limits in your NGINX server block:

location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_buffer_size 128k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
}

504 Gateway Timeout: The stalled upstream

An HTTP 504 Gateway Timeout error occurs when the reverse proxy establishes a successful connection to the upstream application, but the upstream fails to send a complete HTTP response within the proxy's configured time limit.

In simpler terms: The proxy asked for data, the upstream promised it, but the proxy gave up waiting before the upstream delivered.

Common causes of 504 errors

1. Long-running, unindexed database queries

If an API endpoint triggers an unindexed JOIN query on a table with millions of rows, the database query can take 30 to 120 seconds. The reverse proxy's read timeout (defaulting to 60 seconds in NGINX) expires first, generating a 504:

[error] 14202#0: *1520 upstream timed out (110: Connection timed out) while reading response header from upstream

2. Blocking calls to external third-party APIs

If your backend endpoint makes synchronous HTTP calls to payment processors, AI model providers, or shipping APIs that are experiencing an outage or slowdown, your backend worker threads become blocked. Without an aggressive client timeout (e.g., 3–5 seconds) on those outbound calls, your API endpoint will exceed the gateway threshold.

3. Worker pool starvation under high concurrency

When your application server is configured to handle 50 concurrent worker threads, but 1,000 simultaneous requests arrive, 950 requests queue up in the socket backlog. By the time a worker thread becomes available to service request #951, the gateway has already timed out waiting.

500 Internal Server Error vs. 503 Service Unavailable

To complete the diagnostic picture, compare 502 and 504 with 500 and 503:

500 Internal Server Error

A 500 error originates from inside the application code itself, not the reverse proxy. The application framework (e.g., Express, Django, Laravel, Spring Boot) caught an unhandled runtime error (such as a NullPointerException, database authentication failure, or malformed JSON parsing error) and rendered an HTTP 500 error template back to the client.

To fix a 500 error, inspect your application's error monitoring dashboard (e.g., Sentry, Datadog) or stdout application logs.

503 Service Unavailable

A 503 indicates that the server is currently unable to handle the request due to deliberate capacity management. It typically happens when:

• The server is temporarily placed in maintenance mode.
• An API gateway or rate limiter rejects traffic because concurrent request limits were breached.
• Upstream container health checks are failing and the load balancer has removed all healthy pods from the pool.

Whenever possible, a 503 response should include a Retry-After: <seconds> HTTP header indicating when the client may retry.

Master comparison table: 5xx error triage

Status Code	Plain English Meaning	Failing Component	Typical NGINX Log Message	Immediate Action
500	Code crash / unhandled exception	Application codebase	Logged by application, not NGINX	Review application stack traces and exception logs.
502	Upstream dropped or refused connection	Upstream process or daemon	connect() failed (111: Connection refused) or recv() failed: Connection reset	Restart backend service; inspect OOM killer; verify socket permissions.
503	Server overloaded or under maintenance	Load balancer or orchestrator	no live upstreams while connecting to upstream	Scale pod replicas; check Kubernetes readiness probes; disable maintenance mode.
504	Upstream took too long to reply	Database, slow API, or thread pool	upstream timed out (110: Connection timed out)	Optimize database queries; add caching; add timeouts to outbound external API calls.

Step-by-step diagnostic workflow

When an alert triggers for elevated 5xx error rates, follow this rapid triage checklist:

Step 1: Isolate whether the error is global or regional

Run an online ping test and a website performance test across multiple global regions. If the 502/504 only occurs from specific continents, the problem may be an Anycast CDN edge routing problem or a regional database replica lag rather than an origin failure.

Step 2: Benchmark the failing endpoint directly

Use an API speed test to send clean GET or POST requests to the problematic endpoint. Inspect:

• The exact HTTP status code returned.
• The response headers: Look for server, x-cache, cf-ray, or via headers to determine whether Cloudflare, AWS CloudFront, or your origin NGINX generated the error page.
• Time to First Byte (TTFB): If TTFB hovers at exactly 30.0s or 60.0s before failing, you are encountering a gateway timeout (504).

Step 3: Check port reachability and firewall states

If your reverse proxy runs on a separate VM from your application database or microservices, use an online port checker to ensure firewall security groups or iptables rules are not dropping traffic between internal services.

Step 4: Verify DNS and origin IP resolution

Ensure your domain's A and AAAA records point to the correct load balancer IP and that DNS propagation is complete using a DNS test.

SRE best practices to prevent 5xx cascades

1. Implement circuit breakers: When downstream dependencies fail, trip a circuit breaker immediately to return a fast fallback response rather than allowing dozens of threads to hang until a 504 timeout occurs.
2. Set explicit outbound timeouts: Never make an HTTP client call in application code without an explicit timeout: 3000 (3 seconds) option.
3. Configure keep-alive connections to upstreams: In NGINX, configure proxy_http_version 1.1; and proxy_set_header Connection ""; along with an upstream keepalive pool to avoid burning through ephemeral TCP ports under heavy load.
4. Use readiness probes wisely in Kubernetes: Configure liveness probes to detect real deadlocks, but tune initial delays so that cold-starting pods are not killed prematurely during traffic spikes.