How We Debugged a JWT Expiry Bug in 10 Minutes
· Cosyslabs
An e-commerce platform (details anonymized) began receiving reports that users were being logged out randomly, approximately 5 minutes into their sessions. The authentication system used JWTs with a 15-minute access token expiry. The bug affected roughly 30% of users, seemingly at random. Here is how the JWT Decoder surfaced the root cause in under 10 minutes.
The Symptom
Support tickets described a consistent pattern: users would log in successfully, add items to their cart, proceed to checkout, and be redirected to the login page. The frontend error logs showed:
API Error: 401 Unauthorized
Response: {"error": "Token expired", "code": "JWT_EXPIRED"}
The tokens were being rejected before the 15-minute window expired. Users were being told their session ended when they had been active for only 4–5 minutes.
Initial Investigation (Wrong Direction)
The team initially suspected a race condition in the token refresh logic. They spent two hours reviewing the React frontend code that managed access tokens in memory. No obvious bug was found. The refresh logic appeared correct:
// Token refresh logic — seemed fine on review
async function getValidToken() {
const token = getStoredAccessToken();
const payload = decodeTokenPayload(token);
// Refresh if less than 2 minutes remaining
if (payload.exp - Date.now() / 1000 < 120) {
return await refreshAccessToken();
}
return token;
}
Using the JWT Decoder
A developer pasted a JWT token from a failing request into the JWT Decoder Tool. The decoded payload immediately revealed:
{
"sub": "user_8473",
"iss": "https://auth.company.com",
"aud": "https://api.company.com",
"iat": 1749996400,
"exp": 1749997300,
"jti": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
}
The Unix Timestamp Converter showed:
iat: 1749996400 → 2026-06-15T11:00:00Z (token issued at)
exp: 1749997300 → 2026-06-15T11:15:00Z (token expires at)
At first glance, this looked correct — 15 minutes between iat and exp.
But then the developer checked the current server time using the tool:
Current time: 1749997005 → 2026-06-15T11:10:05Z
The token was issued at 11:00:00 and it was now 11:10:05. The 15-minute window should still be open. But the API was rejecting it.
The Root Cause: Clock Skew Between Containers
The developer pulled another JWT from a request that had succeeded moments before the failure. Comparing the two iat values:
Successful request: iat = 1749996400 (11:00:00Z)
Failed request: iat = 1749996400 (11:00:00Z)
Same token. But the API was rejecting it during some requests and accepting it during others.
The API ran on three container instances behind a load balancer. The developer checked the system clocks on each container:
# Container 1
$ date -u
Mon Jun 15 11:10:05 UTC 2026
# Container 2
$ date -u
Mon Jun 15 11:10:05 UTC 2026
# Container 3
$ date -u
Mon Jun 15 11:25:12 UTC 2026 ← 15 minutes ahead!
Container 3 had a misconfigured NTP (Network Time Protocol) client. Its clock was 15 minutes ahead of actual time. When requests routed to Container 3, exp (11:15:00) < server_time (11:25:12) — the token appeared expired.
The 30% failure rate matched the 1-in-3 probability of a request landing on Container 3.
The Fix
Two actions:
- Immediate: restart the NTP service on Container 3 to resync the clock
- Structural: add NTP monitoring to the container health checks and alerting
# Added to container startup check
healthcheck:
test: |
MAX_SKEW=5 # seconds
NTP_OFFSET=$(chronyc tracking | grep 'System time' | awk '{print $4}')
[ $(echo "$NTP_OFFSET < $MAX_SKEW" | bc) -eq 1 ]
interval: 60s
timeout: 5s
retries: 3
The team also added a 30-second clock skew tolerance to the JWT validation:
// Add leeway for minor clock differences between services
jwt.verify(token, PUBLIC_KEY, {
algorithms: ["RS256"],
clockTolerance: 30, // seconds of leeway
});
What Made the JWT Decoder Essential
Without the JWT Decoder, the investigation would have required:
- Adding debug logging to production
- Capturing and logging raw tokens
- Running Node.js scripts to decode tokens manually
The JWT Decoder showed the exact iat, exp, and all claims in seconds — no code required, no tokens left in logs. The Unix Timestamp Converter converted the raw Unix timestamps to human-readable times instantly, making it clear when the token was issued versus when it should expire.
The total investigation time from first token paste to root cause identification: 9 minutes.
Lessons Learned
- Clock synchronization is a silent failure mode: NTP drift does not trigger alerts by itself but breaks time-sensitive operations
- JWT expiry bugs are not always in your code: the JWT structure was correct; the infrastructure was wrong
- Browser tools eliminate the need to decode tokens in code: decoding a token takes 30 seconds with the right tool
- Short-lived tokens amplify clock skew impact: a 15-minute token with 15 minutes of drift will always fail; a 24-hour token would have been unaffected
- Add clock skew tolerance: the JWT spec allows for a configurable tolerance window precisely because distributed systems have imperfect clocks