Logged-out at random — one pod's clock is 8 seconds behind
Service: auth-middleware (JWT validation, behind load balancer with 6 pods)
Symptom: ~1 in 6 authenticated requests returns 401 with 'token issued in the future'. Re-logging in works once. Users report the pattern feels random.
The code that's running in prod
# auth_middleware.py
# Validates JWTs at every authenticated endpoint. Rejects tokens
# whose 'iat' (issued at) timestamp is in the future to defeat
# replay-via-future-stamped attacks.
import time
import jwt
def validate_token(token, secret):
payload = jwt.decode(token, secret, algorithms=["HS256"])
now = time.time()
if payload["iat"] > now:
raise jwt.InvalidIssuedAtError("token issued in the future")
if payload["exp"] < now:
raise jwt.ExpiredSignatureError("token expired")
return payload
Log dashboard (last 60 seconds)
10:21:14
INFO
[auth-svc]
Issued token for user u_88 with iat=1719224474 (10:21:14 UTC)