IncidentPaged 02:18 UTC, Saturday (3rd time this week)
asyncio worker hits 'Too many open files' after 45 minutes
Service: background scraper-worker (asyncio)
Symptom: After 40-50 minutes of uptime, every request fails with `OSError: [Errno 24] Too many open files`. SIGTERM + restart clears it for another 45 minutes.
The code that's running in prod
# scraper_worker.py
# Polls a list of upstream feeds every 60 seconds, parses each
# response, persists deltas. Long-running asyncio worker.
import aiohttp
async def fetch_one(url):
"""Return the response body, or None on any error."""
session = aiohttp.ClientSession()
try:
async with session.get(url) as resp:
return await resp.text()
except Exception as e:
print(f"[ERROR] {url}: {e}")
return None
async def poll_loop(feeds):
while True:
for url in feeds:
body = await fetch_one(url)
if body:
process(body)
await asyncio.sleep(60)
Log dashboard (last 60 seconds)
Pick the root cause
Four candidates. Three are plausible but mistake symptom for cause. Pick what you'd write in the postmortem timeline.