engineering postmortem

Your Observability Dashboard Is Throttling the Agent It Watches — an Async Latency Postmortem

A P99 latency spike every few minutes — CPU, QPS, error rate all flat, Agent code unchanged. The culprit no one suspects: the observability dashboard built to watch the Agent was choking it. The postmortem — how one sync call freezes a single-threaded async event loop, the two-line fix (to_thread + TTL cache), and 10 event-loop probes you can add to your async service this week.