Degraded retrieval latency on us-east cluster
Apr 23 · 14:08–14:26 UTC · 18 minDetected,p99 retrieval latency rose above the 800ms threshold; pager fired.
Investigating,engineer on-call confirmed elevated tail latency; isolated to one shard.
Identified,a noisy-neighbour query was rebuilding an index; rate-limit applied.
Mitigated,p99 returned below threshold; no failed requests, only slow ones.
Monitoring,system stable; declared resolved.