Latency stability
Check LLM latency stability
Repeated probes reveal whether this endpoint looks like one stable upstream.
How this check works
An LLM latency variance check repeats the same request and compares delay. Stable latency looks like one upstream. Large swings can mean queues, extra hops, or a model switch.
01Probe
Repeat the same request on the same endpoint.
02Inspect
Compare first-token delay across those repeats.
03Conclude
Flag high variance that looks like queues, extra hops, or a model switch.
- How to read the result
- One fast reply can hide a slower backup path. This is a stability signal, not a throughput ranking.
- When to run this check
- Use this when answers feel sometimes instant and sometimes stalled. A single speed test can miss that swing.
Recent reports
Public reports that included this check.
- sub.callai.onegpt-5.6-solCautionReport timeAug 18
- sub.callai.onegpt-5.6-solBlockedReport timeAug 18
- ai.databyte.co.iddatabyte-m1NormalReport timeAug 18
- ai.databyte.co.iddeepseek-v4-flashNormalReport timeAug 18
- ai.databyte.co.idMiniMax-M3CautionReport timeAug 18
- tokengate-cqt9ivzs.manus.spaceclaude-opus-5CautionReport timeAug 17
FAQ
What is an LLM latency variance check?
It repeats the same request and looks at how much latency changes. The point is stability, not one lucky fast reply.
What can high variance mean?
It can mean queueing, multi-routing, cold starts, or silent model switching. It is a signal, not a full root-cause report.
Is this the same as a speed benchmark?
No. A speed test ranks throughput and first-token latency. This check looks at how stable those delays stay across repeats.