Fernbank
Fintech
p95 from 4.1s to 780 ms
p95 latency, down from 4.1 s
Fernbank’s support agent was fast in testing and slow at 2pm. Failover fixed what capacity planning could not.
The problem
Median latency looked fine. p95 did not. Between 1pm and 4pm the single provider they called degraded, and 6% of sessions took over four seconds. Users abandoned those sessions at three times the normal rate.
What they did
They set a one second latency budget with failover enabled, and turned on streaming so the first token arrives while the rest is still generating.
What changed
p95 dropped to 780 ms and stayed there through the afternoon. Abandoned sessions fell 34%. They did not add a single line of retry logic.
Our p95 was a support problem long before it was an engineering problem. Nobody could see it because the median looked healthy.
Tomás Berger
Head of Engineering, Fernbank
By the numbers
34% fewer abandoned sessions
0 retry code written
2 providers in rotation
1M tokens a month, free forever.
No card, no trial clock, no sales call. Move to a paid plan when the invoice makes sense, or don’t.