04 · Work — 02
Crash triage that replaced a paid subscription
A system running across hundreds of sites produced more crash reports than anyone could read, and sorting them by hand cost a morning. The paid tool that summarised them billed for everything it read, and the bill grew with the site count.
- Of 148 reports filtered before anyone read them
- 64
- Real bugs discarded by mistake
- 0
Node.js · SQLite · Sentry API · structured LLM output · cost guardrails
The problem in the room
Hundreds of sites produce a large volume of crash reports, most of which are not bugs: a tablet losing wifi, a device out of storage, a known quirk on old hardware. They sit in the same list as the reports that matter, and separating them by hand costs a morning.
The paid tool used to summarise that list billed for everything it read, and the bill grew with the site count.
What we did
The order of operations is the design. Plain rules run first, so dropped connections, storage errors and known device quirks are filtered out before a model sees them — only reports that might be fixable bugs are worth paying to analyse.
Every crash is fingerprinted and cached, so the same problem reported from thirty devices is analysed once. The whole run sits under a hard spending cap and stops when it is reached.
The tool is open source under MIT and was validated on the same 400-site system.
What it does now
In the measured run, 64 of 148 reports were filtered before a person or a model saw them, with no real bug discarded. Per-report analysis costs between $0.15 and $0.27, in place of a subscription that charged regardless of what it read.