A price collector I run had already gone silent for a full day once, for a reason I've written about elsewhere: a target site swapped its front end, and "zero new listings" looked exactly like an ordinary quiet day because nothing about it counted as an error. Fixing the immediate parser was the easy part. The harder question was what to build so that the next front-end change wouldn't repeat the exact same silence.
An alarm that fires once, not never and not constantly
The naive fix — alert whenever a selector returns nothing — has an obvious failure mode: it fires once and then keeps firing on every single run afterward, and a warning that repeats forever gets ignored exactly like one that never fires at all. What went in instead is a small tracker attached to each site adapter that watches whether its selectors are returning the shape and count it expects, and reports a single tagged warning the first time a given selector type drifts in a run — not one line per broken record, one line per broken pattern. The goal wasn't more warnings, it was one legible warning where there used to be zero.
A second, quieter change went in alongside it: validation used to treat a record as all-or-nothing — if any field failed to parse, the whole listing was dropped. Now required and optional fields are judged separately. A listing missing an optional field loses that one field and survives; only a missing required field drops the record. A partial break in one price format now shows up as a gap in the data instead of erasing listings that were otherwise perfectly readable — which matters because the first version of this bug would have looked, again, like fewer listings existing that day rather than like a parsing problem.
Testing the fallback by breaking it on purpose
The pipeline also gained a second way to fetch a page: if the fast, plain page fetch comes back with nothing usable, it now falls back to a full browser-rendered fetch instead of simply failing. Trusting that this actually works meant not waiting for the next real outage to find out — the static fetch path was deliberately disabled to simulate exactly the kind of failure that had gone unnoticed before, and the fallback picked up cleanly, returning the same forty records within about two seconds.
One more thing turned up while re-checking real listings against the rebuilt pipeline: two listings carried no advertising marker at all and matched nothing in the existing catalog — single items from a marketplace add-on with no proper listing page of their own, where the price had to come from a different fallback field entirely, not the one every other listing on the site used. It's a small case, two records out of thousands, but it's exactly the kind of edge that a validated fallback catches and a silent failure would have simply dropped.
None of this prevents the next front-end change — nothing can promise that. What changed is what happens in the hour after it: instead of a green dashboard and a warning nobody reads, there's now one line that says which pattern broke, a fetch path that tries a second way before giving up, and a record that survives a partial parsing failure instead of vanishing along with it. Building the alarm for the exact silence you just found, and then forcing that silence again on purpose to see if the alarm actually rings, is the only way to know it does.