A pipeline that reads images and writes structured descriptions of them had one of its readers moved to a different machine — a change that turned out to need exactly one row updated in a database, nothing redeployed. The worker checks that value on every job, so the switch took effect without a restart. Round-trip time for the new machine came in around 7 to 12 seconds against roughly 50 seconds for the reader it replaced, with parsing succeeding on all 33 test images. On paper it was a clean win.
The screen that showed which model was doing the reading disagreed. It kept displaying the name of the reader that had just been retired, because the header pulled that name from a fixed setting rather than from the job that had actually just run. Anyone looking at a result that seemed off would go looking for the cause in the wrong reader entirely — the display wasn't lying about the system's intent, it was lying about what had actually happened, which is a worse kind of wrong because it still looks authoritative. The fix was to stop asking the header what it was configured to use and start asking the most recent job what had actually served it, through a small status endpoint built for exactly that question.
Two readers agree only on the easy cases
With two readers now genuinely available, I ran the same six images through both and compared their answers on two specific measurements — camera height and camera angle. They matched on four of six for each, and the images where they agreed were the unremarkable, straightforwardly framed ones. On one image described in its own generation prompt as shot from a low angle, one reader called it low-and-downward and the other called it elevated-and-upward — opposite readings, each internally consistent with nothing else it said about the same image. Neither was obviously right.
The more useful signal showed up somewhere else. On two separate images, both readers independently converged on a description that flatly contradicted what the image had actually been generated to show — both said "facing away" for a pose the prompt specified as facing forward. When two readers that never see each other's answers land on the same wrong description, that stops looking like reader noise and starts looking like the generator not doing what it was asked. Disagreement between readers points at a weak axis to stop trusting; agreement on a description that contradicts the source prompt points somewhere else entirely — at the thing being read, not the one reading it.
The context window had far more room than assumed
Separately, a proposal to shrink the context window fourfold to save memory got tested rather than argued about. Tallying actual token usage across several thousand real requests, the largest single exchange on record — input and output combined — came to under 2,500 tokens, against a full window of over 16,000. Running two real images through the smaller window directly produced identical structured output to the larger one, field for field. The one honest caveat: a couple of individual values shifted slightly between window sizes on repeat runs, which could just as easily be ordinary sampling noise as anything about the window itself — distinguishing the two would need the same image run many times at each setting, which hadn't been done yet.
The two problems rhyme. A display built to show configuration will keep telling you the truth about a setting long after that setting stopped being the truth about what ran. Reading the log of what a system actually did, rather than what it was told to do, is a different question — and often the only one worth asking when a result looks wrong.