I use a vision model as a measuring instrument: it looks at each generated image and fills in a small form — camera height, camera pitch, and so on. The research question behind it was “which phrasings produce a usable low-angle shot?” When I scored the instrument against 30 images I had labelled by hand, one line in its prompt looked suspicious.
The line
The first sentence the model received was: Research question this measurement serves: …, followed by the question. The function's own docstring said nothing but the image should go in. I had written the principle and broken it two lines below.
In effect I had told the judge what I hoped to find and then asked, “is this it?” The scores showed it. Low-angle recall was 100% on every version of the reader, which sounds wonderful, and precision was between 9.5% and 12.5%. The model wasn't finding low angles. It was calling nearly everything a low angle, so it couldn't miss one. Up to half of the high-angle photos got labelled low.
Measuring the size of the leak
Removing the line was easy. I wanted to know how much it had mattered, so I read the same 30 images twice with the same model: one arm with the question, one without. For scoring I used the 27 images where my own label wasn't “unclear.”
Agreement with my labels was 44.4% without the question and 11.1% with it. Twenty-five of the 30 images changed their answer depending on that one sentence. What matters more is the direction: every change moved toward “low angle,” and none went the other way. With the question, the model produced 13 low angles; without it, zero. Four of the five high-angle calls were erased into “unclear.”
The version without the question isn't good either. 44.4% is far below the 80% threshold I'd set, and it missed both of the real low-angle images. But the two failures differ in kind. The leaked version is biased: it pushes results in a direction, which flips conclusions. The clean version is merely dull, which weakens them. If there is an order in which to fix things, biased comes first.
How the experiment was kept honest
- Same bytes for both arms. Each image was fetched once and the identical data was sent twice, so a resize step couldn't differ between arms. Only the instruction varied.
- Arms alternated per image, rather than running one whole arm first. If the gateway slowed down or the model changed midway, the drift wouldn't land on one arm only.
- The full prompt is stored with each result. “The version without the question was better” becomes an argument later, and by then the code has changed.
- Experiments live in their own table. Half of the 60 rows were deliberately contaminated. Had they gone into the main observations, the aggregate views would have counted the contaminated values as properties of the images, so a test now enforces that the experiment code can't write there.
Limits I'm keeping in view
Only 2 of the 27 images are truly low-angle, so every precision and recall figure in this study hangs on those two. I'm not grading the reader with these numbers. What I measured is the difference between two arms on the same images, model and fields, and that difference is valid on its own terms. Whether the model can't see low angles or the sample simply has too few of them is a question this experiment cannot answer; more low-angle labels come first.
The replacement instruction is deliberately dull: judge every field from the image alone, no particular answer is expected, and the ordinary value is as correct as any other. That last clause matters. Bias comes from knowing what is being looked for, and also from the mood of a task where only special answers seem welcome.