Try it yourself
The sequence below contains the thesis in miniature. The query is abstracted: a drawing matched to photographic evidence, with no pixel in common. The target is then occluded, behind other players in identical navy. Finally it is absent, outside the frame altogether, and the identity must survive an interval in which the camera observes nothing relevant to it. Nine seconds of broadcast hockey at 3 frames per second, exactly the frames put to the vision–language models discussed in the thesis. See how you get on.
a drawing, not a photograph
Most people manage this, particularly when stepping through the frames in order. Contemporary vision–language models, given the same twenty-five frames and the same question, do not. The failure is not perceptual: every frame is sharp and the jersey is legible in the first one. Two things make it hard. Granularity: asked which of several candidates conceals an object, a model will often answer correctly, but its competence is verbal and coarse, whereas these tasks demand a pixel-level mask, an ordered ranking, or a continuous direction. Compounding: the sequence is not difficult in one respect but in three at once, and it demands all three in order, on the same target.