How Does an AI See an Image? Why It Reads the Screenshot but Misses the Tiny Number

How does an AI see an image?

A model never sees an image as a surface of pixels. A preprocessor cuts it into a grid of fixed tiles, the patches. Each tile is compressed into a handful of numbers and read like a token. The model processes the scene, not the individual picture elements. That is why the fine print falls through the grid first.

The same model reads out the screenshot of an error message word for word. On the next image, a dense dashboard, it misses the small number in the corner. No glitch. The model never saw the image as pixels.

This is again the brilliant, fast, occasionally-wrong intern from this series. Strong at recognizing the scene. Blind in a surprising spot. With the tiny number, the blindness begins.

The Model Reads a Grid, Not a Photo

An image never reaches the model as a surface of pixels. Before that, a preprocessor cuts it into a grid of small tiles. Each tile is compressed into a handful of numbers, a short profile. The model then reads these profiles in order, the same way it reads text as tokens (see What Is a Token).

Like a grid of thumbnail-sized previews. The scene as a whole is clearly recognizable. But the fine print inside a single tile vanished into the profile long ago. The intern does not look at the photo. It skims the grid and says what is on it. Good for the big picture, blind to the fine text.

This picture has a name: the model is a Vision-Language Model, VLM for short. The tiles are called patches, and each becomes an image token.

A VLM like this is one building block in the series. Here is where we are:

Under the Hood

A Vision EncoderVision EncoderThe image part of a multimodal model. It breaks the image into fixed patches and translates each into a vector that the language model can process like a token.breaks the image into fixed patches, around 14 to 16 pixels per edge in common architectures. Each patch is linearly projected into a vector, along with information about its position in the grid. These image tokens run through a Vision TransformerVision TransformerA Transformer architecture for images. It treats image patches like a sequence of tokens and lets them compute over each other via attention, exactly like a language model with word tokens.and a projection layer that translates them into the input space of the language model. From there, the model treats them like tokens: attention runs over text and image tokens together.

A number to go with it: Claude has processed images at high resolution since Opus 4.7, up to 2576 pixels on the long edge, with up to roughly 4,784 tokens per image at full resolution. Before that, it capped out at around 1,600 tokens. So a high-resolution image costs as much as several pages of text.

Where It Breaks

If the grid is too coarse for the detail, the detail is lost. Tiny text, dense tables, a small number in the dashboard: all of it blurs into the profile of its tile before the model even reads. Uploading the whole page shrunk down makes it worse, because then even more content shares even fewer tiles.

For mid-sized businesses, this has two consequences. First, cost: images cost tokens, and at full resolution, many. A pipeline that sends hundreds of receipts or screenshots through a model pays per image, and the bill grows with resolution. Second, reliability: “the model did read the screenshot” does not mean it read every number on it.

The remedies are simple. Crop the relevant section and send it at good resolution, instead of shrinking the whole page. Budget image tokens like text tokens. And for critical numbers, never blindly trust that the fine print made it through.

A model does not see an image, but a grid of tiles that turned into numbers. It reads the scene, not the pixels. And the fine print falls through the grid first.

Next term: how many of these tokens, image and text alike, a model takes in at once, and why that limit decides cost and errors.

Frequently Asked Questions

How does an AI process an image? A preprocessor cuts the image into a grid of fixed tiles, called patches. Each tile is compressed into a vector of numbers and treated like a token. The model then reads these image tokens, never the individual pixels.

Why does an AI model miss small details in a screenshot? Because fine details blur inside the tile before the model reads. If the grid is too coarse for a tiny number or small text, the detail is lost in the tile’s numeric profile. The model reads the scene, not every pixel.

Does an image cost more than text when using AI? Often yes. Images are converted into tokens, and a high-resolution image can cost as many tokens as several pages of text. With Claude, since Opus 4.7 it is up to roughly 4,784 tokens per image at full resolution. A pipeline with many images pays per image, and the cost grows with resolution.

How do I get reliable results when the AI should read numbers from an image? Crop the relevant section and send it at good resolution, instead of shrinking the whole page. Budget image tokens like text tokens. And for critical numbers, never blindly trust that the fine print made it through; verify it.

Part of the AI Engineering, explained series. Related: What Is a Token and What Does an AI Project Cost in the Mittelstand.