{"error":"illegal base64 data at input byte 4"} from Ollama's /api/generate or /api/chat means a string in the images array is not plain base64. In the test, a data:image/png;base64, prefix failed at byte 4 and a file name (red.png) at byte 3. Send the bare base64 of the file's bytes, with no prefix.

In PowerShell, [Convert]::ToBase64String([IO.File]::ReadAllBytes("red.png")) gives the string to put in images. Drop the data:...;base64, part that browser code and other APIs add, and do not pass a path.

The error

The body file img-dataurl.json, then the request:

{"model":"qwen2.5:0.5b","prompt":"What colour is this image?","stream":false,"images":["data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAIAAABLbSncAAAAEklEQVR4nGP4z8CAFWEXHbQSACj/P8Fu7N9hAAAAAElFTkSuQmCC"]}
PS> curl.exe -s -i -d "@img-dataurl.json" http://127.0.0.1:15191/api/generate
HTTP/1.1 400 Bad Request
Content-Type: application/json; charset=utf-8
...
Content-Length: 47

{"error":"illegal base64 data at input byte 4"}

With a file name in the array instead ("images":["red.png"]):

PS> curl.exe -s -i -d "@img-path.json" http://127.0.0.1:15191/api/generate
HTTP/1.1 400 Bad Request
...
{"error":"illegal base64 data at input byte 3"}

Why it happens

The images field takes base64 text, and the server answered 400 when the string did not decode as base64. The byte number matches the position of the first character that is not a base64 character, counted from zero: in data:image/png that is the colon at position 4, and in red.png it is the dot at position 3. The same data URL sent to /api/chat inside a message's images array gave the same input byte 4 answer.

The fix

PS> $b64 = [Convert]::ToBase64String([IO.File]::ReadAllBytes("red.png"))
...
PS> [IO.File]::WriteAllText("$PWD\chat-img2.json", '{"model":"qwen2.5:0.5b","stream":false,"messages":[{"role":"user","content":"What colour is this image?","images":["' + $b64 + '"]}]}')
PS> curl.exe -s -i -d "@chat-img2.json" http://127.0.0.1:15191/api/chat
HTTP/1.1 400 Bad Request
Content-Type: application/json; charset=utf-8
...
Content-Length: 85

{"error":"Multimodal data provided, but model does not support multimodal requests."}

With the bare base64 string the base64 error is gone, on /api/chat and on /api/generate alike. The next answer in this test is about the model: qwen2.5:0.5b takes text only, and no image model was on the test server. The base64 error did not come back for this string; with a model that accepts images, this is the request shape to try. If you build the string in another language, encode the raw file bytes, not a path and not an already-encoded data URL.

How it was reproduced

Ollama 0.40.14 (ollama --version) on Windows 11, run as a separate ollama serve on 127.0.0.1:15191 with its own models folder. The image was an 8 by 8 red PNG of 75 bytes written by a short Python script. Requests went through curl.exe from Windows PowerShell 5.1 with ASCII body files, three to /api/generate (data URL, file name, bare base64) and two to /api/chat (data URL, bare base64).

Frequently asked

How do I send an image to the Ollama API?
Put the base64 of the image file's bytes, with no data: prefix, in the images array of /api/generate or of a message in /api/chat. In PowerShell, [Convert]::ToBase64String([IO.File]::ReadAllBytes("red.png")) gives that string. The request must also name a model that accepts images; the test had none, so the next answer was Multimodal data provided, but model does not support multimodal requests.
Why does Ollama say illegal base64 data at input byte 4?
The string starts with data:image/png;base64, and the colon at position 4 is not a base64 character. Remove everything up to and including the comma.
What does Multimodal data provided, but model does not support multimodal requests mean?
The image string decoded, but the model named in the request takes text only. In the test it came from qwen2.5:0.5b; send the request to a model that accepts images.

More decoded errors in the Fixes category. The other Ollama endpoints are in the Ollama API guide.