Ollama API: 'Multimodal data provided, but model does not support multimodal requests.'

{"error":"Multimodal data provided, but model does not support multimodal requests."} is Ollama's HTTP 400 answer when a request carries an images array and the model it names reads text only. The image itself is fine; drop the images field, or send the image to a model built to read images.

Check the model first with ollama show <model>: the Capabilities block lists what it accepts. qwen2.5:0.5b showed completion and tools, its image request was refused, and the same request without the image was answered with HTTP 200.

The error

PS> curl.exe -s -w "\nHTTP %{http_code}\n" http://127.0.0.1:15190/api/chat -d "@chat.json"
{"error":"Multimodal data provided, but model does not support multimodal requests."}
HTTP 400
PS> curl.exe -s -w "\nHTTP %{http_code}\n" http://127.0.0.1:15190/api/generate -d "@gen.json"
{"error":"Multimodal data provided, but model does not support multimodal requests."}
HTTP 400

The bodies were chat.json with "messages":[{"role":"user","content":"What is in this image?","images":["<base64>"]}] and gen.json with "prompt" and the same images array, both naming qwen2.5:0.5b.

Why it happens

The image was a valid PNG encoded as plain base64, and the words of the answer are about the model, not the data. /api/chat and /api/generate gave the same words and the same status. The text model did not answer the text and skip the image: the whole request came back as an error, with no message or response field.

ollama run behaves differently. With a file path inside the prompt, ollama run qwen2.5:0.5b "What is in this image? .\red.png" printed no error and exited with 0; the model answered the text and said it could not see images. So on the command line a text model gives you a polite paragraph instead of a refusal, and the path reached it as plain words.

The fix

PS> ollama show qwen2.5:0.5b
...
  Capabilities
    completion
    tools
...

{"model":"qwen2.5:0.5b","stream":false,"messages":[{"role":"user","content":"Say hi in three words"}]}

PS> curl.exe -s -w "\nHTTP %{http_code}\n" http://127.0.0.1:15190/api/chat -d "@chat-ok.json"
{"model":"qwen2.5:0.5b",...,"message":{"role":"assistant","content":"Hello, how are you?"},"done":true,"done_reason":"stop",...
HTTP 200

Either keep the model and send text, as above, or keep the image and change the model field to a model built for images. The test server had no such model and none was pulled, so the second route was not run here; read the Capabilities block of the model you pull before you rely on it.

How it was reproduced

On Windows 11 with Ollama 0.40.16, against a separate test server on 127.0.0.1:15190 with its own models folder. A 16 by 16 PNG was made in PowerShell and encoded with [Convert]::ToBase64String; the JSON bodies were written to files with [IO.File]::WriteAllText and sent with curl.exe -d "@file" to /api/chat and /api/generate. The same model was then asked a text-only question, and ollama run was given the file path in its prompt.

Frequently asked

What does 'Multimodal data provided, but model does not support multimodal requests' mean?

The request carried an images array and the model named in it accepts text only. Ollama answered HTTP 400 on both /api/chat and /api/generate. Remove the images, or send them to a model built to read images.

How do I check whether an Ollama model supports images?

Run ollama show with the model name and read the Capabilities block. qwen2.5:0.5b listed completion and tools, and a request with an image to it was refused with HTTP 400.

Why does ollama run not complain when I give a text model an image path?

In the test with Ollama 0.40.16 the text model received the path as part of the prompt, answered that it could not see images, and the command exited with 0.

More decoded errors in the Fixes category. If the server answers about base64 before it gets to the model, see Ollama API: 'illegal base64 data at input byte'.