Is Ollama Using My GPU? ollama ps, the Server Log, nvidia-smi and Why It Falls Back to the CPU

The ten-second answer: run a prompt, then ollama ps while the model is still loaded, and read the PROCESSOR column. 100% GPU means every layer sits in video memory, 100% CPU means none does, and a split such as 51%/49% CPU/GPU means the model did not fit. The rest of this post is for when the answer is not the one you wanted: which lines of the server log say why, what nvidia-smi can and cannot show on Windows, the three causes that were reproduced here, and how much speed the GPU is worth, measured.

Everything was captured on one PC on 2026-10-30: Ollama 0.40.11 for Windows, an NVIDIA GeForce RTX 3080 Ti Laptop GPU with 16 GB, driver 616.56, 32 GB of system memory, plus Ollama 0.34.2 as a service in WSL and the ollama/ollama image (0.40.8) in Docker Desktop. The CPU-only runs used a second ollama serve started for the test on its own port, so the PC's own tray app was used as it is and nothing in it was changed.

What ollama ps shows

All on the GPU, the usual case for a 3B model on a 16 GB card:

PS> ollama ps
NAME          ID              SIZE      PROCESSOR    CONTEXT    RUNNER      UNTIL
qwen2.5:3b    357c53fb659c    2.7 GB    100% GPU     4096       llamacpp    4 minutes from now

Nothing on the GPU, from the test server with both GPU backends hidden:

NAME            ID              SIZE      PROCESSOR    CONTEXT    RUNNER      UNTIL
qwen2.5:0.5b    a8b0c5157701    682 MB    100% CPU     4096       llamacpp    4 minutes from now

A split, here forced with the num_gpu option so that only 24 of 49 layers went to the card, while a second model that had been pushed off the GPU sat entirely in system memory:

NAME           ID              SIZE      PROCESSOR          CONTEXT    RUNNER      UNTIL
qwen2.5:14b    7cdf5a0187d5    12 GB     51%/49% CPU/GPU    4096       llamacpp    4 minutes from now
qwen2.5:3b     357c53fb659c    2.7 GB    100% CPU           4096       llamacpp    3 minutes from now

Three details that trip people up. SIZE is the loaded size, not the file: qwen2.5:3b is 1.9 GB in ollama list and 2.7 GB here, because the context cache is counted. The FAQ example shows five columns; 0.40.11 adds CONTEXT and RUNNER, and the 0.34.2 build in WSL prints CONTEXT without RUNNER. And before the first prompt the command prints only its header, which says nothing about the GPU. The API gives the same answer as two numbers, size and size_vram, from which the percentage is computed; the CPU case above reads "size":2716000582 with "size_vram":0.

The server log, line by line

On Windows the log is %LOCALAPPDATA%\Ollama\server.log; the troubleshooting page says "The most recent server logs will be in" server.log and older ones in server-#.log. Every start of the server writes the discovery result:

msg="discovering available GPUs..."
msg="inference compute" id=0 filter_id=0 library=CUDA compute=8.6 name=CUDA0 description="NVIDIA GeForce RTX 3080 Ti Laptop GPU" libdirs=ollama,cuda_v13 driver=13.4 pci_id=0000:01:00.0 type=discrete total="16.0 GiB" available="14.9 GiB"
msg="vram-based default context" total_vram="16.0 GiB" default_num_ctx=4096

That is the card, the library it will use, its compute capability, the CUDA driver version and the free VRAM, and the default context Ollama picked from that VRAM. When no GPU is usable the same line names the CPU instead, and the default context is chosen from a VRAM of zero. This is the test server with both device variables set:

msg="user overrode visible devices" CUDA_VISIBLE_DEVICES=-1
msg="user overrode visible devices" GGML_VK_VISIBLE_DEVICES=-1
msg="if GPUs are not correctly discovered, unset and try again"
msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="31.8 GiB" available="7.4 GiB"
msg="vram-based default context" total_vram="0 B" default_num_ctx=4096

No line saying "no compatible GPUs" appeared in any log of this run, on 0.40.11 or on 0.40.8 in Docker; id=cpu library=cpu is the whole message. Then, at every model load, Ollama 0.40 starts one llama.cpp llama-server.exe per model and the log records the memory plan and the verdict:

msg="starting llama-server" cmd="C:\\Users\\...\\llama-server.exe --model C:\\Users\\...\\.ollama\\models\\blobs\\sha256-5ee4f07cdb9beadbbb293e85803c569b01bd37ed059d2715faa7bb405f31caa6 --port 61561 --host 127.0.0.1 --no-webui --offline -c 16384 -np 4 ...
msg="gpu memory" id=0 library=CUDA available="14.4 GiB" free="14.9 GiB" minimum="457.0 MiB" overhead="0 B"
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 16383 MiB):
  Device 0: NVIDIA GeForce RTX 3080 Ti Laptop GPU, compute capability 8.6, VMM: yes, VRAM: 16383 MiB
load_backend: loaded CUDA backend from C:\Users\...\AppData\Local\Programs\Ollama\lib\ollama\cuda_v13\ggml-cuda.dll
common_params_fit_impl: projected to use 2579 MiB of device memory vs. 15239 MiB of free device memory
common_params_fit_impl: will leave 12659 >= 1024 MiB of free device memory, no changes needed
load_tensors: offloaded 37/37 layers to GPU
msg="llama-server started in 2.02 seconds"
slot print_timing: id  0 | task 0 |        eval time =      67.10 ms /     9 tokens (    8.39 ms per token,   119.23 tokens per second)

offloaded 37/37 layers to GPU is the line to search for: N of N is all on the card, fewer is a split, and the CPU-only server printed no offloaded line at all, only load_tensors: CPU model buffer size = 345.65 MiB. The last line is the speed of one request as llama.cpp measured it. When the plan does not fit, the same block shows the arithmetic. This is qwen2.5:14b asked for a 32,768-token context:

common_params_fit_impl: context size set by user to 131072 -> no change
common_params_fit_impl: cannot meet free memory target of 1024 MiB, need to reduce device memory by 18662 MiB
common_params_fit_impl:   - CUDA0 (NVIDIA GeForce RTX 3080 Ti Laptop GPU): 20 layers,  13943 MiB used,   1295 MiB free
load_tensors: offloaded 20/49 layers to GPU
llama_kv_cache:        CPU KV buffer size = 14848.00 MiB

NAME           ID              SIZE     PROCESSOR          CONTEXT    RUNNER      UNTIL
qwen2.5:14b    7cdf5a0187d5    35 GB    59%/41% CPU/GPU    32768      llamacpp    4 minutes from now

A 9 GB model became a 35 GB load, 20 of 49 layers on the card and a 14.5 GB context cache in system RAM, and the load took 47 seconds. Why the context reads 131072 rather than 32768 is explained under parallel slots below. One more line worth knowing: when a new model does not fit beside the loaded ones, Ollama unloads them first and says so:

msg="llama-server model predicted to exceed available memory, evicting" predicted="11.4 GiB" predicted_num_ctx=16384 num_batch=512 available="12.4 GiB" gpu_free="12.4 GiB" system_free="4.6 GiB" system_limited=false

nvidia-smi on Windows

The process doing the work is llama-server.exe, a child of the tray app's ollama.exe, and it appears in the process table with type C (compute) among the C+G entries of browsers and the desktop:

|    0   N/A  N/A           24156    C+G   ...Chrome\Application\chrome.exe      N/A      |
|    0   N/A  N/A           42432      C   ...a\lib\ollama\llama-server.exe      N/A      |

The per-process memory column reads N/A under the Windows display driver model, in the table and in the --query-compute-apps form alike, so you cannot read how much VRAM one model holds from it. The total is readable: memory.used was 3,448 MiB with the tray app idle and 14,831 MiB with qwen2.5:14b loaded. The other entries matter too. Browsers, the desktop and other apps hold VRAM, and so does another Ollama; at one point in this run a second server held about 3 GB, and the scheduler counted available="12.4 GiB" instead of 14.9 before the next load.

Three reasons for the CPU, reproduced

A device variable hides the card. The GPU page says that to "force CPU usage, use an invalid GPU ID" such as -1 in CUDA_VISIBLE_DEVICES. On Windows 0.40.11 that alone did not give the CPU. With only CUDA_VISIBLE_DEVICES=-1 set, the test server found the same card again through its Vulkan backend, which the page says is enabled by default when installed:

msg="inference compute" id=0 filter_id=0 library=Vulkan compute=0.0 name=Vulkan0 description="NVIDIA GeForce RTX 3080 Ti Laptop GPU" libdirs=ollama,vulkan driver=0.0 pci_id=0000:01:00.0 type=discrete total="15.8 GiB" available="15.0 GiB"

and ran qwen2.5:0.5b at 100% GPU from ggml-vulkan.dll. The CPU needed the second switch as well, GGML_VK_VISIBLE_DEVICES=-1, which the same page lists as the way "you can disable all Vulkan GPUs". The lesson runs the other way for most readers: if the log says user overrode visible devices, something in your environment set one of those variables, and that is why the card is missing. Neither variable appears in ollama serve --help; the server config line at the top of the log prints both.

A Docker container started without the GPU. Same image, same model folder, two runs on Docker Desktop with the WSL 2 backend. Without --gpus all (the container names are the test's):

PS> docker run -d --name howto-gpu-cpu -p 127.0.0.1:15192:11434 -e OLLAMA_NOPRUNE=1 --mount type=bind,source=C:\Users\...\howto-ollama-a\models,target=/root/.ollama/models ollama/ollama:latest
msg="Listening on [::]:11434 (version 0.40.8)"
msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="15.5 GiB" available="15.1 GiB"
qwen2.5:0.5b    a8b0c5157701    486 MB    100% CPU     4096       llamacpp    4 minutes from now

With it:

PS> docker run -d --gpus all --name howto-gpu-nv -p 127.0.0.1:15193:11434 -e OLLAMA_NOPRUNE=1 --mount type=bind,source=C:\Users\...\howto-ollama-a\models,target=/root/.ollama/models ollama/ollama:latest
msg="inference compute" id=0 filter_id=0 library=CUDA compute=8.6 name=CUDA0 description="NVIDIA GeForce RTX 3080 Ti Laptop GPU" libdirs=ollama,cuda_v13 driver=13.4 pci_id=0000:01:00.0 type=discrete total="16.0 GiB" available="14.9 GiB"
load_tensors: offloaded 25/25 layers to GPU
qwen2.5:0.5b    a8b0c5157701    483 MB    100% GPU     4096       llamacpp    4 minutes from now

The documented command is docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama, and the troubleshooting page's first check for containers is docker run --gpus all ubuntu nvidia-smi: "if this doesn't work, Ollama won't be able to see your NVIDIA GPU".

The model and its context do not fit. No model on this PC is larger than the 16 GB card by file, so the splits above were produced two ways: a 32k context on the 14B model, and the num_gpu option, which sets how many layers go to the card. num_gpu 0 in a request forces that model onto the CPU entirely, and the tray app obliged with 100% CPU and "size_vram":0. The option is accepted by the API, and absent from the Modelfile parameter table and the option list on docs.ollama.com; it appears once, in an example in the GitHub copy of the API reference. The remaining documented cause, a driver that is too old or a card below compute capability 5.0, could not be reproduced with one modern card. The GPU page states the floor: "Ollama supports Nvidia GPUs with compute capability 5.0+ and driver version 550 and newer." The Windows page asks for "NVIDIA 551.61 or newer Drivers if you have an NVIDIA card".

How much the GPU is worth, measured

Same prompt everywhere, Count from 1 to 300, separated by commas., with temperature 0, seed 42 and num_predict 200, so every answer is exactly 200 tokens. Tokens per second is eval_count over eval_duration from the response; seconds per answer is one curl call timed with Measure-Command after the model was loaded. The CPU is the laptop's i9-12900HX; yours will differ.

Model	Where	PROCESSOR	Tokens/s	Seconds per answer
qwen2.5:0.5b	test server, CUDA	100% GPU	419.1	0.55
qwen2.5:0.5b	test server, CUDA and Vulkan hidden	100% CPU	101.4	2.05
qwen2.5:3b	tray app	100% GPU	162.1	1.33
qwen2.5:3b	tray app, num_gpu 0	100% CPU	24.8	8.17
qwen2.5:14b	tray app	100% GPU	47.7	4.36
qwen2.5:14b	tray app, num_gpu 24	51%/49% CPU/GPU	10.5	19.27

The 3B model runs 6.5 times faster on the card, the half-split 14B model 4.5 times slower than the whole one, and a 200-token answer that takes 1.3 seconds takes 8 on the CPU. The request that made the CPU row, for anyone who wants to repeat it against a running tray app:

{"model": "qwen2.5:3b", "prompt": "Count from 1 to 300, separated by commas.", "stream": false, "options": {"temperature": 0, "seed": 42, "num_predict": 200, "num_gpu": 0}}

The context-length page puts the advice in one sentence: "For best performance, use the maximum context length for a model, and avoid offloading the model to CPU." A split is not a failure, it is a 4x slowdown you did not ask for.

The setting that changes every memory number: OLLAMA_NUM_PARALLEL

The load line above shows -c 16384 -np 4 for a 4,096-token context. That is because this PC has OLLAMA_NUM_PARALLEL=4 in its user environment, and llama-server sizes its context cache for four requests at once: 16,384 cells for a 4k context, 131,072 for the 32k request, which is where the 35 GB came from. The FAQ defines the variable as "The maximum number of parallel requests each model will process at the same time, default 1." and says the required RAM scales with it. A test server started without the variable confirmed the default, OLLAMA_NUM_PARALLEL:1 and -c 4096 -np 1, and its cache for the 0.5B model was a quarter of the size:

llama_kv_cache:    Vulkan0 KV buffer size =   192.00 MiB
llama_kv_cache:      CUDA0 KV buffer size =    48.00 MiB

So the 32k split in this post is partly this PC's setting; with the default of 1 the cache would have been a quarter of 14.5 GB, and the model might have fitted. That was not measured. If you set the variable for throughput, remember that it multiplies the context cache of every model you load.

WSL

wsl -d Ubuntu-24.04 -- ollama ps asks the service inside the distribution, which has its own 127.0.0.1:11434 and its own models, not the Windows tray app. The distribution's journal shows the same discovery line with library=CUDA compute=8.6 through the Windows driver, and a run of llama3.1:8b there read:

llama3.1:8b    46e0c10c039e    5.9 GB    100% GPU     4096       24 hours from now
load_tensors: offloaded 33/33 layers to GPU

The 24 hours is that service's own OLLAMA_KEEP_ALIVE, not a default. One observation, not investigated: the WSL service reported 14.9 GiB free while Windows nvidia-smi showed about 8 GB free at the same moment, so its view of VRAM did not include what Windows processes held.

Frequently asked

How do I check if Ollama is using the GPU?

Run a prompt, then ollama ps within five minutes and read the PROCESSOR column: 100% GPU, 100% CPU, or a split such as 51%/49% CPU/GPU. The server log confirms it: an inference compute line with library=CUDA at startup, and load_tensors: offloaded N/N layers to GPU at each load. The API's /api/ps returns the same as size and size_vram.

Why is Ollama using the CPU instead of my GPU?

The three causes reproduced here: a CUDA_VISIBLE_DEVICES or GGML_VK_VISIBLE_DEVICES variable hiding the card (the log says user overrode visible devices), a Docker container started without --gpus all, and a model whose weights plus context cache exceed free VRAM, which gives a split rather than CPU. A driver older than 550, or a card below compute capability 5.0, is the documented fourth.

How much faster is Ollama on a GPU than on a CPU?

On this PC, 6.5 times for qwen2.5:3b (162 against 25 tokens per second) and 4 times for qwen2.5:0.5b; a 14B model split half and half ran 4.5 times slower than fully on the card. A 200-token answer took 1.3 seconds on the GPU and 8 on the CPU. The CPU figures are for an i9-12900HX and will differ on other machines.

The port and OLLAMA_HOST have their own post, Ollama Default Port 11434; the models folder and OLLAMA_MODELS are in Where Ollama Stores Models; the API fields used for the speed figures are in The Ollama API with curl and from .NET; and the Linux service itself is set up in How to Install Ollama on WSL and Linux.