From Prompt to Private: A Clinic's Own AI, on Its Own Server

Three seasons built ClinicLive: a clinic books appointments, checks patients in at a kiosk, calls them from a board on the wall, and carries the queue in the patient's pocket. Every one of those features runs on the clinic's own server. This fourth season adds the feature every clinic owner now asks for, an AI assistant, and keeps the same rule: nothing about a patient leaves the building. That rules out every cloud AI on day one. So we install our own, with free open-weight models, on hardware a small clinic can afford, and we measure everything, including the parts that disappoint.

Why the clinic cannot use a cloud AI

The obvious way to add an assistant is an API call to a hosted model. It is also the way this clinic cannot take. Every useful question a receptionist or a patient will ask touches patient data: who is waiting, what did the doctor note last time, is my report ready. Sending that to a third party means consent forms, data-processing agreements, a residency question for every provider, and a conversation with the clinic's owner that ends with "so where does it go?" and no comfortable answer. Some clinics will accept that trade. Ours will not, and it is a common enough position that it deserves a proper build rather than a shrug.

Privacy is the reason that decides it, but two more make the decision easier. Cost: a hosted model bills per token, forever, and a busy front desk asks a lot of small questions; a local model bills once, in hardware, and then costs electricity. Availability: the clinic's internet connection is not the clinic's strongest asset, and an assistant that vanishes when the line drops is worse than none. The thing you give up is real and should be said plainly: the best hosted models are better than anything that fits in 16 GB of video memory. This season is about whether "good enough, and private" is good enough for a clinic's questions. The evaluation in Part 10 answers that with numbers, not adjectives.

What "private AI" actually is

Strip the marketing and there are four boxes, and this season builds one per phase.

• A model file. Open-weight models such as Llama, Qwen and Gemma are published as downloadable weights. Quantized to four bits, a capable one is five to nine gigabytes. That is the whole "AI": a file.
• A runtime that serves it. Ollama loads the file onto the GPU (or the CPU, slowly) and exposes a plain HTTP API on localhost. It is the piece most tutorials stop at. Parts 2 to 4 pick the model, size the machine, and put the runtime on a Linux server properly, which means the port is never reachable from outside.
• Your application talking to it. ClinicLive is a Blazor Server app on .NET 10 with PostgreSQL. Part 5 adds a chat service and a chat panel in the staff pages using Microsoft.Extensions.AI and OllamaSharp, streaming tokens as they arrive.
• Your documents, made searchable. A model knows nothing about Sunrise Family Clinic's fees, hours or cancellation policy. Parts 6 and 7 chunk the clinic's own documents, turn them into vectors with an embedding model, store them in the PostgreSQL the app already runs, using the pgvector extension, and hand the right passages to the model with each question. This is retrieval-augmented generation, and it is where the "medical records" embellishment above gets corrected: the model answers from the passages or says it does not know.

On top of those four, Parts 8 and 9 build the two faces: a staff assistant that can also ask the live queue a question through tool calling, and a patient assistant on the kiosk and in the phone app that sees public documents only. Parts 10 to 12 test it, run it, and take stock.

The first run: ten minutes on a laptop

Before planning anything we ran the cheapest possible experiment, because the numbers decide the plan. The dev machine is a laptop with an RTX 3080 Ti and 16 GB of video memory. Ollama installs with one command on Windows and runs as a background service bound to 127.0.0.1:11434; nothing listens on the network.

winget install Ollama.Ollama
ollama pull llama3.1:8b
ollama pull qwen2.5:14b
ollama pull gemma3:12b
ollama pull nomic-embed-text

Three chat models at the default four-bit quantization, 4.9, 9.0 and 8.1 GB, plus the small embedding model for the knowledge work later. Then five clinic-style prompts to each: a plain explanation, a question answered only from a short clinic information sheet, a question the sheet cannot answer, a request for JSON, and a request for a function call. Medians over the five, on the GPU:

All three fit the card with room for the working memory a conversation needs, and all three write faster than anyone reads. The interesting results were the answers, not the speeds. Asked about a 70-year-old's follow-up visit, where the sheet lists both a senior fee and a follow-up fee and never says which wins, Qwen chose the follow-up fee and explained why, the other two chose the senior fee, and Llama added "since you are a first-time patient" to a question that said the opposite. Asked about an insurance card the sheet never mentions, all three said they did not know, which is the behaviour the patient assistant will depend on. Asked for JSON, Qwen and Gemma returned clean JSON and Llama wrapped it in prose. Asked for a function call, only Qwen produced one a program could execute; the other two passed the names of the arguments instead of values. Part 2 turns this into a proper choice, but the spike already says which model the tool-calling part will lean on.

One more run, with the GPU switched off and four CPU threads, stands in for the cheapest server a clinic might buy: the 8B model generated 9 tokens a second, which is readable, but read the incoming prompt at only 38 tokens a second. That number, not generation, is what decides "minimum" hardware: a question with a page of clinic documents attached waits forty seconds for its first word on that machine and under a second on the GPU. Part 3 measures a real four-core cloud server and turns this into a buy-or-rent table.

Two assistants, one knowledge base

The staff assistant lives inside the staff pages, behind the existing login. It answers from every clinic document, including internal procedures and price lists, and it can ask the application a live question such as who has waited the longest, which is where tool calling earns its place. The patient assistant lives on the kiosk and in the phone app from season three, reaches the server through the same thin API, and sees public documents only: hours, what to bring, fees, reports. Both share one retrieval pipeline and one model; what differs is which documents each may see and how each is allowed to speak. Keeping that distinction in the data, not in the prompt, is the design decision of Part 7.

Architecture, in one breath: the browser or the phone talks to ClinicLive; ClinicLive talks to Ollama on localhost and to PostgreSQL; nothing else talks to Ollama, ever. On the server, nginx fronts ClinicLive as it always has, and the model's port is not in any nginx configuration at all.

The toolchain

Ollama 0.34 as the runtime, on Windows for development and Ubuntu on the server. Llama 3.1 8B, Gemma 3 12B and Qwen 2.5 14B as candidates, nomic-embed-text for embeddings. .NET 10, Blazor Server, PostgreSQL 18 with pgvector, Microsoft.Extensions.AI and OllamaSharp on the application side. Claude Code as the pair programmer, as in every season, with every prompt preserved in the commit messages of the public repo. No API keys anywhere in the stack: the only account this season needs is the one that downloads model files.

What this season does not cover

Fine-tuning: the clinic's knowledge changes weekly, and retrieval handles that where a fine-tuned model would need retraining. Multi-GPU or cluster setups: one card, one clinic. Medical advice: the assistant answers about the clinic, never about a condition, and the patient assistant refuses that category outright by design. Voice: text only. And any hosted model for comparison, because the point is that the clinic never sends a token out; the evaluation in Part 10 compares local models with each other.