Every team adding an assistant to an application faces the same fork: call a hosted frontier model over an API, or run an open-weight model on your own hardware. Both are good engineering. They win on different questions, and the questions are about your data, your users and your bill, not about which model is smarter. Here is the decision as the ClinicLive season made it, with the numbers that decided it.
Start with the data, because it decides first
If the questions your assistant answers touch data you are not allowed to send to a third party, patient records, employee files, unreleased financials, legal matters, the decision is made before any benchmark: local, or no assistant. Consent and data-processing agreements can make a hosted model lawful; they do not make it comfortable for a clinic owner who is asked "so where does it go?". If the data is public, or already lives with a cloud provider, this consideration disappears and the choice becomes cost and quality.
Quality: the honest gap
The best hosted models are better than anything that fits in 16 GB of video memory. On open-ended reasoning, long documents, code and unusual languages, the gap is wide. On grounded question answering over your own documents, the kind of work Part 7 built, it is narrow: a 14-billion-parameter model that is handed the right paragraph and told to cite it answers as well as a frontier model does on the same paragraph, and refuses as well when told how. The evaluation in Part 10 puts a number on that for the clinic's questions. The rule of thumb: the more your assistant relies on retrieval and the less on the model's own knowledge, the smaller the quality gap.
Cost: two different shapes
A hosted model bills per token, forever, in proportion to use. A local model bills once in hardware and then in electricity, regardless of use. Part 3 measured the local side: about 35 W idle, about 120 W above idle while answering. A busy clinic day of five hundred questions is under a tenth of a kilowatt-hour of answering; the idle draw over a month is the larger number, roughly 25 kWh. Against that, five hundred grounded questions a day at around a thousand tokens each is fifteen million tokens a month, which at current hosted prices for a mid-tier model is real money every month and grows with every new user. The crossover for a small team is measured in months, and the used-GPU market moves it earlier.
Availability and latency
Local answers do not depend on the internet connection and do not queue behind other customers. Part 3 showed four simultaneous users on one 16 GB card with each seeing 26 tokens a second, and a first word inside three seconds. A hosted API is faster per token and slower to first byte across a wide-area link, and it is down when the link is down. For a front desk, the link is the weaker part.
Operations: what you take on
With a hosted model you operate a key. With a local model you operate a server: a runtime to update, models to swap, a GPU to keep cool, a backup for the knowledge base, logs to read. Part 11 shows that this is a few hours a quarter for a clinic-sized deployment, not a job, but it is not nothing, and somebody has to be that person.
The decision table
| If this is true | Choose |
|---|---|
| The data cannot leave the building | Local. Nothing else matters. |
| Answers come mostly from your own documents | Local is good enough; the gap is small. |
| Answers need broad world knowledge or long reasoning | Hosted, or a much bigger local card. |
| Usage is steady, all day, many users | Local pays back fastest. |
| Usage is rare or bursty, few users | Hosted; a warm GPU idling is waste. |
| Nobody can own a server | Hosted, with a data agreement, or no assistant. |
| The connection is unreliable | Local. |
The clinic's answer
Patient data, grounded questions, steady daytime use, a flaky connection, and an owner who can point at the box: every row says local. That is why season four exists. A design agency asking an assistant to draft proposals from public research would read the same table and choose hosted, and would be right.
Related: what server a clinic needs, and the other decision post in this style, Blazor Server versus WebAssembly versus Hybrid.