There is something satisfying about a local model. It is right there. It can help with a repetitive job without every little request heading out to another service. It is a good fit for tasks where privacy, predictable cost, or being able to keep working through a connection problem matters.

That does not make it a religion.

Use the right tool for the job

A smaller local model can be great at sorting, extracting, labeling, drafting from a template, or handling a bounded internal helper job. It may not be the best choice for a hard reasoning problem, a fuzzy research question, or a job where getting the answer wrong is expensive.

The mistake is acting like one model has to win every contest. That is how you end up using a screwdriver to drive a fence post because you are proud of the screwdriver.

I still want proof

When I change models, I want the same job run against the same example cases. I want to see what was correct, what was missed, what it cost, how long it took, and what a person had to fix afterward. “It felt smarter” is not a test plan.

  • Use local for the contained jobs it can do well.
  • Use a stronger hosted model when the problem actually needs it.
  • Keep sensitive material out of prompts unless the setup is specifically approved to handle it.
  • Measure the work, not the model marketing.

Running something locally is useful because it gives me another option. The point is not to win a nerd argument about where the model lives. The point is to get reliable work done without being stupid about cost or risk.

Related: what a stronger model helped finish and why memory is not authority.