← back to the archiveCover illustration for “Open-weights model: an AI model you can download and run”
TERM8mo ago·Published · Updated ·by Andy Padia

Open-weights model: an AI model you can download and run

In short: An open-weights model publishes its trained parameters so you can run, fine-tune and host it yourself, which is not the same thing as being open source.

An open-weights model is an AI model whose trained parameters, the weights, are published for download, so anyone can run, fine-tune or host it under the terms of its licence. The Hugging Face Hub, the usual place to find them, listed about 3.25 million models in November 2026. Open weights is narrower than open source, because training data and code are often held back.

Open-weights model in plain words

A recipe-free ready meal you are allowed to take home, reheat and add spices to, but not always to sell, and you do not get the original recipe.

How it works

The developer publishes weight files, a model card and a licence, usually on Hugging Face. You run them with inference servers such as vLLM, llama.cpp or Ollama, on your own GPUs or a cloud provider. Quantised versions let larger models run on smaller hardware. You become responsible for hosting, scaling, safety settings and updates.

Example

ollama run qwen3:8b

As an illustration, a team with strict data-residency rules runs an open-weights model in its own Indian cloud region, so customer documents never leave its environment.

Where you'll meet it

  • Model families such as Llama, Mistral, Qwen, DeepSeek, Gemma and Kimi
  • On-premises and private-cloud AI deployments
  • Local development with Ollama or LM Studio
  • Procurement debates about sovereignty, cost and licensing

What it changes for you

For architects, open weights give control over data, latency, cost at scale and model versions, in exchange for operations work. For legal and procurement, licences matter: some restrict commercial use, user counts or use cases, and some raise jurisdiction questions. For developers, it makes fine-tuning and deep customisation possible.

Open-weights vs open source

Open weights means the trained model is downloadable. Open source asks for more. The Open Source Initiative's Open Source AI Definition 1.0 requires the weights, the complete code used to train and run the system, and sufficiently detailed information about the training data, all under terms that let you use, study, modify and share the system.

Common mistake

My view: assuming self-hosting is cheaper. At low volume, API pricing usually wins once GPUs, engineering time and on-call are counted. Self-host for control or very high volume, and measure cost per task before deciding.

Foundation model · Quantization · Inference server · Small language model · Fine-tuning

From the archive: Add the operating cost to the open-source shortlist

Self-host for control or very high volume, and measure cost per task before you decide.

#field-guide#open-weights#models#infrastructure#procurement

Sources

  1. opensource.org/ai/open-source-ai-definition
  2. huggingface.co/models
← older drop
Agent memory: how AI agents remember across sessions
newer drop →
Golden dataset: the reference cases your AI must pass

related drops

explore all 557 drops →
← back to the archiveday 175