← back to the archiveCover illustration for “Foundation model: one big model, adapted to many tasks”
TERM7mo ago·Published · Updated ·by Andy Padia

Foundation model: one big model, adapted to many tasks

In short: A foundation model is a large AI model trained on broad data and then adapted to many tasks, from chat to code to images, by prompting, fine-tuning or tools.

A foundation model is a large AI model trained on broad data at scale that can then be adapted, through prompting, fine-tuning or tool use, to a wide range of tasks. The term comes from a Stanford paper submitted in August 2021, whose examples were BERT, DALL-E and GPT-3. Large language models are the best-known type, and the EU AI Act regulates the closely related category of general-purpose AI models.

Foundation model in plain words

A broadly educated graduate rather than a specialist trained for one job. With a short briefing, they can do many different jobs reasonably well.

How it works

The model is pre-trained once, at great expense, on general data. Many products then build on it without retraining from scratch: some through prompts and retrieval, some by fine-tuning a copy, some by distilling it into a smaller model.

What flows downstream is both good and bad. Improvements in the base model reach everything built on it, and the Stanford paper warns that its defects are inherited by all the adapted models too.

Example

As an illustration, one foundation model can power a customer-support bot through a system prompt and RAG, a contract classifier through fine-tuning, and a coding assistant through an agent harness, all without training a new model from zero.

Where you'll meet it

  • Model families from OpenAI, Anthropic, Google, Meta, Mistral, DeepSeek, Alibaba and others
  • Model catalogues on the large cloud platforms
  • Regulation and procurement documents
  • Open-weights hubs such as Hugging Face

What it changes for you

For product teams, the question changes from "should we train a model?" to "which foundation model, and how do we adapt it?". For architects, dependence on a few model providers becomes a supply-chain and portability concern. For buyers in the EU, the European Commission's questions and answers on general-purpose AI models say provider obligations, including technical documentation for downstream providers and a summary of training content, apply from 2 August 2025. Ask your provider for that documentation.

Foundation model vs fine-tuned model

The foundation model is the general base. A fine-tuned model is a copy adapted to a narrower task, style or domain.

Common mistake

My view: hard-wiring one provider's model throughout a product. Model versions are replaced and retired on the provider's schedule. Keep a thin abstraction and your own evals, so switching is a measured decision rather than a rewrite.

Large language model · Fine-tuning · Open-weights model · Distillation · EU AI Act

From the archive: How to choose an LLM for your actual workload

Keep a thin abstraction and your own evals, so changing models is a measured decision.

#field-guide#models#fundamentals#llm

Sources

  1. arxiv.org/abs/2108.07258
  2. digital-strategy.ec.europa.eu/en/faqs/general-purpose-ai-models-ai-act-questions-answers
← older drop
Distillation: teaching a small model from a big one
newer drop →
Plan mode: let the coding agent think before it edits

related drops

explore all 557 drops →
← back to the archiveday 175