How to Choose the Right Model

The four axes

Why do you care? Because “just use the best model” is the single most common way to burn money in this field — and “just use the cheapest one” is the most common way to ship a feature that almost works. Ten minutes here saves you both.

Every model choice is a balance of four things:

1. Capability — can it do the task at all, and how well?
2. Cost — what does a million requests cost you?
3. Speed — how long does the user wait for the first token? (This is called latency, and what people actually feel is time to first token: how soon the first words appear on screen.)
4. Context size — how much text fits in one request? (Context window: the model’s working memory for a single request, measured in tokens — chunks of text about 3-4 characters long.)

The trap beginners fall into: optimizing one axis and ignoring the rest. The most capable model is often 10x more expensive and 3x slower than one that would do the job 95% as well. For most products, that 5% difference in quality is invisible to users — but the cost and latency differences are very visible to your budget and your UX.

Frontier vs workhorse models

Most providers now split their lineup into two tiers, and the vocabulary is worth knowing:

• Frontier models are the flagships: the smartest, the slowest, the most expensive. Think Claude Opus, GPT-4-class, Gemini’s biggest tier. Use them for hard reasoning, complex code, multi-step planning — tasks where failure is expensive.
• Workhorse models (sometimes called “mini” or “flash”) are smaller, much cheaper, several times faster. Classification (putting things into categories), extraction (pulling structured fields out of text), summarization, simple rewrites, chat — this is where 80% of real production traffic lives.

A useful mental model: frontier models are your senior engineer you call in for the hard problem. Workhorse models are the team that handles the daily queue. You would not pay your senior engineer to sort tickets all day.

Rough numbers to calibrate: workhorse models often cost on the order of tens of cents per million input tokens; frontier models can be 10-30x that. Latency differs in the same direction.

The decision checklist

Run every new feature through these questions, in order:

1. What is the actual task? Summarize? Classify into 5 categories? Answer questions over documents? Write user-facing copy? The narrower the task, the smaller the model you can afford.
2. What does failure cost? A wrong movie recommendation is a shrug. A wrong answer in a medical assistant is a lawsuit. High cost of failure pushes you toward the frontier.
3. How much context does the task need? Count the tokens honestly: system prompt + documents + history + expected answer. If it is 200k tokens, you need a long-context model. If it is 2k, almost anything works.
4. What is the latency budget? A chat interface needs the first token in well under a second of perceived response. A nightly batch job — code that chews through a pile of work overnight with nobody watching a spinner — does not care at all.
5. What is the volume? 100 requests a day: pick anything, the bill will be trivial. 10 million requests a day: the per-token price is now a business decision.

When a small, cheap model is enough

Small models are shockingly good at narrow tasks. Use them when:

• The output is structured: extraction, classification, JSON (a machine-readable text format) of a fixed shape.
• The task is repetitive: the same kind of transformation applied to many inputs.
• You can verify cheaply: a regex check (a pattern match, like “does this look like an order number”), schema validation, or a human in the loop anyway.
• The prompt fits easily in a small context window.

A classic pattern: a small model handles 95% of requests, and escalates the hard, ambiguous cases to a frontier model. This is called a router: a cheap model decides, “can I handle this?”, and only the tricky cases get handed upstairs. It can cut your LLM bill by an order of magnitude while barely affecting quality.

When you should pay for the frontier

• Open-ended reasoning and planning, especially multi-step.
• Code generation where correctness matters.
• Long, subtle documents where a missed detail matters.
• Cold-start: when you do not yet know how hard your task is, prototype with the frontier model first, then try to downgrade — downgrading a working prototype is far easier than upgrading a broken one.

Try it: the decision as a pipeline

Check yourself