Choosing a model

A kind of input or output is called a modality. Text, images, audio and video are modalities. A multimodal model handles more than one.

Check both directions. Reading images and making images are different skills. They often come from different models. For photos of receipts, you need a model that accepts image input.

Media is not free. Images and audio are turned into tokens, too. One image can cost as much as a page of text, or more. It depends on the provider and the image size. Sometimes a chain of models is cheaper or better. For example: speech-to-text, then a text model, then text-to-speech.

This is the built-in version of "let it think first" from Prompts that work. The thinking is made of ordinary tokens. Some apps show it, some show a summary, and some hide it. In every case, it is billed as output tokens. It also takes up room in the context window.

The setting has different names, like effort, reasoning effort or thinking level. The values are often low, medium and high. More effort usually means slower answers and more tokens. It is a guide, not an exact limit. Even on high effort, a model thinks less about an easy question.

Thinking is worth it when a careful person would need pen and paper: math, logic, finding bugs, weighing evidence. It is a waste for looking up a fact or brainstorming names.

Time to first token includes several things: the network, any queue at the provider, and the model reading your input. Then comes any thinking it does first.

Here is an example. The first token comes after 0.5 seconds. Then 400 tokens arrive at 80 tokens per second, which takes 5 seconds. So the total is 0.5 + 5 = 5.5 seconds.

Many teams route requests. Every request goes to a small, fast model first. Only the hard ones go up to a larger or thinking model. The router can be a simple rule or a small model. It can also be a model that only picks from a list.

Priya's team looked at a week of real questions. Most were simple, like "Where is the leave form?" These moved to a small model with thinking off. Answers started in under a second, and the cost per question fell a lot. A few hard questions went to the large model on high effort. There, waiting twenty seconds was fine.

Before you choose, check a few more things. How big is the context window? Can it use tools? What is the price per million tokens? Where does your data go (Safe AI at work)?

Here are some model families that come in sizes, as of October 2026. Claude has Fable, Opus, Sonnet and Haiku. GPT comes in several levels too, from a top model to a cheaper, faster one. Gemini has a top model, Flash and Flash-Lite. Open-weight families include Llama, Qwen, Mistral, Gemma and gpt-oss. Closed providers usually do not say how big their models are. So the name of the size is your guide. Names change every few months. The trade-offs stay the same.