LLMs, Coding Assistants, and Tokens

Three ideas carry this whole series: what a language model does, what the assistant around it adds, and why text volume sets time and cost.

What a language model does

A large language model is trained on large collections of text. Given your text, it predicts a likely continuation.

It doesn’t think like a person, and it doesn’t look things up by itself. Alone it takes text in and returns text out — it can’t read your files, run tests, or ship anything without software around it.

What the assistant adds

That software is the coding assistant: the program around the model that supplies tools, instructions, and a working method. In a project it lets the model read files, edit them, and run checks.

The ones I use: Oh My Pi (OMP) on Pi, OpenCode, Claude Code, and Codex CLI.

An assistant handles four jobs the model can’t do alone:

1. Tools: reading a file or running a command when the model needs it.
2. Context: choosing which files and history to send.
3. Instructions: supplying standing rules, like the ones in AGENTS.md.
4. Errors: returning a failure so the model can try a fix.

Tokens and cost

Every call is measured in tokens — small chunks of text, roughly a few characters each in English. You pay in both directions:

Input (what the model reads)

Your prompt plus history, instructions, and attached files. Re-sending a big file every turn adds up, so send only what the current task needs.

Output (what the model writes)

The text it generates. A long answer costs more than a short one.

Context window limits

Each model caps total input-plus-output held at once. Past that ceiling, older text drops out of view. Being selective isn’t tidiness — it’s what keeps long sessions working.

How instructions steer output

Two layers:

• System prompt: background rules from the app or assistant.
• Your prompt: the task, plus limits on shape and tone.

Specific beats vague. “Five sentences, friendly tone” gives the model a target; “make this better” doesn’t.

With that model in mind, the next question is where a model belongs in your build — and where ordinary code does the job better.