Start at the bottom
Why firms build before they understand
Most professional services firms jump straight to the tool. They install a chatbot, connect it to a data source, and wonder why it gives wrong answers, costs too much to run, or both. The problem is almost never the technology. It is building before you understand the foundation.
This piece covers the three concepts that sit underneath every AI build: what AI actually is, what a large language model (LLM) does, and what makes something an agent rather than just a chat box. Get these right and every decision that follows — which model to use, how to write a prompt, how much it will cost — becomes considerably easier.
Not all the same thing
AI, machine learning, and what your firm actually calls “AI”
Artificial intelligence has existed as a field for decades. It refers broadly to systems that let computers simulate human reasoning and decision-making. The confusion starts because AI gets used as a catch-all for several very different techniques.
Within AI sits machine learning. It does not operate on explicit rules written by a programmer. Instead, computers are exposed to data and use algorithms to find patterns within it, and those patterns drive decisions. The system learns from examples rather than instructions. Inside machine learning sits deep learning, which uses more advanced neural networks to detect complex patterns. And within deep learning sits generative AI — the technology that powers tools such as ChatGPT.
This distinction matters in practice. When an insurance firm prices a policy, it is using machine learning to predict the probability of a claim — not generative AI. When a CCTV camera reads a number plate, that is AI-powered image recognition — also not generative AI. Knowing the difference stops you applying the wrong solution to the right problem, and stops vendors selling you the wrong one.
Metered in tokens
What an LLM actually does
A large language model is a specific type of generative AI trained on large volumes of plain text. Its job is to predict what comes next: given a sequence of text, it calculates which token is most likely to follow.
A token is the basic unit of processing for an LLM — roughly three-quarters of a typical English word. It can be a word, part of a word, a character, or a punctuation mark. Tokens matter for two reasons:
- They determine cost. Every interaction is metered in tokens — both the text you send in (input tokens) and the text the model generates (output tokens). A simple six-word question to an agent that checks a customer database and refers to conversation memory can consume close to 10,000 tokens. At scale, that adds up quickly.
- They define how much the model can hold in mind at once. Every model has a context window — a limit on how many tokens it can process in a single interaction. Exceed that limit and earlier parts of the conversation effectively disappear.
The practical upshot: a well-written prompt reduces token consumption and improves output quality at the same time.
A well-written prompt is not a soft skill. It is an engineering decision with a direct line to your running costs.
Where the money goes
Model choice is a commercial decision, not a technical one
Not all models are equal in cost or capability, and the gap between them is significant. A model such as GPT-4.5 can cost in the region of $75 per million tokens. Other models with comparable capability for many business tasks cost $40 or less for the same volume. Open-source models hosted on third-party infrastructure can drop that cost to under $1 per million tokens.
The difference is not academic. An agent running continuously — answering queries on your website, processing client data, generating reports — can accumulate hundreds of thousands of tokens per hour. At high-cost model pricing, that becomes a substantial operating expense. At open-source pricing, it may cost almost nothing.
The right model for a given task depends on three factors: the complexity of the task, the volume of interactions, and the latency required. A customer-facing agent that must respond instantly under load has different requirements to an overnight document processing pipeline. Most firms over-specify on model and under-specify on prompt quality. That is where the money goes.
This is exactly the kind of decision the KWA framework is built to surface. Before you reach the Agents layer — where the model choices and cost structures live — you need the Workflow layer documented and the Knowledge layer clean. Deploying an agent on top of messy, undocumented processes does not automate the work. It automates the confusion, at cost.
Actions, not answers
What makes something an agent
A chatbot answers questions. An agent takes actions. The distinction is not about the interface. An agent has access to tools — defined functions it can call to retrieve or write data. It has a memory system so it can refer back to earlier exchanges. And it has a prompt that defines its role, its rules, its available tools, and the workflow it follows when a query arrives.
A practical example: asked “how many customers do we have?”, an agent does not guess. It calls a tool to query a customer list, processes the result, refers to memory for conversation context, and returns an answer with the correct count. That sequence consumed nearly 10,000 tokens. Every step in the chain — the system prompt, the tool call, the memory retrieval, the generated response — contributes to the token count and therefore the cost.
Building an agent well requires four things defined in the prompt before anything else runs:
- Role — what the agent is and what it is responsible for
- Tools — what it can call, and in what order
- Behaviour — how it handles edge cases, tone, format, and escalation
- Workflow — the sequence it follows to process a query from start to finish
The quality of that prompt is the quality of the agent. A generic prompt produces an unpredictable agent. A precise prompt produces a reliable one.
The real villain
Building before you understand the foundations
The AI hype cycle has convinced many firms that the hard part is finding the right tool. It is not. The tools are accessible, the costs are transparent, and the interfaces are straightforward. What is genuinely hard — and what most vendors skip entirely — is the groundwork.
Firms that move through the 5 Steps for AI Leadership in sequence do not run into the problems above. They Align on what AI is actually for before touching a platform. They Activate their team on basic tools before building custom agents. They Amplify what works before scaling it. They Accelerate the workflows where the return is clear. They Govern access, data handling, and costs before anything goes near a client.
Firms that skip directly to step four — and most do — end up with agents that cost too much, answer unreliably, and get quietly switched off. The technology was never the problem.