Start here
The demo works. The real brief does not.
Most AI agents do not fail because the technology is wrong. They fail because the instructions are vague, the model is misconfigured, or nobody has thought clearly about what the agent is actually being asked to do. Three fundamentals separate a reliable agent from an expensive demo: tokens, prompts, and context.
If you are a Managing Partner, COO, or Delivery Director who watched a vendor demo go well and then saw the same agent fall apart on a real brief, this is why.
The villain
Shipping agents without structure
The AI vendor ecosystem is full of demos that work on clean, simple inputs. Ask a textbook question and the agent performs beautifully. Give it a messy real-world brief — partial data, ambiguous client names, a request for a board-ready format — and it hallucinates, produces an inconsistent result, or quietly decides to do something different each time.
The root cause is almost always the same: the agent shipped without a proper system prompt, without an understanding of how tokens affect cost and quality, and without a clear picture of what the model’s context window actually holds when it runs. These are not advanced problems. They are foundational ones, and they are entirely fixable.
Expert firms should not be losing billable hours to agents that behave differently every time because no one wrote a proper specification.
Tokens
The unit of cost, and the first thing to understand
Large language models do not read text the way a human does. They process it in chunks called tokens — in English, roughly three-quarters of a word, so a fifteen-word sentence consumes around fifteen tokens. Longer prompts mean more tokens; more tokens mean higher costs. Two practical implications follow.
- Model selection changes your cost dramatically. GPT-4.1 charges around $3 per million input tokens; GPT-4.1 mini roughly a third of that; GPT-5 Pro runs to $15 per million input tokens and $120 per million output tokens. Choosing the wrong model for a routine query can mean paying five to ten times more for the same result. The model dropdown is a financial decision, not just a technical one.
- Your actual prompt is rarely just your words. In a recent production test, a seven-word query — “show me the latest orders from client Alfie” — consumed over 113,000 tokens once the agent had pulled its memory, invoked its tools, and assembled the full context. The output was only 340 tokens. That is normal in a properly built agent; the lesson is that cost tracking must account for what the system sends, not what the user types.
You do not need to obsess over exact counts. You do need to understand that longer context, more tools, and richer memory all raise consumption — and that model choice has an outsized effect on the bill.
Prompts
The instructions your agent cannot function without
A fuzzy prompt produces unpredictable results. That is not a quirk of AI; it is a direct consequence of how these models work — they predict the most likely next token given everything in the context. Vague context, vague prediction.
The most reliable structure for professional services use cases is the Role–Action–Outcome framework.
- Role. What should the agent act as? This sets the knowledge base, tone, and expectations the model draws on. “You are a senior data analyst writing for executives” tells the model something very different from no role at all.
- Action. What is the specific task? It must be clear and testable. “Show me the latest orders from client Alfie” is clear. “Help with orders” is not.
- Outcome. What format should the result take? “Return a markdown table” eliminates a whole class of formatting decisions the model would otherwise make arbitrarily.
Bundle those three into a single system prompt and you get fewer tangents and more board-ready answers. This maps directly to the Workflow layer of the KWA framework: before you can automate a task reliably, the workflow needs to be documented and standardised. A system prompt is that written specification. Skipping it is like hiring a junior and providing no onboarding — they might get the right answer occasionally, but you cannot count on it.
In a controlled test, an agent was run twice on the same query, once with a full system prompt and once without. With the prompt it returned a clean markdown table. Without it, it found the same data but presented it inconsistently, made its own decisions about what to include, and could not be relied upon to produce the same format twice. That inconsistency is the real cost — not the wasted tokens, but the wasted senior time reviewing outputs you cannot trust, and the client credibility at stake when a deliverable misses the mark.
Context
Your agent’s working memory
Context is everything the model can see when it generates a response: your system prompt, the current conversation history, any documents loaded into the session, and the results of tool calls already made. It is bounded by the context window — a hard limit on how many tokens the model can hold at once — and larger windows cost more.
Managing context well matters for two reasons: it saves money, and it produces more accurate results by keeping the model working with the right information rather than an overloaded set of inputs. For agents in production, that means being deliberate about what goes in the system prompt, what you retrieve via tools, and how much history you carry forward. A well-designed agent loads precisely what it needs for each task — no more.
Tools
The agent’s connection to your data
Tools are how agents interact with external systems — the equivalent of the software your team already uses: a CRM for client information, a spreadsheet for order data, a calculator for figures. Each tool needs a clear description of what it does, what inputs it expects, and what it returns. Without that, the agent may call the right tool the wrong way, or fail to call it at all.
In a Google Sheets integration, for example, adding explicit instructions to match customers on exact IDs rather than approximate names is what separates a reliable lookup from a hallucinated one. A calculator needs almost no description because its function is self-evident; a tool querying live order history needs precise instructions about date ranges, ID formats, and what to do when a record is not found. Every tool is different, and every tool description needs to reflect that.
In practice
What a production-ready prompt specifies
A full system prompt for a customer support agent handling order queries will typically specify:
- The agent’s role and the business it represents.
- Each available tool, with instructions on how to use it.
- Behavioural rules — always confirm the question before answering, use British English, make one tool call per turn.
- The exact workflow — how to identify a customer, list their orders, surface individual order lines, and handle missing data.
That level of specification is what drives reliable output. It is directly comparable to the documented processes and tool training you would give a new team member — the key difference being that these models do not retain knowledge between sessions by default, so the instructions must be provided every time. Getting this right is the Workflow layer of KWA in action: standardised processes, documented precisely enough that an agent can execute them end to end.
The real question
Three questions before any deployment
If you are evaluating AI agents for client-facing or operational use, three questions are worth asking before you deploy. First, has someone written a proper system prompt — with role, action, and outcome specified — rather than relying on the agent to figure it out? Second, is the model appropriate for the task volume and budget, or was it chosen because it appeared in a demo? Third, is context being managed deliberately, or is the agent accumulating tokens with no clear rationale for what it holds?
None of these require a technical team to answer. They are operational decisions, and they sit firmly within the Activate and Amplify stages of the 5 Steps for AI Leadership: getting the team using AI in daily work, and capturing the practices that work so they spread across the firm rather than staying with one person.
Prompt engineering is not a developer skill. It is a specification skill. The same discipline that makes a good process document or a well-scoped brief makes a good system prompt. The firms that get reliable output from AI agents are the ones that treat prompt design with the same rigour they bring to any operational procedure — because a reliable agent is not a technology achievement, it is a management one.