Steve Mitchell – Page 2 – Steve's AI Diaries

The Next Shadow IT Isn’t Software. It’s Agents.

June 13, 2026 by Steve Mitchell

If you’ve been in technology long enough, you remember the rise of Shadow IT.

It never started with rebellion. Nobody woke up and decided to undermine corporate governance. A department needed something. IT couldn’t move fast enough. Someone built a spreadsheet. Someone else signed up for a SaaS platform on a company card. A manager ran an Access database for three years before anyone in the centre knew it existed.

Each decision made sense in isolation. The problem only became visible when you zoomed out. Suddenly nobody knew what systems existed, who owned them, what data they held, or what would happen if they disappeared tomorrow. The organisation hadn’t intentionally designed an architecture. It had accidentally accumulated one.

We are about to do exactly the same thing with AI agents.

A few weeks ago I wrote about a question a colleague asked while watching me work: “How do you know what your agents are going to build?” That post was about the missing specification layer between intent and implementation.

This is the follow-on question. What happens when those agents leave development and enter production? What happens when there are hundreds of them?

I sit on an AI committee at work. I see the race for agentic capability from the inside. The energy is real and the pressure is genuine — teams across every function want agents, want them now, and are building them faster than any central function can review. That’s not a complaint. The capability is real and the business cases are solid.

But I’ve noticed something. The teams that will win this race aren’t the ones deploying the most agents. They’re the ones who can keep deploying because they built the operating model before they needed it. Everyone else will hit a wall: a question they can’t answer, an audit they can’t pass, a failure they can’t explain. And at that point the agents stop until someone builds the governance they skipped.

Agent sprawl starts with success

The dangerous thing about agents is that useful ones are easy to justify.

A claims team deploys one to triage work. Customer service deploys one to prepare responses. Engineering builds one to review pull requests. Finance builds one to reconcile reports. Each has a business case. Each saves time. Each makes someone’s life easier.

That is exactly why they spread.

Bad agents die quickly. Useful agents multiply.

Before anyone notices, the organisation hasn’t deployed an agent. It has deployed an estate.

One agent is a use case. Ten agents are a portfolio. Hundreds of agents scattered across business units, vendor platforms, and local scripts are an estate. Estates do not run on vibes. They need mechanisms.

The question nobody can answer

I run a personal AI infrastructure: Jarvis on a Raspberry Pi, Hermes on a Hetzner server, monitoring agents watching both. Even at that scale I’ve had to make deliberate decisions about agent identity, access scope, and what happens when something breaks. I decommissioned one entire runtime when I couldn’t confidently answer basic questions about what it was doing. Painful call. Right call.

Now multiply that by a department operating in a regulated industry.

Six months after a workflow has been running, an auditor asks: which agent prepared this recommendation? What data did it use? What did it ignore? What policy applied, and what did the reviewer actually see before approving the output?

And the room goes quiet.

Not because anything went wrong. Because nobody designed the system to remember. That is the failure mode that keeps enterprise architects awake at night. Not rogue AI. Missing evidence.

The harness tools solve the wrong problem

When teams do think about governance, they reach for observability tools. LangSmith. Langfuse. Tracing integrations inside LangChain. These are genuinely useful. They tell you what happened: which tools the agent called, what the prompt looked like, where it failed, how long it took.

But observability tells you what happened. It does not tell you whether it should have happened.

Those are different questions. Logging that an agent accessed a production database is observability. Preventing that agent from accessing the database unless it has been explicitly granted permission, in that environment, at that stage of its lifecycle, by someone with authority to grant it: that is governance. No harness tool does the second thing.

The result is organisations with excellent visibility into what their agents are doing and no mechanism for controlling whether they should be doing it. The dashboard is green. Whether that means the right things are happening is a different question entirely.

Three surfaces, not one

Most governance conversations stop at agents. That is too narrow.

Agents are the obvious starting point, but an agent without version control is a liability. If the one running today behaves differently from the one running last month because someone changed the system prompt, and you cannot reconstruct what the original was doing, you do not have a production system. You have a guess with a nice interface.

Agents need what software has had for decades: source control, environments, promotion gates, and the ability to roll back. An actual lifecycle: draft, test, staging, production, monitoring, and retirement. Including retirement. An agent with no active owner, no current use case, and persistent access to production systems is a risk sitting quietly in your infrastructure.

Skills are the discrete capabilities agents call on: the function that searches your knowledge base, the one that classifies an intent, the one that drafts a response. They’re often shared across agents, and that is where it gets interesting. If a shared skill changes, every agent using it changes. If it has a bug, every agent inherits it. Skills need versioning, ownership, and controlled promotion. They are code. Treat them that way.

Tools are the connections to real systems: databases, APIs, CRMs, payment platforms. A tool is where an agent stops reading and starts acting. Tool access needs to be explicit, scoped, and auditable. Not “the agent can access the claims database.” Which agent? Which environment? Which scope? Granted by whom, reviewed when?

Capability is not permission. Confidence is not clearance. The level of control required scales with how close a tool gets to systems of record. An agent answering from approved documentation is one risk profile. An agent executing transactions is another entirely.

Agents need an SDLC

No engineering team ships code to production without source control, a review process, environment gates, and a way to audit what changed. The code running your business has owners. It has history. It has a path from idea to production that somebody can reconstruct.

Your agents are code. Your skills are code. Your tool integrations are code.

The argument against treating them that way is speed: “We’ll add governance once we’ve proven the value.” That logic holds until something unexpected happens in production and nobody can explain why. At that point governance stops being optional. It becomes the difference between being able to answer a question and not.

Before the next agent goes live: who owns it, what environment does it run in, what can it access, what lifecycle stage is it in, and how was it approved for production? If those five questions don’t have answers, the agent might be useful. It is not ready for production.

What a real control plane covers

A genuine control plane is not a dashboard. It is the layer that makes basic questions answerable.

What agents exist, who owns each one, where they run, what model they use, what workflow they support. What skills those agents draw on. What tools they can call and in which environments. Who approved each agent for production and when. What changed between versions. Which agents are retired and why.

Beyond inventory, it handles three things that should never blur together: what the agent can see, what it can do, and what it is permitted to decide without human approval. An agent may need broad context to prepare useful work and still have no authority to act on anything without sign-off. Letting those boundaries drift is how workflows accumulate permissions nobody intended to grant.

It also handles human review properly. A human in the loop is not governance if the reviewer sees a polished summary and a green button. Real oversight means seeing the sources, the proposed action, the downstream impact, and having a genuine path to reject or escalate. Without those, human approval is theater with better UX.

And it handles traceability: not just logs, but the reconstructible path showing which sources the agent used, which tools it called, what the human approved, and what changed downstream. Autonomy without traceability is operational debt with better marketing.

The window is still open

Most organisations are somewhere in the middle of this right now. Agents are live, useful, and multiplying. The governance conversation is either not happening or stalling because someone thinks it will slow things down.

That window will not stay open. As agents move closer to systems of record, the cost of not having a control plane increases. Vendors will mature. Regulators will catch up. And internally, the teams that built governance early will be the ones with room to keep moving. They can expand autonomy because they can verify it is working. They can answer the audit question because they designed for it. They stay in the race.

The ones that didn’t will be retrofitting governance onto a live estate, agent by agent, skill by skill, tool by tool, while trying not to break anything people now depend on. That is a slow, expensive way to fall behind.

We have seen this before. We called it Shadow IT. We spent years cleaning it up.

The estate is already forming. The question is whether you will be able to govern it when it matters.

I run Jarvis and Hermes, a personal AI infrastructure across a Raspberry Pi and Hetzner, as a way to stay close to how these systems actually behave. Most of what I write here comes from things I’ve had to figure out the hard way.

How Do You Know What Your Agents are Going to Build?

May 9, 2026 by Steve Mitchell

A couple of weeks ago I was in our Chicago office, working with a group of colleagues as we looked to leverage our agentic framework on one of our premier products. The goal of the week was a thought experiment: using our agentic framework, could we leverage AI to completely modernise the tech stack. Ordinarily this would be 12 months of work. We had 5 days.

During the very first session, my colleague Jim Pucci was going through a PRD, a project requirements document I use to drive agentic development through FADE, our framework for keeping agentic development on the rails across long sessions. The PRD was solid. Requirements were clear. The agent was about to start building.

Jim read it, looked up, and asked:

“These are the requirements. How do you know what you’re going to build?”

I didn’t have a clean answer. And the more I sat with that question, the more I realised it had exposed a gap in my own framework.

What FADE actually does

FADE injects context. It locks in the frame the agent works inside: vision, constraints, coding standards, architectural principles, the things you’ve learned from previous sessions. It’s why I can hand an agentic developer a complex task at 5pm on a Monday afternoon and have it work all night so we can pick up again on Tuesday morning to working software to review, built to standard.

FADE.md sets the frame. progress.md tracks state. learned.md captures memory across sessions. It works. I’ve shipped real things with it.

But Jim’s question was about something FADE doesn’t do. The frame tells the agent the world it’s building in. The PRD tells it what the business wants and why. Neither tells you, before a line of code is written, what is actually going to be built.

That decision: the architecture, the structure, the patterns, the schemas, the contracts between modules. It’s happening at runtime, inside the reasoning engine, on the fly. Two FADE runs against the same PRD, with the same frame, can produce materially different builds. You only find out which one you got after the fact.

The role we forgot

In a normal engineering team, this isn’t how it works. The PRD is a contract between the business and the architect. It says what the business wants. The architect’s job is to translate that into a spec: how it will be built, what the structural commitments are, which patterns will be used. That spec is then the contract between the architect and the developer. The developer implements the spec faithfully, not re-derives the architecture from requirements every time.

We’ve had this model for decades. Classical software engineering with the roles named honestly.

What’s new is that the agent has been quietly doing the architect’s job inside the reasoning engine, without anyone signing off on it. FADE gave it enough frame to do that job reasonably well. That’s why it works at all. But it never made the architect’s output a visible artifact. The build decisions are real. They have consequences. Right now they’re invisible until they’re already code.

Jim’s question lands because in human teams, the architect’s work is written down and reviewed before the build starts. With agents, we skipped that step and called it productivity.

What plan mode gets right, and where it stops

GitHub Copilot and Cursor have plan modes now. Before the agent touches your code, it tells you what it’s going to do. You can push back. That’s a step forward.

But it’s still one developer, in one session, reviewing what one agent is about to do. The plan isn’t a document. The architect doesn’t see it before the work starts. Security doesn’t sign off on the auth model. QA finds out what was built when there’s something to test. One person saw the decisions. Then they became code.

For a solo project, fine. For a team in financial services, insurance, or healthcare, that’s a problem.

These industries have governance processes and audit requirements for real reasons. When something goes wrong, the question isn’t “what did you build.” It’s who reviewed the approach, when, and what they approved. A plan mode session doesn’t produce that evidence. A private FADE.md doesn’t either.

The gap isn’t developer visibility. Developers have that. The gap is that there’s no written spec, reviewed and agreed before the build starts, that the whole team can sign off on. Not just the developer. The architect, security, QA — the people whose names go on the approval. In a regulated environment, where clients need demonstrable evidence that what you built is secure, testable, supportable, and built to scale, “the agent had a plan mode” isn’t an answer anyone will accept.

What the spec layer should look like

If FADE’s evolution is a spec layer between PRD and build, a few things follow.

The spec is owned by the architect role. That might be a human, or agent-assisted, or the agent drafting while a human signs off. The form doesn’t matter much. Somebody has to be the architect. Accountability sits there. The agent can propose; the architect signs.

The spec generator isn’t a blank-page exercise. It’s grounded in three inputs:

The PRD, for intent.
The standards, for constraints. The FADE frame already covers most of this.
The existing codebase, for context. What patterns are already in use. What the auth model is. Where the seams are. What the team has already committed to.

A spec produced against the codebase fits the codebase. It can’t propose an architecture that contradicts what already exists without that contradiction being visible. The architect reviewing it sees “this reuses the existing pattern” or “this proposes a new one” as an explicit decision, not a runtime accident.

That makes the spec a negotiation surface. The agent proposes, the architect pushes back. “You’ve reused the auth module, good, but you’ve bypassed the rate limiter, fix it.” That conversation is worth having before the code exists, not after.

The bar I’d want to hold it to

Here’s a test for whether the spec layer is doing its job: a different agent, or the same agent in a fresh session, should be able to pick up the spec and produce substantially the same build.

If they can, the spec is committing the architectural decisions properly. If they can’t, the spec is too thin and too much is still being decided at runtime.

That’s a falsifiable bar. You can actually tell whether you’re winning.

What I haven’t worked out

I’m not going to pretend I’ve solved this. There’s a stack of open questions:

Granularity. Is the spec one document per PRD, or per feature, or per module? Probably depends on size, but I haven’t drawn the line yet.
Sync. A spec drifts from the PRD on one side and from the code on the other. How do you keep all three honest as the build evolves?
Codebase reading. This is the hard bit. A spec generator is only as good as its ability to actually understand the codebase it’s reading. Most agents read code shallowly: grep for keywords, miss the actual structure. A spec generator that reads badly will produce specs that look grounded but aren’t, which is worse than no spec at all because it hides the architecture decisions behind a veneer of rigour.

That last one is the real engineering challenge. It’s the next post.

The principle

Agents can be developers. Agents can assist architects. But somebody has to be the architect. Their output has to be visible.

A PRD tells you what the business wants. It doesn’t tell you what you’re going to build. Until that decision is written down and signed off, you haven’t authorised anything. You’ve just hoped the reasoning engine picks well.

FADE gave me the frame. Jim gave me the question. The spec layer is what comes next.

Token Management Is the Number One Skill You Need to Learn Right Now

April 26, 2026April 23, 2026 by Steve Mitchell

The flat-rate era is over.

Anthropic quietly killed bundled enterprise tokens in February. Your seat fee used to cover a token allowance. Now it covers platform access. Tokens are billed separately at API rates, on top of whatever you’re paying per user. Their own help centre confirms it; The Register broke the full story. OpenAI followed in April, officially announcing that Codex was moving from per-message credits to token-based metering. Two of the biggest AI providers, both moving in the same direction, within two months of each other.

This is not a pricing footnote. It is a structural change in what it costs to build with AI.

If you are running AI agents, integrating LLMs into a product, or just using Claude heavily at work, token efficiency is now a direct financial skill. Not a nice-to-have. Every bloated prompt, every unnecessary tool call, every context window you failed to trim hits your bill. The people who learn this now have a compounding cost advantage over everyone who doesn’t.

Here is what I have learned building and running six production AI agents.

First, understand what just changed

Anthropic’s Opus 4.7, released last week, shipped with a new tokenizer. The rate card is unchanged. The real cost is not.

The new tokenizer produces up to 35% more tokens for the same input text. Your prompt costs 35% more to run on Opus 4.7 than it did on Opus 4.6, at the same price-per-token. If you benchmarked your costs on the old model and assumed they’d carry over, they won’t. Test your actual workloads.

This is the pattern to watch: providers change tokenizers, context pricing brackets, and billing structures without changing headline rates. The number on the pricing page stays the same. Your bill does not.

The tips

1. Prompt caching is the single biggest lever

Both Anthropic and OpenAI offer cache-based pricing. Anthropic’s prompt cache cuts cached input token costs by 90%. If you have a system prompt, reference documents, or long context that stays the same across requests, cache it. One setup. Ninety percent reduction on every subsequent call that hits the cache.

Most people using the API are not using this. It is the highest-ROI change you can make.

The rule: anything that appears in every request should be cached. System prompts, persona instructions, knowledge base chunks, code files you’re asking the model to reason about. The cache TTL on Anthropic is five minutes. Build your calls to stay warm.

Structure matters for cache hits. Both OpenAI and Anthropic cache from the start of the prompt forward. Put fixed content first: system instructions, tool schemas, reference documents. Put the changing user-specific content last. A prompt that has dynamic content in the middle breaks the cache for everything that follows it.

2. Right-size the model for each task

Opus costs five times more than Haiku at input, and five times more at output. Claude Sonnet sits between them.

Haiku is fast and cheap. It is entirely capable of routing, classification, summarisation, simple extraction, and structured output generation. Routing an agent decision through Haiku to determine whether a task needs Opus or can be handled locally is not premature optimisation. It is cost architecture.

The mistake is using the most capable model for everything because it feels safer. A planner that decides whether to fetch a file does not need Opus. A model writing a novel does. Know the difference.

I covered the full case for multi-model workflows in Not All AI Is Equal — Stop Pretending It Is — the benchmarks and the practical routing logic are there if you want the detail.

3. Use the Batch API for anything that isn’t real-time

Anthropic’s Message Batches API runs requests asynchronously and returns results within 24 hours at exactly 50% off standard token prices. OpenAI has an equivalent.

If you are running nightly analytics, weekly report generation, bulk data enrichment, or any processing where a human is not waiting on the response, there is no reason to pay full price. Half-price tokens, same quality, same models. The only cost is latency.

I use this for Paper Ritual’s weekly analytics runs. The agent processes a batch of Etsy performance data overnight. The report lands in Telegram by morning. The tokens cost half what they would in real-time mode.

4. Know your context breakpoints

GPT-5.4 introduced a short/long context pricing split. Below the threshold, input tokens cost $2.50 per million. Above it, $5.00. Same model, same output quality, double the input price once you cross the line.

Anthropic’s pricing is currently flat across context sizes, but the pattern is worth knowing. Before assuming a long context call costs the same as a short one, check the current pricing page for the model you are using. Tokenizer changes and pricing bracket changes happen without fanfare.

5. Trim your context window actively

The default behaviour of most LLM frameworks is to pass the entire conversation history on every request. That is fine for short conversations. For agents that run for multiple turns, it is a quiet cost multiplier.

Every input token costs money. Context from turn 1 that is no longer relevant to what the agent is doing now should not be in the prompt at turn 20. The fix: summarise and compress. After a defined number of turns, distil earlier context into a summary and drop the raw messages. The model still has the relevant history. You stop paying for redundant tokens.

In ZeroClaw, Anthropic’s agentic runtime, this is handled automatically above a threshold. If you are rolling your own agent loop, build this in from the start.

6. Control output length deliberately

Output tokens are priced higher than input tokens. On Claude Opus 4.6, input costs $5.00 per million tokens and output costs $25.00.

Tell the model how long its response should be. Set max_tokens in your API call. Use stop sequences when you only need a specific field or a yes/no answer. Ask for a two-sentence summary rather than a full analysis when a full analysis is not what you need.

A model that naturally writes long responses will write long responses unless you tell it not to. Every sentence you didn’t need costs five times more than a sentence of input.

Structured outputs take this further. Asking the model to respond in JSON with a fixed schema, or to use a bullet list instead of prose, constrains how much it can say. Open-ended prose invites padding. A schema does not. Use the structured output parameter in your API call where the task allows it.

7. Put verbose instructions in cached system prompts, not per-request

If you are passing “you are an expert assistant, think step by step, respond in JSON with the following schema…” as part of every user message, you are paying full price for those tokens on every call. Put all persistent instructions in the system prompt and cache it. They cost 90% less on every subsequent request.

This also includes any in-context examples you pass to guide output format. One cache. Permanent discount.

8. Turn down reasoning effort on routine tasks

OpenAI’s reasoning models expose a reasoning.effort parameter. Anthropic’s extended thinking has an equivalent effort control. Both let you dial how much internal reasoning the model runs before answering.

High effort is appropriate when the task is genuinely hard: multi-step planning, complex code generation, tasks where quality visibly improves with more thought. It is not appropriate for extraction, classification, rewriting, or summarisation. Those tasks do not benefit from extended reasoning and you are paying for tokens the model spent thinking, not just tokens in the final response.

Set effort to low by default. Raise it selectively when you have evidence the task needs it.

One thing to watch on OpenAI: reasoning tokens consume context window and budget even when they are not shown in the final answer. If you are watching output tokens and the numbers seem high, check whether reasoning is running in the background.

9. Break complex tasks into stages

One giant prompt that asks the model to extract, reason, transform, and generate all at once is usually more expensive than breaking that work into smaller sequential steps. Each stage operates on only the context it needs. None of them carry the dead weight of the others.

The counterintuitive result: more API calls often means lower total cost. A pipeline that extracts structured data cheaply with Haiku, then passes only that structured result to Sonnet for reasoning, costs less than asking Opus to do everything from raw input in a single call.

Design your pipelines as pipelines. Not as monolithic prompts.

10. Combine tool calls where you can

In an agent loop, every tool call consumes input tokens (the tool call request), output tokens (the tool call content), and then more input tokens when the result is passed back to the model as context.

Agents that make many small, sequential tool calls can accumulate significant token overhead from the scaffolding alone. Where you can, batch operations into single calls. Fetch and summarise in one step rather than two. Retrieve and filter before passing to the model rather than passing raw and asking the model to filter.

This is harder to retrofit than to design in from the start. Think about it early.

11. Test your prompts against the actual tokenizer

Different models tokenize differently. The Opus 4.7 tokenizer change is the most recent example, but tokenizer differences between models have always existed. A prompt that costs X tokens on one model does not necessarily cost X tokens on another.

OpenAI has an official tokenizer at platform.openai.com/tokenizer — paste your prompt and see exactly how it breaks down. Anthropic doesn’t have a first-party equivalent; their token counting is API-based, but claudetokenizer.com is a third-party tool that uses the official API and gives you accurate counts across Claude models. Before optimising, measure. The gains you think you’re getting from shorter prompts may not be what you expect if you haven’t checked what the tokenizer actually does with your text.

12. Build cost visibility into your stack from day one

You cannot manage what you cannot see.

In my agent stack — which I wrote about in A Day in the Life of an Agent — every agent reports token consumption to Prometheus via Pushgateway. I can see which agent is burning the most tokens, which tasks are expensive, and whether a prompt change actually reduced costs or just shifted them. The observability is not optional: it’s how I know whether an optimisation worked.

At minimum, log input and output token counts per request. Aggregate by agent, by task type, and by model. Surface the top ten most expensive operations. You will find the waste quickly once it is visible.

The compounding problem

Agents make this worse than standard API usage.

A user sending a single query to a chatbot makes one API call. An agent completing a complex task might make twenty. Paper Ritual — an autonomous Etsy business running on a Raspberry Pi — makes dozens of API calls per daily run: research, pricing decisions, listing generation, analytics. Each tool call, each planning step, each verification loop is a separate API call with its own token cost. Inefficiency that costs $0.01 per user query costs $0.20 per agent task. At scale, that gap is the difference between a viable product and a product that bleeds money.

Token efficiency matters most in agentic systems. That is exactly where most people are not thinking about it yet.

The other agent-specific failure mode: runaway loops. An agent that retries, re-reads context, or gets stuck in a reasoning loop can burn through token budgets in minutes. Hard-cap your iteration count. Add explicit stopping conditions before the agent starts, not as an afterthought. Log token usage per step so you can see where a task went expensive. Agents don’t fail because they’re unintelligent. They often fail because nobody put a ceiling on how much thinking they were allowed to do.

The shift is permanent

The pricing shift from flat-rate to usage-based is not temporary. Both Anthropic and OpenAI have moved in the same direction. Every AI provider will follow, because subsidised flat-rate AI usage is not sustainable at the token volumes that real production workloads generate.

The developers who learn token management now will build cheaper, faster, and with more headroom than those who learn it later when the bill is already large.

Start with prompt caching. It takes one afternoon and the cost reduction is immediate.

Token prices correct as of April 2026. Check the current pricing pages before optimising for specific numbers. They change.