Tag
AI Engineer
7 posts.
Write the Dataset Before You Write the Agent
How to Build a Production AI Agent · part 1
Most agents are three boxes and no way to know if they are still right. This series builds the rest. First step: label the dataset before the agent exists.
Your LLM Judge Is Not Neutral. It Has Opinions You Never Asked For.
Agents in Production · part 11
LLM judges have measured biases toward length and confident wrong answers. Check yours against real human labels before it gates anything you actually ship.
A Prompt Edit Is a Deploy. Treat It Like One.
Agents in Production · part 10
A prompt edit changes what your agent does for every user, just like code. Give it review, tests, staged rollout, and a rollback button, before it needs one.
Model Routing Is an Optimisation, Not an Architecture
Agents in Production · part 9
Routing is a cost optimisation, not day-one architecture. Build it once you have production data showing which requests a cheaper model actually handles well.
Your Tool Schema Is a Prompt, Not an API
Agents in Production · part 8
A tool schema is read by the model on every call, not once by an engineer. Design the contract, the errors, and the catalog size, because MCP will not.
The Pipeline That Feeds Your Agent Is Broken
Agents in Production · part 7
Production AI data pipelines have four jobs: extraction, normalization, enrichment, indexing. Most teams build only the last one and ship stale answers.
Most Production RAG Is Quietly Wrong
Agents in Production · part 6
Part 6 of a series on what actually goes into production agentic systems.