The Trends of Engineering in AI: Prompt, Context, Harness, Loop, and Evaluation
What's the difference between prompt engineering, context engineering, harness engineering, loop engineering, and evaluation engineering? A side-by-side comparison with diagrams for AI engineers building production systems. One night my AI agent Hermes, which I run 24/7 on my own hardware, spent 47 turns trying to “fix” a script. Every turn it ran the same broken command, got the same error, apologized and tried again. I sat there watching the token counter climb like a Tatkal queue at 10 AM, and I genuinely could not tell what had gone wrong. Was the prompt bad? Was it missing context? Was a tool broken? Was the loop just never going to stop? I work in AI observability, and I was staring at my own agent unable to name which layer had failed. That night is what this post is about. A production AI agent is not one skill, it is five, and when something breaks at 2 AM you need to know exactly which one to blame. Everyone talks about prompt engineering. Fewer people talk about context engineering. Almost nobody talks about harness, loop or evaluation engineering, even though they’re the difference between a demo that works once and a system that survives a night alone on my server. An AI observability architect who doesn’t understand the full stack of agent engineering is just a person who reads dashboards. So I sat down and split it into the five disciplines that actually make up a production agent: what each one is, how they differ and how they fit together. Before the definitions, one metaphor ties all five together: a worker doing a job at a workbench. I reuse it for every discipline so the boundaries stay sharp. Keep that worker in your head. The whole post hangs off it. What it is: Writing the instructions that tell the model what to do, how to do it, and what format to return. The job: Craft system prompts, few-shot examples, output format constraints, and chain-of-thought scaffolding that produce reliable outputs from the model. There are different styles of prompting (zero-shot, few-shot, chain-of-thought, tree-of-thought) and different techniques (role prompting, constraint prompting, format prompting) that can be combined in various ways. What you’re optimizing for: Failure mode: Example: “You are a Python debugging assistant. Given a traceback and the relevant code, identify the root cause and suggest a fix. Always output: 1) Root cause, 2) Fix (as a code block), 3) Explanation. Do not hallucinate functions that don’t exist in the code.” What it is: Deciding what information goes into the model’s context window and in what order. In the metaphor: the briefing packet you hand the worker for this task. Not the whole library, just the pages that matter right now. The job: Build retrieval pipelines (RAG), manage conversation history, compress old context, inject relevant documents, and structure the information so the model can use it effectively. What you’re optimizing for: Failure mode: Example: Instead of passing an entire 500-page PDF, you chunk it into sections, embed each chunk, retrieve the top-5 most relevant sections based on the query, and inject only those, with a summary of what was excluded so the model knows the full document exists. What it is: Building the infrastructure that wraps the model, tool definitions, API integrations, execution environments, and safety rails. In the metaphor: the workbench, the power tools and the safety guards. It is what the worker can reach and operate, nothing more. This is the static part: it exists before the agent runs a single step. The job: Define the tool schemas the model can call. Wire up the terminal, file system, web browser, and external APIs. Enforce permissions, rate limits, and sandboxing. Handle retries, timeouts, and error recovery. What you’re optimizing for: Failure mode: The agent calls a tool that doesn’t exist, or a tool that exists but returns an error the model can’t interpret. The agent spirals into retry loops or hallucinates a successful outcome. Half of Hermes’ early “successes” were it cheerfully telling me a task was done while the tool had actually thrown a stack trace it never bothered to read. Example: Defining a What it is: Designing the agent’s reasoning loop, how it decides what to do next, when to stop, and when to ask for help. In the metaphor: the work rhythm. If the harness is the machine on the bench, the loop is the worker pressing “go” again and again, checking the result each time, until the piece is finished or the foreman gets called. This is the dynamic part: it only exists while the agent is running. Peter Steinberger, the creator of OpenClaw, put the same idea more provocatively in June 2026: “You shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents.” His framing adds a sharp point I love: a good loop corrects itself against objective signals (tests, type checkers, linters, runtime errors) rather than against your patience. That is the difference between an agent that runs while you sleep and one that needs you babysitting every turn. The job: Choose the loop pattern (ReAct, Plan-and-Execute, Tree of Thoughts). Set the max turns. Implement context compression when the conversation gets long. Decide when the agent should pause for human approval vs. proceed autonomously. What you’re optimizing for: Failure mode: Example: A ReAct loop that caps at 90 turns, compresses context at 50% utilization, and falls back to asking the user for clarification after 3 consecutive failed tool calls on the same sub-task. What it is: Designing the systems that tell you whether the other four are working. The job: Build eval datasets. Configure LLM-as-a-Judge pipelines. Run regression tests on every model change. Aggregate scores, track trends, and set alerting thresholds for quality degradation. What you’re optimizing for: Failure mode: Example: A Langfuse evaluator that classifies every agent trace by query type (coding, research, ops), scores output quality on a 1-5 rubric, and alerts when the research query quality score drops below 3.5 for two consecutive days. This is the part that trips everyone up, including me. Prompt and Evaluation are easy to tell apart. The blur is between Context, Harness and Loop, because all three feel like “stuff around the model.” Here is the cleanest way I have found to separate them. Or, borrowing a line I like from LangChain: prompt shapes behavior, context shapes reasoning, harness shapes execution. The loop is the slice of execution that repeats. Here is the trick that finally made it click. Take one task, “fix the failing test,” and one symptom, “the agent didn’t fix it.” Who gets paged depends entirely on why: Same task. Same surface failure. Three different engineers fix it. Once you can sort a failure into KNOW, DO or DECIDE, the blur is gone. Harness and Loop blur the most, so be precise. Honestly, the industry itself is not fully settled here: LangChain treats the loop as one component inside the harness, while others treat Loop Engineering as a separate layer on top. The split I use: The one-liner that sticks: the harness runs the agent once, the loop runs the harness until the job is done. Forget the big architecture diagram for a second. Here is a single turn of the agent, with each discipline owning exactly one segment: They do not overlap. Context is the arrow going in. Harness is the arrow going out to the world and back. Loop is the circle plus the exit gate. That is the whole difference in one picture. Each discipline feeds into the next, and Evaluation wraps around all of them. Here’s what a production AI agent system looks like when all five are in place, from the user’s question to the final answer: The dotted arrows are the key insight. Evaluation Engineering doesn’t just sit at the end, it feeds back into every other discipline. When eval scores drop, you don’t know which layer is broken until you instrument all five. I’ve seen each of these failure modes in production. They’re not hypothetical. Evaluation Engineering is the most skipped and the most dangerous to skip. Every other discipline has a visible failure mode, the output is wrong, the agent crashes, costs spike. Evaluation failure is invisible. Everything looks fine until a user complains. Most teams today are at different maturity levels for each discipline: Prompt Engineering is high (everyone does it), Context Engineering is medium (RAG is mainstream), Harness Engineering is growing, Loop Engineering is early, and Evaluation Engineering is the lowest, often skipped entirely. The gap between Prompt Engineering (everyone does it) and Evaluation Engineering (almost nobody does it well) is where most production AI systems fail silently. This is also where the most interesting work is happening right now, LLM-as-a-Judge, automated regression pipelines, and observability-driven development. I run an AI agent (Hermes) 24/7 on my own infrastructure. Here’s how the five disciplines map to real decisions I make: If you’re building AI systems, you’re doing all five of these whether you call them “engineering” or not. The question is whether you’re doing them deliberately. Prompt Engineering is table stakes. Context Engineering is where most teams spend their RAG effort. Harness Engineering is where agent frameworks add value. Loop Engineering is where most teams have the most room to grow. Evaluation Engineering is where most teams are flying blind. If you can’t measure it, you can’t improve it. Start with Evaluation Engineering, even a simple LLM-as-a-Judge over your existing traces will tell you more about your system than months of intuition.
The Five Disciplines
1. Prompt Engineering
2. Context Engineering
3. Harness Engineering
read_file tool with a clear schema (path: string, offset: int, limit: int), a check_fn that verifies the file exists before the tool is exposed to the model, and an error handler that returns structured JSON the model can act on instead of a raw stack trace.4. Loop Engineering
5. Evaluation Engineering
How are Context, Harness, and Loop different?
One word each
Discipline One word The question it answers Context KNOW What does the model get to see? Harness DO What can the model actually touch in the real world? Loop DECIDE How many times, in what order, and when does it stop? The same bug, three different owners
run_tests tool, or the tool crashed. Harness bug: the capability was missing.Harness vs Loop: the machine and the “go” button
One agent turn, three zones
flowchart LR
IN["Context
what the model SEES
prompt + retrieved facts"]
M["Model
picks the next action"]
H["Harness
what the model CAN DO
runs the tool, returns result"]
D{"Loop
goal met?"}
OUT["Done"]
IN --> M
M -->|"action"| H
H -->|"result"| D
D -->|"no: feed result back"| M
D -->|"yes: stop"| OUT
style IN fill:#1a1a2e,stroke:#0f3460,color:#fff
style M fill:#1a1a2e,stroke:#888888,color:#fff
style H fill:#1a1a2e,stroke:#16a085,color:#fff
style D fill:#1a1a2e,stroke:#e9a23b,color:#fff
style OUT fill:#1a1a2e,stroke:#2d6a4f,color:#fff
Side-by-Side Comparison
Dimension Prompt Engineering Context Engineering Harness Engineering Loop Engineering Evaluation Engineering What you build Instructions for the model Information pipeline into the model Tool infrastructure around the model Reasoning cycle for the model Measurement system over the model Core artifact System prompt, few-shot examples RAG pipeline, context compressor, chunking strategy Tool schemas, API adapters, sandbox Agent loop (ReAct, Plan-Execute, ToT) Eval datasets, judge prompts, score dashboards Key question “What should the model do?” “What does the model need to know?” “What can the model do?” “How does the model decide what to do next?” “Is the model actually doing it well?” Optimizes for Output quality, instruction adherence Signal-to-noise ratio, retrieval precision Tool reliability, execution safety Task completion, loop efficiency Evaluation accuracy, regression detection Failure mode Over-prompting, instruction dilution Context stuffing, retrieval noise Tool errors, missing safety rails Infinite loops, premature stopping False confidence, measuring the wrong thing Measured by Eval scores, format compliance Retrieval precision/recall, token cost Tool call success rate, latency Turn count, completion rate Judge-human agreement, alert precision When it breaks Outputs are wrong format, off-topic Model hallucinates from missing context Agent can’t act on its environment Agent gets stuck or loops forever You don’t know quality is degrading Who typically owns it Prompt engineer, ML engineer RAG engineer, data engineer Platform engineer, infra engineer Agent engineer, ML engineer AI observability engineer, QA engineer Maturity in industry High (everyone does it) Medium (RAG is mainstream) Growing (agent frameworks emerging) Early (most teams have simple loops) Lowest (most teams skip it entirely) How They Fit Together
flowchart TB
USER[User asks a question
or gives a task]
subgraph MODEL_SIDE[Inside the AI system]
PROMPT[Prompt Engineering
Defines the instructions
What should the model do?]
CONTEXT[Context Engineering
Selects useful information
What does the model need to know?]
MODEL[LLM / AI Model
Generates text, reasoning,
tool calls, or decisions]
LOOP[Loop Engineering
Controls the work cycle
Think, act, retry, stop,
or ask for help]
HARNESS[Harness Engineering
Connects the model to the world
Tools, APIs, files,
permissions, safety]
end
OUTPUT[Final answer
or completed action]
EVAL[Evaluation Engineering
Measures quality after the run
Did it work well?
Did anything regress?]
USER -- task --> PROMPT
PROMPT -- instructions --> MODEL
CONTEXT -- useful facts --> MODEL
MODEL -- proposed next step --> LOOP
LOOP -- action request --> HARNESS
HARNESS -- tool result --> LOOP
LOOP -- done --> OUTPUT
OUTPUT -- full run trace --> EVAL
EVAL -. improves instructions .-> PROMPT
EVAL -. improves retrieved context .-> CONTEXT
EVAL -. improves tools and safety .-> HARNESS
EVAL -. improves agent behavior .-> LOOP
style USER fill:#f8f9fa,stroke:#333,color:#111
style OUTPUT fill:#f8f9fa,stroke:#333,color:#111
style PROMPT fill:#2b2d42,stroke:#ef233c,color:#fff
style CONTEXT fill:#1d3557,stroke:#457b9d,color:#fff
style MODEL fill:#111827,stroke:#fca311,color:#fff
style LOOP fill:#3a0ca3,stroke:#7209b7,color:#fff
style HARNESS fill:#14213d,stroke:#06d6a0,color:#fff
What Happens When You Skip One?
Skip This What Happens Prompt Engineering Model outputs inconsistent formats, ignores constraints, hallucinates expected behavior Context Engineering Model answers from stale or irrelevant information. Token costs 5-10x higher than necessary Harness Engineering Agent can’t interact with the real world. Every tool call is a potential crash Loop Engineering Agent runs forever on simple tasks, or stops too early on complex ones Evaluation Engineering You ship a model update and quality drops 20% and nobody notices for a week Where Is the Industry?
How I Apply This
check_fn that verifies requirements before the tool is exposed to the model. Destructive commands require approval.The Bottom Line
Related Articles