Lesson 01 · Harness Engineering

What Is an Agent Harness?

The architecture of a tool-calling autonomous agent, using Hermes as the reference

The Big Picture

An agent harness is the software that wraps an LLM and turns it from a stateless text-completion function into an autonomous system that can act. The LLM is the brain; the harness is everything else — the hands, the eyes, the memory, the loop that keeps it going.

Here's the simplest possible picture of Hermes:

flowchart TB USER[("👤 User")] ENTRY["CLI or Gateway"] CORE["Agent Core\nconversation loop"] LLM["🧠 LLM API"] RESP["Response back"] USER --> ENTRY ENTRY --> CORE CORE --> LLM LLM --> RESP style CORE fill:#dbeafe,stroke:#2563eb,stroke-width:3px style LLM fill:#fef9c3,stroke:#ca8a04,stroke-width:2px
Figure 1: A simple overview. The core calls the LLM (repeatedly for tool calls), then returns the final answer. The detailed loop is in Figure 2.

The loop is the heart. It sends messages to the LLM, executes any tools the LLM requests, feeds results back, and repeats until the LLM produces text. That's the entire agent. Let's walk through each piece.

The Agent Loop

The AIAgent class (run_agent.py) runs a single loop that powers everything:

flowchart TB START(["User sends message"]) BUILD["Build system prompt
identity + tools + memory + context"] CALL["Call LLM
messages + tool schemas"] TOOL{"Has tool_calls?"} VALIDATE["Validate tool names & JSON
reject hallucinations"] DISPATCH["Execute tool
append result to messages"] DONE["Final response
plain text, no tools"] START --> BUILD BUILD --> CALL CALL --> TOOL TOOL -->|"yes"| VALIDATE VALIDATE --> DISPATCH DISPATCH -->|"loop back"| CALL TOOL -->|"no"| DONE style CALL fill:#dbeafe,stroke:#2563eb,stroke-width:2px style TOOL fill:#fef3cd,stroke:#d97706,stroke-width:2px style DONE fill:#f0fdf4,stroke:#16a34a,stroke-width:2px
Figure 2: The conversation loop. LLM is called, tools are dispatched if requested, results are fed back, repeat until text-only response.

From the actual source:

while (api_call_count < agent.max_iterations
       and agent.iteration_budget.remaining > 0):

    # Call the LLM
    assistant_message = agent._call_model(messages, ...)

    # Tool calls → execute, append results, loop
    if assistant_message.tool_calls:
        agent._execute_tool_calls(assistant_message, messages, ...)
        continue

    # Plain text → final answer
    if assistant_message.content:
        final_response = assistant_message.content
        break
Key Insight

The LLM decides which tools to call and when to stop. The harness just makes the calls, validates results, and loops. The "intelligence" is the model's tool selection. The harness is the capability surface + persistence.

The Tool Registry

Instead of hardcoding tool dispatch with if/else chains, every tool self-registers at import time into a central registry:

flowchart TB DIR["tools/ directory"] REG(["ToolRegistry
singleton"]) MT["model_tools.py
query + dispatch"] LLM2["LLM receives
tool schemas in API call"] DIR -->|"import time"| REG REG -->|"get_all_tool_names()"| MT REG -->|"get_entry(name)"| MT REG -->|"dispatch(name, args)"| MT MT -->|"inject schemas"| LLM2 style REG fill:#dbeafe,stroke:#2563eb,stroke-width:3px style MT fill:#f0fdf4,stroke:#16a34a,stroke-width:2px
Figure 3: Tool registry. Each tools/*.py calls registry.register() at import time. model_tools.py queries the registry to build tool schemas and dispatches calls.

Each tool is a single file that calls registry.register():

# tools/terminal_tool.py — one file per tool
from tools.registry import registry

def check_requirements() -> bool:
    return True  # terminal always available

registry.register(
    name="terminal",
    toolset="terminal",
    schema={
        "name": "terminal",
        "description": "Execute shell commands",
        "parameters": { ... }
    },
    handler=lambda args, **kw: terminal_tool(
        command=args.get("command", ""), **kw
    ),
    check_fn=check_requirements,
)
Why This Pattern Matters

Adding a tool = one file, zero core changes. Tools can be conditionally available via check_fn. MCP server tools register the same way. The tool list sent to the LLM is dynamically assembled per-session based on enabled toolsets.

System Prompt Assembly

Before every conversation, Hermes builds a system prompt from multiple sources. This prompt stays byte-stable for the whole conversation — changing it mid-flow kills prompt caching.

flowchart TB A["Identity + Memory\nSOUL.md · MEMORY.md · USER.md"] B["Capabilities\nTool schemas · Skills index"] C["Session Context\nContext files · Platform hints"] COMBINE(["Prompt Builder"]) OUTPUT["One byte-stable system prompt"] A --> B --> C --> COMBINE --> OUTPUT style COMBINE fill:#dbeafe,stroke:#2563eb,stroke-width:3px style OUTPUT fill:#f0fdf4,stroke:#16a34a,stroke-width:2px
Figure 4: Six sources feed into one prompt. The prompt is assembled once per session and never changes — this keeps the LLM's cache prefix valid.

Session Persistence

Every message — user, assistant, tool result — goes into SQLite with FTS5 full-text search. This enables session resume and cross-session recall.

flowchart TB LOOP2["Agent writes message data"] DB["SQLite database"] FTS["FTS5 full-text index"] SEARCHER["Cross-session search"] COMPRESSOR["Context compression"] LOOP2 --> DB DB --> FTS FTS --> SEARCHER DB --> COMPRESSOR style DB fill:#faf5ff,stroke:#9333ea,stroke-width:3px style FTS fill:#f0fdf4,stroke:#16a34a,stroke-width:2px
Figure 5: SQLite + FTS5. Every message stored. Cross-session search via FTS5. Context compression splits sessions via parent_session_id chains.

The Gateway

The gateway is a long-running daemon that connects to 20+ messaging platforms. Each platform is one adapter file. The same agent core serves them all.

flowchart TB GR["GatewayRunner"] AD["Platform Adapters"] CORE2["AIAgent Core"] GR --> AD AD --> CORE2 style CORE2 fill:#dbeafe,stroke:#2563eb,stroke-width:3px style GR fill:#fef9c3,stroke:#ca8a04,stroke-width:2px
Figure 6: One agent core behind a gateway. Platform adapters handle I/O; the agent doesn't know it's on Discord vs Telegram.

Memory & Skills

Two persistence systems make Hermes self-improving across sessions. Memory is declarative (facts about you). Skills are procedural (reusable workflows).

flowchart TB S1["Session 1
solves complex task"] CREATE["skill_manage
action='create'"] SKILL["SKILL.md
stored in skills/"] S2["Session 2
similar task appears"] LOAD["skill_view()
loads procedure"] APPLY["Follows proven steps
avoids earlier mistakes"] S1 --> CREATE CREATE --> SKILL SKILL --> S2 S2 --> LOAD LOAD --> APPLY style SKILL fill:#fef9c3,stroke:#ca8a04,stroke-width:2px style S1 fill:#f0fdf4,stroke:#16a34a style S2 fill:#dbeafe,stroke:#2563eb
Figure 7: The self-improvement loop. What the agent learns in one session is available in the next — no code changes needed.

The Five Invariants

If you build something like this, these rules must never be broken:

1. Prompt Caching Is Sacred

The system prompt + tool schemas must be byte-stable for the life of a conversation. Change it and every turn pays full token cost.

2. Strict Role Alternation

user → assistant → tool → assistant → tool → assistant(text). Never two same-role messages in a row. Tool results are the tool role.

3. Narrow Core

Every core tool ships on every API call. New capability belongs at the edges: skill → plugin → MCP server → new core tool (last resort).

4. Config ≠ Secrets

Behavioral settings in config.yaml. Credentials in .env. Never invent a new env var for non-secret settings.

5. Tools Return JSON Strings

Every handler returns json.dumps(...). Uniform protocol means results can be truncated, sanitized, and appended to messages consistently.

What's Next

What pattern makes adding a tool as simple as writing one file with zero core changes?
A central if/else chain in model_tools.py
Each tool file self-registers into a singleton registry at import time
A JSON config file listing all tools
All tools come from MCP servers