The Conversation Loop

Your tangible win

After this lesson, you should be able to trace a Hermes turn from memory: append user → build API copy → call model → validate tool calls → execute tools → append tool results → repeat until text.

The loop spine

The agent loop is not magic. It is a bounded state machine. Each model call either produces tool calls or final text. Tool calls keep the loop alive; final text exits.

The real loop guard in Hermes is two budgets: a per-agent max_iterations and a shared iteration_budget. There is also a one-turn grace path for graceful summaries.

# agent/conversation_loop.py, around lines 675-700
while (api_call_count < agent.max_iterations
       and agent.iteration_budget.remaining > 0) \
       or agent._budget_grace_call:

    api_call_count += 1

    if agent._budget_grace_call:
        agent._budget_grace_call = False
    elif not agent.iteration_budget.consume():
        _turn_exit_reason = "budget_exhausted"
        break

Design point: budget is consumed at the API-call boundary, not at the tool boundary. The expensive thing is another model step.

Stable prompt, changing user context

Hermes protects prompt caching by building the system prompt once per session and replaying it byte-stably. Runtime context from memory/plugins is injected into the current user message copy, not into the system prompt.

# agent/conversation_loop.py, around lines 482-495
if agent._cached_system_prompt is None:
    _restore_or_build_system_prompt(agent, system_message, conversation_history)

active_system_prompt = agent._cached_system_prompt

# around lines 566-575
# Plugin context is appended to the current user message,
# never the system prompt. This preserves the prompt cache prefix.

Message role rhythm

Tool calling has a strict transcript rhythm. The assistant asks for tools; the harness answers each tool call with a tool message; then the model sees those results on the next call.

Hermes repairs malformed history before each API call and fills missing tool results on errors. That is defensive harness engineering: never let a malformed transcript poison future turns.

Tool validation before execution

The model can hallucinate tool names or broken JSON. Hermes validates before dispatch. Invalid names are repaired or returned to the model as tool errors; broken JSON is retried before recovery.

# agent/conversation_loop.py, around lines 3286-3518
if assistant_message.tool_calls:
    for tc in assistant_message.tool_calls:
        if tc.function.name not in agent.valid_tool_names:
            repaired = agent._repair_tool_call(tc.function.name)
            if repaired:
                tc.function.name = repaired

    # validate JSON arguments...
    assistant_msg = agent._build_assistant_message(assistant_message, finish_reason)
    messages.append(assistant_msg)
    agent._execute_tool_calls(assistant_message, messages, task_id, api_call_count)
    continue

Recovery is part of the loop

A production harness is mostly edge cases. Hermes treats empty responses, provider errors, context overflow, and budget exhaustion as loop states rather than surprises.

# Empty after tools → add a synthetic nudge, then continue
# agent/conversation_loop.py, around lines 3675-3740
if _prior_was_tool and not agent._post_tool_empty_retried:
    agent._post_tool_empty_retried = True
    messages.append({
        "role": "user",
        "content": "You just executed tool calls but returned an empty response...",
    })
    continue

API failures go through agent/error_classifier.py, which classifies errors as auth, billing, rate limit, overload, timeout, context overflow, payload too large, model not found, format error, and more. Retry delay uses jittered exponential backoff from agent/retry_utils.py to avoid synchronized retry storms.

# agent/retry_utils.py
def jittered_backoff(attempt, base_delay=5.0, max_delay=120.0):
    delay = min(base_delay * (2 ** (attempt - 1)), max_delay)
    jitter = rng.uniform(0, 0.5 * delay)
    return delay + jitter

Compression happens inside the loop

Long sessions do not just fail at the context window. Hermes estimates request size before calls and also checks after tool execution. If the context is too large, it compresses the middle while protecting the beginning and recent tail.

# agent/conversation_loop.py, around lines 3563-3600
_real_tokens = agent.context_compressor.last_prompt_tokens

if agent.compression_enabled and _compressor.should_compress(_real_tokens):
    messages, active_system_prompt = agent._compress_context(
        messages,
        system_message,
        approx_tokens=agent.context_compressor.last_prompt_tokens,
        task_id=effective_task_id,
    )

Interactive trace

Tap each state. Your job is to feel where the transcript mutates.

Retrieval check

What to remember

• The loop has two exits: final text or exhausted/failed budget handling.
• The system prompt is stable; runtime context is injected into the user-message copy.
• Tool-call messages are transcript objects, not just events.
• Recovery paths are first-class: validate, retry, nudge, compress, fallback, summarize.
• A minimal harness can be built with this exact loop before adding memory, gateway, or skills.