Conversation Loop Cheat Sheet

One-turn algorithm

messages.append(user)
if cached_system_prompt is None:
    build_or_restore_system_prompt()

while budget_remaining:
    api_messages = copy(messages)
    inject_system_prompt(api_messages)
    inject_ephemeral_context_into_current_user(api_messages)
    sanitize_role_order_and_tool_json(api_messages)

    assistant = call_model(api_messages, tools)

    if assistant.tool_calls:
        validate_names_and_json(assistant.tool_calls)
        messages.append(assistant_with_tool_calls)
        for call in assistant.tool_calls:
            result = dispatch_tool(call.name, call.args)
            messages.append(tool_result(call.id, result))
        maybe_compress_context()
        continue

    if assistant.content:
        messages.append(assistant_final)
        persist_session(messages)
        return assistant.content

    recover_empty_response()

Core invariants

• Stable system prompt: build once per session, replay byte-stably.
• Ephemeral context: memory/plugin recall goes into the current user-message API copy.
• Role rhythm: user → assistant(tool_calls) → tool → assistant.
• Tool ID contract: each tool result references the exact assistant tool_call_id.
• Validation first: repair or reject bad tool names and invalid JSON before execution.
• Recovery branches: retry, nudge, compress, fallback, summarize.

Source map