目标：
做一个面向内部工程文档的 RAG 问答服务。

背景：
文档位于 docs/ 和 Notion 导出目录；用户会询问部署、排障和接口规范。

范围：
- 支持文档导入、语义分块、向量化、检索和回答
- 回答必须附来源标题和 URL；无依据时明确拒答

非目标：
- 本期不做多租户；用户权限只保留接口和元数据字段

技术约束：
- 后端使用 FastAPI；向量库使用 pgvector
- 模型调用封装在 app/llm.py；引入重型框架前先说明理由

验收标准：
- pytest 通过；新增至少 3 个 RAG eval case
- 命中文档时返回可定位引用；无依据问题不得编造

执行方式：
先阅读仓库并给出可验证的子任务、风险、实现顺序和测试方案；
等待确认后再修改代码，完成时汇总改动、测试结果与剩余风险。

rag-project/
  app/ingest.py          # 解析、清洗、分块与入库
  app/retrieval.py       # 过滤、召回与重排
  app/answer.py          # 证据组装、引用与拒答
  app/api.py             # 服务接口
  app/schemas.py         # Chunk、Citation 等数据结构
  evals/cases.jsonl      # 问题、期望来源与判分规则
  tests/test_retrieval.py
  tests/test_grounding.py
  tools/internal-kb-mcp/server.ts
  .codex/config.toml
  .codex/hooks/
  AGENTS.md

from openai import OpenAI

client = OpenAI()
store = client.vector_stores.create(name="engineering-kb")

with open("docs/runbook.pdf", "rb") as f:
    uploaded = client.files.create(file=f, purpose="assistants")

client.vector_stores.files.create(
    vector_store_id=store.id, file_id=uploaded.id
)
# 查询前轮询文件状态，确认其已经 completed。
response = client.responses.create(
    model="<MODEL_ID>",
    input="部署失败时先检查什么？请给出来源。",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [store.id],
        "max_num_results": 6,
    }],
    include=["file_search_call.results"],
)
print(response.output_text)

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";

const server = new McpServer(
  { name: "internal-kb", version: "1.0.0" },
  { instructions: "Search before answering project questions; cite title and URL." }
);

server.registerTool("search_docs", {
  title: "Search internal documentation",
  description: "Use before answering project knowledge questions.",
  inputSchema: {
    query: z.string().min(3),
    source: z.string().optional(),
    top_k: z.number().int().min(1).max(20).default(6),
  },
  annotations: { readOnlyHint: true, destructiveHint: false, openWorldHint: false },
}, async ({ query, source, top_k }) => {
  const chunks = await searchIndex({ query, source, top_k });
  return {
    structuredContent: { chunks },
    content: [{ type: "text", text: JSON.stringify(chunks) }],
  };
});

await server.connect(new StdioServerTransport());

[mcp_servers.internal_kb]
command = "node"
args = ["./tools/internal-kb-mcp/server.js"]
env_vars = ["KB_API_TOKEN"]
enabled_tools = ["search_docs", "get_doc", "list_sources", "create_eval_case"]
default_tools_approval_mode = "prompt"
startup_timeout_sec = 20
tool_timeout_sec = 30

[mcp_servers.internal_kb.env]
KB_API_URL = "https://kb.example.com"

[mcp_servers.internal_kb.tools.search_docs]
approval_mode = "approve"

## RAG workflow
- 回答项目知识问题前，优先调用 internal_kb.search_docs。
- 每项事实必须引用 source title 和 URL，并保留 doc_id 便于追溯。
- 证据低于阈值、来源冲突或 ACL 拒绝时，不得猜测，应说明缺口。
- 需要全文时再调用 get_doc，避免无目的地读取大量内容。
- create_eval_case 会写入仓库，只有在用户确认后才能调用。
- 修改检索策略后运行 tests 与 evals，并汇报 Recall@k 和失败案例。

[[hooks.UserPromptSubmit]]
[[hooks.UserPromptSubmit.hooks]]
type = "command"
command = 'python3 "$(git rev-parse --show-toplevel)/.codex/hooks/security_policy.py"'
command_windows = 'py -3 C:\\path\\to\\repo\\.codex\\hooks\\security_policy.py'
timeout = 10

[[hooks.PreToolUse]]
matcher = "^Bash$|^apply_patch$|^mcp__internal_kb__.*$"
[[hooks.PreToolUse.hooks]]
type = "command"
command = 'python3 "$(git rev-parse --show-toplevel)/.codex/hooks/security_policy.py"'
command_windows = 'py -3 C:\\path\\to\\repo\\.codex\\hooks\\security_policy.py'
timeout = 10
statusMessage = "Running safety policy"

[[hooks.PostToolUse]]
matcher = "^Bash$|^mcp__internal_kb__.*$"
[[hooks.PostToolUse.hooks]]
type = "command"
command = 'python3 "$(git rev-parse --show-toplevel)/.codex/hooks/post_tool_review.py"'
command_windows = 'py -3 C:\\path\\to\\repo\\.codex\\hooks\\post_tool_review.py'
timeout = 20

import json
import re
import sys

data = json.load(sys.stdin)
event = data.get("hook_event_name", "")

SECRET = re.compile(
    r"(?i)(api[_-]?key|access[_-]?token|secret|password)\s*[:=]\s*['\"]?[A-Za-z0-9_./+-]{12,}"
)
DANGEROUS = [
    re.compile(r"(?i)\brm\s+-rf\b"),
    re.compile(r"(?i)\bRemove-Item\b.*\b-Recurse\b"),
    re.compile(r"(?i)\b(DROP\s+DATABASE|TRUNCATE\s+TABLE)\b"),
    re.compile(r"(?i)(^|[ /\\])\.env($|[ ./'\"])"),
]

if event == "UserPromptSubmit":
    prompt = data.get("prompt", "")
    if SECRET.search(prompt):
        print(json.dumps({
            "decision": "block",
            "reason": "提示中疑似包含密钥；请先脱敏。",
        }, ensure_ascii=False))
        raise SystemExit(0)

if event == "PreToolUse":
    tool_input = data.get("tool_input") or {}
    payload = str(tool_input.get("command", tool_input))
    if any(rule.search(payload) for rule in DANGEROUS):
        print(json.dumps({
            "hookSpecificOutput": {
                "hookEventName": "PreToolUse",
                "permissionDecision": "deny",
                "permissionDecisionReason": "命中破坏性操作策略。",
            }
        }, ensure_ascii=False))
        raise SystemExit(0)

print("{}")

import json, re, sys

data = json.load(sys.stdin)
response = json.dumps(data.get("tool_response", ""), ensure_ascii=False)
failed = re.search(r"(?i)(pytest.*FAILED|Traceback|secret detected)", response)

if failed:
    print(json.dumps({
        "decision": "block",
        "reason": "工具输出包含失败或泄密信号，请检查并修复。",
    }, ensure_ascii=False))
else:
    print("{}")

What is an agent? What can an agent do? How to use an agent

Key cases: OpenClaw · Hermes · Claude Code · Codex

Core thread: From “model capabilities” to “a task system that can be entrusted, verified, and governed”

Content summary

This report focuses on three questions: what is an agent (Agent), what an agent can do, and how to use an agent. The report proposes that an agent is not a separate model or chat interface, but a task system that is composed of a model and Harness (model execution and interaction layer) and connects to the environment through observation and action interfaces; goals, context, tools, memory, runtime and governance determine whether this system can advance an answer into a verifiable task.

Capability analysis shows that agents have demonstrated consistent value in research synthesis, document and data processing, software engineering, digital operations, continuous automation, and multi-agent collaboration. Their suitability still depends on whether results are verifiable, actions are reversible, permissions are limited, and responsibilities are clear. The report provides methods for task specification, permission classification, verification design, memory layering, skill packaging, and evaluation, emphasizing: define completion before authorizing execution.

The product section uses OpenClaw, Hermes, Claude Code and Codex as four observation samples. Among them, Claude Code and Codex are the focus, starting from entrance, configuration, context, extension, multi-agent, long task, security, typical workflow and team implementation respectively, and absorb the key points of two Chinese tutorials provided by users. Finally, the selection coordinates, organizational readiness, 90-day roadmap and supporting templates are given.

Executive Summary: Ten Key Takeaways

1. Agent is not a synonym for model, but a task system that combines goals, models, tools, memory, runtime and governance.

2. The key change in the agent is that the interaction unit changes from “generating an answer” to “delivering a verifiable task within the boundary”.

3. The value of a capability depends on the quality of the feedback. Strong feedback such as code testing, file rendering, and structured readback make engineering and knowledge work mature earlier.

4. Applicability should be judged by four conditions: results can be verified, actions can be revoked, authority can be limited, and responsibilities can be clear.

5. Effective use starts with task engineering: first write down the deliverables, scope, constraints, validation, permissions and failure handling, then discuss the degree of autonomy.

6. Long-term availability relies on context, memory, skills, tools, and multi-agent orchestration, but each additional layer of capabilities also adds new governance objects.

7. Security is not the end audit, but the complete chain from untrusted input, minimum permissions, sandbox, approval, secret protection to audit and rollback.

8. OpenClaw and Hermes embody the resident personal agent route; Claude Code and Codex embody the engineering route with code base and verifiable changes as the core.

9. The selection of Claude Code and Codex should not stop at the function list. Comparative evaluation should be carried out based on real tasks, unified environment, clear budget and failure classification.

10. The order of organizational implementation should be task inventory - controlled pilot - capability solidification - large-scale operation; autonomy can only be gradually enlarged with evidence.

One-sentence conclusion｜Agent’s competitiveness is ultimately not “whether it can act”, but whether it can continuously deliver verifiable results under controllable costs and risks.

Research Questions and Report Structure

This material does not equate the AI agent with a certain model or product, but rather regards it as a new task execution system: the model is responsible for understanding and decision-making, the tool connects to the real environment, the memory maintains the state, the runtime carries continuous execution, and governance restricts authority and responsibility.

This article focuses on three core questions: What is an agent? What can an agent do? How to use AI agents safely and effectively?

The full text discusses paradigm changes and system definition, capability map and applicable boundaries, task engineering and governance, four product cases, as well as model selection, organizational implementation and trend judgment.

The core judgment of the report is that the value of an AI agent is not to “chat for others”, but to turn tasks into operational units that are entrustable, observable, verifiable, reusable and manageable.

Table of Contents

• Content summary
• Executive Summary: Ten Key Takeaways
• Research question and report structure
• Table of Contents
• 1. Introduction: From dialogue paradigm to task delegation
  • 1.1 Why now: from answering questions to delegating tasks
  • 1.2 Why technology and organizational conditions mature at the same time
  • 1.3 Three core questions and four observation samples
  • 1.4 Evidence methods, version boundaries and reading paths
• 2. What is an agent: system structure, operational loop and autonomous boundary
  • 2.1 A definition that can be used throughout the text
  • 2.2 Six basic components: goals, models, tools, memory, runtime and governance
    • 2.2.1 Model—Harness—Environment: Three-layer boundary of the agent
    • 2.2.2 From Prompt Engineering to Harness Engineering
  • 2.3 Operational loop: plan, act, observe, correct and transfer
  • 2.4 Chatbots, Fixed Workflows and Agents
  • 2.5 Degrees of Autonomy and Human Control Points
  • 2.6 Single agent, multi-agent and organization collaboration
• 3. What the agent can do: Capability map and scenario maturity
  • 3.1 Six types of relatively mature capabilities
  • 3.2 Research, Reporting and Knowledge Work
  • 3.3 Digital Operation and Business Collaboration
  • 3.4 Software Engineering and Verifiable Changes
  • 3.5 Data Analysis, Tabulation and Multimodal Delivery
  • 3.6 Continuous automation and background work
  • 3.7 Multi-agent collaboration
  • 3.8 Scenario Maturity Matrix: Value, Verifiability and Impact
• 4. Capability boundaries, failure modes and evaluation methods
  • 4.1 When not to use agents
  • 4.2 Cognitive Failure: hallucinations, Misunderstood Goals, and Overconfidence
  • 4.3 Execution failure: tool errors, state drift and partial completion
  • 4.4 Security Failures: Prompt Injection, Unauthorized Access, and Supply Chains
  • 4.5 Evaluation Framework: Task Success, Evidence, Controllability and Cost
    • 4.5.1 Capability upper limit and service reliability should be measured separately
    • 4.5.2 From final results to trajectories, boundaries and first error attribution
    • 4.5.3 Establishing a sustainable Agent evaluation closed loop
  • 4.6 ROI vs. Total Cost of Ownership
  • 4.7 Manual intervention, stop conditions and responsibility rollback
• 5. How to use agents: task engineering and human-agent collaboration
  • 5.1 Start with the right task
  • 5.2 Upgrade prompts to task instructions
  • 5.3 Five steps from read-only pilot to continuous automation
  • 5.4 Plans, Milestones and Task Status
  • 5.5 Write verification into “Definition of Done”
  • 5.6 human-agent interaction: Feedback should be directed towards rules, evidence and trade-offs
  • 5.7 Context and long-term task management
  • 5.8 Failure recovery, review and process upgrade
• 6. Critical Infrastructure: Context, Memory, Tools and Multi-Agents
  • 6.1 Context Engineering: Determining what the current task sees
    • 6.1.1 Stable prefix, dynamic trajectory and task status bar
    • 6.1.2 Compression, isolation and progressive disclosure are three different methods
  • 6.2 Persistent instructions: write stable rules to each task
  • 6.3 Memory stratification: working memory, project facts and long-term experience
    • 6.3.1 Distinguish between user memory and organizational knowledge base
    • 6.3.2 RAG is a retrieval system and a knowledge organization system
    • 6.3.3 Knowledge writing, updating and publishing require independent governance
  • 6.4 Skills: Turn procedural knowledge into reusable abilities
  • 6.5 Tools, MCP and external system connections
    • 6.5.1 Five types of tools and two design routes
    • 6.5.2 Tool contracts must make model parameters consistent with real execution parameters
  • 6.6 Hooks and Deterministic Control
  • 6.7 Subagents, Agent Teams and Delegation Boundaries
    • 6.7.1 Multi-Agent classification axis: context and topology
    • 6.7.2 Collaboration failures need to be handled explicitly during the design phase
  • 6.8 Worktree, cloud environment and background operation
  • 6.9 Observability: Logs, Traces, Costs and Audits
• 7. Security and Governance: Aligning Capabilities, Permissions and Responsibilities
  • 7.1 Threat Model: Who can input, what the Agent can do, and who the results affect
  • 7.2 Minimum Privilege and Capability Classification
  • 7.3 Sandbox, Approval, Rules and Rollback
  • 7.4 Credentials, Privacy and Data Lifecycle
  • 7.5 Scaling and Supply Chain Security
  • 7.6 Auditing, Accountability and Incident Response
  • 7.7 Organizational Policies and Managed Configuration
  • 7.8 Governance Maturity from Pilot to Production
• 8. OpenClaw in-depth analysis: self-hosted multi-channel personal agent
  • 8.1 Product Positioning: Multi-channel entrance for persistent personal agent
  • 8.2 Core Architecture: Channels—Gateway—Agent—Nodes
  • 8.3 Channels, Sessions and Multi-Agent Routing
  • 8.4 Tools, Skills, Plugins and Capability Extensions
  • 8.5 Memory, Workspaces and Automation
  • 8.6 Control UI, CLI and Mobile Nodes
  • 8.7 Usage path: from single-channel read-only pilot to resident operation
  • 8.8 Security model, applicable scenarios and boundaries
• 9. In-depth analysis of Hermes Agent: a learnable open personal agent
  • 9.1 Product positioning: open personal agent capable of accumulating experience
  • 9.2 Running modes: CLI, desktop, message gateway and remote environment
  • 9.3 Memory, Context Files and SOUL.md
  • 9.4 Skills and built-in learning loops
  • 9.5 Tools, MCP, Browsers and Multimodal Capabilities
  • 9.6 Subagents, message channels and scheduled tasks
  • 9.7 Sandboxes, Approvals and Security Boundaries
  • 9.8 Usage paths, migration and selection recommendations
• 10. Claude Code in-depth analysis
  • 10.1 Product positioning and four types of user interfaces
  • 10.2 Engineering closed loop from problem to verifiable change
  • 10.3 How to issue high-quality engineering tasks
  • 10.4 Context windows, compression and session continuation
  • 10.5 CLAUDE.md, Rules and Persistent Project Directives
  • 10.6 Auto Memory, Session History and Knowledge Precipitation
  • 10.7 Skills: On-demand loading of knowledge and workflow
  • 10.8 MCP: Connecting Issue, Monitoring, Database and Design Systems
  • 10.9 Subagents and Agent Teams
  • 10.10 Hooks: Deterministic Automation and Security Gates
  • 10.11 Plugins and Marketplace: Team Distribution Layer
  • 10.12 Permissions, Security and Trusted Execution
  • 10.13 Team adopts route and asset system
  • 10.14 Case: Cross-file login defect repair
• 11. Codex in-depth analysis
  • 11.1 Product Positioning: Organizing Engineering Work Between Projects, Tools, and the Cloud
  • 11.2 Local, Worktree and Cloud: How to choose between the three types of environments
  • 11.3 CLI, IDE, desktop and cloud control planes
  • Standard Engineering Closed Loop of 11.4 Codex
  • 11.5 AGENTS.md: Command chain for global and project rules
  • 11.6 Local Memories: Division of Soft Memories and Hard Rules
  • 11.7 Plan, Goal and Long Tasks
  • 11.8 Worktrees and Handoff: local parallel file isolation
  • 11.9 Subagents: Concurrent roles of exploration, testing and review
  • 11.10 Skills, Plugins, MCP and Hooks
  • 11.11 Use Codex to build RAG and project knowledge base
    • 11.11.1 RAG’s six-layer engineering architecture
    • 11.11.2 Write Prompt into executable project requirements
    • 11.11.3 Use OpenAI file_search to quickly build a managed knowledge base
    • 11.11.4 Let Codex be implemented in stages and end with evaluation
  • 11.12 MCP Server: Integrating knowledge and tools into workflow
    • 11.12.1 Define the tool contract first, then write the server
    • 11.12.2 A runnable skeleton: TypeScript MCP Server
    • 11.12.3 Configure MCP in the project and use AGENTS.md to solidify the workflow
  • 11.13 Hooks: Security Interception and Automatic Inspection
    • 11.13.1 Which checkpoint should events be placed on
    • 11.13.2 Configuring three key Hooks
    • 11.13.3 Implementing security policies using JSON input and output
    • 11.13.4 Boundary and online checking of Hooks
  • 11.14 Cloud, external integration and reproducible environments
  • 11.15 Scheduled Tasks and Continuous Tasks
  • 11.16 Sandbox, Approvals, Rules and Networking
  • 11.17 Team landing route
  • 11.18 Case: Complete dependency upgrade and regression verification in parallel
• 12. Product comparison, selection and combination use
  • 12.1 Selection principles: matching of tasks, environment, ecology and governance
  • 12.2 Unified comparison of four products
  • 12.3 Key differences between Claude Code and Codex
  • 12.4 Decision tree: Select product route from task entry
  • 12.5 Unified Measured Design: How to compare without being misled by demonstrations
  • 12.6 Combining Usage and Migration Strategies
• 13. Organizational implementation: from pilot to large-scale operation
  • 13.1 Readiness: Processes, Data, Tools, Validation and Governance
  • 13.2 Pilot combination: one high-frequency task, one high-value task, one governance task
  • 13.3 90-Day Roadmap
  • 13.4 Roles and Responsibilities: Shared Accountability among Business, Engineering, Security and Platform
  • 13.5 Organizational Knowledge Assets: Rules, Skills, Tools, and Assessment Sets
    • 13.5.1 From operational trajectories to four types of capability carriers
    • 13.5.2 Online execution and offline evolution must be separated
  • 13.6 Indicators and Operational Rhythm
  • 13.7 People Capabilities and Change Management
  • 13.8 Scale model: platformization, federal governance and continuous optimization
• 14. Trends, Conclusions and Action Proposals
  • 14.1 Competition shifts from model capabilities to complete system capabilities
    • 14.1.1 Models absorb Harness, but do not destroy Harness
  • 14.2 Interactions shift from conversations to ongoing tasks with digital colleagues
  • 14.3 Skills, Plugins and tool protocols become the capability distribution layer
  • 14.4 Multimodality and Computer Use Expanded Work Objects
  • 14.5 Security and governance will become product capabilities
  • 14.6 Answer three questions: system, task and method
• Appendix A: Agent task specification Template
• Appendix B: Engineering Agent Task Acceptance Checklist
  • Task and Scope
  • Change quality
  • Verification
  • Security
  • Transfer
• Appendix C: Agent Security and Governance Checklist
• Appendix D: Agent Evaluation Scale and Pilot Records
• Appendix E: Glossary of core terms
• Appendix F: References and Further Reading

1. Introduction: From dialogue paradigm to task delegation

The rise of AI agents does not stem from a mutation in the capabilities of a single model, but is the result of the joint maturation of models, tools, operating environments and organizational empowerment. The collaborative unit between humans and AI is gradually shifting from one-time answers to task delivery with boundaries and evidence.

Understanding this paradigm shift helps avoid reducing agents to chatbots or marketing labels and establishes a common starting point for subsequent discussions of system definition, capability boundaries, and governance.

1.1 Why now: from answering questions to delegating tasks

Chatbots often work around conversational turns: users ask questions and the system returns answers. No matter how long the answer is, the main product is still the text. The agent works around the goal and status: the user gives the desired result, allowable range and acceptance conditions, and the system decides the next step on its own during the running process, uses tools to obtain facts or changes the environment, and continuously judges whether it is close to completion.

This distinction can be understood using three interaction paradigms. Search is “I look for information”, dialogue is “I ask you questions”, and the agent is “I give the task to the system, but retain key control points”. AI agents do not require complete autonomy; many reliable systems embed humans into closed execution loops through manual approval, revocable operations, and staged acceptance.

Judgment criteria | If the system can only generate suggestions but cannot continue to act based on real environment feedback, it is closer to an assistant; if the system can continuously select steps, call tools and verify results within the constraints, it only has the characteristics of an AI agent.

The rise of agents is not driven solely by the growth of model parameters. What really changes the shape of products is when models begin to steadily call search, code execution, browsers, file systems, and enterprise interfaces, and can retain state, read feedback, and continue to modify in a single task. When reasoning capabilities are combined with an operational environment, artificial intelligence changes from a content generator to a process participant.

This change can be summarized by three interaction paradigms: search leaves information to humans to filter, dialogue leaves interpretation and generation to models, and agents entrust a constrained work to the system. Delegation does not mean giving up control, but shifting the focus from each operational step to goal setting, key decisions, permission approval and final acceptance.

For organizations, the most important signal is not whether a demonstration can run for dozens of steps, but whether the delivery unit has changed: in the past, model calls or chat seats were purchased, and now task templates, tool interfaces, validation scripts, permission policies, and running records are being built. These assets jointly determine whether the agent can enter real work.

The industrial significance of Agent lies in transforming the collaborative unit between humans and AI from “answers” into “boundary and evidence-based tasks.”

Data basis: OpenAI “A practical guide to building agents”; Anthropic “Trustworthy agents in practice”

1.2 Why technology and organizational conditions mature at the same time

The model capability addresses “whether it can be understood and made decisions”, the tool protocol addresses “whether it can reach the real system”, the runtime addresses “whether it can continue to work”, the verification basis addresses “whether it knows whether it is done correctly”, and organizational readiness determines “whether it can be authorized”. If one of the five is missing, the agent will easily stay in the demonstration stage.

Software engineering matures first because code libraries, compilers, tests, static checks, and version control naturally provide feedback. Research and knowledge work subsequently expanded as web pages, documents, tables, and presentations became actionable objects. Business processes advance more slowly, not because the model cannot write text, but because identity, data, approval, auditing, and exception handling are more complex.

Therefore, adopting an agent is not an automatic upgrade after purchasing a more powerful model. Organizations need to simultaneously sort out processes, data interfaces, acceptance standards, and responsibility chains, and transform vague job activities into tasks that can be delegated, observed, and rolled back.

The implementation of AI agents is the result of the joint maturation of model capabilities and tissue engineering.

1.3 Three core questions and four observation samples

This report first answers three basic questions. First, what is an agent and how to distinguish it from chatbots, fixed workflows and ordinary models; second, what tasks can an agent undertake, which capabilities are relatively mature, and which are still limited by the environment and verification; third, how to use an agent and how to organize task descriptions, permissions, verification and manual supervision into runnable methods.

OpenClaw and Hermes represent the resident personal agent runtime route, emphasizing message channels, long-term memory, skills and continuous operation; Claude Code and Codex represent the engineering agent route, forming verifiable changes around the code base, terminal, IDE, desktop and cloud environment. The four samples are not a full market ranking, but are used to observe how different products combine models, tools, status, and governance.

The report returns to common issues after product analysis: how to layer memories, how to skill repetitive processes, how to connect external systems, how to collaborate with multiple agents, how to audit background tasks, and how to enlarge permissions step by step. This reduces the product menu to a portable methodology.

Products are observation samples, and what really needs to be refined is a systematic approach that can be reused across products.

1.4 Evidence methods, version boundaries and reading paths

Product capabilities are mainly based on official documents, product pages and update logs; the “Claude Code Complete Tutorial” and “Codex Complete Solution” provided by users are used to supplement Chinese operation paths, interface understanding and practical experience; manufacturer research and independent papers are used to discuss effects, security and trends. Different evidence answers different questions. Operation tutorials cannot be used to prove optimal performance, nor can marketing cases be used to replace evaluations under uniform conditions.

Intelligent products change rapidly. The report clarifies the data retrieval date as August 10, 2026, and marks features that are preview, experimental, turned off by default, or platform-limited. Version numbers, pricing, platform support, quotas, and security policies should still be re-verified before actual deployment.

Managers can focus on reading Chapters 1-4, 7, 11-13; readers who want to establish usage methods can focus on reading Chapters 5-7; those who are concerned about personal agents can read Chapter 8; engineering teams should focus on reading Chapter 9 Claude Code and Chapter 10 Codex, and directly use the appendix template to conduct pilots.

Layering “product facts, practical experience, and evidence of effects” is a prerequisite for maintaining report reliability.

2. What is an AI agent: system structure, operational loop and autonomous boundary

An agent is not a separate model, but a task system composed of goals, models, tools, memory, runtime and governance. It continuously acts around a goal and adjusts next steps based on feedback from the environment.

The system definition must not only explain how the agent selects and performs actions, but also must explain the degree of autonomy, artificial control points, and responsibility boundaries.

2.1 A definition that can be used throughout the text

This report defines an AI agent as a system that, within the boundaries of constraints and permissions, can independently select steps around goals on behalf of the user, use tools, maintain task status, and correct actions based on environmental feedback. This definition deliberately places the “model” in the system, rather than equating the model itself with the agent.

The engineering quality of an agent depends on how these components are combined. Stronger models may reduce hallucinations, but they cannot replace correct tool interfaces, test environments, permission policies, and audit records. In turn, a system with moderate model capabilities but a clear operating environment, stable tools, and strict acceptance is often more reliable than a system with “stronger models but fuzzy boundaries.”

“Representing the user” in the definition emphasizes the authorization relationship, “centering around the goal” emphasizes task orientation, “independent selection of steps” emphasizes that the path is not completely hard-coded, “using tools and maintaining status” emphasizes that the system can be exposed to the real environment, “correcting based on feedback” emphasizes closed loop, and “within the authority boundary” incorporates security and governance into the definition itself.

This definition allows for the existence of different degrees of autonomy. A system that can only read the repository and propose modification plans can be a low-autonomous agent; a system that can modify, test and submit for review in an isolated workspace has a higher degree of autonomy; a system that is online for a long time and can be executed across applications requires stricter identity, permissions and auditing.

An agent is a constrained system that continuously acts around a goal and corrects itself based on feedback from the environment.

Data basis: OpenAI “A practical guide to building agents”; Anthropic “Trustworthy agents in practice”

2.2 Six basic components: goals, models, tools, memory, runtime and governance

The target layer translates user intentions into deliverables, scope, and stopping conditions; the model layer is responsible for understanding, reasoning, and next-step selection; the tool layer provides reading and action interfaces; the memory layer saves the current status, project facts, and reusable experiences; the runtime is responsible for execution, scheduling, isolation, and recovery; the governance layer defines permissions, approvals, audits, and responsibilities.

Six components have shortcomings. No matter how strong the model is, if the tool interface does not have clear parameters and error returns, the system will still be misused; no matter how many tools there are, if there are no acceptance criteria, the system may package failure as completion; no matter how long the memory is, if the content is outdated or contaminated, errors will be magnified; no matter how long the running time is, if there is no stop condition, costs and risks will only accumulate.

When evaluating a product, questions should be asked along each of the six components, rather than just comparing model names. It is necessary to know whether the model can be switched, whether the tool is controllable, whether the memory can be viewed and corrected, whether the operating environment can be reproduced, whether key actions are approved, and whether evidence is left during the entire process.

Components	Questions Answered	Common Failures
Goals	What to deliver and when to stop?	Task scope drift
Model	How to understand and choose the next step?	Errors in reasoning or fact
Tools	What can I read and do?	Miscall, interface failure
Memory	How to preserve current state and long-term facts?	Obsolescence, contamination, leakage
Runtime	Where is it executed and how is it restored?	Inconsistent environment, disconnected tasks
Governance	Who authorizes, approves, and is responsible?	Ultra vires, non-traceable

Agent quality is determined by the weakest system link, not by the model rankings alone.

2.2.1 Model—Harness—Environment: Three-layer boundary of the agent

The six basic components are suitable for making a construction list; from the perspective of operating mechanism, the agent can also be compressed into a three-layer relationship: Model is responsible for understanding, reasoning and selecting the next step, Harness is responsible for constructing context, exposing tools, maintaining loops and states, implementing permissions and verification, and Environment saves the real state and performs transfers. The model and Harness together form an AI agent, and the environment is located outside the AI agent.

This boundary clarifies three often confused issues. First, the model does not directly “see the world”, it can only process observations written by Harness into the context; second, the model does not directly change the world, it can only make tool calls, which are then executed by the action interface; third, environmental feedback is the source of truth, and the model’s readme cannot be used to replace file differences, database status, web page receipts or test results.

So any Agent can be checked with three questions: What can it see, what is it allowed to do, and how can it prove that it does it right? The observation range determines the cognitive boundary, the action range determines the capability and risk boundaries, and the verification mechanism determines whether the system can move from demonstration to reliable operation.

Hierarchy	Core responsibilities	Main engineering objects	Typical failures
Model	Understanding goals, reasoning, choosing next steps	Model, reasoning budget, output protocol	Misunderstandings, hallucinations, wrong strategies
Harness	Organizational context, tools, loops, state, permissions, and validation	Prompts, Skills, MCP, Hooks, sandboxes, reviews	Context pollution, tool misuse, state loss
Environment	Save the real state and respond to actions	Files, databases, web pages, APIs, applications and users	Concurrent changes, irreversible side effects, incomplete receipts

Table 2-1: Responsibilities and failure boundaries of Model-Harness-Environment

Data basis: Chapter 2 of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

2.2.2 From Prompt Engineering to Harness Engineering

Prompts are still important, but the reliability of production-level agents cannot rely solely on a longer system prompt. Harness Engineering focuses on the complete control loop: how to select and trim observations, how to provide stable tool contracts to the model, how to save task state, when to compress the context, which actions must be approved, how to correct or roll back after failure, and what evidence is used to decide to stop.

This shift also explains why the same model performs significantly differently in different products: products wrap the model in different context structures, tool sets, permission mechanisms, execution environments, and evaluation closed loops. The model determines the potential strategy space, and Harness converts potential into repeatable delivery.

The stronger model will absorb some of the strategies originally written in Harness, but Harness will not disappear. As capabilities expand, new observation interfaces, action interfaces, business constraints, and verification responsibilities will continue to be moved out to the system layer. The engineering focus will continue to shift from “teaching the model to complete one step” to “organizing the model to safely complete longer tasks.”

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

2.3 Operational loop: plan, act, observe, correct and transfer

The basic closed loop of an AI agent can be written as “understand the goal - formulate a plan - perform actions - observe the results - revise the plan - complete or hand over.” Each tool call brings new facts back to the system: whether the command succeeded, whether the file was changed, whether the test passed, whether the interface returned the expected data. The model proceeds based on these observations rather than assuming that the previous step has been successful.

Reliability comes from feedback and verification, not one-shot reasoning. For software tasks, observations may be compilation output, test logs, static check results, and diff; for research tasks, observations may be source content, timestamps, data consistency, and reference checks; for business processes, observations may be system receipts, status fields, and approval results.

Figure 2-1: The coding agent forms a verifiable feedback closed loop between “human-interface-model-environment”

Source: Anthropic, Building effective agents (retrieval date: 2026-08-10)

1. Understand the task: Identify the final deliverable, audience, limitations and irreversible risks.

2. Form a plan: Break the goal into executable and checkable intermediate nodes.

3. Call tools: read, calculate, edit or call external services within the scope of authorization.

4. Observation results: Return real evidence such as command output, file differences, and interface responses to the system.

5. Correction and retry: Adjust the path according to the cause of failure, and do not package failure as success.

6. Acceptance and handover: Compare the completion standards to summarize the results, risks and next steps.

Each step in closing the loop should produce a state change or new evidence. A plan is not a static list, but a current hypothesis; actions must fall into the environment through tools; observations must read the real results of commands, files, interfaces, or pages; corrections must change the path based on the cause of failure; handovers must describe completion, incompleteness, and risk against acceptance criteria.

Reliable agents handle failures explicitly. Command timeouts, test failures, page structure changes, permission denials, and origin conflicts should not be condensed into a “done.” The system needs to retain failure context to determine whether to retry, change tools, narrow the scope, or request human input.

The running closed loop also explains why engineering agents develop rapidly: compilers, tests, diffs, and version control constitute strong feedback; while value judgments, strategic choices, and high-risk operations in the real world lack cheap, objective feedback, so manual participation is more needed.

Reliability comes from observation-correction-verification, rather than one-time generation that looks reasonable.

2.4 Chatbots, Fixed Workflows and Agents

The three are not mutually exclusive product categories, but different control methods. Chatbots are suitable for explanations, suggestions and lightweight generation; fixed workflows are suitable for tasks with stable rules, clear paths and high predictability; agents are suitable for tasks whose paths cannot be completely preset and require judgment based on new information in the middle.

Real products often use hybrid architectures. The agent is responsible for understanding fuzzy targets, finding information and proposing solutions; the fixed workflow is responsible for payment, production release, external sending, data deletion and other high-risk links; the chat interface is responsible for explaining the process and accepting manual decisions. Giving all steps to the agent or writing them into fixed processes is usually not the optimal solution.

A chat interface does not automatically imply an agent, nor does a non-chat interface imply the absence of an agent. The key is who controls the steps: a chatbot typically generates answers; a fixed workflow follows a preset path; an agent chooses the next step based on the current state. Actual systems can use three control methods simultaneously in the same product.

High-impact actions are suitable for preserving deterministic workflows. For example, the agent can collect refund materials, determine whether it is in compliance with the policy, and draft opinions, but the real payment goes through the rules engine and manual approval; the agent can prepare changes for production release, but the deployment system performs fixed checks and permission control.

Type	Step Control	Suitable for Task	Key Risks
Chatbot	User pushes round by round	Explanation, generation, single round of Q&A	The answer seems correct
Fixed workflow	Developer preset path	Stable, repetitive, clear rules	Insufficient exception coverage
Agent	The system selects the next step according to status	Open path, multi-step task	Goal drift and permission risk
Hybrid system	Intelligent judgment + determined execution	High-value real process	Complex boundary design

The most reliable architecture is usually a combination of “intelligent judgment + deterministic execution + manual control points”.

2.5 Degree of autonomy and Human Control Points

“Whether it is an AI agent” does not equal “whether it is fully autonomous.” A more useful way is to break down the degree of autonomy into five levels: read-only analysis, controlled writing, tool execution, background tasks, and multi-agent collaboration. Each level should have independent acceptance gates, rather than automatically receiving higher permissions just because they performed well in the previous stage.

Responsibility remains with the organization and users. Agents can make judgments, perform steps and provide evidence, but who authorizes access to data, who approves high-impact operations, who publishes externally, and who bears compliance responsibilities must be clear in the process. The value of human oversight is not just a “last look” but deciding which nodes must be confirmed by a human.

The degree of autonomy includes at least five dimensions: readable data range, executable tools, writable systems, sustainable time, and self-delegable subtasks. Different dimensions can be configured independently, and agents should not be simply divided into “automatic” and “non-automatic”.

Human control points should be placed at high-leverage nodes: goals and success criteria require human confirmation; when multiple plans involve business trade-offs, human choice is required; permission upgrades, irreversible actions, and external releases require human approval; and independent review is required when the results have a significant impact. Low-risk, reversible, well-verified steps can be automated.

As the system matures, controls can shift from incremental approval to management by exception, but only if there are stable success rates, failure visibility, and audit records. Eliminating manual validation should not precede establishing technical boundaries and evaluation evidence.

Autonomy is not a single switch, but a set of empowerment dimensions matched to risk.

2.6 Single agent, multi-agent and organizational collaboration

Single agents are easiest to maintain consistent goals, context, and responsibilities and should be the default starting point. Splitting multiple agents only makes sense when the prompt logic is too complex, when the tools are highly similar making selection difficult, when certain side tasks generate a lot of context, or when the tasks can be clearly parallelized.

Common structures include the manager model and the peer-to-peer model. In the manager mode, the main agent holds user goals and delegates tasks such as research, testing, and security reviews as tools; in the peer-to-peer mode, multiple sessions share tasks and communicate with each other, which is more suitable for complex work that requires discussion and negotiation.

The costs of multi-agent include repeated reads, increased costs, version conflicts, inconsistent conclusions, and blurred merge responsibilities. When splitting, you need to define inputs, outputs, tools, editable scope and acceptance criteria for each role, and ultimately still need a clear merger.

Multi-agent is a complexity tool, not a demonstration of ability; when it can be solved with one agent, use one first.

3. What the agent can do: Capability map and scenario maturity

Agents are already able to participate in research synthesis, document and data processing, software engineering, digital operations, continuous automation and multi-agent collaboration, but the maturity of different scenarios varies significantly.

Judge value not just by the sequential steps in the demo, but also by looking at the deliverables, the strength of the feedback, the cost of verification, and the impact of errors.

3.1 Six types of relatively mature abilities

At present, the more mature AI agent capabilities can be summarized into six categories. What they have in common is not “the ability to generate content”, but the ability to organize multiple sources of input, multiple intermediate steps and verification actions into a complete task.

The common characteristics of mature capabilities are that inputs can be obtained, steps can be executed, and results can be verified. It does not matter whether a system “can summarize”, but rather whether it can find material, distinguish sources, generate deliverables, check formatting and evidence, and account for limitations when information is missing.

Competency Domain	Typical Deliverables	Verification Methods	Maturity
Research synthesis	Presentation/report with sources	Source coverage, citation verification	Higher
Documents and data	Word/PPT/forms	Rendering, formulas, sampling verification	Higher
Software Engineering	Code Change and Testing	Diff, Testing, Static Checking	Higher
Digital operations	Cross-system recording or configuration	Readback, screenshots, logs	Medium
Continuous automation	Regular monitoring and notification	Operation history, abnormal alarms	Medium
Open reality action	External irreversible results	Often require manual confirmation	Caution

Using a complete task chain to describe capabilities is closer to the real value than using verbs such as “search, summarize, write code”.

3.2 Research, reporting and knowledge work

Research agents can generate search strategies around questions, browse websites, documents, update logs, and business company pages, extract key facts, mark sources and times, and organize the evidence into reports, tables, or presentations. The value of this type of task is to reduce data handling and format conversion, allowing people to focus on problem setting, evidence weight and conclusion judgment.

However, when studying AI agents, the most common problem is that “the references exist but the content does not support the conclusion.” Therefore, acceptance cannot just be based on whether there is a link, but whether the source directly supports the sentence in question, whether the information is out of date, whether different sources contradict each other, and whether commercial promotion and independent evidence are confused.

High-quality research agents need to include search scope, source priority, publication date, cross-corroboration, and citation locations in their tasks. For commercial products, official websites, documents, and update logs are often more timely than papers; for effects and risks, independent research, standards, and actual measurements are more important.

Humans should focus on research questions, weight of evidence, and conclusion boundaries. Agents can cover more sources and unify formats, but they cannot replace judgments about conflicts of interest, sample representativeness, and causality.

The core of studying AI agents is not to capture more information, but to establish a traceable chain of evidence.

3.3 Digital operation and business collaboration

When the agent connects to browsers, emails, calendars, databases and business APIs, it can read work orders and customer records, determine the type of problem, generate handling suggestions, update system fields and arrange follow-up. It can also turn meeting minutes into tasks, organize feedback from multiple channels into priority lists, or clean tabular data into usable analysis results.

Value often comes from cross-system connections. For example, customer complaint handling is not about writing a reply, but about reading historical tickets, identifying problems, finding policies, formulating solutions, updating CRM, creating schedules and maintaining audits. The closer it is to real business, the more important it is to distinguish between “generating a draft” and “formally sending”, “proposing updates” and “actual writing”.

Cross-system tasks typically include reading state, interpreting unstructured content, selecting processing paths, updating records, and scheduling follow-up. Agents are good at handling exception branches outside the rules, but write and outgoing actions should be drafted, suggested, approved, or automatically executed based on the impact level.

The difficulty in business adoption lies in identity and status consistency. The system must know who is currently represented, which version of the data is read, whether the action is truly successful, whether repeated execution is idempotent, and how to compensate after failure.

The value of business intelligence lies in the cross-system closed loop, and the risks are also concentrated in identity, writing and outgoing.

3.4 Software Engineering and Verifiable Changes

Software engineering is a field where agent capabilities are relatively mature because the code base provides structured context and the compiler, testing, static checking, and Git provide strong feedback. Engineering agents can start from Issues, error logs or requirement descriptions, locate relevant code, modify across files, run tests, check diffs, and form commits or PRs.

The real value is not in “generating a piece of code”, but in completing a reviewable engineering closed loop: it can explain what was changed, why it was changed, what was verified, and what risks there are. Both Claude Code and Codex are strengthening this complete way of working from exploration to verification.

The code base provides structured context, the toolchain provides high-density feedback, and version control provides review and rollback. The agent can complete code positioning, cross-file modification, testing, static inspection, diff review and PR preparation, and the results can be accepted more objectively.

Engineering agents still require domain knowledge. Requirement implications, compatibility strategies, architectural trade-offs, and production risks are often beyond test coverage. People who are familiar with the system can give better boundaries, evidence, and feedback, and therefore produce higher quality results.

The strongest thing about engineering agents is not to generate code, but to incorporate code changes into verifiable engineering processes.

3.5 Data analysis, tables and multi-modal delivery

Data tasks can start from raw tables or exported files and complete field identification, anomaly checking, calculations, visualization and report delivery. Multi-modal capabilities enable the system to read screenshots, design drafts and scanned documents and write analysis results into spreadsheets, Word or presentations.

The key to this type of task is to maintain the data lineage: the original data is not overwritten, the cleaning rules can be recorded, the formulas and results can be recalculated, and the graphs and text conclusions are consistent. Visual delivery also requires rendering checks to avoid occlusions, truncation, or font substitutions on the page if the file structure is correct.

For statistical conclusions, agents should report sample size, caliber, missing values, and uncertainty, and not just output a seemingly precise number. For analyzes that require specialized models, agents should call proven code or tools rather than relying on language model mental arithmetic.

Multimodal agents extend analysis into final delivery, but data lineage and visual acceptance must be preserved.

3.6 Continuous automation and background work

When task descriptions, tool permissions, and acceptance criteria have stabilized, the agent can go into the background or run on a schedule: checking daily error logs, summarizing weekly code changes, tracking PR status, scanning for dependency updates, generating operational reports, or monitoring specific metrics. Compared with traditional timed scripts, the agent can handle format changes and abnormal branches; compared with real-time conversations, it can continue to advance unattended.

The bar for continuous automation is higher because mistakes happen repeatedly. Before formal scheduling, you must manually run several rounds to confirm whether the input is stable, whether the failure is visible, whether the results are reviewable, and whether the stopping conditions are clear. One of the most common adoption mistakes is handing over high-privilege work to a timed agent without observation.

The background agent can handle input format changes and abnormal branches, but errors will also amplify with frequency. Before formal scheduling, multiple rounds should be manually run to record normal, unchanged, partial failures, and insufficient permissions behaviors, and then gradually reduce the frequency of manual observation.

Each background task must define a reporting strategy: what changes are worthy of notification, whether to be silent when there are no changes, how many consecutive failures to stop, where to write the results, and who is responsible for recovery. Simply setting a timed reminder without operational responsibility turns a one-time risk into an ongoing risk.

The prerequisites for continuous automation are stable task descriptions, narrow permissions, observable failures, and clear accountability.

3.7 Multi-agent collaboration

Multi-agent is suitable for tasks that can be parallelized and independently accepted. For example, one agent studies the market, another analyzes technical solutions, the third does security review, and the main agent finally summarizes. In engineering scenarios, different sub-agents can be allowed to explore the code, supplement tests and review risks respectively.

More parallelism is not always better. Subagents increase cost, context synchronization, and merging difficulty. Parallelism only really improves time when tasks have few dependencies on each other, delivery boundaries are clear, and failures do not contaminate each other. For highly coupled code modifications, it is more secure to have a main agent complete the overall design first, and then assign independent subtasks.

Parallelism is best suited for work with independent information sources, independent file scopes, or independent evaluation dimensions. Research can be divided by source or hypothesis, development can be divided by module, and review can be divided by security, performance, testing, and maintainability. The main agent is responsible for unifying the problem, eliminating conflicts, and forming final conclusions.

If multiple agents need to constantly read each other’s intermediate results, or modify the same core file at the same time, parallelism is usually not faster. At this time, one role should first complete the architecture and interface design, and then split the stable boundary.

Tasks can be independently accepted, which is the basic condition for multi-agent to generate net benefits.

3.8 Scenario Maturity Matrix: Value, Verifiability and Impact

Scenario maturity should be determined by task conditions rather than product hype. Whether input is stable, whether environmental feedback is timely, whether deliverables can be independently verified, whether actions are revocable, and whether permissions can be restricted are key variables in judging whether the degree of autonomy can be improved.

Research synthesis and software engineering typically have stronger feedback, making it easier to establish end-to-end processes; cross-system business operations may be of higher value, but identity, approval, and exception handling are more complex; and open reality actions should retain greater human control once they are irreversible.

Maturity is not a fixed attribute. By supplementing structured input, readback interfaces, test scripts, simulation environments and rollback mechanisms, the same task can be gradually upgraded from “assistance only” to “controlled execution”. Conversely, environmental changes or verification failures can also reduce maturity.

Pilot records should save both success and failure conditions to avoid treating a single successful operation as a universal capability. Scenario maturity is organizationally meaningful only if it can be delivered reliably across different inputs, exceptions, and permission boundaries.

When converting maturity judgments into pilot plans, you should first select tasks that are narrow in scope, provide fast feedback, and have manual baselines. It is not advisable to change data sources, business rules, and execution tools at the same time during the pilot, otherwise it will be difficult to attribute success or failure.

The pilot must record at least four sets of indicators: completion quality, time and cost, manual intervention location, risks and anomalies. The completion rate must be counted separately from the correctness of the results; just because a task seems to be completed does not mean that the deliverables have met business standards.

Increased autonomy can then be stratified by risk. Low-risk steps are allowed to be executed automatically, medium-risk steps are set for sampling review, and high-risk and irreversible actions are reserved for sequential approval. Every increase in level of autonomy must be supported by new verification evidence.

Therefore, a truly replicable scenario is not a beautiful demo, but a set of task products that include input contracts, execution boundaries, acceptance criteria, exception handling, and responsible persons. Organizations should engineer tasks first before discussing whether to expand agent authority.

High-value, easily verifiable, low-impact tasks are best suited for piloting first, such as open source research, small-scale test completion, internal reporting, and read-only diagnostics. High-value, easily verifiable but high-impact tasks are suitable for agent execution and human approval, such as production release preparation and customer refund recommendations.

Tasks that are difficult to verify and have high impact should not be automated. Agents are more suitable as analysis assistants or solution generators. Tasks with low value and stable rules should give priority to using scripts and traditional workflows to avoid introducing additional costs for “intelligence”.

Maturity is not a fixed attribute. Through supplementary testing, structured data, sandboxing, approval and rollback, originally higher-risk scenarios can be gradually transformed into delegated tasks.

Task characteristics	Applicability judgment	Suggested methods
Verifiable results, revocable actions, limited permissions	Priority pilot	Allow end-to-end execution
The results are verifiable but the impact is large	Conditionally applicable	Key node approval
Judgment is subjective and evidence is incomplete	Auxiliary use	Generates options and evidence, does not replace decision-making
High risk, irreversible, unclear responsibilities	No entrustment for the time being	Retain human leadership

Transforming validation and boundaries first, and then increasing the degree of autonomy is often more effective than pursuing a stronger model directly.

4. Capability boundaries, failure modes and evaluation methods

Agents can expand the range of tasks that can be automated and introduce new failure modes such as goal drift, factual errors, tool misuse, environmental inconsistencies, and unauthorized access.

Only when failure mechanisms, takeover conditions, and evaluation indicators are incorporated into system design can capability boundaries be transformed from abstract reminders into executable management rules.

4.1 When not to use agents

Tasks with fixed rules, stable paths, and low-cost programming should prioritize the use of scripts or workflows; tasks that cannot verify the results should first establish a checking mechanism; high-impact tasks such as payments, production changes, external sending, and data deletion should not be fully autonomous in the absence of approval; tasks that expose a large number of credentials or sensitive data should first reduce permissions and isolate the environment.

Scenario selection | Don’t just write isolated actions such as “summarizing, classifying, and writing code”; only by writing input, multiple intermediate steps, final products, and verification methods into complete tasks can you take advantage of the agent.

If the task path can be written as a deterministic script at low cost, traditional automation is usually cheaper and more stable; if there is no way to judge whether the result is right or wrong, the agent cannot be improved through closed loop; if the action is high-impact and irreversible, automatic execution should not be authorized without approval.

Not using it does not mean completely excluding agents. It can undertake preliminary material sorting, plan comparison, risk warning and draft generation, leaving the final action to a fixed system or person. The key is to get capabilities into the right parts, rather than automating complete processes for the sake of automating them.

Scenario boundaries determine project success more than model capabilities.

4.2 Cognitive Failure: hallucinations, Misunderstood Goals, and Overconfidence

Agents may cite non-existent sources, misread user intent, mistake partial successes for overall successes, or reach firm conclusions when there is insufficient evidence. Because it performs multiple steps, early misunderstandings can be magnified later.

Reducing cognitive failure requires writing goals, don’ts, and acceptances clearly, requiring exploration before action, using external evidence and structured output, and allowing the system to request input when critical information is missing, from conflicting sources, or with insufficient confidence. For high-stakes judgments, independent review should be arranged rather than having the same agent self-confirm.

Long chains of agents amplify early misunderstandings, so clarification and planning gates must be placed up front.

4.3 Execution failure: tool error, status drift and partial completion

The tool may time out, return truncated results, modify incorrect files, duplicate submissions, or only partially complete. Web interfaces change, API status updates occur concurrently, and long tasks may lose critical context after session compression. The execution system must treat return codes, file differences, and external receipts as sources of truth.

Consider idempotence, transactions, and compensation for write actions: is it safe to repeat, how to recognize partial success, and how to roll back after failure. Task status should be logged explicitly rather than just existing in the conversation. Phased acceptance prevents errors from propagating over long links.

The real danger is not that the tool reports an error, but that the system still reports success after the tool fails.

4.4 Security failure: prompt injection, unauthorized access and supply chain

Web pages, emails, issues, code comments, documentation, and MCP returns may contain malicious or misleading instructions. If the agent confuses these contents with user authorization, it may leak information, invoke high-privilege tools, or alter long-term memory.

Third-party Skills, Plugins, Hooks, and MCP servers are both capability sources and software supply chains. Risk control requires content isolation, tool least privilege, secret protection, source review, version pinning, network restrictions, and approval of high-impact actions, rather than just reminding the model to “ignore malicious instructions.”

Prompt injection is a permissions and data-flow problem, not merely a prompting problem.

4.5 Evaluation Framework: Task Success, Evidence, Controllability and Cost

Task success rate should be calculated based on complete acceptance, rather than self-reported by the model; evidence quality checks whether tests, citations, receipts, and diffs directly support the conclusion; controllability focuses on whether failures are visible, whether manual intervention points are reasonable, and whether misoperations are recoverable; costs are also calculated for model calls, manual reviews, infrastructure, and rework.

The evaluation set should be from real tasks, covering normal samples, boundary samples, missing inputs, insufficient permissions and malicious inputs. A single demonstration can only illustrate the possibility. Stable results from multiple consecutive rounds, different operators and different times support promotion.

Dimensions	Core indicators	Evidence
Task success	End-to-end success rate, first-time pass rate	Deliverables and acceptance records
Correctness	Facts/tests/calculated error rates	Quotes, tests, spot checks
Efficiency	Labor time, total delay, cost	Control group and log
Robustness	Abnormal recovery, number of retries	Failure classification
Security	unauthorized action, sensitive data, approval hits	Audit records
Usability	Taking over burden, clarity of instructions	User feedback

To evaluate Agent, you need to look at the complete task and running evidence, not just the final text.

4.5.1 Capability upper limit and service reliability should be measured separately

Pass@k answers the upper limit of the ability of “try k times independently and succeed at least once” and is suitable for exploring whether the model is possible to complete the task; Best@k focuses on whether the best result can be selected from multiple candidates. Real business is more concerned about continuous reliability: whether the same process is successful k times in a row can be observed by multiplying Pass consecutive@k or the approximate success probability.

The meanings of the two indicators are not interchangeable. A system with a single success rate of 90% seems to be very high; but if a business link contains multiple independent key steps, or is required to be correct ten times in a row, the overall reliability will be significantly reduced. The pilot report should also give the success of a single task, the success of continuous operations, the manual takeover rate and the recovery cost after failure.

Assessment perspective	Questions to answer	Applicable stages	Misunderstandings that need to be prevented
Pass@k	Can you succeed at least once in multiple attempts	Capability exploration, model comparison	Does not represent stable production performance
Continuous success	Whether multiple consecutive runs always meet the standards	Business launch, SLA evaluation	Too few samples will overestimate reliability
Manual takeover rate	How many tasks require people to supplement or correct errors	Piloting and operation	Low takeover may come from the risk of omissions
Recovery cost	How long does it take to locate, rollback and rerun after failure	High-impact process	Don’t just count model costs

Table 4-1: Complementary indicators of capacity upper limit and business reliability

4.5.2 From final results to trajectories, boundaries and first error attribution

The end-to-end result evaluation determines whether the task is finally completed; the trajectory evaluation further checks what information the model saw, why a certain tool was selected, whether the permissions are compliant, whether the real receipt is read, and whether it is corrected correctly after failure. Looking only at the final answer will miss “getting it right by accident”, and looking only at the reasoning process may be divorced from the real result, so the two must be combined.

Trajectory prefix evaluation is suitable for testing boundary judgment: truncate the run at a critical node and let the system decide the next step based on the information available at that time. For example, whether the current command covers historical rules, whether dangerous actions should be confirmed again, and whether low-confidence searches should be rejected. It can turn occasional incidents into stable regression use cases.

First error attribution requires identifying the first node in the trajectory that is significant enough to cause a failure and distinguishing between task specification, context, model, tool, environment, permissions, or validation issues. When the root cause is unclear, teams tend to use longer prompts to cover up tool defects, or use stronger models to cover up unreproducible circumstances.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

4.5.3 Establish a sustainable Agent evaluation closed loop

The complete evaluation infrastructure consists of reproducible environments, task datasets, automated verifiers, LLM-as-a-Judge, rubrics, trace recording, and problem analysis. The executable facts are given priority with deterministic verifiers such as code, tests, amounts, and status fields; style, completeness, and policy explanations can be reviewed by models with scoring standards, but the evidence position must be retained and “insufficient evidence, reject scoring” is allowed.

Real failures should be entered into the evaluation set: retained input, environment version, tool output, first error node, final impact and repair method. After the system is modified, the target use case is run first, and then the general regression and safety boundary set are run; the gray scale is expanded only after passing the test. Evaluation is not an acceptance activity at the end of the project, but an ongoing engineering cycle of observation-hypothesis-experiment-validation-new hypothesis.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

4.6 ROI and total cost of ownership

Benefits from agents may come from shortened cycle times, improved coverage, reduced waits, standardized delivery, and freed up expert attention. The cost is not only model fees, but also includes tool construction, environmental maintenance, permission review, evaluation, manual review, failure recovery and extended supply chain management.

The most overlooked are the costs of rework and supervision. If users must review all output line by line, nominal automation translates into a new review burden. A good process will automate verification, focus differences, and explicitly mark high-risk parts, so that humans only review the parts that really require judgment.

The core of ROI is not how much less money is spent on models, but how much total cost each reliable delivery requires.

4.7 Manual intervention, stop conditions and responsibility rollback

Manual intervention should be triggered in at least three types of situations: continuous failure exceeds the threshold, action impact is high and irreversible, and key facts or authorization are unclear. The system should hand over the current goal, completed steps, evidence, errors, and suggested options to avoid people having to understand it from scratch.

Stop conditions not only prevent risks but also control costs. Can be stopped by maximum steps, maximum time, fee cap, number of failures, permission denied, or conflicting evidence. Stopping is not a cover for failure, but an integral part of the reliability of an agent’s system.

Being able to stop at the right time and hand over clearly is an important ability of a mature agent.

5. How to use agents: task engineering and human-agent collaboration

Effective use of agents starts with task engineering, not with a longer prompt. Deliverables, scope, constraints, permissions, validation, and failure handling collectively determine whether a task can be reliably delegated.

The role of humans has also shifted from step-by-step operations to goal setting, key decisions, permission approval and final acceptance, while low-risk and verifiable steps are continued by the system.

5.1 Start with the right task

The best pilot is not the grandest and most important task, but a task with “real value, many steps, controllable risks, and verifiable results.” For example: generate a research report with references based on public information; fix a small range of bugs based on test failures; read work orders and generate processing suggestions to be manually confirmed; summarize changes to the code base every week but do not automatically publish them.

Four questions to ask when selecting a task: Is the final deliverable clear? Are real tools and data available? Is there an objective acceptance method? If an agent makes a mistake, can it be undone or discovered by a human in time? If the answer to more than two of the four questions is no, the scope should be narrowed down first.

An ideal pilot has four conditions: real value, many steps, controllable risks, and verifiable results. Tasks should be small enough to recover from failure, but large enough to demonstrate cross-step value. Publicly available data research, small-scale bug fixes, test completion, and internal reporting are often better than directly automating emails or operating production systems.

First choose a verifiable small closed loop, and then gradually expand the tasks and permissions.

5.2 Upgrade prompts to task instructions

A single cue word is often insufficient for complex tasks. A reliable task specification contains at least six types of information: goals, context, scope, prohibitions, evidence and acceptance, and stopping conditions. It is the equivalent of giving the executive a brief project charter.

A task specification does not require every step to be spelled out. Its function is to clarify the “results and boundaries” and allow the agent to choose its own path within the boundaries. For exploratory tasks, the plan can be allowed to change dynamically, but phased summary and acceptance nodes must be maintained.

The task specification describes the results and boundaries, not all the steps. For exploratory tasks, plan changes can be allowed, but phased summaries must be required; for engineering tasks, verification commands and unmodifiable items must be listed; for business tasks, drafts, suggestions, writing, and external sending must be distinguished.

Field	What to write	Example
Goals	Problems to be solved	Complete special report
Deliverables	Files, formats, audiences	80+ page Word report
Scope	Inclusions and Exclusions	Four Products, Focus on CC/Codex
Constraints	Time, tools, sources, style	Official information is preferred
Validation	How to judge completion	Number of pages, table of contents, rendering checks
Permissions	Allow/Require confirmation/Prohibit	Read-only search; write local files
Failure handling	When to retry or upgrade	Marking when sources conflict

Complex tasks require a micro-project charter, not a wish.

5.3 Five steps from read-only pilot to continuous automation

1. Read-only pilot: Let the agent read, analyze, organize and propose solutions first, without writing to key systems.

2. Controlled writing: Only directories and files necessary for the task are opened, and all changes are reviewed through diff or version control.

3. Tool execution: Allows running low-risk tools such as testing, formatting, and querying, while maintaining manual approval for high-impact actions.

4. Skilling: Encapsulate processes that have been verified many times as Skills, rules or project instructions to reduce re-interpretation each time.

5. Background and parallelism: Enable scheduled tasks, cloud long tasks or multi-agent after the process is stable, and strengthen auditing.

The evidence of “passing” must be defined individually for each level. For example, when upgrading from read-only analysis to controlled writing, you should at least confirm that the agent can correctly identify the file range, will not accidentally modify irrelevant content, can run basic verification, and clearly report failures. Upgrading is a risk decision, not a product feature switch.

Evidence of success is redefined at each level. The read-only stage looks at fact and range identification, the write stage looks at diff and rollback, the tool stage looks at commands and receipts, the skilling stage looks at cross-sample stability, and the background stage looks at unattended failure and auditing. Only when the previous level is stable can you enter the next level.

Privilege escalation should be driven by operational evidence and should not be driven by product feature availability.

5.4 Plans, Milestones and Task Status

The plan should list task breakdown, dependencies, risks, validations, and stopping points. It is not to show that the agent “can plan”, but to allow people to judge the direction before investing a lot of execution costs. High-uncertainty tasks should be scheduled for exploration milestones first, and high-risk tasks should be scheduled for environment and rollback verification first.

Each milestone requires a status update: what was accomplished, what the evidence is, why the plan was adjusted, what are the next steps. Status should be written to a resumable summary or project file, and you cannot rely solely on long conversations.

The value of planning is to expose errors of direction in advance and provide recovery points for long tasks.

5.5 Write verification into “Definition of Done”

The criteria for completing an agent’s task cannot be just “generate content” or “modify code.” Research reports should check that references directly support the conclusions, source dates are valid, and numbers are consistent; engineering tasks should run tests, static checks, formatting, and review diffs; business tasks should check system receipts, field status, and approval records.

It is best to ask the agent to answer four questions upon delivery: What did it do? What is the basis? How to verify? What’s left unfinished or at risk? These four questions can significantly reduce the situation of “surface completion but actual unverification”.

Research report check citation and time, engineering task operation test and review diff, data analysis recalculation formula and caliber, business operation confirmation system receipt and approval. Validation must be part of the task, not an optional additional step after completion.

If there is no “how to know when you get it right” in the task description, it is easy for the agent to only optimize the superficial completion.

5.6 human-agent interaction: Feedback should be directed towards rules, evidence and trade-offs

Effective feedback should indicate which fact is incorrect, what constraint was violated, what evidence is missing, or what trade-offs are needed. For repeated errors, the project instructions or skills should be updated; for mechanical force, it should be changed to Hooks or rules; for one-time preferences, just keep them in the current task.

Users should also require agents to explicitly present uncertainties and rejected solutions. This way, follow-up personnel can understand the decision-making process and determine whether the failure resulted from the input, tool, model, or environment.

High-quality collaboration is not about more rounds of chatting, but about settling corrections into the appropriate system layer.

5.7 Context and long-term task management

More context is not always better. Large amounts of logs, search results, and duplicate files can crowd out a model’s attention. Long tasks should form a short summary at each milestone to record the current goal, completed, key decisions, unfinished items, verification status and next steps. When you need to delve deeper into a certain side of the task, you can use sub-agents to isolate high-capacity exploration.

Stable facts should be kept separate from interim procedures. Project build commands, architectural constraints, and review rules are suitable for writing into CLAUDE.md or AGENTS.md; reusable multi-step processes are suitable for making Skills; session processes and logs are only retained in the current task; automatic memory is more suitable for recording experience and preferences, rather than assuming hard rules that must be executed.

Long tasks should periodically compress logs and intermediate exploration, retaining only goals, key facts, decisions, completed, verification and next steps. Stable project rules are written into CLAUDE.md or AGENTS.md, repetitive processes are written into Skills, and the current session log should not permanently occupy the context.

The goal of context management is to maintain relevance, not to cram all information into the model forever.

5.8 Failure recovery, review and process upgrade

After failure, first differentiate between target misunderstandings, missing information, tool errors, environmental differences, insufficient permissions, and verification gaps. Different reasons correspond to different improvements: clarifying tasks, supplementing data, fixing tools, fixing the environment, adjusting permissions, or increasing acceptance. Simply rerunning the same prompt will often repeat the same problem.

Review should decide where to put improvements: project rules, skills, hooks, tests, sandbox configuration or organizational processes. Only when errors are transformed into reusable improvements can the agent system truly have a learning effect.

Precipitating failure into rules, tests, and skills is more reliable than asking the model to “pay attention next time.”

6. Critical infrastructure: context, memory, tools and multi-agent

To undertake long-term, repetitive and parallel tasks, agents must rely on infrastructure such as context management, hierarchical memory, tool interfaces, skill assets and multi-agent orchestration.

These mechanisms respectively solve the problems of information supply, state continuation, real action, method reuse and task division. They also bring new versions, permissions and coordination costs.

6.1 Context Engineering: Determining what the current task sees

Context is the work material that the agent can currently directly use, including user goals, system instructions, project rules, read files, tool results, and stage summaries. Contextual quality depends on relevance and structure, not sheer length. Reams of duplicate logs, irrelevant documentation, and broad tool descriptions dilute the information that really matters.

Context engineering answers four questions: which content must always be loaded, which content is loaded by file path, which content is only loaded when the task is triggered, and which content should be read by the sub-agent and returned with a summary. Stable and short rules are placed in the resident layer, domain materials and long processes are loaded on demand, and side tasks use isolated contexts.

Condensation and summarization should occur at natural milestones. The summary at least retains the goal, completed, key facts, decision, verification, unfinished, and next step; if only “done a lot of work” is retained, subsequent agents cannot reliably recover.

Context engineering pursues high signal-to-noise ratio and recoverable states, rather than the maximum amount of input.

6.1.1 Stable prefix, dynamic trajectory and task status bar

The context can be divided into stable prefixes and dynamic trajectories. The stable prefix includes system rules, core tool definitions and project-level constraints. It should be kept byte-level stable as much as possible to reuse KV Cache or server-side Prompt Cache; dynamic tracks append user messages, tool calls, results and stage summaries according to time. Frequently rewriting prefixes or cramming all tools into the context at once increases both cost and selection noise.

Long tasks also require a short, structured task status bar that continuously records goals, current stage, verified facts, open issues, risks, budget, and next steps. Rather than replacing the full trajectory, it allows the model to quickly recover “where it is now” at each round. The status bar should be updated by Harness based on the real operation to avoid relying solely on model free summary.

6.1.2 Compression, isolation and progressive disclosure are three different methods

Compression is suitable for completed history, folding a large number of processes into goals, decisions, evidence and remaining work; context isolation is suitable for search, long log analysis or special review, allowing sub-agents to return only conclusions and evidence; progressive disclosure is suitable for tools and Skills, loading the metadata directory first, and then reading detailed instructions or scripts after hitting the task. The three handle the contextual growth of time, task and capability catalogs respectively.

Roots of trust must be preserved before compression: current user directives, permission boundaries, outstanding commitments, key origins, and verification status. If the abstract only retains the conclusion and does not retain the location of evidence, subsequent rounds will mistake unverified intermediate judgments as facts.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

6.2 Persistent instructions: write stable rules to each task

Persistent directives are good for storing repository structure, build commands, coding style, don’ts, and delivery formats that you need to know every time you work on it. They should be short, actionable, versionable, and reviewable by the team. Vague statements such as “pay attention to quality” should be rewritten into specific actions and verification commands.

Documents at different levels should be consistent. Global rules define a common bottom line for an individual or organization, project rules define repository conventions, and subdirectory rules handle specific languages ​​or components. When conflicts arise, rely on clear priorities and more specific scope, rather than copying the same sentence into multiple places.

Persistent directives are not enforcers. If the rule must be unconditionally true, it should be checked with permissions, command rules, Hook or CI. Otherwise it remains within the context that the model needs to be interpreted and followed.

Persistent instructions preserve “know it every time”, and enforcement mechanisms guarantee “must do it every time.”

6.3 Memory stratification: working memory, project facts and long-term experience

Working memory serves the current task, project facts are maintained by humans and put into version control, automatic memory distills preferences and experiences from history, and conversational retrieval looks up past processes when needed. The writers, life cycles and credibility of the four are different and cannot replace each other.

Memory needs governance. Users should be able to view, correct, delete and export; secrets should not enter long-term memory; automatically generated experiences must have a source and update time; hard rules cannot only exist in model memory. Expired memory will cause hidden errors. The longer the memory, the more cleaning and conflict handling are needed.

The ideal memory writing has clear triggers: recurring project facts go into the rules file, proven multi-step methods go into the skill, personal preferences go into the user memory, and the in-betweens of the current task are archived or compressed after the task is completed.

Hierarchy	Content	Life cycle	Governance points
Session state	Current plan, tool results	Single task	Prevent context overflow
Project memory	repository/process agreement	Project period	Editable and traceable
User preferences	Stable expression and collaboration preferences	Cross-task	Requires confirmation and error correction
Organizational knowledge	Policies, standards, cases	Long term	Versions and permissions
Skill memory	Reusable steps and scripts	Long term	Test, release, rollback

The value of memory is not in “never forgotten”, but in being traceable, correctable and used on demand.

6.3.1 Distinguish between user memory and organizational knowledge base

User memory serves the individual scale, saving preferences, relationships, historical decisions, and proven habits; the knowledge base serves the team or organizational scale, saving policies, products, projects, and professional materials. The former emphasizes individual control, privacy, and right to correction, while the latter emphasizes source, permissions, versions, coverage, and consistency. Both can be retrieved, but write policies and governance responsibilities are different.

The reliable memory life cycle includes: reading according to relevance at the beginning of the task; recording candidate facts during operation rather than writing immediately and permanently; judging whether to add, update, delete or remain unchanged based on source, time, conflict and privacy in the background; retaining the provenance and verification time after writing. Simple notes are suitable for atomic facts, structured cards are suitable for updateable attributes, narrative records are suitable for event background, and procedural experiences are more suitable for Skills or executable scripts.

6.3.2 RAG is a retrieval system and a knowledge organization system

The basic RAG link includes collection and cleaning, semantic segmentation, indexing, retrieval, rearrangement, generation and reference. Dense retrieval is good at semantic approximation, while sparse retrieval such as BM25 is good at proper nouns and precise word forms. Production systems often use mixed retrieval and then rearrange. Chunking should not only be cut by a fixed number of words, but also preserve metadata such as title, path, time, document version, and access control.

When data size and problem complexity increase, flat vector indexing is not always sufficient. Hierarchical summarization can support drilling down from global topics to the original text, knowledge graphs are suitable for entity relationships and multi-hop problems, and file system directories can be used as natural interpretable levels. A practical principle is to place short, stable structural overviews in context, and to load long, thin pieces of evidence on demand through retrieval.

Agentic RAG hands retrieval as a tool to the Agent: the system first analyzes the problem, rewrites the query, selects a knowledge source, and then determines whether to complete the search, change filter conditions, read the full text, or stop based on the results. It improves the coverage of complex problems and also introduces more loops and costs, so it must be evaluated jointly with recall, citation correctness, rejection performance and retrieval trajectory. Section 11.11 gives a project path to implement this architecture using Codex.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

6.3.3 Knowledge writing, updating and publishing require independent management

The knowledge base is not a one-time import project. Incremental updates are responsible for absorbing new documents and new evidence, and periodic sorting is responsible for deduplication, merging conflicts, eliminating expired content, and rebuilding indexes; for high-impact knowledge such as policy, medical, legal, financial, or production operations, candidate updates should be independently reviewed before being released.

Retrieval permissions must be performed based on user identity and document ACL on the backend and cannot rely solely on Prompt reminders. Deletion, withdrawal and version replacement should simultaneously affect the original text, index, cache and citation; the answer should return the source title, positioning information and version time. If no sufficient basis is found, the answer should be explicitly rejected.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

6.4 Skills: Turn procedural knowledge into reusable abilities

Skills are suitable for encapsulating repetitive processes and domain knowledge that require judgment, such as code review, publishing, research retrieval, and data cleaning. A complete Skill should contain a clear description, input requirements, steps, output, verification, stopping conditions, and reference templates, scripts, and examples as needed.

The trigger description determines whether the agent will correctly select the Skill. If the description is too broad, it will be triggered by mistake; if the description is too narrow, the scene will be missed; if the boundaries of multiple Skills overlap, it will make selection difficult. Teams should build examples using real tasks to test automatic triggering, manual invocation, failure handling, and contextual costs.

Figure 6-1: Context loading method of persistent instructions, Skills, MCP, Subagents and Hooks

Source: Claude Code official document, Extend Claude Code (retrieval date: 2026-08-10)

Scripts and dependencies in Skills are part of the supply chain. The source and execution content of third-party Skills must be reviewed before installation. Internal Skills must be versioned, responsible persons designated, and change records kept.

Skills are procedural memories and software assets that need to be tested and managed.

6.5 Tools, MCP and external system connections

Tools transform model decisions into verifiable actions. Data tools read information, action tools change systems, and orchestration tools manage tasks and delegation. Tool names, descriptions, parameters, permissions, and error returns must be clear; too many similar tools will make selection difficult, and broad tools will increase risks.

MCP provides standard connections for external services, but the connection itself does not equal security. Authentication, authorization scope, transport, data retention, tool write capabilities, output size, and auditing need to be evaluated. Content from MCP may also carry hint injection and should be considered untrusted data.

Skills and MCP solve different problems: MCP provides connections, and Skills provide the knowledge and processes on how to effectively use connections. When combining the two, permission control should be left at the tool layer, and business steps and acceptance should be left at the Skill layer.

Tool Levels	Typical Permissions	Default Controls
L0 read-only	Search and read public information	Automatic
L1 local revocable	Write workspace, run tests	Sandbox + log
L2 external writing	Work orders, databases, email drafts	Range limit + readback
L3 high impact	Release, payment, deletion, permission change	Strong approval + double review

The tool determines the scope of reality that the Agent can touch, and also determines the possible impact of errors.

6.5.1 Five types of tools and two design routes

Tools can be divided into five categories according to system functions: awareness tools read web pages, files, databases and multi-modal input; execution tools modify the environment; collaboration tools delegate or exchange structured messages; event trigger tools send timers, webhooks and external state changes into the runtime; user communication tools are responsible for clarification, approval, progress and delivery. The first two categories determine the basic action closed loop, and the last three categories determine whether the Agent can work asynchronously, long-term, and collaboratively.

There are two complementary lines of tool design. High risks, strong compliance, and high-frequency fixed actions are suitable for special tools, which use narrow parameters, server-side authentication, and clear receipts to reduce the degree of freedom; exploration, data conversion, and long-tail combinations are suitable for general-purpose executors and skills, and use code to combine multiple simple primitives into new capabilities. The choice is not based on “which one is more intelligent”, but whether the results can be verified, whether the actions can be limited, and whether failures can be recovered.

6.5.2 The tool contract must make the model parameters consistent with the real execution parameters

A usable tool contract should define name, purpose, input types, value ranges, default values, permissions, timeouts, idempotence, side effects, structured returns, and error codes. The Schema seen by the model must be consistent with the actual execution of the server; if the description is written as “read-only” but the implementation allows writing, or the parameter name is reinterpreted in the gateway, all upper-layer security judgments will be invalid.

After the number of tools increases, active discovery and progressive disclosure should be adopted: first let the Agent discover related servers or tools based on natural language requirements, and then load only the complete definitions of a small number of candidates. MCP unifies the discovery and calling interfaces, but does not automatically solve tool selection, version compatibility, identity authorization and prompt injection. These are still the joint responsibilities of Harness and the server.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

6.6 Hooks and deterministic control

Hooks are suitable for actions that must occur regardless of the model’s judgment, such as formatting after editing, blocking dangerous prefixes before commands, session end record summaries, additional review before permission requests, and re-injecting key states after context compression. Its value is to convert natural language suggestions into deterministic mechanisms.

Hooks themselves can also fail or be abused. Timeouts, exit codes, concurrency, and logs need to be handled; Hooks from plug-ins are equivalent to automatically executing code, and scripts, networks, and secret access should be reviewed. All complex business judgments should not be stuffed into Hooks, otherwise it will form an implicit workflow that is difficult to observe.

The mechanical actions that need to be guaranteed to occur are handed over to Hook, and the processes that need to be understood and made are handed over to Skills or AI agents.

6.7 Subagents, Agent Teams and Delegation Boundaries

Subagent is suitable for reading a large amount of material, performing independent testing or professional review, and returning the summary to the main session; Agent Team is suitable for multiple independent sessions to share tasks, communicate with each other and negotiate with each other. The former emphasizes context isolation, while the latter emphasizes collaborative autonomy.

Delegation instructions should include goals, inputs, tools, editable scope, output format, and acceptance. The main agent cannot delegate final responsibility altogether; it has to check the subresults for evidence, conflicts, and coverage, and decide whether to merge them.

The amount of concurrency should be determined by task separability, model cost, environment resources, and coalescing capabilities. Multiple agents jointly modify a set of strongly coupled files, often resulting in more rework than sequential execution.

Delegation isolates context and responsibility boundaries rather than duplicating more chat windows.

6.7.1 Multi-Agent classification axis: context and topology

Multi-Agent systems can be understood along two axes. In terms of context, the complete history can be shared, structured task status can be shared, or results can only be exchanged after complete isolation; in terms of topology, sequential pipelines, managers-workers, peer-to-peer negotiation, or decentralized networks can be used. The more sharing, the lower the handover costs but the higher the risk of contamination and leakage; the stronger the isolation, the clearer the specialization but requiring stricter interfaces.

The condition for multi-Agent to be truly valuable is to obtain a parallel environment, independent information sources, professional tools or mutual verification that is difficult for a single agent to obtain at the same time, rather than simply adding inference copies. If all actors read the same context, use the same model, and repeat the same path, the result may only amplify common blind spots.

6.7.2 Collaboration failure needs to be handled explicitly in the design phase

Common failures include concurrent write conflicts, error cascading, homogeneous conclusions, buck-passing, communication loops, and “understanding debt.” Understanding debt refers to the increasing difficulty for humans to explain how multiple agents form the final result; it accumulates as the level, message volume, and implicit state increase.

Controls include: setting owners or locks on shared resources; replacing free chat with structured task orders and product interfaces; limiting delegation depth, rounds, and budgets; letting key roles use differentiated information sources; requiring managers to take responsibility for verification of merged results; and retaining cancellations, timeouts, and manual takeovers. Multi-Agent optimizes information acquisition, execution parallelism and independent verification, and cannot replace clear task decomposition.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

6.8 Worktree, cloud environment and background operation

Worktree provides an independent working copy in the same Git project to resolve local parallel modification conflicts; the cloud environment provides a remote, configurable and sustainable execution space; background tasks focus on temporal persistence. The three can be combined, but the problems they solve are different.

The isolated environment must have the dependencies, configuration, and testing required to complete the task. Ignored files, secrets, and local services cannot exist by default; environmental differences should be managed through setup scripts and explicit variables. There should be a process for handover, review, branching, and cleanup after tasks are completed.

Parallel reliability comes from environment and file isolation first, and model concurrency second.

6.9 Observability: Logs, Traces, Costs and Audits

Observability includes, at a minimum, input versions, schedule changes, tool calls, permission requests, file differences, external receipts, test results, models and costs, human intervention, and final status. Without this information, failures cannot be located and successes cannot be reproduced.

Logs should be distinguished between debugging, auditing and business evidence. Debug logs can be detailed but have a retention period; audit records must be tamper-proof and associated with identities; business evidence should directly support delivery conclusions. Sensitive data requires desensitization and access control, and full context cannot be saved indefinitely for observability.

Agents without running evidence can only be demonstrated and cannot be governed.

7. Security and governance: Align capabilities, authority and responsibilities

Agent security is not an end-point audit after task completion, but a continuous control process throughout input, planning, execution, verification, release and review.

Untrusted content, identity and permissions, network access, secret data, extended supply chain, approval, auditing and rollback need to form an interconnected governance chain.

7.1 Threat model: who can input, what the agent can do, and who the results affect

Agent risks arise from the combination of data and permissions. The same model has lower risk in read-only public profile tasks, but completely different risks when connecting mailboxes, terminals, and production systems. The threat model should map out how external content enters the context, what tools the model can call, how credentials are provided, and where results are written.

In particular, distinguish between trusted instructions and untrusted content. User and organizational policies can authorize actions. Web pages, emails, issues, and documents only provide data and cannot be upgraded to authorization just because “please execute” appears in the content.

Before we can discuss agent security, we must first draw a clear picture of the data flow and permission flow.

7.2 Minimum permissions and capability classification

At the beginning of the task, only the necessary directories, commands, domain names and accounts are opened. Reading does not mean risk-free, and reading sensitive data may also lead to leakage; writing must be distinguished between drafts, test environments, and formal systems; outgoing, payment, deletion, and production changes must be controlled separately.

Permissions can be enlarged step by step with evidence, but cannot be permanently upgraded at the discretion of the agent. Short-term tasks use short-term credentials and temporary scope, and are revoked after completion; multiple projects use independent environments and identities to prevent an agent from crossing irrelevant boundaries.

Delegation should be minimized around the current task, not maximized around product functionality.

7.3 Sandbox, Approval, Rules and Rollback

Governing agents need to distinguish four concepts. Permissions determine what identity the agent has; the sandbox determines which files, processes, and networks can technically be accessed; approval determines who confirms when crossing boundaries; and audit records describe what has been done and what tools and data have been used. They are complementary but not substitutes for each other.

The principle of least privilege should be implemented into specific configurations: only open the directories, commands, domain names and accounts necessary for this task; close unnecessary networks by default; provide sensitive credentials in a short-term, limited-scope manner; set up manual confirmation for payment, publishing, deletion, external sending and production operations; retain diff, logs or system receipts for all important changes.

Bottom line of governance｜Sandbox answers “what can be technically touched”, and approval answers “who will confirm when crossing the boundary”. Both must be designed simultaneously; relying solely on model recognition prompt injection or self-restraint is not enough.

The sandbox answers “what is technically accessible”, the approval answers “who confirms when crossing the boundary”, the rules allow, ask or prohibit specific commands or tool prefixes, and the rollback ensures recovery after an error occurs. The four together form a closed loop, and no one item can replace the others.

Low-risk baselines are typically workspace writes, restricted networks, and on-demand approvals. Full access plus never approval removes primary control and should only be used in clearly isolated, recoverable, and authorized environments. Automatic approval changes the reviewer and should not be misinterpreted as expanding the sandbox boundaries.

Stage	Critical Controls	Evidence
Input	Source trust, injection isolation, sensitive classification	Input tags
Planning	Scope, authority, stopping conditions	task specification
Execution	Sandbox, Least Privilege, Approval	Tool Log
Verification	Independent inspection, readback, reconciliation	Testing and acceptance
Release	Manual approval, rollback preparation	Approval record
Review	Failure classification, indicators and improvements	Evaluation report

Boundaries, approvals, rules, and rollbacks are four different control planes.

7.4 Credentials, Privacy and Data Lifecycle

Credentials should be provided on demand via the environment or secret manager to avoid writing prompts, project files, memories and logs. Use minimal scope, short-lived, revocable tokens, and let different tools have different identities. Configure the local and cloud environments separately, and full personal credentials cannot be copied.

Data governance also includes input sources, cross-border and compliance requirements, supplier retention policies, training usage, export and deletion. Automated memories, session histories, and tool logs may hold sensitive content and must have mechanisms for review, correction, retention, and cleanup.

The agent’s data boundaries cover context, tools, memories, logs, and providers, not just input boxes.

7.5 Scaling and Supply Chain Security

Extensions can bring directives, scripts, binaries, dependencies, network connections, and automatically triggered behaviors. The review should focus on sources, maintainers, versions, release records, permissions, data flows, installation scripts, and update mechanisms. What is installable on the market does not equal what the organization allows.

Internal extensions also require code review, testing, signing or verification, version locking, owners, and rollbacks. High-privilege environments should use allow lists and verify updates first in an isolated environment to avoid automatic upgrades that change behavior.

The agent extension ecosystem should be managed as a software supply chain, not as a collection of prompt materials.

7.6 Audit, Accountability and Incident Response

Each key action should be able to answer who initiated it, which agent and model was used, what data was read, what tool was called, what approval was obtained, what object was modified, and how to verify. Outbound delivery and production changes also need to be associated with the business person in charge.

Incident response plans should cover credential leaks, accidental deletions, unauthorized writes, malicious extensions, memory pollution, and out-of-control background tasks. Requires the ability to immediately stop tasks, revoke tokens, isolate environments, restore data, preserve evidence, and review control gaps.

Responsibility cannot be attributed to “AI”; organizations must maintain traceable chains of authorization and disposition processes.

7.7 Organizational Policies and Managed Configurations

Organizational policies should define available products and models, allowed data to connect to, default sandboxes and approvals, extension sources, background task permissions, log retention, and high-risk actions. Policies must be implemented through managed configuration and technical controls, not just training documents.

At the same time, an actionable exception process must be provided. While excessive rigidity pushes employees to bypass controls, clear mechanisms for application, review, time limits, and revocation make governance an efficiency infrastructure.

Good governance uses default configurations to reduce risk and exception processes to maintain business speed.

7.8 Governance maturity from pilot to production

Personal experiments focus on public or desensitized data, read-only and isolated environments; team pilots establish task templates, project rules, extended review and manual acceptance; controlled production adds identity, auditing, incident response and service responsibilities; the scale-up phase is unified through managed configuration, evaluation platform and asset catalog.

Each stage has exit conditions. If the task success rate is unstable, the failure is not visible, the extension is not maintained, or the permissions cannot be reclaimed, it should not enter a higher stage. Governance maturity needs to increase simultaneously with the degree of autonomy.

Stage	Features	Upgrade conditions
Explore	Personal trial, manual step-by-step confirmation	Task templates and basic logs
Controlled pilot	Limited data/tools/team	Stable evaluation and failure classification
Process integration	Access to real systems, exception approval	Identity, auditing, rollback
Scale operation	Multi-team, multi-agent, continuous monitoring	Responsibility system and portfolio governance

First establish controllable operation, then expand coverage; first prove stability, then reduce labor.

8. OpenClaw in-depth analysis: self-hosted multi-channel personal agent

OpenClaw combines message channels, Gateways, Agents, tools, memories and continuous operations into a long-term AI agent portal that can be controlled by individuals.

Its value lies not in providing another chat interface, but in enabling users to initiate tasks from their mobile phones or daily messaging platforms, and have an Agent in a self-hosted environment continue execution and return results. At the same time, always-on, remote triggering and host permissions also make it necessary to manage it as an infrastructure rather than an ordinary application.

8.1 Product positioning: multi-channel entrance for persistent personal agent

OpenClaw officially describes the product as a self-hosted gateway that connects AI agents across channels such as Discord, Google Chat, iMessage, Matrix, Microsoft Teams, Signal, Slack, Telegram, WhatsApp, Zalo, and more. A Gateway can serve multiple channel plug-ins at the same time and deliver messages to the corresponding Agent.

This positioning separates the model from the runtime. The model is responsible for understanding and reasoning, and OpenClaw is responsible for entry, session, routing, tool connection, status, and operation and maintenance. Users can choose different model providers and assign different Agents to different workspaces or senders.

Therefore, when evaluating OpenClaw, you can’t just ask about the quality of the answers, but also whether the Gateway is stable, whether the channel identity is reliable, whether the session can be isolated, how the background tasks are restored, whether the tool permissions are controllable, and whether the deployer can maintain it for a long time.

OpenClaw’s core product is a personally controllable Agent Gateway, not a new base model.

Data basis: OpenClaw official documents Overview, Agents, Capabilities, Gateway Security, retrieved as of 2026-08-10.

8.2 Core architecture: Channels—Gateway—Agent—Nodes

The basic link of OpenClaw can be summarized as Channels → Gateway → Agent. Channel plug-ins are responsible for receiving and sending messages. Gateway is the single source of truth for sessions, routes, and channel connections. Agents combine models and context to determine actions and invoke tools, Skills, or Plugins.

Gateway also connects CLI, Web Control UI, desktop and mobile Nodes. The Control UI provides chat, configuration, session, and node management; nodes expose specific device capabilities to the Gateway through pairing. The entire system is both a data plane and a control plane.

Figure 8-1　OpenClaw uses Gateway to connect message channels, Agents, control interfaces and mobile nodes

Source: OpenClaw official document, Overview (retrieval date: 2026-08-10)

Layers	Main responsibilities	Key objects	Governance focus
Entrance layer	Receiving and returning messages	Channel plug-ins, WebChat	Sender and group allow lists
Control layer	Sessions, routing, connections and state	Gateway, Control UI	Authentication, network exposure, logging and recovery
Agent layer	Understand tasks and organize calls	Agent, session, workspace	Context, isolation and model configuration
Capability layer	Execution actions and reuse processes	Tools, Skills, Plugins	Minimum permissions, sources and versions
Device layer	Providing remote device capabilities	Desktop and mobile Nodes	Pairing, tokens, and execution approvals

The advantage of this structure is that channels and capabilities can be expanded independently, but the disadvantage is that the Gateway becomes a high-value single point. Configuration files, channel tokens, model keys, plugins, and logs are all centralized around it, and deployment requires backup, update, monitoring, and recovery solutions.

Understanding the control plane status of Gateway is the starting point for understanding OpenClaw’s capabilities and risks.

8.3 Channels, sessions and multi-agent routing

Multi-channel solves the problem of where users reach the Agent. Multi-session and multi-agent routing solve the problem of which trust boundary different inputs enter. Work accounts, personal accounts, home groups, and public communities should not share the same context, memory, and tool permissions by default.

OpenClaw can organize isolation sessions by agent, workspace, sender and other dimensions. A reasonable approach is to have separate Agents and working directories for different uses, and to set up allow lists and matching processes for private messages, groups, and channels. Facts that need to be shared across channels should be written explicitly rather than directly merging the entire history.

Multi-Agent routing can also separate research, home automation, engineering tasks, or operational notifications. However, routing labels are only selectors and cannot replace authentication; who can trigger high-authority tools is still determined by channel identity, Gateway permissions, and tool policies.

Multi-channels bring accessibility, and multi-session and multi-Agent can transform accessibility into controllability.

8.4 Tools, Skills, Plugins and Capability Extensions

OpenClaw divides extensions into different levels. Tools are structured actions that can be called by the model, such as files, terminals, web, browsers, messages, automation, and node operations; Skills provide on-demand loading work methods to Agent; Plugins can add tools, channels, model providers, Hooks, and other runtime code.

This layering helps to use the principle of minimal surface: one-time actions use existing Tools, stable processes are precipitated into Skills, and Plugins are only introduced when new runtime code or distribution is really needed. Not every need should be solved by installing a new extension.

Scaling is also a major supply chain risk. Skills may reference scripts and external dependencies, and Plugins run within the Gateway process. Teams should use source allow lists, pin precise versions, review unpacked content, isolate testing, and update regressions to avoid automatically treating community directories as trusted.

Expand the upper limit of ecological improvement capabilities and bring the security of the software supply chain into personal intelligence.

8.5 Memory, Workspace and Automation

Long-term operations require separation of the current session, stable workspace, and long-term facts. The session saves the task process, the workspace provides files, instructions, and tool environments, and the memory should only save information that is valuable and confirmed across sessions. Loading the entire history forever wastes context and amplifies false memories.

Cron, webhooks and background events enable OpenClaw to generate scheduled briefings, reminders, patrols or message forwarding. The key to automation is not whether it can be triggered, but whether the trigger source is trustworthy, whether the task is idempotent, whether failure is notified, whether retries are limited, and whether the results can be read back.

For long-term tasks, the minimum state required for the next run should be saved, stopping conditions and manual takeover points should be clear. Tasks involving external sending, file modification, or equipment control should be retained for approval or review before action.

The stronger the long-term state and background operation, the more observable, undoable and auditable it needs to be.

8.6 Control UI, CLI and mobile nodes

Control UI is the management portal in the browser for chat, configuration, sessions, and nodes; CLI is responsible for installation, onboarding, diagnostics, security auditing, and automation; desktop or mobile Nodes provide device-side capabilities through pairing. The three share the Gateway state, but are oriented to different operators and risks.

Nodes can expand scenarios such as camera, voice, Canvas or system execution, which also means that Gateway can remotely call real devices. Node pairing, token rotation, offline queues, command approval, and device removal must form a unified process.

Figure 8-2: Control UI/mobile interactive interface shown in OpenClaw official document

Source: OpenClaw official document, Overview (retrieval date: 2026-08-10)

The control interface should not be directly exposed to the unprotected public network. Prefer remote access using a private network or controlled tunnel, and verify reverse proxy, authentication, and WebSocket configurations. The management entrance and the ordinary message entrance should use different permissions.

Nodes are not ordinary clients, but remote capability boundaries that Gateway can call.

8.7 Usage path: from single-channel read-only pilot to permanent operation

The current official fast path includes installing OpenClaw, running onboarding and starting the Gateway service, and then interacting through the dashboard or a messaging channel. Commands and platform requirements change rapidly, and the actual deployment should be based on the official documents and release notes at that time.

A more prudent approach to adoption is divided into four steps. The first step is to use a separate system account, a separate directory, and a channel to only connect low-risk models with read-only tools; the second step is to establish sender allow lists, session isolation, and logs; the third step is to enable a small number of vetted Skills, memories, or cron; and the fourth step is to evaluate write operations, nodes, and more channels.

Each step must be completed with real tasks to record quality, manual intervention, continuous operation, fault recovery and safety incidents. Verify backups, stop services, revoke tokens, and restore configurations before scaling rather than waiting until the Gateway goes out of control and then designing emergency procedures.

OpenClaw should be enabled in stages by permissions and operating radius, rather than connecting all personal environments at once.

8.8 Security model, applicable scenarios and boundaries

The official OpenClaw security documentation explicitly adopts the personal assistant trust model: each Gateway corresponds to a trusted operator boundary. It is not an adversarial multi-tenant security boundary where multiple mutually distrustful users share the same Agent and host permissions.

If you have mixed trusts or different organizations, you should split the Gateway, credentials, and OS users or hosts first. Channels use private messages and group permission lists, tools follow least privileges, perform setup approvals, plug-ins have fixed sources and versions, and the network remains minimally exposed.

Official inspection methods such as openclaw security audit, –deep, –fix and –json are provided to discover open group policies, tool explosion radius, execution approval drift, network exposure and browser control risks. Audits are diagnostic tools and are not a substitute for an organization’s own threat model.

OpenClaw is suitable for personal assistants, cross-channel task portals, controlled home or work automation, and remote triggering of engineering agents; it is not suitable for direct use as a public high-privilege robot without isolation and dedicated governance.

OpenClaw’s value comes from persistent connectivity, and its main risk comes from persistent connectivity.

9. Hermes Agent in-depth analysis: learnable open personal agent

Hermes Agent emphasizes the accumulation of experience through persistent memory, Context Files and automatically generated or improved Skills, so that an open, multi-model Agent can form stronger personalization and programmatic reuse in long-term use.

Compared with OpenClaw’s Gateway-centric route, Hermes emphasizes learning loops, open tools, and multiple execution backends. It also covers message channels and ongoing tasks, but “self-improvement” does not mean that there is no need for supervision; the more memories, skills, and tools can change subsequent behavior, the more reviews, versions, and safety boundaries are needed.

9.1 Product positioning: open personal agent that can accumulate experience

Hermes Agent is open sourced by Nous Research and is officially described as a self-improving AI agent with a built-in learning loop: Create Skills from experience, improve during use, and write information worth retaining into cross-session memory. It’s not a coding copilot that only relies on an IDE, nor is it a chat shell centered around a single API.

It is important to understand this positioning accurately. The so-called improvements mainly occur at the context selection, memory, procedural Skills and tool orchestration layers. It does not mean that the model weights will be automatically trained locally, nor does it guarantee that every automatic write will improve the quality.

Figure 9-1 CLI interface layout in Hermes Agent official documentation: status banner, conversation area and fixed input area

Source: Hermes Agent official documentation, CLI Interface (retrieval date: 2026-08-10)

As of August 10, 2026, the official product page and GitHub release label the current version as v0.20.0. This release continues to expand voice, quote, A2A, webhook, and desktop capabilities, and illustrates that the product surface is still changing at a rapid pace.

The core value of Hermes is to turn personal experience into reusable assets; the core risk is that wrong experiences may also be amplified in the long term.

Data basis: Hermes Agent official documentation, Nous Research product page and GitHub v0.20.0 release, retrieved as of 2026-08-10.

9.2 Running state: CLI, desktop, message gateway and remote environment

Hermes offers a CLI, desktop application, and messaging gateway accessible from platforms such as Telegram, Discord, Slack, WhatsApp, Signal, Email, and more. It can run on a PC, VPS, GPU cluster or remote execution backend and connect to multiple model providers.

Different interfaces are not independent products, but entrances to the same Agent state. Configuration, context, memory, skills, tools, and scheduled tasks all work together to determine behavior. Maintaining continuity across interfaces is valuable, but it is also necessary to avoid sharing borderless states of personal, work, and team identities.

Layer	Main Responsibilities	Represented Objects	Main Risks
Interaction layer	Unified multi-entry session	CLI, Desktop, message gateway, Voice	Identity obfuscation and context threading
Knowledge layer	Saving facts, background and procedures	Memory, Context Files, SOUL.md, Skills	Accumulation of errors and leakage of secrets
Action layer	Calling tools and external systems	Tools, MCP, Browser, Code	unauthorized access, side effects and supply chain
Collaboration layer	Delegation and continuous operation	Subagents, Cron, Webhooks, A2A	Unattended and consolidated responsibilities
Execution layer	Isolated running environment	Local, Docker, SSH, Singularity, Modal	Environment drift and boundary configuration

Open models and execution environments facilitate experimentation, cost control, and customization, but can also introduce model differences, dependency drift, and run reproduction issues. The team pilot should fix the version, model, tool set and backend, and then compare the task effects.

The openness of Hermes expands the scope of deployment and makes environment management a part of use.

9.3 Memory, Context Files and SOUL.md

Hermes separates long-term facts, project context, personalities and historical processes into different assets. Memory holds user, project, or preference information that remains valuable across sessions; Context Files provide a stable context for a specific working directory; SOUL.md describes the default personality and expression; and History Search is used to review past sessions when needed.

This layering avoids loading the entire history for each conversation. Stable rules are placed in reviewable files, factual memories are retrieved on demand, and temporary task processes remain in session. What really matters is not how much you can save, but whether you can retrieve the right facts at the right time.

Figure 9-2 File division and capacity boundaries of Hermes Agent’s built-in persistent memory

Source: Hermes Agent official document, Persistent Memory (retrieval date: 2026-08-10)

Memory writes must allow viewing, correction, and deletion. Secrets, short-term state, unverified inferences, and data that may belong to another project should not enter long-term memory; team scenarios also need to preserve source, time, and owner.

The quality of memory depends on selection, verification and error correction, not the quantity saved.

9.4 Skills and built-in learning loops

Skills is Hermes’ procedural memory layer. Agents can generate Skills from successful experiences, or they can improve existing Skills during use, solidifying tool sequences, inspection steps, scripts, templates, and failure handling into methods that can be reused in the future.

Programmatic reuse can significantly reduce repeated exploration, but it can also turn accidental successes, dangerous commands, or outdated dependencies into long-term behavior. A high-quality skill should have clear trigger conditions, inputs, outputs, permissions, verification, owners, versions, and rollbacks.

Automatically created skills should first enter candidate or review status, be tested with representative tasks in an isolated environment, and then be promoted to default capabilities. If the memory or skill is corrected, a regression task should be used to verify that the error has truly disappeared.

Only when the learning cycle is combined with evaluation and version management can it become reliable reuse.

9.5 Tools, MCP, browsers and multi-modal capabilities

Hermes official documentation currently lists more than 60 built-in tools and covers tasks such as terminal, file, web search, browser, code execution, image and voice. Toolsets can be enabled on a per-scenario basis to avoid indiscriminately exposing all capabilities to the model.

MCP allows Hermes to connect to external tools and data sources. Each MCP server should use dedicated identities, narrow parameters, and minimal permissions; text from external systems is still untrusted input and cannot be treated as system commands just because the entry tool returns.

v0.20.0 enhances live voice, provenance citations, A2A v1.0, signed outbound webhooks and desktop artifacts. These capabilities expand interaction and collaboration, but still require individual verification of stability, data flow, and permissions when adopted by organizations.

Tool breadth determines what can be done, permissions and validation determine whether it should be done.

9.6 Subagents, message channels and scheduled tasks

Subagents from Hermes can undertake research, testing and data processing in independent sessions, terminals or scripts, reducing main context noise. The main agent is responsible for splitting targets, providing materials, setting tool boundaries, and validating returned results.

A messaging gateway lets users initiate tasks and receive results remotely from their daily platform, and natural language scheduling schedules reports, backups, and briefings. Delegation, messaging and Cron combine to form continuous work, but also connect identity, delivery and error recovery.

Unattended tasks should use a narrow toolset, limit the maximum run time and number of retries, and set clear notifications. When multiple Subagents operate on a shared directory, independent workspaces or clear read/write scopes should be used, and the final merge responsibility must be borne by the main task.

Delegation and scheduling improve throughput, but cannot replace task boundaries, evidence, and responsible persons.

9.7 Sandbox, Approval and Security Boundaries

Hermes supports local, Docker, SSH, Singularity, Modal and other execution backends. Different backends determine how file systems, processes, networks, GPUs, and secrets are exposed; backend selection should be based on task risk and reproducibility requirements, not just launch convenience.

Command approval, authorization, DM pairing and container isolation control different levels of risks respectively. The sandbox restricts the environment, allowing approval to retain human decision-making, channel authorization restricts who can initiate tasks, and secret management restricts what credentials can be obtained during execution.

For high-impact tasks, it is recommended to use dedicated accounts and isolated backends, default to read-only or restrict writing to the root directory, tighten outbound networking, and log tool calls and manual approvals. Scheduled tasks should also verify stop and resume to prevent background tasks from continuously retrying after failure.

The sandbox provides environmental boundaries and approvals provide decision boundaries, and both must be designed simultaneously.

9.8 Usage path, migration and selection suggestions

With Hermes, you can start with the CLI and read-only tools: fixed models, project directories and Context Files, establish a quality baseline with a set of repetitive tasks; then enable auditable memory, then test individual Skills, MCPs or Subagents; and finally connect to message channels, Cron and high-privilege execution.

The official migration path is provided to import settings, memories, skills, allowed lists, message configurations and some keys from OpenClaw, and supports dry-run. Migration should not mechanically copy all assets. Secrets, expired memories, and unvetted Skills should be excluded before permissions are revalidated.

If the core requirements are mature multi-channel gateways, session routing and device nodes, priority can be given to evaluating OpenClaw; if the core requirements are open tools, multi-execution backends, memory and skill learning loops, priority can be given to evaluating Hermes. Both require the deployer to undertake updates, scaling reviews, and security operations.

Hermes is suitable for users who are willing to manage long-term memory and open tools, but is not suitable for treating “self-improvement” as unsupervised.

10. Claude Code in-depth analysis

Claude Code embeds model capabilities into code libraries, terminals, IDEs, desktops, and browsers to form a continuous engineering closed loop of requirement understanding, code modification, real testing, difference review, and delivery.

Its value comes not only from generated code, but also from the reusable engineering system composed of CLAUDE.md, Skills, MCP, Hooks, Subagents, Agent Teams and plug-ins.

10.1 Product positioning and four types of user interfaces

Claude Code officially describes it as an agentic coding tool that can read code libraries, edit files, run commands, and integrate development tools. It covers terminals, IDEs, desktop applications and browsers; different interfaces share underlying engineering capabilities, but have different focuses on local files, cloud environments, parallel tasks and visual review.

Understanding Claude Code as a “code completion tool” would underestimate its value. Completion focuses on the next piece of code near the current file, while the engineering agent focuses on the complete life cycle of a target in the code base: understanding the requirements, locating the implementation, modifying multiple files, running real tools, continuing to fix based on failures, reviewing differences and preparing for delivery.

The “Claude Code Complete Tutorial” provided by the user shows the specific operation path in the Chinese environment, including context management, CLAUDE.md, Auto Memory, Skills, MCP, Subagents, Hooks and plug-ins. The report explains these paths in the official product architecture to avoid mistakenly writing an author’s personal thresholds or convenient practices into hard product rules.

As of the search date, Claude Code officially describes it as an agentic coding tool that can read code libraries, edit files, run commands, and integrate with development tools. It can be used in terminals, IDEs, desktops, and browsers. When choosing an interface, you should first consider where the code and dependencies are, whether local services are required, whether the task will run for a long time, and how the results will be reviewed.

The terminal is suitable for complete command line loops, the IDE is suitable for collaboration around the current file and symbols, the desktop is suitable for managing multiple projects and sessions, and the web is suitable for remote and background tasks. The differences between interfaces are not simple preferences, but differences in permissions, files, networks, and ongoing operating environments.

Choosing the Claude Code interface is essentially selecting the execution environment and control surface of the task.

Data basis: Claude Code Docs: Overview; user provided “Claude Code Complete Tutorial”

10.2 Engineering closed loop from problem to verifiable change

The most valuable thing about Claude Code that can form a team consensus is not a certain command, but a set of standard closed loops. The following takes “Fixing a cross-file login status bug” as an example.

1. Clarify the phenomenon and scope: Provide error logs, reproduction steps, affected versions, and unchangeable behaviors.

Figure 10-1 Claude Code’s agent closed loop: collect context, perform actions, verify results, and make corrections in a loop

Source: Claude Code official document, How Claude Code works (retrieval date: 2026-08-10)

1. Explore the code base: locate entrances, status flows, related tests and historical implementations, and do not rush to edit.

2. Form a plan: describe root cause assumptions, expected modification files, compatibility risks and verification methods.

3. Implement minimal changes: Keep changes focused and avoid refactoring irrelevant code.

4. Run verification: Run targeted tests first, then run broader checks of affected modules.

5. Check the diff: Make sure there are no secrets, debug code, extraneous formatting changes, and missing files.

6. Prepare for delivery: Summarize root causes, modifications, validations, remaining risks, create commit or PR draft.

The key responsibility of people in a closed loop is to identify scope and trade-offs. The agent can provide multiple implementation solutions, but interface compatibility, migration rhythm, performance budget and product behavior often require business or architectural judgment. High-quality collaboration does not mean having people write every line of code by hand, but rather letting people place decisions at high-leverage nodes.

The team should fix the closed loop as a common operating language: first clarify the phenomenon, scope and immutable behavior, then locate the code and tests, form root cause hypotheses and plans, implement minimal modifications, run targeted and extended verification, check diff, and finally summarize the basis, evidence and remaining risks.

Without real validation, Claude Code may still generate code that looks reasonable but is incompatible. Test failures, missing dependencies, and environment differences should be presented directly; at the end of the task, not only the results should be given, but also the verifications that have not been run and the trade-offs that require human judgment should be listed.

Stages	Agent Actions	Human Control Points	Verifiable Evidence
Understand	Read repository, rules, issues	Confirm target scope	Plan/reference files
Modify	Edit code and configuration	High-impact decisions	diff
Verify	Run tests and checks	Determine exceptions	Test logs
Review	Explain changes and risks	Approve merge	Review records
Handover	Organizing commits/PRs	Deciding to publish	Commit and rollback instructions

The delivery unit of Claude Code should be “changes with proof of verification”, not code snippets.

10.3 How to issue high-quality engineering tasks

Engineering task input should include goals, phenomena, boundaries, constraints, and completion criteria. Don’t just write “Fix login bug”, you can write: “Reproduce the problem that the page still displays logged in after logging out; locate the root cause and make minimal repairs; do not change the login API; supplement regression testing; run auth module test and lint; finally give a diff summary and unresolved risks.”

Context can be composed of a variety of materials: local files, images, terminal errors, log fragments, Issue descriptions, and multi-line requirements. When providing materials, explain their status, such as “logs are evidence of the production environment”, “screenshots are used to reproduce visual problems”, “design documents are only for reference, existing APIs must not be modified”.

Practical suggestions | First ask Claude Code to “read-only explore and give a plan”, and then allow writing after confirming the range. For large tasks, write the completion criteria of each milestone into the same session to avoid the results only staying in code generation.

High-quality tasks should describe how the problem is reproduced, the scope of the impact, existing logs, interfaces that cannot be changed, directories that are allowed to be modified, test commands and final delivery. Requires read-only exploration and planning for large tasks first, which can significantly reduce the agent’s extensive revision of erroneous assumptions.

The status of pictures, logs, issues, and design documents must be clearly stated to prevent the system from treating the reference solution as a requirement that must be implemented. Separate approval points should be set when it comes to new dependencies, database migrations, public APIs, and production configurations.

Level	Example writing
Goal	Fix the problem that the import process crashes under an empty header
Scope	Only modify the importer and related tests; do not change the public API
Constraints	Comply with CLAUDE.md; do not access production data
Verification	Reproduction failure; unit testing; complete testing; description of compatibility
Stop	Ask first if you need to change the data model or have insufficient permissions

The closer the input is to the project brief, the closer the output is to merging changes.

10.4 Context windows, compression and session continuation

The current session context is Claude Code’s working memory and contains targets, open or read files, tool results, logs, and conversation history. Context is not infinite, and excessive log and search output can dilute the information that really matters. You can understand the composition through interfaces such as /context, and compress the history through staged summary or /compact.

Compression is not about simply “deleting old messages” but about refining key facts, decisions and progress before continuing. In practice, it should be compressed at natural milestones: the root cause has been confirmed, the solution has been decided, and the first phase of verification is completed. In the tutorial, the author may actively compress at a certain occupancy ratio. This can be regarded as a personal habit, but it should not be written as a unified threshold for the product.

When the task is completely redirected, the current context should be cleared to avoid interference with old goals; when the task needs to be continued, the historical session should be restored. The stage summary should save at least: current goals, modified files, verification results, unfinished items, rejected solutions and next steps.

Long logs, repeated searches, and irrelevant files increase contextual pressure. Natural milestones include root cause identification, solution decision, phase verification completion, and task transition. Forming summaries at these locations is more reliable than waiting until the context is close to its limits and then passively compressing it.

The continuation summary should save the goals, modified files, key decisions, test results, incomplete and next steps. The specific occupancy ratio in the tutorial can be regarded as a personal habit, but should not be used as a unified threshold for the product; the compression timing should be determined based on the task structure and information quality.

The goal of compression is to preserve task state, not simply reduce word count.

10.5 CLAUDE.md, Rules and Persistent Project Directives

CLAUDE.md is best suited for holding engineering facts that are stable across tasks, such as build and test commands, directory responsibilities, architectural constraints, dependency policies, interfaces that prohibit modification, and delivery requirements. Temporary requirements, one-time debugging processes, and easily expired states should not be accumulated in the long term.

Large code bases can maintain rules hierarchically by directory: the root directory describes global constraints, and subdirectories supplement local conventions. More specific rules should serve to clarify boundaries and not allow the agent to guess priorities through duplication and conflict. The team needs to regularly clean up expired rules.

Project instructions should be kept as short and executable as possible. Instead of writing a slogan like “maintain high quality”, it is better to clarify the test commands, formatting tools, directories that allow modification and compatibility requirements. Rule changes should also be reviewed like code and verified with representative tasks.

CLAUDE.md is used to save project facts and work engagements that should be known for each session. The official document distinguishes between organizational policies, user instructions, project instructions, and local instructions. Project-level files are suitable for inclusion in version control, recording build and test commands, directory structures, architectural constraints, coding styles, and review checklists; local files are suitable for saving preferences that are related to individual machines but should not be shared.

A useful CLAUDE.md should be specific, concise, and conflict-free. Writing “Modify JavaScript and run npm test” is more executable than “Test carefully”; writing “API handlers are located in src/api/handlers/” is more helpful than “Keep the structure clear”. Files that are too long will occupy context and reduce compliance. Multi-step operations should be migrated to Skills. Specific directory rules can be configured using path ranges.

CLAUDE.md is the context, not the enforcer. If an action must be prohibited regardless of the model’s judgment, it should be intercepted through permission rules or PreToolUse Hook. The team needs to regularly clean up outdated instructions to avoid conflicting files in the root directory and subdirectories.

CLAUDE.md is suitable for core project facts, build commands, and rules that always apply; path-scoped Rules are suitable for specific languages, directories, or components; longer references and processes should be moved to Skills. The official extension guide recommends keeping CLAUDE.md focused to avoid everything going into context every time.

Teams need to treat directives as code assets: review changes, clean up obsolete content, resolve conflicts, and verify compliance with real tasks. Security hard constraints still require hooks, permissions, or CIs and should not just be declared in natural language.

CLAUDE.md is a summary of the project constitution, not a knowledge repository or enforcer.

Data basis: Claude Code Docs: Features Overview; Memory / CLAUDE.md

10.6 Auto Memory, conversation history and knowledge accumulation

Auto Memory is automatically recorded by Claude based on corrections, preferences, and work processes, and is suitable for saving build commands, debugging clues, and recurring experiences. It is complementary to CLAUDE.md: the former is a study note written by Claude, and the latter is a project agreement that is clearly maintained by humans. Both enter the context of each session.

The benefit of automatic memory is that it reduces repeated interpretations, but the risk is that erroneous experiences may be retained for a long time. Therefore, users should be able to view, edit, and delete memories; secrets should not be placed in memories; and auditable project directives or permission mechanisms should still be used for security requirements and team norms that must be followed.

Auto Memory is good for saving recurring build threads, preferences, and project experiences, but automatically written content may be out of date or wrong. Users need to be able to review and correct, secrets should not be allowed to slip into memory. For the team bottom line, reviewable rules still need to be written.

Knowledge precipitation should be selected: one-time debugging details remain in the session, recurring experiences enter the memory, stable rules enter CLAUDE.md, multi-step methods enter Skill, and mechanical inspections enter Hook. Layering can reduce contextual noise.

Putting experiences into the right tier is more important than simply increasing long-term memory.

10.7 Skills: Load knowledge and workflow on demand

Skills are suitable for hosting methods that occur repeatedly but do not necessarily occupy the main context each time, such as release checks, database migration reviews, failure recovery, and dependency upgrades. Description files define trigger conditions and steps, and scripts, templates, and reference materials are loaded as on-demand resources.

A high-quality skill should clearly describe applicable scenarios, inputs, outputs, preconditions, allowed tools, verification methods, and failure handling. A trigger description that is too broad will cause miscalls, and a trigger description that is too narrow will be difficult to reuse; both need to be calibrated through real tasks.

Skills should be tested in a clean environment before release, and the responsible person, version, dependencies, and rollback methods should be recorded. Teams also need to check scripts and external resources within them to prevent skill packs from becoming hidden entry points to bypass project rules or introduce supply chain risks.

Skills use SKILL.md to encapsulate instructions, resources and optional scripts. Claude can be automatically loaded according to the description, or can be called directly by the user via /skill-name. Unlike CLAUDE.md, the Skill body only enters context when it is actually used, making it suitable for longer checklists, release steps, review processes and domain knowledge.

A good Skill should explain when to use it, what input it requires, what steps it takes, what output it produces, how it is verified, and when it stops. Support files can include templates, samples, scripts, and reference materials. Personal skills are suitable for reuse across projects, and project skills are suitable for sharing with the repository. When mature, they can be loaded into plug-ins for distribution.

For example, the “review uncommitted changes” skill can first dynamically read git diff and then output risks from the five dimensions of functional correctness, testing, secrets, performance and compatibility. Teams can build review sets with real PRs and continuously iterate on trigger descriptions and checklists, rather than judging quality based on how they feel after one use.

Claude Code Skills can be used as reference knowledge or callable processes. By default, the complete content is only loaded during use. The official extension guide considers it the most flexible extension: it can be called by user command, selected by Claude based on description, and can be run in an isolated context.

The team should prioritize converting repeatedly copied processes and long references into Skills to avoid CLAUDE.md expansion. Skill descriptions must be able to distinguish adjacent scenarios, steps must include verification and stopping, and scripts must be reviewed.

Skills allow project knowledge to enter context on demand, and also allow repetitive work to have versions and evaluation objects.

Data basis: Claude Code Docs: Skills; Features Overview

10.8 MCP: Connect Issue, Monitoring, Database and Design Systems

MCP exposes external system capabilities as structured tools, but successful connection does not mean unlimited authorization. Each tool should have narrow parameters, clear side effects, stable error returns, and readable results that enable the agent to verify the true state.

Connections to issues, monitoring, and databases should use dedicated identities with minimal permissions, and write operations should be restricted by default or subject to approval. Text from external systems is still untrusted input and cannot be treated as system commands just because it is returned through MCP.

MCP allows Claude Code to connect issue tracking, monitoring, databases, design, messaging and custom APIs. When teams often copy Jira, Sentry, Figma or Slack content into conversations, MCP can turn “copy and paste” into an authorizable and callable tool interface.

After connecting the tool, tasks can be closed-looped across systems: read Jira Issues, view Sentry errors, modify code, run tests, create PRs on GitHub, and generate communication drafts to be manually confirmed. MCP also expands the attack surface because the server may return external untrusted content and may also have write capabilities.

Before using MCP, you should confirm the server source, transmission method, OAuth scope, available tools, output upper limit and audit method. Set tools such as database writing, message sending, and production operations to ask or prohibit each time; set range filtering for tools with large output to prevent the context from being overwhelmed by data.

MCP can reduce copy and paste, allowing Claude Code to directly read Issues, Sentry, databases, designs and messaging systems. A real closed loop may start with a bug ticket, read monitoring evidence, modify the code, run tests, and then generate PR and communication drafts.

Review server origin, OAuth scope, tool write capabilities, output size, and logs before connecting. The content returned by the external system is not trustworthy, and production writing and external sending should be subject to approval. Skills can supplement the domain knowledge of how to use MCP, but they cannot replace the permissions of the MCP layer.

MCP expands the context and radius of action and also brings external system risks into the engineering session.

10.9 Subagents and Agent Teams

Subagent is suitable for well-bounded exploration, testing, or review tasks. The main session should provide the goals, necessary materials, allowed tools, and desired outputs, while the sub-agent works in an isolated context and returns evidence, avoiding bringing all the process noise back to the main thread.

Agent Teams consists of multiple independent sessions sharing tasks and messages, which is suitable for complex work that can truly be done in parallel and requires mutual discussion. It’s still a collaboration method that needs to be carefully evaluated; if tasks cannot be clearly split, the communication, conflict, and merge costs may outweigh the speed gains.

Subagent is a professional agent that performs specific side tasks in an independent context. It’s suitable for jobs that generate large amounts of search results, logs, or file content, with only a summary returned to the main session. Each Subagent can be configured with specialized system prompts, tools, permissions, models and persistent memory.

Typical usage includes Explore to locate code, Test to supplement and run tests, and Security Reviewer to check risks. Only tasks with clear output boundaries are suitable for delegation; multiple Subagents editing strongly coupled files at the same time will increase conflicts. If multiple full sessions need to be parallel, coordinated, or long-running, consider background Agents or Agent Teams instead of cramming all the parallelism into a single session.

Cost control is equally important. Faster, cheaper models are available for exploration, and stronger models are used for critical architecture and final review. Subagents should be restricted to tools and directories and should not inherit by default more permissions than are required for the task.

Subagents run in an independent context and return summaries, suitable for exploration, testing and review; Agent Teams consists of multiple independent Claude Code sessions sharing tasks and communicating directly, suitable for competing hypotheses and complex collaboration. The official documentation also indicates that Agent Teams is experimental and is turned off by default. Actual deployment should verify the current limitations.

The main session retains global goals and merge responsibilities. Subroles should define tools, directories, and outputs to prevent multiple agents from modifying strongly coupled files at the same time. Parallel benefits need to be measured in terms of actual cycles, costs and rework.

Use Subagent to isolate the context first, and only consider Agent Team when negotiation between sessions is required.

Data basis: Claude Code Docs: Subagents; Agent Teams; Features Overview

10.10 Hooks: Deterministic Automation and Security Doors

Hooks are suitable for checks that must be performed reliably, such as command blocking, formatting, testing, logging, and notifications. They are triggered by life cycle events and do not rely on model ad hoc judgments, so they can serve as deterministic safety gates in the agent process.

Hook’s failure strategy needs to be clear: when a blocking check fails, subsequent actions should be stopped and the reason returned, while a prompt check should record the problem but allow continuation. Scripts must set timeouts, output limits, and diagnosable errors to avoid becoming stuck in the task.

Teams should test and review Hooks alongside normal project scripts. Hooks involving file modifications, network requests, or secrets require higher-privilege review, and the triggering timing should be ensured so that repeated executions do not produce side effects.

Hooks are commands, prompts, agents or HTTP actions that are automatically run at specific nodes in the Claude Code life cycle. They provide deterministic controls: automatic formatting after editing, running lint and tests before committing, preventing modification of protected files, re-injecting critical information after context compression, and sending notifications when tasks are completed.

Hooks are suitable for hard constraints and mechanical actions, but are not suitable for carrying complex natural language judgments. A Hook must handle timeouts, exit codes, and failure messages; the script itself needs to be included in version control and code review. Hooks from third-party plug-ins are equivalent to automatically executed code and should be reviewed before installation.

The official extension guide clearly distinguishes Hook from Skill: Hook is bound to be triggered when matching events, and is suitable for lint, blocking dangerous commands, logs and notifications; Skill is explained by Claude and is suitable for processes that require reasoning and knowledge. PreToolUse can turn “don’t read secrets” into technical blocks.

Hook output may enter the context, and the script may access files and networks. Output, timeout and failure modes must be controlled during design. Before installing third-party Hooks, it will be reviewed as automatically executing code.

The mechanical control that must be guaranteed is written as Hook, not just as prompt.

Data basis: Claude Code Docs: Hooks; Features Overview

10.11 Plugins and Marketplace: Team Distribution Layer

Plug-ins can package Skills, Agents, Hooks, MCPs, and other components into an installable, versionable package. Standalone .claude/configuration is suitable for personal or single-project experiments; plug-ins are more suitable when capabilities need to be reused across projects, shared by teams, version updated, or distributed through the Marketplace.

Before installing a plug-in, you should check what commands, Agents, Skills, Hooks, MCP/LSP servers, context costs, and update times it contains. The official Marketplace provides a curated directory, but any extension may bring instructions, scripts, dependencies and network access. “On the market” cannot be understood as being suitable for all environments of the organization.

The mature path is usually: first create a Skill or Hook in the project and use real tasks to verify it; then combine related capabilities into plug-ins; and finally establish versions, change records, responsible persons, rollback methods, and allowed installation scope.

Plugin packages multiple extensions into installable units, suitable for distribution across repositories, teams, and organizations. Namespaces can reduce skill name conflicts, and Marketplace provides discovery and update paths. The mature process usually first verifies a single Skill or Hook within the project, and then combines it into a plug-in.

The team should establish allowed sources, versions, owners, change records, permission descriptions, and rollbacks for the plugin. Market presence does not equate to risk assessment by the organization; in particular, automatic hooks, MCP servers, and installation dependencies should be checked.

Plug-ins are software products and require release engineering and supply chain governance.

10.12 Permissions, Security and Trusted Execution

Claude Code runs on a permissions basis by default. Reading is generally lower risk, editing, command execution, and external tools require more explicit authorization; sandboxing can limit the file system and network scope of Bash commands. The goal of security is not to make the agent never make mistakes, but to limit mistakes to recoverable and auditable areas.

Tip injection may come from issues, web pages, documents, code comments, emails, or MCP returns. You cannot rely solely on models to identify attacks. Treat external content as untrusted input, restrict credentials and networks, review high-impact commands, verify critical file changes, and use VMs or isolated environments to run third-party scripts when necessary.

Teams also have to deal with supply chain issues when using it: Skills, Plugins, Hooks, and MCP servers may execute code or access data. Establishing mechanisms for allowing sources, version locking, review, and updates is more reliable than simply training users to “click carefully.”

Claude Code can read code, execute commands and connect to external systems. The risk comes from the combination of untrusted content and high-privilege tools. Issues, web pages, code comments and MCP results may induce the system to overstep its authority and cannot rely solely on model self-identification.

Security baselines include read-before-write, minimal directories and commands, restricted networks, covert isolation, high-impact action confirmation, running third-party scripts in isolation, extended source review, and diff and test evidence. Permissions should match the task stage rather than authorizing the entire machine at once.

The closer the capabilities are to the real development environment, the more technical boundaries must be used to protect human intentions.

Data basis: Claude Code Docs: Security

10.13 The team adopts the route and asset system

To transform Claude Code from a personal productivity tool into a sustainable engineering capability, the team first needs to build code base baselines, task baselines, and extension baselines separately. The three types of baselines respectively answer “what the system should comply with”, “how to accept the delivery” and “which capabilities can be called repeatedly”.

The code base baseline should include CLAUDE.md, build and test commands, directory boundaries, dependency policies, and sensitive information rules; the task baseline should include representative tasks such as defect fixes, test supplements, document updates, and dependency upgrades, and retain success and failure samples.

Skills, Hooks, MCPs, Subagents, and plugins are part of the extended baseline. They need to have clear ownership, sources, versions, permissions, testing and rollback methods to avoid personal scripts evolving into the team’s critical path without review.

The Claude Code team pilot can start with a code base and two types of tasks: fixing small bugs, adding tests, or updating documentation. First create the project CLAUDE.md and basic verification commands, requiring all deliveries to contain diff summaries and test evidence. After two to four weeks, the high-frequency processes will be made into Skills, the necessary checks will be made into Hooks, and the external system will be connected through the controlled MCP.

• Before starting: Confirm the repository status, task scope, allowed directory modifications, dependency installation rules and prohibited items.

• When exploring: read first and then write, record root cause hypotheses, and do not regard the first solution as a conclusion.

• When modifying: keep the diff to a minimum, do not modify irrelevant files, and do not hide failures.

• When validating: run targeted tests, affected module checks, formatting and static analysis.

• Upon delivery: Description of modifications, basis, verification, residual risks and content requiring human judgment.

• When expanding: Third-party Skills, Plugins, Hooks, and MCP first review the sources, scripts, permissions, and update mechanisms.

In the first stage, select one or two repositories and small tasks, improve CLAUDE.md, build tests and delivery formats; in the second stage, make repetitive processes into Skills and necessary checks into Hooks; in the third stage, connect controlled MCP and professional Subagents; in the fourth stage, plug-ins and parallel teams will be promoted.

The asset catalog should record rules, Skills, Hooks, MCP, plug-ins, responsible persons, versions and evaluation results. Extensions without an accountable person will become obsolete and eventually become a hidden risk.

Claude Code relies on reusable engineering assets to scale, rather than everyone figuring it out on their own.

10.14 Case: Cross-file login defect repair

Before starting the case, a baseline should be established: reproduce the problem of still displaying the login status after exiting without modifying the code, and save the failed steps, console output and related test results. Only if the baseline is stable will there be comparable evidence for subsequent repairs.

The exploration phase can check cache invalidation logic, routing guards, and authentication status synchronization in parallel, but the modification phase should return to a single main line, giving priority to solutions with the smallest impact and that can be covered by automated tests. Parallel exploration does not mean parallel writing to the same set of files.

Verification needs to start with directly relevant unit tests and then extend to scenarios such as cross-tab pages, refreshes, network interruptions, and mobile terminals. If the test exposes problems with the data model or public interface, automatic modifications should be suspended, the evidence should be compiled, and the architectural choices should be left to the maintainers to decide.

The deliverable should not only include code diffs, but also include reproduction methods, root cause determination, modification scope, executed tests, risks that are still not covered, and recommended rollback paths. The reviewer should be able to understand the chain of evidence without replaying the entire session.

If the fix relies on new browser behavior, third-party authentication services, or unconfirmed product rules, Claude Code should clearly mark these as external assumptions. The system can continue to prepare scenarios and tests, but unconfirmed assumptions should not be written as established facts.

The task requires reproducing the problem that the page still displays the login status after logging out, without changing the authentication API, supplementing the regression test, running the auth module test and lint, and giving a diff summary. The first round of read-only exploration locates the status cache, routing guards and related tests to form two root cause hypotheses; modifications will be made after confirming the compatibility requirements.

During implementation, the main session modifies the cache invalidation logic, tests Subagent to supplement exit and cross-tab use cases, and PostToolUse Hook runs formatting and local lint. The test failure revealed that the analog clock was not restored, and the system corrected the test fixture and re-verified. The final delivery lists the root cause, three modifications, test commands, uncovered mobile risks and rollback methods.

The case shows that the most valuable thing is not to generate the correct patch once, but that the agent can converge through real failures and focus human judgment on interface behavior and risk trade-offs.

The engineering closed loop wraps the uncertainty of the model in an observable, correctable, and reviewable process.

11. Codex in-depth analysis

Codex organizes engineering work through CLI, IDE, desktop applications, work trees and cloud tasks, enabling local real-time collaboration, isolated parallel execution and background long tasks to be connected in the same system.

Understanding Codex requires looking at project rules, memory, plans, work trees, sub-agents, extension mechanisms, sandboxes and approvals at the same time, rather than just looking at it as a single terminal tool.

11.1 Product Positioning: Organizing Engineering Work Between Projects, Tools, and the Cloud

Codex is OpenAI’s engineering agent for software development. It can understand code bases, edit files, run local tools, review changes, and handle long tasks in desktop applications, CLIs, IDEs, and cloud environments. The current product experience is integrating with ChatGPT desktop, web, and Work capabilities, but Codex remains centered around code libraries, development tools, and verifiable changes.

The “Codex Complete Solution” provided by users emphasizes the “front-end/back-end” relationship between Local and Worktree, context and three types of persistent information, large-scale project plans, Skills/CLI/plug-ins/MCP, and automated tasks. This report retains these easy-to-understand Chinese operating frameworks, while using official documents to distinguish environment boundaries, instructions and memories, sandboxes and approvals.

Figure 11-1 The official product release page of Codex application

Source: OpenAI, Introducing the Codex app (Retrieval date: 2026-08-10)

Codex understands code bases, edits files, runs tools, reviews changes and handles long tasks on-premises and in the cloud. Desktop applications place more emphasis on projects, multiple chats, Worktrees, Handoffs, and background work; CLI and IDE are close to the existing development cycle; Cloud provides remote isolation environments and integration portals.

The “Codex Complete Solution” provided by users uses “front-end Local, back-end Worktree, and remote Cloud” to explain environmental relationships, which is suitable for operational understanding; the report also calibrates current mechanisms such as AGENTS.md, Subagents, Worktrees, sandboxes, and approvals according to official documents.

The core of Codex is to organize engineering goals, project rules, execution environment and verification evidence into the same task.

Data basis: OpenAI Codex official document; users provide “Codex Complete Solution”

11.2 Local, Worktree and Cloud: How to choose between the three types of environments

In desktop applications, Codex tasks can run in Local, Worktree or Cloud environments. Local directly operates the current project directory, which is suitable for rapid alternation between humans and agents, tasks that require local debugging or rely on the current machine status; Worktree creates an isolated Git working copy on the same computer, which is suitable for multiple tasks in parallel without interfering with the main workspace; Cloud runs in a configured remote environment, suitable for long background tasks and multiple independent attempts.

The tutorial uses “front desk and back desk” to explain Local and Worktree are very intuitive: Local is the front desk that people are operating, and Worktree is the background that does not interrupt the current work. But Worktree is not in the cloud; both Local and Worktree are on the local machine. Cloud is an independent remote environment.

Local is suitable for tasks that depend on the current machine state, rapid human-agent iteration, and local debugging; Worktree is suitable for parallel tasks in the same repository without directly interfering with the main workspace; Cloud is suitable for independently configured, long-running, remotely delegated work. Local and Worktree run on the local machine, while Cloud provides remote execution.

Environment selection checks for dependencies, secrets, networks, ignored files, services, and validation tools. Data and permissions cannot be ignored just because a task is running in a standalone environment, nor can you assume that locally available state will automatically appear in Worktree or Cloud.

Form	Suitable for the task	Environmental characteristics	Collaboration methods
CLI	Quick modification of current repository	Local terminal	Instant chat
IDE	Delegate while coding	Editor context	Partial review
Desktop application	Parallel tasks and work trees	Isolation checkout	Task management and handover
Cloud tasks	Long-term, background, parallelizable	Isolated cloud environment	Asynchronous recycling results

Choose the right environment first, then choose a model and how to work.

11.3 CLI, IDE, desktop and cloud control plane

Codex CLI is suitable for focused loops in the terminal: launch in a repository directory, explore the code, plan changes, edit files, run local tools, and review commands and diffs in the same session. Non-interactive methods such as codex exec can embed scripts and CI, but clear constraints must be set for failure, output format, and permissions.

IDE extensions are suitable for local collaboration around currently open files, selections, and symbols to reduce switching costs; desktop applications are more suitable for managing multiple tasks across projects, reviewing results, and arranging Worktree and Scheduled tasks; Cloud provides an isolation environment for long tasks, which can be initiated from the Web, GitHub, Linear, or Slack. After completion, the summary and diff are reviewed and then merged.

Multiple interfaces are not duplicate products, but different control surfaces for the same engineering effort. A common path is: discover the problem and clarify the local context in the IDE, delegate the implementation in Worktree on the desktop, do local reproduction or fine-tuning in the CLI, and finally review diffs and PRs on the desktop or GitHub.

CLI is suitable for command line and scripted project loops, IDE is suitable for current files and symbols, desktop is suitable for cross-project management and parallel tasks, and Cloud is suitable for background execution and external triggering. A task can be handed over between different control planes, but the goals, environment, and verification status should remain consistent.

The common path is to clarify local issues in the IDE, put the implementation into Worktree on the desktop, reproduce and refine it locally on the CLI, and finally review the diff on the desktop or GitHub. Interface switching should serve the evidence and environment and should not cause context rupture.

Multiple control surfaces are not duplicate functions, but entrances to engineering tasks at different stages.

11.4 Codex’s standard engineering closed loop

Codex should also have “verifiable changes” rather than “generated code” as the delivery goal. The following is a recommended closed loop.

1. Write the goal, scope, non-modifiable items, verification commands and stopping conditions at the beginning of the task.

2. Check AGENTS.md, Git status, project structure and existing tests to form read-only exploration results.

3. For complex tasks, enter the plan first, clarify milestones, dependencies and acceptance, and then start writing.

4. Select Local, Worktree or Cloud to ensure that the execution environment is consistent with the required tools and data.

5. Implement minimal changes, run targeted verifications in a timely manner, and make corrections based on real output.

6. Check diff, testing, static analysis, generated files and dependency changes to avoid irrelevant modifications.

7. Develop reviewable deliverables: summary, validation evidence, risks, outstanding items, and consolidation recommendations.

If the task changes its goal midway, the plan should be explicitly reset or a new thread should be started; if it is just continuing the same problem, stay on the same thread to take advantage of the existing context. The key to engineering collaboration is not continuous chat, but consistency in task status, environment, and verification evidence.

Before starting work in Codex, project rules and Git status should be read. Complex tasks should be planned first, and then implemented after selecting Local, Worktree or Cloud. Run real tests and tools during the process, revise based on output; provide diff, validation, outstanding and risk upon delivery.

If the goal changes completely during the task, a new task should be started or the plan should be reset; the continuation of the same problem should remain in the same task and use the existing context. Handoff changes the running position and should not change the completion criteria.

Stages	Agent Actions	Human Control Points	Verifiable Evidence
Understand	Read repository, rules, issues	Confirm target scope	Plan/reference files
Modify	Edit code and configuration	High-impact decisions	diff
Verify	Run tests and checks	Determine exceptions	Test logs
Review	Explain changes and risks	Approve merge	Review records
Handover	Organizing commits/PRs	Deciding to publish	Commit and rollback instructions

The consistency of task status, execution environment and verification evidence is the core of the Codex closed loop.

11.5 AGENTS.md: Command chain of global and project rules

Codex discovers project directives along the directory hierarchy: global configuration provides common constraints, the project root directory defines repository rules, and deeper directories can supplement local requirements; more specific directives have higher priority in the event of conflicts. AGENTS.unauthorized access.md can be used to unauthorized access explicitly.

The current official instructions also give a default command chain size limit, so AGENTS.md should not become an infinitely growing knowledge repository. Build commands, directory boundaries, validation requirements, and security rules are suitable for writing; dynamic progress, long-form background, and one-time task material should be kept in the task context or project documentation.

The team should include AGENTS.md in code review: adding rules should explain the problem to be solved, deleting rules should confirm that there are no dependencies, and directory-level rules should be verified by the real tasks of the corresponding module. The rules themselves also require versions, owners, and failure cleanup mechanisms.

Codex reads AGENTS.md before starting work. The global scope is located in the Codex home directory, which is suitable for general personal preferences; the project scope is discovered layer by layer from the project root directory to the current working directory, and the files closer to the current directory are more specific. AGENTS.unauthorized access.md can overwrite basic files at the same level and is used for special directories or temporary rules.

The project-level AGENTS.md should document the repository structure, build and test commands, dependency management, coding style, prohibitions, review criteria, and delivery format. It’s best suited for rules that must load stably, be reviewed by the team, and versioned with the repository. For code review, Code Review Rules can be defined in the file closest to the relevant code.

AGENTS.md should not be crammed into a long operating manual. Repetitive processes are suitable for Skills, external tools are suitable for MCP or plug-ins, mechanical enforcement is suitable for Hooks or permission rules, and project settings are suitable for config.toml. Putting different types of information on the right surface can reduce contextual costs and conflicts.

The official documentation states that Codex reads AGENTS.md before starting work and builds a chain of instructions from the global scope to the project root and then to the current directory. Files closer to the current directory are more specific, AGENTS.unauthorized access.md can replace the base file at the same level. There is a configuration upper limit for the combination size, and content that is too long should be split.

Project files should document build tests, directory structure, dependency rules, prohibitions, and delivery standards. The team should avoid conflicts between the root directory and subdirectories, use unauthorized access for special temporary requirements and clean up after completion.

AGENTS.md turns team engineering engagements into a stable starting point for every task.

Data basis: OpenAI official document: Custom instructions with AGENTS.md

11.6 Local Memories: Division of Soft Memories and Hard Rules

Local Memories is an optional local memory layer and is turned off by default. When enabled, Codex can extract useful context from eligible historical conversations and update them in the background after the session is idle instead of writing immediately after each session ends. The main files are saved in the memories directory under Codex home.

Each session can control whether existing memories are used and whether the current session becomes input for future memories. Automatic memories will attempt to filter secrets, but manual review is still required before sharing Codex home or exporting memories. Memory files are generated and should not be used as the only source of hard rules for a team.

The most important distinction is “hard rules/soft memories”: AGENTS.md is a working convention that is explicitly maintained, and Local Memories are references extracted from history. Rules such as tests that must be run, prohibiting file deletion, etc. should be left in AGENTS.md, permissions, or Hooks, rather than just relying on memory.

Local Memories are suitable for extracting reference context from historical tasks, but should not be the only source of hard rules for the team. Tasks can choose whether to use memories and become future memory inputs; secrets and personal information need to be checked before sharing or exporting.

Must-run tests, prohibited actions, and architectural bottom lines are written into AGENTS.md, Rules, or Hooks; experience, preferences, and history clues can go into Memories. Regularly review outdated and conflicting content to prevent the long-term impact of incorrect experiences.

Hard rules are human-maintained and reviewable, and soft recall helps reduce repetitive explanations.

11.7 Plan, Goal and Long Tasks

The number one risk with complex projects is not slow execution, but the wrong direction. The Plan phase should confirm users, scope, dependencies, architectural constraints, milestones and completion criteria through questions and answers. A plan is not a decorative task list, but the basis for subsequent judgments about “what to do next and when to complete it.”

For work that requires many steps, the official documentation recommends giving clear results, constraints, and definitions of completion, and leaving related work in the same chat. Desktop applications can use Goal mode to track ongoing goals; long tasks on the Web can write results and review criteria directly into the prompt.

Long tasks require milestones rather than infinite loops. Each milestone should produce verifiable artifacts such as architectural plans, runnable prototypes, test coverage, migration scripts, and rollback instructions. When faced with conflicting evidence, insufficient authority, or critical decisions, pause and ask for judgment.

For complex tasks, first confirm the scope, dependencies, architecture, milestones and completion standards through Plan. Plans should be updated with evidence rather than generated once and then used again. Long-term goals require stage products, budgets, and stopping conditions to avoid infinite advancement.

The Goal or long task mechanism in the desktop environment is suitable for continuous tracking, but the critical status must still be written to the recoverable summary. When encountering insufficient authority, conflicting evidence, and business trade-offs, you should request manual input instead of expanding the scope independently.

The reliability of long tasks comes from milestones and status, not from longer single runs.

11.8 Worktrees and Handoff: local parallel file isolation

Worktree in the desktop app creates independent checkouts for tasks and shares Git metadata with the main repository. Tasks can read and modify files in parallel without directly occupying the main workspace, but users still need to understand branch, commit, and merge relationships.

The work tree is usually in detached HEAD, and when completed, the task should be moved back to the local workspace via Handoff or explicitly created a branch. Since the same branch cannot be checked out in both working trees at the same time, the target branch and uncommitted changes need to be checked before handover.

Configuration that is ignored by Git but required for task running can be copied to the new worktree via the .worktreeinclude mechanism. The mechanism should not replicate secrets or machine-specific state; environment files, dependency caches, and artifacts require separate security policies.

Worktree uses Git worktree to create independent checkouts for each task, so that multiple Codex sessions can work on the same project in parallel without directly modifying the main workspace. They share Git metadata but have independent copies of files, suitable for implementing different functions, comparing solutions and running background tasks.

Codex-managed Worktrees usually start from the selected branch HEAD and can apply local uncommitted changes to the new Worktree. Files that are ignored but required for operation can be selectively copied via .worktreeinclude. Don’t copy all .env and secrets indiscriminately; only include the paths that the task really needs.

Common pitfalls of Worktree include the inability to check out the same branch in multiple worktrees at the same time, dependencies and caches occupying disks, failure to run due to missing files, and failure to transfer results back to Local after completion. Handoff is used to move sessions between Local and Worktree, which is safer than manually preempting the same branch.

The official documentation points out that Worktree in the desktop application allows multiple independent chats to run in parallel on the same Git project without interfering with each other; it has independent file copies and shares Git metadata. Handoff is used to safely move tasks and code between Local and associated Worktree.

Files that are ignored but necessary for operation can be selectively copied via .worktreeinclude, secrets cannot be copied indiscriminately. The same branch cannot be checked out in multiple worktrees at the same time. After the results are completed, a branch must be created, transferred to Local, or verified in the original worktree.

Worktree solves the problem of parallel modification isolation, and Handoff solves the safe handover between the frontend and the backend.

Data basis: OpenAI official document: Worktrees

11.9 Subagents: Parallel roles of exploration, testing and review

Codex Subagents are suitable for side tasks that can be independently accepted, such as code base exploration, independent testing, security review, and data retrieval. The main agent maintains global goals and merges responsibilities, and the sub-agent returns results after working in an independent context.

The parallel strategy should follow “minimum dependencies, clear delivery, and independent verification.” For example, one sub-agent only outputs the call chain map, another supplements the test, the third reviews the safety, and the main agent finally decides on changes. Multiple sub-agents directly editing the same core file will increase conflicts and cognitive costs.

The sub-agent inherits the current sandbox policy, and tool permissions are still limited by tasks. Teams should document concurrency counts, model costs, failure rates, and merged rework, and not treat “more Agents” as an automatic source of efficiency.

Codex Subagents are suitable for side tasks with clear output such as code base exploration, test completion, security review and data retrieval. The main agent is responsible for global goals and merging, and the sub-agents return evidence and summaries.

When configuring roles, the task scope and tools should be limited to prevent multiple sub-agents from modifying the same core file. The official mechanism and currently available interfaces may change with versions. Actual concurrency and configuration should be subject to the documentation and the current environment.

The value of Subagent is to isolate high-volume side tasks, not to spread the main task responsibilities.

Data basis: OpenAI official document: Subagents

11.10 Skills, Plugins, MCP and Hooks

The extension mechanism of Codex can be distinguished by responsibilities: Skills describe reusable methods, Plugins are responsible for packaging and distribution, MCP connects to external systems, and Hooks perform deterministic control at life cycle nodes. Mixing different responsibilities into one giant script reduces auditability.

The order in which extensions are introduced should start with the smallest surface. One-time requirements are left in the task description first, stable project rules are entered into AGENTS.md, and repeated processes are deposited into Skills; MCP and Plugin are added only when distribution by external systems or organizations is really needed.

Every extension should have a source, owner, version, permissions, testing and rollback records. When upgrading, check not only whether functionality is available, but also whether tool parameters, network access, credential scopes, and default behavior have changed.

The principle of minimal surface helps to keep the system clear: one-time requirements are placed in prompt or thread context; repository rules are placed in AGENTS.md; repeated processes are placed in Skills; when unified installation and external connections are required, Plugin/MCP is placed; mandatory life cycle actions are placed in Hooks. Mixed requirements can be split without forcing one file to take on all responsibilities.

Skills follow progressive loading and read main instructions, references and scripts as needed; plug-ins are suitable for sharing across projects and teams; MCP allows Codex to access third-party documents and tools; Hooks perform mechanical control before and after tool calls, commands or file editing. Any extension may expand permissions and requires review of source code, dependencies, configuration and data scope before installation.

One-time requirements are left in the prompt, repository rules are written into AGENTS.md, repeated processes are made into Skills, Plugin is used for cross-project combination and distribution, external tools are connected through MCP, and mechanical controls that must be triggered use Hooks. Different layers can be combined, but responsibilities should be kept clear.

Plug-ins and MCP will expand the boundaries between tools and data, and Hooks may automatically execute code. Organizations need extended manifests, provenance reviews, version management, permission clarification, and rollbacks.

Putting requirements into the smallest, clearest scaling surface reduces context and governance complexity.

11.11 Use Codex to build RAG and project knowledge base

In knowledge-intensive projects, Codex is more suitable to be positioned as the “project agent layer”: it reads requirements and repositories, disassembles data ingestion, retrieval, generation, evaluation and delivery tasks, calls tools to implement code and verifies the results with tests; the RAG runtime that truly provides services to users should still be borne by deterministic components that are deployable, observable and auditable. The relationship between the two is not to “let Codex replace the knowledge base”, but to let Codex organize and continuously improve the knowledge base project.

Recommended main line: Requirements Prompt → Codex dismantling and planning → Inventory of data and code → Implement ingestion and retrieval → Access generation → Evaluation and security reinforcement → Delivery, and deposit stable rules into AGENTS.md, Skill or Plugin. Each step must have inputs, products, verification methods, and stop conditions.

11.11.1 RAG’s six-layer engineering architecture

RAG (Retrieval-Augmented Generation) is not “direct question and answer after uploading the file”, but a complete link from data governance to evidence-based answers. The following table gives the minimum floor-standing structure.

Level	Key tasks	Typical products	Main verification
1 data source	Access PDF, Markdown, Notion/Confluence export, database, work order and code repository; identify timeliness and permissions.	Source directory, collection list, ACL and update time fields	Accessibility, permission correctness, incremental update
2 Cleaning	Remove headers, footers and duplicate content, restore title levels, tables and code blocks, and unify coding and formatting.	Cleaning documents, parsing reports, failure queues	Sampling accuracy, missing rate, repetition rate
3 Chunks	Split by chapters and semantic boundaries, retaining title, path, time, author, authority and source URL.	Chunk structure and metadata specifications	Complete boundaries, locateable references, and no excessive shredding
4 Index	Choose a managed vector store, or a self-managed index such as pgvector, Qdrant, Chroma, etc.	Indexing tasks, vectors and keyword fields	Status completion, coverage, update and delete consistent
5 Search	Query rewriting, metadata filtering, Top-k, hybrid search and rerank; do permission filtering first.	Retrieval API, sorting strategy, cache	Recall@k, source accuracy, latency
6 Generation	Provide evidence, citations, confidence rules, and rejection conditions to the model; avoid evidence extrapolation.	Answer API, reference objects, rejection templates	Evidence-basedness, citation accuracy rate, rejection rate and manual review

In terms of selection, OpenAI managed file_search can complete file storage, semantics and keyword retrieval with less code, and is suitable for prototypes and rapid delivery; self-managed pgvector, Qdrant or Chroma are more convenient for customizing blocking, sorting, permissions and deployment boundaries, but they need to bear the burden of index life cycle, backup, monitoring and capacity management. The selection criteria should be data compliance, customization requirements, operation and maintenance capabilities, and cost, rather than simply pursuing the number of components.

11.11.2 Write Prompt into executable project requirements

Don’t just write “Make me a RAG.” Prompts should be written as small PRDs so that Codex can break down tasks, identify boundaries, and design acceptance accordingly. The following templates can be used directly as project startup prompts.

Code List 11-1　RAG Project Requirements Prompt

Goal:
Build a RAG question-answering service for internal engineering documents.

Background:
Documents live under docs/ and a Notion export directory. Users will ask about deployment, troubleshooting, and API specifications.

Scope:
- Support document ingestion, semantic chunking, vectorization, retrieval, and answering
- Every answer must cite a source title and URL; explicitly refuse when evidence is insufficient

Non-goals:
- Multi-tenancy is out of scope for this iteration; retain user-permission interfaces and metadata fields only

Technical constraints:
- Use FastAPI for the backend and pgvector for the vector store
- Wrap model calls in app/llm.py; explain the need before introducing a heavyweight framework

Acceptance criteria:
- pytest passes; add at least three RAG evaluation cases
- Return traceable citations when documents are retrieved; never fabricate an unsupported answer

Execution:
Read the repository first and propose verifiable subtasks, risks, implementation order, and a test plan.
Wait for confirmation before modifying code. On completion, summarize changes, test results, and remaining risks.

The recommended repository layout separates data ingestion, retrieval, answering, evaluation, and tool access to avoid cramming all logic into one script. AGENTS.md documents long-term engineering constraints, .codex/config.toml connects project tools, and .codex/hooks/ handles deterministic checks.

Code List 11-2 Minimum maintainable RAG project directory

rag-project/
  app/ingest.py          # Parse, clean, chunk, and ingest documents
  app/retrieval.py       # Filter, retrieve, and rerank
  app/answer.py          # Assemble evidence, citations, and refusals
  app/api.py             # Service API
  app/schemas.py         # Data structures such as Chunk and Citation
  evals/cases.jsonl      # Questions, expected sources, and scoring rules
  tests/test_retrieval.py
  tests/test_grounding.py
  tools/internal-kb-mcp/server.ts
  .codex/config.toml
  .codex/hooks/
  AGENTS.md

11.11.3 Use OpenAI file_search to quickly build a managed knowledge base

The minimum link of the hosting solution is: create a vector store, upload the file, add the file to the storage, wait for the index status to change to completed, and then retrieve it through the file_search tool of the Responses API. The production environment also needs to persist file and storage IDs, record index status and failed retries, and establish a complete life cycle for deletion, version updates, permission filtering, and reference display. Keys should come from environment variables or a key management system and should not be written to code or reports.

Code Listing 11-3: Minimal Python example using vector store and file_search

from openai import OpenAI

client = OpenAI()
store = client.vector_stores.create(name="engineering-kb")

with open("docs/runbook.pdf", "rb") as f:
    uploaded = client.files.create(file=f, purpose="assistants")

client.vector_stores.files.create(
    vector_store_id=store.id, file_id=uploaded.id
)
# Poll the file status and confirm it is completed before querying.
response = client.responses.create(
    model="<MODEL_ID>",
    input="What should I check first when deployment fails? Cite the source.",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [store.id],
        "max_num_results": 6,
    }],
    include=["file_search_call.results"],
)
print(response.output_text)

If the knowledge has department, project or confidentiality level boundaries, the ACL should be saved in the index metadata, and server-side authorization and metadata filtering should be performed before retrieval occurs; you cannot rely on prompts to ask the model to “do not read unauthorized content.” The number of search results can be controlled by max_num_results. When evaluating, you can use include to return the search results and locate the recall and answer questions.

Source: OpenAI, “File search”, https://developers.openai.com/api/docs/guides/tools-file-search/ (verification date: 2026-08-19).

11.11.4 Let Codex be implemented in stages and end with evaluation

A solid implementation sequence is: the first step is to read-only inventory of data, permissions and existing code; the second step is to complete ingestion and repeatable indexing; the third step is to establish a keyword or vector search baseline; the fourth step is to add answers, citations and rejections; the fifth step is to use a fixed question set for retrieval and generation evaluation; the sixth step is to perform performance, security, monitoring and deployment reinforcement. Each stage requires Codex to run minimal verification before moving on to the next stage.

Acceptance should not just be based on the answer “looks like”. The retrieval layer records at least Recall@k, hit source, permission filtering and delay; the generation layer checks whether the answer is supported by evidence, whether the reference can locate the original text, and whether to reject the answer if there is no basis; the engineering layer checks whether the index can be rebuilt, the document can be updated and deleted, failure can be retried, and the log does not leak sensitive information. Write representative failure cases into evals/cases.jsonl so that subsequent modifications will not degrade repeatedly.

11.12 MCP Server: Integrate knowledge and tools into workflow

The value of MCP (Model Context Protocol) is to package knowledge and actions that are not available locally in Codex into a tool with clear boundaries, stable input and output, and auditability. The RAG service is still responsible for retrieval, permissions, and data governance; the MCP Server only exposes the narrow interface required to complete the task. The relationship can be summarized as: MCP provides capabilities, AGENTS.md or Skill stipulates when and in what order to use capabilities, and Hooks and approval mechanisms are responsible for hard constraints.

11.12.1 Define the tool contract first, then write the server

Tools should “do one thing at a time”, and the name and description should be such that the model can be selected correctly; reading and writing actions are separated, and writing tools require confirmation by default. Permission verification must be performed on the server side. Annotations such as readOnlyHint and destructiveHint only help the client understand the risks and cannot replace authentication.

Tools	Main Inputs	Outputs	Permissions and Approvals
search_docs	query, filters, top_k	snippet, title, URL, score and document ID	read-only; do ACL filtering before querying
get_doc	doc_id	Text or controlled summary, metadata	Read-only; verify document permissions again
list_sources	Optional source_type	Available corpora and update time	Read-only; hide unauthorized corpora
create_eval_case	question, expected_sources	Evaluation case ID and save location	Write; default prompt/manual confirmation

11.12.2 A runnable skeleton: TypeScript MCP Server

The following example uses the official MCP SDK, connects to Codex through standard input and output, and registers the read-only search_docs tool. searchIndex is the project’s own search adapter where identification, ACL filtering, timeouts, audit logging and backend error handling should be done. The returned structure provides both structuredContent and text content for easy model consumption and programmatic processing.

Code List 11-4　Minimal implementation of internal-kb MCP Server

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";

const server = new McpServer(
  { name: "internal-kb", version: "1.0.0" },
  { instructions: "Search before answering project questions; cite title and URL." }
);

server.registerTool("search_docs", {
  title: "Search internal documentation",
  description: "Use before answering project knowledge questions.",
  inputSchema: {
    query: z.string().min(3),
    source: z.string().optional(),
    top_k: z.number().int().min(1).max(20).default(6),
  },
  annotations: { readOnlyHint: true, destructiveHint: false, openWorldHint: false },
}, async ({ query, source, top_k }) => {
  const chunks = await searchIndex({ query, source, top_k });
  return {
    structuredContent: { chunks },
    content: [{ type: "text", text: JSON.stringify(chunks) }],
  };
});

await server.connect(new StdioServerTransport());

The actual server should also register get_doc, list_sources, and create_eval_case, and define the input length, pagination, timeout, error code, and audit fields for each tool. Remote services can be exposed using Streamable HTTP /mcp; use MCP Inspector or unit tests to verify normal input, illegal input, permission denials, and backend timeouts before going live.

11.12.3 Configure MCP in the project and use AGENTS.md to solidify the workflow

Codex’s MCP configuration can be placed in user-level ~/.codex/config.toml, and trusted projects can also use project-level .codex/config.toml. The following example restricts the visible tools and automatically approves read-only searches and keeps the default query for write-based create_eval_case. Tokens are only read from environment variables. Project-level configuration is shared with the repository, so it should only be enabled in trusted projects and avoid submitting real keys.

Code List 11-5　Project level .codex/config.toml

[mcp_servers.internal_kb]
command = "node"
args = ["./tools/internal-kb-mcp/server.js"]
env_vars = ["KB_API_TOKEN"]
enabled_tools = ["search_docs", "get_doc", "list_sources", "create_eval_case"]
default_tools_approval_mode = "prompt"
startup_timeout_sec = 20
tool_timeout_sec = 30

[mcp_servers.internal_kb.env]
KB_API_URL = "https://kb.example.com"

[mcp_servers.internal_kb.tools.search_docs]
approval_mode = "approve"

If Codex is not started from the repository root directory, you should change server.js to a verified stable absolute path or use a wrapper script to parse the project root directory. After the configuration is completed, run codex mcp list to check the discovery status, and then test search, full text retrieval, low-relevance results, unauthorized documents, timeout, and write approval respectively.

Code Listing 11-6: RAG tool rules written into AGENTS.md

## RAG workflow
- Call internal_kb.search_docs before answering questions about project knowledge.
- Cite the source title and URL for every factual claim, and retain doc_id for traceability.
- If evidence is below the threshold, sources conflict, or the ACL denies access, do not guess; explain the gap.
- Call get_doc only when full text is needed; avoid reading large amounts of content without a purpose.
- create_eval_case writes to the repository and may be called only after user confirmation.
- After changing retrieval strategy, run tests and evals, then report Recall@k and failure cases.

Source: OpenAI, “Model Context Protocol” and “Build an MCP server”, https://developers.openai.com/codex/mcp/; https://developers.openai.com/plugins/build/mcp-server/ (verification date: 2026-08-19).

11.13 Hooks: Security interception and automatic inspection

Hooks are deterministic commands that run at key points in the Codex life cycle and are suitable for executing “must happen” rules: screening keys when prompted to submit, blocking dangerous actions before calling Bash, apply_patch or MCP tools, checking for failure and leak signals after the tool ends, and requiring additional testing and summary when Codex is ready to stop. They turn natural language constraints into testable programmatic guardrails.

11.13.1 Which checkpoint should the event be placed at?

Events	Trigger timing	Typical uses	Actions that can be taken
UserPromptSubmit	After user prompt is submitted, before Codex processing	Detect API key, production password; supplement fixed project context	Block prompt or add context
PreToolUse	Before the execution of Bash, apply_patch, MCP and other tools	Intercept recursive deletions, destructive database commands, and modify sensitive files	allow, deny or modify input
PermissionRequest	Codex when requesting approval	Map project risk rating to automatically allow, deny, or ask normally	allow, deny, ask
PostToolUse	After a tool has completed and returned results	Identify test failures, exception stacks, leaked secrets, or missing artifacts	Feedback and prevent further use of results; side effects cannot be undone
Stop	Codex is ready to end the task	Confirm that testing, lint, review and change summary are completed	Allow to end, or ask to continue the round

11.13.2 Configure three key Hooks

Hook configuration is written to config.toml the same as MCP. matcher uses regular matching tool names; Windows can provide command_windows separately. Next, point the prompt screening, pre-tool strategy, and post-tool review to the in-project script. The Windows paths in the examples are placeholders and must be replaced with the verified absolute path to the repository when used.

Code List 11-7　Hooks configuration in config.toml

[[hooks.UserPromptSubmit]]
[[hooks.UserPromptSubmit.hooks]]
type = "command"
command = 'python3 "$(git rev-parse --show-toplevel)/.codex/hooks/security_policy.py"'
command_windows = 'py -3 C:\\path\\to\\repo\\.codex\\hooks\\security_policy.py'
timeout = 10

[[hooks.PreToolUse]]
matcher = "^Bash$|^apply_patch$|^mcp__internal_kb__.*$"
[[hooks.PreToolUse.hooks]]
type = "command"
command = 'python3 "$(git rev-parse --show-toplevel)/.codex/hooks/security_policy.py"'
command_windows = 'py -3 C:\\path\\to\\repo\\.codex\\hooks\\security_policy.py'
timeout = 10
statusMessage = "Running safety policy"

[[hooks.PostToolUse]]
matcher = "^Bash$|^mcp__internal_kb__.*$"
[[hooks.PostToolUse.hooks]]
type = "command"
command = 'python3 "$(git rev-parse --show-toplevel)/.codex/hooks/post_tool_review.py"'
command_windows = 'py -3 C:\\path\\to\\repo\\.codex\\hooks\\post_tool_review.py'
timeout = 20

11.13.3 Implement security policy using JSON input and output

Codex passes JSON to the Hook via standard input. PreToolUse reads tool_name and tool_input; for Bash and apply_patch, the command or patch text is usually located in tool_input.command. The following script handles both user prompts and pre-tool checks, and gives blocking results that are recognized by Codex. Real projects should classify rules, record hit audits, and set controlled exceptions for allowed operation and maintenance operations, rather than relying on keywords to expand indefinitely.

Code List 11-8　.codex/hooks/security_policy.py

import json
import re
import sys

data = json.load(sys.stdin)
event = data.get("hook_event_name", "")

SECRET = re.compile(
    r"(?i)(api[_-]?key|access[_-]?token|secret|password)\s*[:=]\s*['\"]?[A-Za-z0-9_./+-]{12,}"
)
DANGEROUS = [
    re.compile(r"(?i)\brm\s+-rf\b"),
    re.compile(r"(?i)\bRemove-Item\b.*\b-Recurse\b"),
    re.compile(r"(?i)\b(DROP\s+DATABASE|TRUNCATE\s+TABLE)\b"),
    re.compile(r"(?i)(^|[ /\\])\.env($|[ ./'\"])"),
]

if event == "UserPromptSubmit":
    prompt = data.get("prompt", "")
    if SECRET.search(prompt):
        print(json.dumps({
            "decision": "block",
            "reason": "The prompt may contain a secret; redact it before continuing.",
        }, ensure_ascii=False))
        raise SystemExit(0)

if event == "PreToolUse":
    tool_input = data.get("tool_input") or {}
    payload = str(tool_input.get("command", tool_input))
    if any(rule.search(payload) for rule in DANGEROUS):
        print(json.dumps({
            "hookSpecificOutput": {
                "hookEventName": "PreToolUse",
                "permissionDecision": "deny",
                "permissionDecisionReason": "Blocked by the destructive-operation policy.",
            }
        }, ensure_ascii=False))
        raise SystemExit(0)

print("{}")

PostToolUse should parse tool_response instead of second-guessing command intent. If you find pytest FAILED, Traceback, secret scanning alarms or missing expected files, you can output decision=block and reason, feedback the failure to Codex and ask for repair; however, the action has already occurred, and PostToolUse cannot undo the side effects. Stop Hook can also return decision=block when there is no recent successful test record, requiring the test to be run again and the results reported. Test status should be derived from CI, timestamps, or machine-readable records and cannot be inferred from model summaries alone.

Code Listing 11-9　Core check of post_tool_review.py

import json, re, sys

data = json.load(sys.stdin)
response = json.dumps(data.get("tool_response", ""), ensure_ascii=False)
failed = re.search(r"(?i)(pytest.*FAILED|Traceback|secret detected)", response)

if failed:
    print(json.dumps({
        "decision": "block",
        "reason": "Tool output contains a failure or secret-leak signal; inspect and fix it.",
    }, ensure_ascii=False))
else:
    print("{}")

11.13.4 Boundary and online checking of Hooks

Hooks are strong engineered guardrails, but not complete security perimeters. Part of the call path may not go through the local function tool Hook; PostToolUse occurs later than the side effect. High-risk projects must use sandbox, least privileges, network or domain name restrictions, MCP server authentication, manual approval, branch protection, and CI at the same time. The Hook script itself should also enter version control, code review, and automated testing, and security rules should be modified with the same caution as production policies.

At least verify before going online: prompts containing test keys can be blocked and not echoed; recursive deletions, database destructive statements and sensitive file modifications can be rejected before execution; read-only search_docs runs normally and is subject to ACL constraints; test failures can trigger post-feedback; Stop rules will not loop infinitely; Hook timeout or exception behavior complies with the team’s failure opening and closing policy; logs can be associated with sessions, tools and decisions but do not record secrets.

Source: OpenAI, “Hooks”, https://developers.openai.com/codex/hooks/ (verification date: 2026-08-19).

11.14 Cloud, external integration and reproducible environment

Codex Cloud runs tasks in an isolated environment, suitable for allowing multiple long jobs to continue in parallel. Before official use, dependencies, tools, environment variables, and secrets must be configured for the repository to ensure that the task environment is reproducible. Once the results are complete, the summary and diff should be reviewed, subsequent revisions requested if necessary, and a PR created.

Cloud tasks can be initiated from portals such as Web, GitHub, Linear, or Slack. Integration improves the efficiency of tasks entering the system, but it also brings identity and context issues: who has the authority to trigger tasks, which repositories are allowed to access, whether external messages are trustworthy, where results are returned, and who is responsible for merging all require organizational rules.

Files, credentials, and networks are not the same in the cloud as on-premises. Don’t assume that locally runnable commands must exist in the cloud, and don’t copy secrets directly from your personal machine. Environment configuration, minimum scope credentials, and domain name allowlists should be used.

Cloud tasks require explicit dependencies, setup scripts, environment variables, secrets, and network rules. After completion, review the summary and diff before merging. External integration improves the efficiency of task entry, but it must also be clear who can trigger, which repositories are accessed, where the results are returned, and who is responsible for merging.

The cloud and personal machine status are different. Don’t assume that local services, uncommitted files, and personal credentials exist; environment configuration should be reproducible, secrets should use the smallest scope, and networking should be limited to domain names required for the task.

The value of Cloud comes from isolation and reproducibility, not from moving the local environment to the remote location.

11.15 Scheduled Tasks and Continuous Work

Continuous tasks should clearly describe the trigger conditions, input range, operating environment, timeout, maximum retry, notification object and stop method. Running in the background does not reduce responsibility, but requires more stable logging, status recording, and exception escalation.

Scheduled tasks should prioritize the use of minimal privileges and a reproducible environment. If a task relies on local uncommitted files, interactive credentials, or external actions that cannot be rolled back, it is not suitable for direct unattended running and should be split into two stages: preparation and approval.

Scheduled Tasks can be created and managed from the web or desktop application. Independent Scheduled tasks start from the saved prompt each time and report in the Scheduled interface; tasks embedded in existing chats can continue to use the context of the chat, suitable for polling long operations, continuous research or PR follow-up. CLI and IDE do not provide Scheduled management interface, but you can test prompts, skills or scripts in it first.

Project-level Scheduled tasks can run in Local or standalone Worktrees. Local will directly modify the main workspace, and Worktree can isolate background changes from unfinished work. When local files are involved, the computer needs to be turned on, the desktop application is running, and the project path is available; Web tasks can use uploading data and connection tools, but cannot directly operate folders on the computer.

Scheduled tasks are unattended, so you should use the narrowest access possible, test manually first, and then observe the first few runs. The task prompt should clearly describe what to do each time, what situations are worth reporting, what to do if there are no changes, and when to stop or request input. Frequent running will amplify costs, false positives, and Worktree usage.

Scheduled tasks are suitable for log summary, PR follow-up, dependency monitoring and periodic reporting. Before formal scheduling, manually test and write down the output, failure threshold, notification responsibility and stop conditions when there are changes and no changes. Project-level tasks should preferentially use independent Worktrees to avoid interfering with local unfinished work.

Frequent scheduling amplifies costs, false alarms, and workspace occupancy. Background tasks are riskier with default permissions and non-interactive policies, so use narrow access, auditable output, and the first few human observations.

Continuous work must have an observation strategy, a failure strategy, and an accountable person.

11.16 Sandbox, Approval, Rules and Network

Codex configures sandbox scope and approval policy separately. The sandbox determines what the system can read, write, or execute, and the approval policy determines which out-of-bounds or high-impact actions must be queried; the two cannot replace each other.

Read-only mode is suitable for inspection and planning, workspace write is suitable for most undone local engineering tasks, and full access should only be used when the risks are understood and the environment is additionally protected. Approvals can be confirmed incrementally from untrusted actions, on demand or in a controlled environment with less interruption.

Networks and credentials need to be governed separately. Even if the workspace is allowed to be written, it does not mean that any domain name should be opened or all user credentials should be inherited. Dependent downloads, external APIs, and code hosting connections should use domain restrictions, dedicated identities, and audit records.

Codex controls “what can be accessed” and “when to ask” separately. Common sandbox modes include read-only, workspace-write, and danger-full-access; approval policies include untrusted, on-request, and never. workspace-write allows editing and executing regular commands in the workspace, which is a low-friction local work baseline; danger-full-access removes boundaries and is only suitable for situations where there is a clear need and the risk can be taken.

Auto-approval or auto-review does not change the sandbox boundaries, it only approves the reviewer of the request. If a process only requires exceptions for a specific command prefix, Rules should be used to allow, ask, or disallow, rather than extending all file and network permissions. Specific writable roots can be added when multiple directories are required.

Network access should be turned off by default or restricted to domain names necessary for the task. Background Scheduled tasks use the default sandbox and may adopt non-interactive approval policies, which poses a higher risk. Organizations can limit the allowed sandbox and approval combinations through management policies such as requirements.toml to prevent personal configurations from bypassing the bottom line.

Security Configuration｜Don’t understand “never approval” as a general option for “higher efficiency”. Unattended tasks should be prioritized with workspace-write, minimal networking, and auditable output; only expand permissions if they are isolable, rollable, and explicitly authorized.

OpenAI official documentation separates sandboxing from approval: read-only, workspace-write, and danger-full-access define access boundaries; untrusted, on-request, and never determine when to stop; reviewer determines whether qualified requests are processed by users or automatic review. Automatic review does not change the scope of the sandbox.

Low-risk local automation usually uses workspace-write and on-request as the baseline, uses specific writable roots across directories, uses Rules for command exceptions, and limits the network to necessary domain names. The combination of danger-full-access and never removes major boundaries and should only be used in explicitly authorized and isolated environments.

Dimensions	Common options	Design issues
Sandbox	Read Only/Workspace Write/Full Access	What the system can touch
Approvals	Untrusted/On Demand/Never	When Must Ask People
Network	Closed/Domain Restricted/Open	External Data and Supply Chain Risks
Credentials	Not provided/injected/inherited by tool	Is it least privileged and auditable
Review	User/Auto Review	Who makes decisions on upgrade requests

“Never ask” is not a safe mode; borders and censors must be configured separately.

Data basis: OpenAI official documents: Sandbox; Agent approvals & security; Rules

11.17 Team landing route

Codex’s team implementation is suitable for layering around projects and execution environments. The project layer is responsible for AGENTS.md, verification commands and code review rules; the environment layer is responsible for the selection of Local, Worktree and Cloud, network and credential policies, and how task results are returned to the main workspace.

Local is suitable for short tasks that require instant judgment and local status, Worktree is suitable for tasks that require multiple modifications in parallel and files must be isolated, and Cloud is suitable for long-running tasks that can be independently verified and can asynchronously recycle results. Environment selection should be part of the task template.

Team operations should also record the success rate, manual takeover time, failure reasons, and rollback costs for each type of environment. Expanding the scope of background tasks and automated scheduling is only appropriate once there is stable evidence of parallelism benefits, verification quality, and governance costs.

Codex implementation should start from the code base baseline rather than installing a large number of plug-ins first. In the first stage, select a repository, create AGENTS.md, confirm the build and test commands, and use Local or CLI to process small tasks; in the second stage, Worktree parallelism and planning mode are introduced; in the third stage, high-frequency processes are made into Skills and Hooks; in the fourth stage, MCP, Cloud and Scheduled tasks are connected.

Organizations need both engineering and governance metrics: task success rate, test pass rate, PR rework rate, average manual intervention, number of privilege escalations, prompt-injection or secret-exposure incidents, automation false-positive rate, and cost. Expanding automation is appropriate only when results are stable, failures are visible, and changes can be rolled back.

• repository baseline: AGENTS.md, build commands, testing, formatting, and code review rules are complete.

• Task baseline: goals, scope, environment, acceptance, and stopping conditions are clearly written.

• Isolation baseline: Parallel tasks use Worktree or Cloud and do not directly preempt the main workspace.

• Validation baseline: Each delivery includes diff, testing and unfinished risks.

• Extended baseline: Skills, Plugins, MCPs and Hooks have sources, responsible persons, versions and rollbacks.

• Automated baseline: manual testing first, manual observation for the first few times, background tasks using minimal permissions.

The first stage improves AGENTS.md, builds testing and delivery formats; the second stage handles small tasks in Local/CLI and introduces Worktree; the third stage turns high-frequency processes into Skills and Hooks; the fourth stage connects controlled MCP and Cloud; and the fifth stage arranges Scheduled tasks and a higher degree of autonomy.

Indicators should cover task success, test pass rates, PR rework, manual intervention, privilege escalation, extension risks, and cost. Team assets should have explicit owners, versions, and review records.

Codex implementation should start from the repository and verification baseline, not from installing more extensions.

11.18 Case: Complete dependency upgrade and regression verification in parallel

The key to this case is not how many tasks to start at the same time, but to first form a constraint set that depends on the upgrade: the target version, compatibility range, prohibited modifications, test matrix, network conditions and rollback criteria should all be clear. Parallelism without constraints will only produce conflicting modifications more quickly.

The exploration task can analyze release notes, affected modules and existing test gaps respectively; the implementation task should modify the lock file and adaptation code in an independent Worktree; the review task should check permissions, security bulletins, generated files and indirect dependency changes. The main task is responsible for merging evidence, rather than repeating the entire process of subtasks.

Verification should be promoted from narrow to broad: first run dependency-related single tests, then perform module integration tests, static checks and builds, and finally cover the key end-to-end paths. If the cloud environment is different from the local environment, differences in running images, system dependencies, network access, and failures should be recorded separately.

When delivering, it is necessary to distinguish between “the code has been updated” and “the upgrade has been accepted.” The former is justified by diffs and build results, the latter also requires evidence of regressions, performance or volume changes, known risks, rollback methods, and maintainer confirmation.

In this approach, the value of Codex is demonstrated by organizing parallel exploration, isolating modifications, and aggregating evidence, while the final decision remains controlled by clear acceptance gates. Parallel capabilities only translate into engineering throughput when combined with reproducible environments and auditable deliverables.

Task splitting must also comply with dependency directions. Research on release notes and security advisories can be carried out independently, but code modifications must wait until the scope of impact is initially clear; end-to-end regressions usually rely on the build to pass. Writing these sequence relationships into task descriptions can prevent subtasks from generating invalid results when prerequisites are missing.

Each parallel task should return a structured summary including key files read, assumptions made, modifications produced, commands run, verification results, and open issues. The main task only accepts conclusions with evidence and arranges for review of conflicting results.

During the merge phase, it is necessary to specifically check whether multiple Worktrees have modified the same lock file, generated file or public configuration. Even if textual conflicts are automatically resolved, semantic version selection and build order may still overwrite each other, so a regeneration and full verification must be performed after the merge.

If the upgrade fails, it should be handled as a diagnosability rather than continuously letting the model try different versions. First distinguish between download or network failures, parser conflicts, compilation interface changes, runtime behavior changes and expiration of the test itself, and then select recovery actions for the corresponding types.

For high-risk dependencies, security bulletins and license changes should be entered into the review log. Codex can help with retrieval and organization, but whether a version meets organizational policies still depends on controlled data sources, software bill of materials, and security team rules, and cannot rely solely on natural language summaries.

Cost estimates should also cover inference, cloud execution, manual review, and failed reruns. If the waiting time saved by parallel tasks is less than the cost of coordination and review, you should return to a single-task closed loop; parallelism is a scheduling strategy and is not the default answer for all engineering tasks.

In the end, stable practices can be settled into dependency upgrade Skills, AGENTS.md rules and CI checks, but task-level judgments should be retained. Automation takes care of repeating steps, and maintainers take care of accepting compatibility trade-offs and release times.

The acceptance matrix should cover both operating systems, runtime versions, and critical functional paths. If the project cannot afford the complete combination, the minimum set can be determined based on production distribution and failure impact, and it should be clear which combinations have not yet been verified; tests that have not been run cannot be recorded as passed.

The rollback plan must also be truly practiced. It is not enough to just keep the old version number. You must also confirm whether the lock file, cache, database migration and generated products can be restored, and prepare for forward repair if necessary. If the rollback will cause data incompatibility, the upgrade must adopt a more strict phased release.

Manual review should focus on the most difficult to prove parts of the model: whether the public interface semantics have changed, whether the default configuration has changed, whether indirect dependencies have introduced new permissions, and whether error handling still meets business expectations. Format and static rules are left to deterministic tools as much as possible.

After the task is completed, the team should add the failure type and manual takeover point to the evaluation set. For the next upgrade, you can run these historical use cases first to determine whether the new model, rule, or execution environment actually reduces the risk, rather than just comparing the speed of task completion.

When the process stabilizes, Codex can take on more preparation, parallel verification, and report compilation work, but release approval should still be decoupled from code generation. Who has production change authority, who confirms business compatibility, and who is responsible for incident response must be made clear in the organizational system.

The task requires upgrading a core dependency, keeping the public API unchanged, fixing type errors, running full tests and providing rollback instructions. The main task first reads AGENTS.md and lock files to form a risk list; one Worktree attempts a minimum version upgrade, and the other only studies upstream changes and tests Subagent to analyze failure distribution.

The main agent compares the results and selects the minimum upgrade path, restricts the network to the packet source and maintains workspace-write. Approval on demand when dependent installation requires external access; after passing the test, Handoff to Local, do final verification in existing development services, review lock files and generated files.

Deliveries include version changes, compatibility modifications, test evidence, platforms not yet covered, and rollback commands. The case shows that Worktree isolation is suitable for comparison solutions, Subagent is suitable for knowledge and testing side tasks, and the final merger still requires unified decision-making.

Codex’s parallelism benefits come from environment isolation and clean handoffs, rather than launching more models simultaneously.

12. Product comparison, selection and combination use

Product selection is not a comparison of the number of functions, but a matter of matching tasks, environment, team ecology and governance conditions. The four products represent different operation entrances, continuity and control methods.

Real task evaluation under a unified dimension can distinguish demonstration advantages and stable delivery capabilities, and can also identify combined use and migration strategies.

12.1 Selection principles: matching of tasks, environment, ecology and governance

When choosing a product, you should start by answering what the task is, where it will run, what tools are needed, and who is responsible for approval and maintenance. OpenClaw is suitable for personal agents who want self-hosting, rich channels, and always online; Hermes is suitable for users who value multiple models, long-term memory, skill accumulation, and experimentation; Claude Code is suitable for teams that want to establish engineering closed loops in multiple development interfaces and reuse processes through rich expansion systems; Codex is suitable for engineering scenarios that require local, Worktree and Cloud collaboration, complex project parallelism, and clear sandbox governance.

Product value depends on task and organizational conditions. Resident personal assistants value channels, memory, and host control; engineering teams value codebase understanding, testing, diffs, environment isolation, and PR processes; research teams value browsing, referencing, long tasks, and multi-format delivery.

The task set and acceptance should be fixed before selection to avoid comparing different products with different tasks. Also calculate the total cost of accounts, models, infrastructure, extended maintenance, permission review, and manual review.

The absolute strongest without deviating from tasks and governance conditions.

12.2 Unified comparison of four products

• Individuals who want to call their own host tools at any time from Telegram/WhatsApp and other channels: Prioritize OpenClaw and consider identity, network and host isolation as prerequisites.

• Researchers or advanced users who want multiple models, self-deployment, long-term memory, and self-accumulation of skills: Check out Hermes, but invest in more versions, security, and skill reviews.

• The team mainly uses the Anthropic ecosystem and hopes to condense the engineering process into CLAUDE.md, Skills, Subagents, Hooks and Plugins: Claude Code to make it more convenient.

• The team needs Local/Worktree/Cloud unified organization, parallel projects, Scheduled tasks and clear sandbox approval: Codex has more advantages.

• If the task is highly certain and risky, don’t abandon a mature workflow for the sake of “agentization”; let the agent handle only the open steps.

OpenClaw highlights multi-channel self-hosted Gateway, Hermes highlights multi-model and procedural learning, Claude Code highlights the engineering closed loop and rich extensions around Claude, and Codex highlights Local/Worktree/Cloud and multi-control plane organization. Their functionality continues to change, and the tables are used to establish current structural understanding and are not intended to serve as long-term unchanging rankings.

Dimensions	OpenClaw	Hermes	Claude Code	Codex
Core positioning	Resident personal gateway	Open self-improvement framework	coding agent	engineering agent platform
Typical entrance	Message channel	Terminal/Message	Terminal/IDE/Desktop	CLI/IDE/Desktop/Cloud
Long term	Strong	Strong	Project/session	Local+Backend
Verification basics	Task customization	Task customization	Test/diff	Test/diff/isolate tasks
Main Governance	Channel and Gateway Permissions	Tools and Memories	Permissions/Hooks/Rules	Sandbox/Approval/Work-Tree

The unified dimension allows product comparison to shift from function stacking to system design.

12.3 Key differences between Claude Code and Codex

The common foundation between the two is far greater than the superficial differences: they both read code libraries, edit files, run commands, integrate with Git and testing, and support persistent instructions, Skills, MCP, Subagents, Hooks and plug-ins. The choice should not only depend on model preferences, but also on the team’s existing accounts, IDE/terminal habits, cloud environment, permission management, and expanded ecosystem.

Claude Code uses CLAUDE.md and Auto Memory to form a “human-written rules + Claude learning” structure. The official document has a very rich description of Skills, plug-in Marketplace and multi-Agent capabilities; Codex uses AGENTS.md and Local Memories to distinguish “hard rules + soft memories”. Worktrees and Local/Cloud environments have become important control surfaces for concurrent engineering, and Scheduled tasks and sandbox/approval configurations are also more prominent.

The safest selection method is to conduct an A/B pilot with the same set of real tasks: a cross-file bug, a repository exploration, a test completion task, a PR review, and a background reporting task. Compare quality of completion, verification evidence, human intervention, permission configuration, and team reusability rather than just build speed or throwaway coding style.

Both can read the code base, edit files, run commands, use Skills, MCP, Hooks and sub-agents. The difference is more reflected in the environmental organization and ecology: Claude Code takes CLAUDE.md, Rules and Anthropic extension system as its core; Codex takes AGENTS.md, Local/Worktree/Cloud, Handoff and OpenAI/ChatGPT control plane as its core.

The selection should not only compare the code quality of one task, but also the stability, context cost, permission prompts, parallel conflicts, extended maintenance and team review habits of multiple runs of the same repository.

Observation Points	Claude Code	Codex	Selection Issues
Configuration entrance	CLAUDE.md, Settings	AGENTS.md, configuration	Where to place team rules
Extension mechanism	Skills, MCP, Hooks, Plugins	Skills, MCP, Hooks, Plugins	Whether deterministic control is required
Parallel collaboration	Subagents, Agent Teams	Subagents, desktop multitasking, cloud tasks	Isolation and coordination costs
Running environment	Mainly local, multiple entries	Local + working tree + cloud	Whether background operation is required
Security model	Permission model and tool control	Separation of sandbox and approval	Who approves risks

Claude Code and Codex have a close common foundation, with the key differences being environmental organization and ecological governance.

12.4 Decision tree: Select product route from task entry

If the main entrance is messaging channels and 24/7 online, evaluate OpenClaw first; if you want multiple models, Profiles, and self-generated procedural knowledge, evaluate Hermes; if the team focuses on Claude ecology and terminal/IDE engineering, evaluate Claude Code; if you need desktop projects, multiple Worktrees, Cloud, and ChatGPT/Codex collaboration, evaluate Codex.

Decision trees only narrow down the scope. Ultimately, representative task A/B pilots are still needed to verify account and data policies, available models, platform support, and organizational managed configurations.

Primary needs	Priority inspection	Next questions
Multiple message channels, resident personal portal	OpenClaw	Self-hosting and channel governance capabilities?
Open framework, long-term memory and customization	Hermes	Can it undertake operation and maintenance?
Deep local code base collaboration	Claude Code/Codex	Team ecology and security requirements?
Parallel background tasks, isolation environment	Codex	Do you need cloud tasks and work trees?
Complex multi-agent collaboration experiment	Claude Code/Codex	Are there clear split and coordination benefits?

First filter by entrance and environment, and then make decisions based on real tasks and management requirements.

12.5 Unified measured design: How to compare without being misled by demonstrations

Actual testing should include at least four types of tasks: locating unfamiliar code bases, small-scale bug fixes, test completion, source research or document updates. Each task provides the same goals, materials, boundaries, and acceptance, documenting the environment, model, permissions, and extensions.

Metrics include complete success rate, test passes, irrelevant changes, correct references, human intervention, runtime, cost, privilege escalation, and failure visibility. Repeat at least multiple rounds and use it by different people to avoid mistaking a single lucky result for an ability.

The review needs to distinguish the impact of the model, product runtime, tools, project descriptions and user actions. Otherwise, even if the winner is compared, there is no way to know which layer to improve.

Steps	Requirements	Output
Topic selection	20-50 real, reproducible tasks	Task set
Blind test	Unify environment, permissions and model budget	Original running record
Acceptance	Pre-defined automatic and manual criteria	Score sheets
Security	Injection, unauthorized access, sensitive data testing	Red team results
Review	Improve by failure type rather than average score	Decision report

Unified tasks and complete evidence are the reliable basis for product selection.

12.6 Combination usage and migration strategy

Organizations can combine products: OpenClaw or Hermes serves as the messaging and scheduling portal, Claude Code or Codex handles code tasks; research agents generate Issues, engineering agents are implemented, and independent review agents are verified. Identity, data transfer, permissions and responsibilities must be clarified before combination.

Transferable assets include task specifications, acceptance criteria, project rules, Skills, MCP interfaces and evaluation sets. The specific configuration formats are different, but the content layer can remain unified. When migrating, migrate rules and tests first, and then migrate automation to avoid directly copying the permissions and implicit dependencies of the old environment.

Making knowledge and verification assets into product-independent layers can reduce lock-in and improve composability.

13. Organizational implementation: from pilot to large-scale operation

The entry of AI agents into organizational processes requires simultaneous preparation of tasks, data, tools, verification, identity, governance and human capabilities. The goal of a pilot is to generate evidence and assets, not just to complete a demonstration.

From task inventory and controlled pilots to capability solidification and large-scale operations, organizations should gradually accumulate rules, skills, tools, evaluation sets and responsibility systems.

13.1 Readiness: Processes, Data, Tools, Validation and Governance

Readiness is not measured by the number of seats purchased. It is necessary to check whether the task has been clearly delivered, whether the data can be obtained legally, whether the tool is stable, whether the verification is automated, whether permissions can be minimized, and whether there is a responsible person for failure. Processes that lack verification should be constructed and accepted first, and processes that lack interfaces should first organize data and tools.

Ready dimension	Not ready signal	Ready signal
Task	Objectives are vague and cannot be accepted	Input and output and boundaries are clear
Data	Confusing sources and unclear permissions	Clear classification, version, and access policies
Tools	Interface is unstable, no readback	Parameters are clear, errors are observable
Governance	Unclear who is responsible	Clear approval, auditing, and rollback
Personnel	Only able to chat and ask questions	Able to write task descriptions and accept them

Agent readiness starts with process and engineering readiness.

13.2 Pilot combination: one high-frequency task, one high-value task, and one governance task

High-frequency tasks are used to observe reuse and cost, such as weekly reports or test completion; high-value tasks are used to observe cross-step capabilities, such as defect fixes or research reports; governance tasks are used to verify prompt injection, permission denial, rollback, and auditing. Combining the three categories is more comprehensive than choosing just one great demo.

The pilot is limited to a small number of teams and repositories, using real but recoverable data. All tasks use unified instructions and acceptance records, register permissions and extensions before running, and record manual intervention and failures after running.

Pilots need to both prove value and proactively verify failure and governance.

13.3 90 Day Roadmap

Days 1-15 sort out tasks and risks and establish evaluation sets; days 16-30 complete project rules, environment and verification baselines; days 31-50 run read-only and controlled write pilots; days 51-65 skill high-frequency processes; days 66-80 supplement expanded reviews, audits and incident responses; days 81-90 make decisions to expand, adjust or stop based on data.

The roadmap must set stage exit conditions. If the success rate, verification coverage, or permission management does not meet the threshold, the current phase should be extended instead of forcing the backend automation to be launched according to the calendar.

Stages	Key tasks	Exit conditions
0-30 days: Discovery	Task inventory, baseline, risk classification, tool screening	Select 3-5 pilots
31-60 days: pilot	task template, sandbox, evaluation, training, weekly review	stable success rate and failure visible
61-90 days: solidification	skills, access process, monitoring, responsibility system	can be copied and passed the governance review

Upgrading the degree of autonomy based on evidence is the main line of the 90-day pilot.

13.4 Roles and Responsibilities: Business, Engineering, Security and Platform are jointly responsible

The business owner defines value and ultimate responsibility, domain experts provide rules and exceptions, the engineering team builds tools and verification, the platform team manages identity, environment and observability, security legal defines boundaries and incident response, and users are responsible for operating within the scope of authorization and providing feedback.

Any background tasks and high-privilege extensions should have clear responsibilities and replacements. Product manufacturers, models and agents cannot become responsible entities, and final authorization and external results still belong to the organization.

Agent operations span processes, software and governance and cannot be left to a single team.

13.5 Organizational knowledge assets: rules, skills, tools, and assessment sets

The asset catalog should contain project instructions, task templates, Skills, Hooks, MCPs, plug-ins, environment settings, assessment sets, and security policies. Each item records the purpose, scope, responsible person, version, dependencies, permissions, evaluation results and retirement date.

High-quality assets come from operational feedback. Rules are updated for repeated errors, Skills are generated for stable processes, Hooks are generated for mechanical checks, tool interfaces are formed across system requirements, and typical success and failure samples are entered into the evaluation set.

The core of scale is not more prompts, but a maintainable Agent asset system.

13.5.1 From operation trajectory to four types of capability carriers

Agent runs will continuously generate trajectories, tool results, user corrections, and evaluation signals. Experience can only form system capabilities after it changes subsequent behavior and is verified. According to the nature of expression, updates can fall into four types of carriers: facts, experiences, and exceptions are written into knowledge; verbalizable judgment principles are written into Prompt or Skill; deterministic processes and hard constraints are written into programs, tools, Hooks, or workflows; only perceptions and strategies that are difficult to express explicitly are considered to be entered into model parameters through supervised fine-tuning, distillation, or reinforcement learning.

The four types of carriers are not a one-way ladder from low to high. External knowledge is easiest to update and audit, and programs are most suitable for strong verification and reuse. Parameter internalization can reduce the burden of inference, but the update cost and rollback are the most difficult. Organizations should give priority to carriers that can be traced, partially modified, and automatically verified. Only capabilities that are stable and repetitive and difficult to carry by external rules will enter post-training.

13.5.2 Online execution and offline evolution must be separated

The stable version Agent in production is responsible for processing tasks and recording evidence, and should not directly rewrite its core rules after an unverified feedback. The offline evolution cycle is responsible for aggregating trajectories, finding common root causes, generating candidate updates, running target regression and security evaluation, and then promoting to a new stable version through versioning, grayscale, monitoring and rollback.

The roots of trust—permission policies, audits, validators, and rollback mechanisms—cannot be lifted by ordinary evolution processes on their own. Candidate knowledge must be marked with sources and applicable boundaries, candidate Prompts or Skills must be tested for sensitivity and conflict, candidate programs must undergo code review and CI, and model updates must check for degradation of old capabilities. The goal of continuous evolution is not to allow the Agent to modify itself infinitely, but to transform real experience into a controlled software release cycle.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

13.6 Indicators and Operational Rhythm

Look at task volume, success, manual intervention and exceptions weekly; look at quality, rework, cost, privilege escalation and extended health monthly; review asset usage, expiration rules, supply chain and scenario benefits quarterly. Metrics should be stratified by task type and risk to avoid averages masking high-risk failures.

Operations meetings should spot-check actual trajectories and deliveries, not just dashboards. Categorize the root causes of failures and identify whether improvements fall on task specifications, models, tools, environments, rules, or organizational processes.

Agent needs to be continuously operated like a software service, rather than no longer maintained after being online.

13.7 People Capabilities and Change Management

The focus of training should not stop at “how to write prompts” but rather teach employees to define delivery, provide context, set boundaries, review evidence, identify uncertainty, and handle permission requests. The knowledge of domain experts determines whether the Agent can make the right choice.

Establish shared cases, office hours, internal support, and safe experimentation spaces to make success stories an asset. Managers should allow small-scale failures and require transparent reviews to prevent employees from hiding Agent use because they are worried about accountability.

Agent amplifies domain capabilities and also amplifies the importance of task definition and judgment capabilities.

13.8 Scale model: platformization, federal governance and continuous optimization

1. Establish terminology consensus: Agent is a system, not a model, nor does it mean complete autonomy.

2. Choose two to three real tasks that are verifiable and rollable for piloting, and avoid just watching demonstrations.

3. Turn the task statement, verification evidence and stopping conditions into unified work specifications.

4. Consolidate stable experience into project instructions, Skills, and Hooks instead of relying on personal prompts.

5. Establish governance baselines for extension sources, sandboxes, approvals, networks, credentials, and background tasks.

Final judgment | The most important ability of an agent is not “how much it can do on its own”, but whether it can continue to produce verifiable results in uncertain tasks and reliably return control to humans at high-risk nodes.

The central platform provides identities, models, environments, logs, reviews, asset catalogs, and managed configurations; business teams have scenarios, rules, and acceptance; security legal affairs provide bottom lines and exception reviews. This federated model avoids duplication of construction by each team and retains domain adaptation.

Scaling does not mean unifying into a super agent. A more reliable direction is to have multiple task systems with clear boundaries, sharing infrastructure and governance. The platform continuously optimizes models, tools and assets through metrics and feedback.

Unifying infrastructure and decentralizing scene ownership is a robust structure for organizational-level agents.

14. Trends, Conclusions and Action Proposals

Agent competition is shifting from single model capabilities to complete system capabilities, and interaction methods are also shifting from short sessions to continuous tasks and digital colleagues.

Multimodality, tool protocols, security governance and organizational knowledge assets will jointly determine the speed of development in the next stage, and the final evaluation criterion will still be whether it can continue to deliver verifiable results.

14.1 Competition shifts from model capabilities to complete system capabilities

The key change in the era of agents is that models gain tools, memory, runtime and governance. The model determines the upper limit of understanding and reasoning, the tool determines what can be done, the memory determines whether the direction can be maintained across steps and sessions, the runtime determines whether the plan can be implemented, and the security mechanism determines how far the system is allowed to enter.

OpenClaw and Hermes show the runtime path of personal agents, and Claude Code and Codex show the paths of engineering agents. All four are converging on memory, skills, plug-ins, delegation, background tasks and permission management, indicating that future differentiation will come more from system integration, reliability and ecology, not just the underlying model.

Models are still an important factor in the upper limit of agents, but the real difference increasingly comes from tool quality, context loading, environment reproduction, feedback loops, extended ecology, parallel organization, and authority governance. Models iterate quickly, and system assets and organizational processes determine whether capabilities can be continuously converted into work results.

The leading Agent products of the future will be more like reliable operating systems than more fancy chat interfaces.

14.1.1 The model will absorb Harness, but will not destroy Harness

Model advancement will gradually internalize capabilities such as format compliance, basic tool selection, and error recovery, reducing some explicit orchestration; at the same time, Harness will move to new frontiers, assuming dynamic business knowledge, identity permissions, long-term status, asynchronous events, real environment verification, and organizational responsibilities. The two form a co-evolution: the running system generates trajectories and evaluations, promoting model and external asset updates; stronger models enable Harness to organize more open tasks.

Therefore, long-term construction should not bet on a certain set of temporary prompts, nor should it be assumed that changing models will automatically solve system problems. Unique data, tool interfaces, evaluation sets, user trust, channels and recoverable business processes should be precipitated as assets outside the model. No matter how the underlying model changes, “what you see, what you can do, and how to verify” are still the three most stable main lines of the Agent project.

Data basis: Relevant chapters of Li Bojie’s “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”.

14.2 Interactions shift from conversations to ongoing tasks with digital colleagues

Instead of just asking questions in one window, users can start tasks, view status, approve key actions, and receive delivery across projects, messages, IDEs, and cloud environments. Agents increasingly have persistent goals and restorable states, but digital colleagues still require job boundaries, work disciplines, and supervision.

What continues to change is the way collaboration is done, while increasing accountability and governance requirements.

14.3 Skills, Plugins and tool protocols become the capability distribution layer

Model capabilities will spread rapidly, and organizational differentiation will come more from own skills, tool interfaces, evaluation sets, and project rules. Plugins and marketplaces reduce distribution costs, but also introduce supply chain, naming conflicts, context costs, and permission expansion.

In the Agent era, organizational knowledge will exist more in the form of executable and measurable assets.

14.4 Multimodality and Computer Use expand working objects

Agents can work across web pages, desktops, documents, forms, presentations, images and voice, and more processes without APIs can be automated. However, interface operations are more fragile than APIs and are easily affected by layout changes, prompt injection, and identity status. Visual verification, action confirmation, and rollback must be strengthened.

After the action radius is expanded, verification and isolation must be upgraded simultaneously.

14.5 Security and governance will become product capabilities

When multiple products can call on tools and work in parallel, organizations will pay more attention to who can define boundaries, how to automate low-risk steps, when to escalate approvals, how to limit expansion, and how to view traces and recover. Rather than hindering adoption, governance is the foundation for reducing approval fatigue and supporting greater autonomy.

True scalable autonomy is built on enforceable boundaries and traceable evidence.

14.6 Answer three questions: system, task and method

What is an agent? It is a system that selects steps around goals within permission boundaries, invokes tools, maintains state, and corrects based on feedback. What can an agent do? It is suitable for complex, cross-tool tasks that require intermediate judgment and verifiable results, especially research, software engineering, data and business collaboration.

How to use the agent? Start with real but recoverable small tasks, use task descriptions to clarify goals and boundaries, write verification into completion standards, run with minimum permissions, accumulate rules, skills and tests, and then gradually enter parallelism, background and organizational scale.

The four products show two routes and common convergence: OpenClaw and Hermes emphasize resident, channel and long-term learning, Claude Code and Codex emphasize engineering closed loop and environmental organization; they all ultimately answer status, reuse, action and governance.

Main line conclusion: from model to system, from answer to task, from demonstration to verifiable operation.

Appendix A: Agent task specification template

This template converts natural language requests into delegated tasks. When using, fill in the goals and acceptance first, and then decide on the tools and degree of autonomy; do not open permissions first and then look for use cases.

Field	What to write	Example
Goals	Problems to be solved	Complete special report
Deliverables	Files, formats, audiences	80+ page Word report
Scope	Inclusions and Exclusions	Four Products, Focus on CC/Codex
Constraints	Time, tools, sources, style	Official information is preferred
Validation	How to judge completion	Number of pages, table of contents, rendering checks
Permissions	Allow/Require confirmation/Prohibit	Read-only search; write local files
Failure handling	When to retry or upgrade	Marking when sources conflict

1. Background and goals | Why to do it; what state to change; what is the relationship with upstream/downstream tasks.

2. Deliverables｜File, format, quantity, audience, location, deadline and version requirements.

3. Scope | Systems that must be included, can be included, explicitly excluded, and cannot be touched.

4. Inputs and Sources | Available data, trusted sources, version dates, and citation requirements.

5. Tools and Permissions | Tools and actions that are allowed to be used automatically, require approval, or are completely prohibited.

6. Acceptance criteria | Correctness, completeness, format, performance, security and manual review criteria.

7. Failure and escalation｜Maximum retries, timeouts, conflicts, insufficient permissions and conditions requiring human decision-making.

8. Handover｜Completion summary, evidence, unfinished items, risks, rollbacks and follow-up responsible persons.

Appendix B: Engineering Agent Task Acceptance Checklist

The acceptance of engineering agents should not only look at the final answer or code differences, but also check whether the process evidence is sufficient to reproduce. The following checklist can be used for task creation, review and handover. The team can be tailored according to risk level, but the responsible person and rollback conditions should not be omitted.

When using it, first confirm the task scope, environment and permissions, then check the changes and verification, and finally check the security and handover. If an item cannot be confirmed, it should be clearly recorded as an unverified fact rather than using model judgments as a substitute for evidence.

For repetitive tasks, the stable checks in the list can be solidified into scripts, hooks, or continuous integration access control; projects that require business judgment should still retain manual signatures or approvals.

Task and Scope

• □ The goals, scope and do’s and don’ts have been clearly written down

• □ repository rules and dependency constraints have been loaded

• □ Baseline problems can be reproduced

• □ High-impact design decisions confirmed

Change quality

• □ diff is directly related to the task

• □ No extraneous formatting or extensive rewriting

• □ Error handling and boundary conditions covered

• □ Public interface, data structure and compatibility impact have been explained

Verify

• □ Targeted test passed

• □ Relevant complete tests passed

• □ Static check, type check or build pass

• □ Failure log is not hidden

• □ Conducted real readback or interface rendering when necessary

Security

• □ No keys and sensitive data are introduced

• □ Newly added dependency sources, licenses and versions have been checked

• □ Command, network and file permissions are consistent with the scope of the task

• □ External input does not directly become a high-privilege command

Transfer

• □ Complete summary and document list accurately

• □ Test commands and results are reproducible

• □ Residual risks, limitations and follow-up items are clearly defined

• □ Submission/PR instructions for easy review

• □ There is a feasible rollback path

Appendix C: Agent Security and Governance Checklist

Object	Inspection Questions	Suggested Evidence/Controls
Targets	Are there range drifts or hidden targets?	Task description, stopping conditions
Input	Can untrusted content affect system commands?	Content isolation, source tags
Data	Is personal, trade secret or regulated data processed?	Classification, desensitization, residence
Identity	Whose identity is used to execute?	Dedicated identities, short-lived credentials
Permissions	Minimized to what is required for the task?	Whitelist, read-only priority
Tools	Are parameters, errors, and side effects clear?	Contract, test, readback
Network	What domain names and downloads can be accessed?	Domain name restrictions, malicious file checks
Memory	Can content be viewed, corrected, expired, and deleted?	Version, source, life cycle
Extension	Where do MCP, Skills, and Plugins come from?	Signed, reviewed, locked version
Approval	Which actions must be decided by humans?	Risk classification, two-person control
Audit	Can every step and every authorization be restored?	Logs, evidence, association ID
Recovery	How to stop, rollback and notify after failure?	Circuit breaker, backup, emergency procedures

Appendix D: Agent Evaluation Scale and Pilot Record

The four records below are examples of filling in, one for each product. They are only used to illustrate the recording method and do not represent the actual evaluation conclusion of any product.

Dimensions	1 point	2 points	3 points	4 points	5 points
Task successful	Not delivered	Partially delivered	Basically completed	Completely passed	Stable one pass
Correctness	Major errors	Many errors	Correctable	Minor flaws	Sufficient evidence
Efficiency	Significantly slower	No gain	Slight gain	Significant savings	Reproducible at scale
Robustness	No recovery in case of failure	Frequent takeover required	Can be restored at prompts	Most automatic recovery	Stable exception handling
Security	unauthorized access/Leakage	Inadequate Controls	Basic Compliance	Complete Controls	Proactively Minimize Risks
Auditability	Unexplainable	Insufficient evidence	Readable process	Complete evidence	Easy to reproduce and audit

Task ID	Task Type	System/Version	Success	Man Minutes	Failure Type	Evidence Link
OC-01	Multi-Channel Read Only	OpenClaw	Success	8	None	Logs, Sources
HE-01	Memory Retrieval	Hermes	Success	10	None	Memory Snapshot
CC-01	Patches and Tests	Claude Code	Success	20	None	diff, test
CX-01	Worktree Parallel	Codex	Success	16	None	Tasks, diff

Appendix E: Glossary of core terms

Terminology	Meaning in this report
Agent	A constrained system that selects steps around a goal, uses tools, maintains state, and corrects based on feedback.
Agentic	The autonomous choice and action characteristics of an intelligent form.
Tool	A structured interface for an agent to read or change the external environment.
Tool calling	A mechanism for model selection tools and generating structured parameters.
Runtime / Runtime	The system that hosts execution, state, scheduling, isolation, and recovery.
Context / Context	The information window of the current task that can be read directly by the model.
Memory	State, facts, and experiences saved across steps or sessions.
Skill / Skill	On-demand loading, reusable and testable task methods and resource packages.
MCP	An open protocol and ecosystem for connecting models and external tools/data.
Hook / Hook	A rule or procedure that executes deterministically on specific lifecycle events.
Plugin	Extension unit that packages skills, connectors, and configurations for distribution.
Subagent / Subagent	A specialized executor with an independent context that is delegated by the main agent.
Agent team	Multiple independent agent sessions collaborate by sharing tasks and messages.
Orchestration	Mechanisms for decomposing, dispatching, coordinating, aggregating, and handling failures.
Sandbox / Sandbox	Execution boundaries that restrict file, network, process, or system access.
Approval	Controls that request explicit human authorization before high-risk actions.
Least privilege / Least privilege	Grant only the minimum privileges required to complete the current task.
Worktree / Working tree	Isolated checkout that shares Git metadata but has a separate working directory.
Prompt injection / Prompt injection	External content attempts to change system objectives or induce unauthorized attacks.
Evals / Evaluations	Measure quality, efficiency, safety and robustness with repeatable tasks and metrics.
Human-in-the-loop	Validation, approval or judgment at the human control point.
Observability / Observability	Understand the running status of the system through logs, traces and indicators.
Rollback / Rollback	Restore erroneous actions to a known safe state.
Handoff	Handover task status, evidence, risks and next steps to humans or other agents.
Definition of Done	Pre-agreed completion and acceptance conditions.

Appendix F: References and Further Reading

The following information provides calibration concepts, product capabilities and practical recommendations. Product capabilities change rapidly, so you should double-check official documentation, change logs, pricing, platform scope, and security policies before implementation.

1. Li Bojie. “In-depth Understanding of AI Agent: Design Principles and Engineering Practice”. Chinese version, 2026. https://github.com/bojieli/ai-agent-book

2. OpenAI. A practical guide to building agents. https://openai.com/business/guides-and-resources/a-practical-guide-to-building-ai-agents/

3. OpenAI Developers. Codex documentation. https://developers.openai.com/codex/

4. OpenAI. AGENTS.md. https://developers.openai.com/codex/guides/agents-md/

5. OpenAI. Git worktrees. https://developers.openai.com/codex/app/worktrees/

6. OpenAI. Sandboxing and approvals. https://developers.openai.com/codex/security/

7. Anthropic. Claude Code overview. https://code.claude.com/docs/en/overview

8. Anthropic. Claude Code features overview. https://code.claude.com/docs/en/features-overview

9. Anthropic. Claude Code security. https://code.claude.com/docs/en/security

10. Anthropic. Building effective agents. https://www.anthropic.com/research/building-effective-agents

11. OpenClaw. Documentation overview. https://docs.openclaw.ai/

12. OpenClaw. Tools, Skills and Plugins overview. https://docs.openclaw.ai/tools/

13. OpenClaw. Gateway security and trust model. https://docs.openclaw.ai/gateway/security/

14. Hermes Agent. Official documentation. https://hermes-agent.nousresearch.com/docs/

15. Nous Research. Hermes Agent product page. https://nousresearch.net/hermes-agent/

16. NousResearch/hermes-agent. Releases. https://github.com/NousResearch/hermes-agent/releases

17. User provided: “Claude Code Complete Tutorial”. https://my.feishu.cn/wiki/Takxwov60iO5OOkOmpEcpOGynac

18. User provided: “Codex Complete Solution”. https://my.feishu.cn/wiki/OCY5wzbGhiLDr8kMulkcLLuSnQd

19. Microsoft Research. AutoGen and multi-agent research materials. https://www.microsoft.com/en-us/research/project/autogen/

20. NIST. AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework

21. OpenAI. File search. https://developers.openai.com/api/docs/guides/tools-file-search/

22. OpenAI. Model Context Protocol. https://developers.openai.com/codex/mcp/

23. OpenAI. Hooks. https://developers.openai.com/codex/hooks/

24. OpenAI. Build an MCP server. https://developers.openai.com/plugins/build/mcp-server/

Translation note: This article uses machine translation. If you notice any errors or ambiguities, please refer to the original Chinese text.