best-of-Agent-Harnesses

What is an agent harness?

A model answers; an agent acts. An agent harness is the runtime that turns one into the other: the model thinks, the harness decides what that thinking is allowed to touch.

Simon Willison's definition of the agent itself is the cleanest: "an LLM agent runs tools in a loop to achieve a goal." The harness is everything around that loop: which tools exist, what needs approval, what the model sees each turn, what survives a crash. Andrej Karpathy named the architecture back in 2023: the model is "the kernel process of a new Operating System", and the harness is the rest of that OS, its scheduler, permissions, and memory. The SWE-agent paper proved the stakes by coining the agent-computer interface: how tools and feedback are presented changes what a model can do, independent of the model. The field's advice has since converged on investing here rather than in framework plumbing, from Anthropic's build-simple guidance to Jerry Liu's argument that the framework era is over and the layers that matter now are skills, tools, and context quality. Those are the layers this list catalogs.

Why harnesses matter

Better models make harnesses more important: more capabilities mean more failure modes, and production needs retry logic, fallbacks, and validation. Harness quality, not just model quality, determines whether agents actually ship. This list ranks projects by relevance to harness concerns (environment, orchestration, lifecycle, guardrails) and by stars/activity.

The benchmark data now backs this up. On SWE-bench Pro, "swapping the agent harness changed pass@1 more than many model upgrades do" (AINews, Aug 8 2026, citing analysis by @joelniklaus). Same model, different harness: 23% to 52% pass@1 on GLM-5.2, and 15% to 36% on Gemma 4 26B. Harness rankings barely transfer across models (rank correlation -0.05), so a small model in the right harness can approach a much larger model in the wrong one.

That is the problem the MCP server in this repo solves. Point your agent at it and it can call recommend or pick_harness to choose a harness matched to your model and task, instead of inheriting whichever harness someone else benchmarked.

The landscape at a glance

Every project in the list, plotted by adoption surface area (the simplicity ↔ capability axis) against GitHub stars. Colors are categories; the largest projects in each tier are labeled.

The same projects placed by how much unsupervised rope they're designed to give (autonomy) and what happens when a run dies (recovery). In the tables below, ★ marks headless-ready projects and ✱ marks durable ones. Both charts regenerate from the list data on every refresh.

How to Pick a Harness

Start with the guide, then the head-to-head decision pages — grounded in the same data as the tables below:

• How to pick a harness: six questions that turn this list into a decision, plus the chart to internalize first (the harness moves scores more than the model)

• How to test-drive a harness: the two-week trial protocol, with a fair setup, tasks from your own repos, seven measurements, and the walk-away test

• OpenClaw vs Hermes — the always-on personal-agent debate: presence vs discipline, plus what the field reports actually say

• Terminal coding agents — opencode vs Codex vs Gemini CLI vs crush vs goose

• Multi-agent orchestration — OpenAI Agents SDK vs CrewAI vs AutoGen vs LangGraph

• Agent memory layers — Mem0 vs Letta vs claude-mem

• Agent sandboxing : what it is, the key concepts, and the field (E2B vs Daytona vs Modal and more)

• Agent evals (SWE-bench vs inspect_ai vs Terminal-Bench)

• Eval and observability platforms (Langfuse vs LangSmith vs Braintrust vs Phoenix)

• Browser agents (browser-use vs Stagehand vs Playwright MCP vs chrome-devtools-mcp)

• Browser infrastructure (Browserbase vs Steel vs Hyperbrowser)

• Claude Code skill packs (superpowers vs GStack vs get-shit-done vs Anthropic Skills)

• Context files for agents (AGENTS.md vs CLAUDE.md vs skills vs MCP tool search)

Pick by use case

Reader's index: pick by what you want to do, not by category. Tag chips (e.g. mcp · memory) next to each row let you cross-filter by capability — see TAGS.md for the full cross-reference.

• I want a turnkey coding agent today — opencode, Cline, Codex, Gemini CLI, OpenHands, crush, Roo Code · see Coding agent products (IDEs, CLIs, full suites)

• I want an always-on personal agent that lives in my chat apps — OpenClaw, Hermes, Khoj, Agent Zero, OpenHarness (HKUDS) · see Personal agent runtimes

• I want to extend Claude Code, Codex, or OpenCode with skills and slash commands — Anthropic Skills, wshobson/agents, superpowers, GStack, pmstack · see Coding harness configs and SDKs

• I want to build my own coding harness from scratch — Claude Agent SDK, Google ADK, AutoHarness, SWE-agent, RepoMaster, claw-code-agent · see Coding harness configs and SDKs

• I want a drop-in memory layer for agents — Mem0, Graphiti (Zep), claude-mem, agentlog, letta · see Plugins, MCPs, CLI tools

• I want to plug hundreds to thousands of tools without context bloat — MCP-Zero, ToolGen, ToolRAG, langgraph-bigtool · see Progressive disclosure harnesses

• I want multi-agent orchestration — openai-agents-python, crewAI, autogen, Microsoft Agent Framework, PraisonAI, agent-squad