Open Source Project #258: Harness² — Google Research's Recursive Agent Harnessing: Keep the Weights Fixed, Optimize the Shell Around the Model

Introduction

"A harness is the system around a model that enables it to act as an agent. Harnessing is the process of optimizing that system."

This is article #258 in the "One Open Source Project a Day" series. Today's project is Harness² (read "Harness Squared") — Google Research's open-source framework for recursive agent harnessing, with 23,451 Stars, 2,354 Forks, an Apache-2.0 license, created on 2026-10-06.

Over the past year it has become clear that the same model can perform very differently depending on the shell it's wrapped in: prompts, tools, memory, skills, orchestration logic. That shell is mostly tuned by hand — try a version, run a few tasks, tweak. Harness² asks: can an agent improve its own harness on a task the harness designers never saw, without ground-truth feedback? Its approach is two levels of recursion, with the model weights completely fixed.

What You'll Learn

• The difference between a harness and harnessing, and the eight editable components Harness² exposes
• Two levels of recursion: task-level Propose–Probe–Compose and the harnessing-level improvement harness
• How parallel and sequential recursion differ, and how to use the --k budget
• Quick start: a local demo needing no model credentials, and the full commands for one LAB benchmark run
• Two things to know before using it: API cost, and network access during evaluation

Prerequisites

• Basic understanding of AI agents and tool use
• Familiarity with coding-agent executors like Claude Code, Codex, or OpenCode helps with how it plugs in
• Running benchmarks locally needs Linux, Python 3.12, and uv; real benchmarks need a Vertex AI project

Project Background

Overview

Harness² is a plug-and-play framework aimed at the open world: agents encounter tasks the harness was not designed for. With model weights fixed, it connects two levels of recursion:

• Task-level recursion: for the current task, refine the execution harness through a contrastive Propose–Probe–Compose step — propose edits, probe their effects on the same task, and compose a refined harness from the behavioral contrasts between executions. It needs no ground-truth feedback.
• Harnessing-level recursion: maintain a persistent improvement harness, updated with the adopted harnesses and evidence-supported lessons from each task, to guide later refinement.

Author / Team

• Developed by: Google Research (the google-research organization). Paper authors include Ruiyao Xu, Yanfei Chen, Zhongying CuiZhu, Bhavana Dalvi Mishra, Yifei Ming, Han Yu, Rujun Han, Chen-Yu Lee, and Tomas Pfister
• Paper: arXiv:2610.14224, "Harness²: Recursive Agent Harnessing for an Open World"
• Primary language: Python
• License: Apache-2.0; code under third_party/, benchmarks, and datasets keep their upstream licenses
• Created: 2026-10-06; last push 2026-11-02
• Disclaimer: the README ends by stating this is not an officially supported Google product and is not eligible for Google's Open Source Software Vulnerability Rewards Program

Project Stats

• ⭐ GitHub Stars: 23,451+
• 🍴 Forks: 2,354+
• 📄 License: Apache-2.0
• 📅 Created: 2026-10-06
• 📬 Open Issues: 163
• 👥 Commit counts per contributor are small and scattered (rudycelekli has 8, most others 1–2), which suggests the main development happened elsewhere and was released as a whole
• 🌐 Website: harness2.github.io

Quick Start

Requires Linux, Python 3.12, and uv.

1. Local Demo (No Model Credentials)

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

The demo runs both recursion modes using deterministic editors and real Python executions, with no model calls. To understand the flow, this is the cheapest place to start.

2. Run a Real Benchmark

You need Node.js 20+, the Google Cloud CLI, and each benchmark's own dependencies (LAB needs Pandoc and Poppler; WorkBuddy-Bench needs Docker). Then configure models:

cp .env.example .env
# fill in your Vertex AI project and model IDs in .env
gcloud auth application-default login

The README's example runs one LAB (legal work) task, then judges and reports:

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
  --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

A few points: --k is the task-level recursion budget; judging and reporting must use the same benchmark, executor, subset, --k, --mode, and seed as the refinement run; add --dry-run to preview task selection without model calls. The README explicitly warns that model-backed runs incur API costs and recommends starting with a small task selection.

Core Deep-Dive

Eight Editable Components: What a Harness Can Actually Change

Harness² breaks the model's surroundings into eight editable components, grouped by runtime role:

The point of this split is that "editing an agent" stops being a vague manual craft and becomes edit operations on a set of defined objects. The optimizer doesn't change the model; it changes these eight things, so results can be saved, reviewed, and reused.

Task-Level Recursion: Propose, Probe, Compose

For a concrete task, one Propose–Probe–Compose step goes like this:

1. Execute: the agent executor runs the task once with the initial harness
2. Propose: a harness editor proposes two kinds of edits — invariant edits and domain edits
3. Probe: each candidate edit is actually run on the same task, and the behavior changes are observed
4. Compose: a harness composer synthesizes the final harness from the behavioral contrasts between those executions

The key is that steps 3 and 4 use contrasts between different executions of the same task rather than a right/wrong judgment, so no ground truth is needed. The cost is direct too: every probe is a real agent execution, so cost grows with the budget.

The task-level budget can scale in two ways:

Harnessing-Level Recursion: Keeping What Was Learned

After each task, the adopted harness and the "evidence-supported lessons" are written into a persistent improvement harness that guides future refinement. In the pipeline figure this is the role of an experience reflector, which reflects on the step and updates the improvement harness used for the next task.

This layer is what separates the framework from "tune from scratch for every task." One caveat: the results the README shows come mainly from a single step, and I didn't see separate numbers for how much this layer's long-term accumulation helps; for fuller ablations you'd need the paper.

The Reported Results, and How to Read Them

The README's claim: across three professional benchmarks and four model–harness configurations, a single Harness² step improves GLM-5 and Gemini 3.5 Flash over their base harnesses, with gains of up to:

• WorkBuddy-Bench: up to +11.5 points
• JobBench: up to +4.5 points
• LAB's partial pass rate: up to +11.8 percentage points

It adds that increasing the task-level recursion budget brings further gains. The supported benchmarks are JobBench (professional work), LAB (legal work), and WorkBuddy-Bench (office, web, and code); executors are OpenCode and Codex; recursion modes are parallel and sequential.

Two things to keep in mind. First, these are "up to" gains, not the size for every configuration and benchmark. Second, these numbers come from the paper and the authors' own evaluation; I haven't reproduced them.

A Must-Read Safety Note: Network Access During Evaluation

The README's Web access section is blunt: executor network access is unrestricted by default, and Codex runs with its sandbox disabled. Public benchmark rubrics may be reachable online, so before evaluating you should configure network restrictions per docs/EXPERIMENTS.md to stop the agent from reaching benchmark answers and grading criteria — otherwise the results can't be trusted.

This is about more than academic honesty. An agent that rewrites its own prompts, guardrails, and scripts, combined with no sandbox and open network access, deserves some thought about what it can touch before you run it on your own machine.

Resources

• 🌟 GitHub: google-research/harness2
• 📄 Paper: arXiv:2610.14224
• 🌐 Website: harness2.github.io
• 📖 Benchmark guides: docs/BENCHMARKS.md
• 🧪 Experiment guide: docs/EXPERIMENTS.md
• ⚙️ Setup guide: docs/SETUP.md

Summary

Harness² turns "tuning an agent" from a craft into an optimization process that can be executed recursively: the model stays fixed, while the shell becomes editable, saveable, and reusable.

Three things worth noting:

The optimization target is the harness, not the model. With weights fixed, eight defined editable components form the search space. That makes results reviewable and portable, and it works for closed models you can't fine-tune.

It improves through behavioral contrast, not ground truth. Propose–Probe–Compose synthesizes a new harness from differences between executions of the same task, which is what lets it face open-world tasks — and also where its cost comes from: every probe is a real execution.

Capability and risk are the same coin. Network access is unrestricted by default, Codex's sandbox is off, and real API costs accrue; the README itself says to configure network restrictions before evaluating. Before plugging it into a real environment, use the credential-free harness2-demo to understand the flow, then start with the smallest task selection. Also note it's research code, the README disclaims it as not an officially supported product, and the repo has 163 open issues.

If you're researching agent self-improvement, comparing how different harnesses perform, or weighing "edit the shell" against "swap the model," this project is a useful reference point.

Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.

Visit my personal site for more insights and interesting products.