October 19, 2026

Self-HPO: Self-Evolving Coding Agents via Harness-Policy Optimization

Zhepei Wei^1,2, Wenlin Yao¹, Baolin Peng¹, Danqing Wang³, Qianhui Wu¹, Hao Cheng¹, Yu Meng², Jianfeng Gao¹

¹Microsoft Research ²University of Virginia ³Carnegie Mellon University

💡

TL;DR. Recursive self-improvement is when a system improves the very thing that improves it. Self-HPO is a small, concrete version of this for coding agents. The policy reads its own failed runs and rewrites its own harness, the code that decides what it sees, how its commands run and when it stops. Then RL trains it inside that harness. The trained policy then writes the next harness, so the harness writer is itself being improved. No external model is required. Starting from Qwen3.5-35B-A3B, Self-HPO takes SWE-Bench Verified from 61.0% to 69.5% and the average over three benchmarks from 57.8% to 64.8%.

Start. Base policy P₀ with the seed harness H₀.

Figure 1. Left: Self-HPO alternates H-steps, where the policy rewrites its harness, and P-steps, where RL trains it under that harness. Each new harness is written by the latest policy. Right: SWE-Bench Verified after each step.

Toward recursive self-improvement

A coding agent is a model (the policy) plus the code around it (the harness). The same policy can solve an issue under one harness and fail it under another.

Most ways of improving an agent rely on something outside it: distilling a stronger model, or having a stronger model or a person rewrite the harness. Recent work that trains both the harness and the weights^{[1, 2]} still relies on an external model to write the harness. The agent only gets as good as what it is handed.

Existing self-improving methods optimize only one side, the policy or the harness. RLVR^[3] trains the policy inside a fixed harness. Self-Harness^[4] lets the policy rewrite its harness, but the policy itself never changes, so it keeps proposing from the same blind spots.

Self-HPO closes the loop. The policy writes its harness, learns inside it, and takes its next tasks from the ones it still fails. Because the harness writer is the policy being trained, each new harness is written by an improved policy. That is the recursive part.

The loop

Overview of the H-step and P-step. — **Figure 2.** **Top (H-step):** the frozen policy looks at the agent's failures on a search set and proposes new harness candidates. **Bottom (P-step):** the policy is trained with RL under the chosen harness. The updated policy becomes the next proposer, and tasks it never solved during training become the next search set.

We start from Qwen3.5-35B-A3B and the mini-SWE-agent^[5] seed harness. Each round has two steps.

H-step: the policy rewrites its harness. It reads its own failed runs and edits the harness, both prompts and runtime logic. It tries a few candidates and keeps the one that solves the most tasks on a search set.
P-step: the policy learns inside it. RL trains the policy inside the new harness, rewarded only by whether the tests pass.

Tasks the policy never solved during training become the next search set, so harness search targets the policy's evolving failure modes. We run two rounds, following our stopping criterion. We also run the same loop with Opus 4.8 writing the harness, which we call HPO, to see what a stronger writer adds.

Does it work?

We train only on Python tasks and evaluate on SWE-Bench Verified (in domain) and on two benchmarks the loop never sees: SWE-Bench Multilingual (nine languages other than Python) and SWE-Bench Pro. Numbers are resolve rates averaged over three runs.

SWE-Bench Verified

61.0 → 69.5

+8.5 points (in-distribution eval)

SWE-Bench Multilingual

54.2 → 59.7

+5.5 (out-of-distribution eval)

SWE-Bench Pro

58.3 → 65.2

+6.9 (out-of-distribution eval)

We compare against policy-only and harness-only optimization under two setups. Without an external model, RLVR^[3] trains the policy in the seed harness, and Self-Harness^[4] lets the base policy rewrite its harness. With Opus 4.8, SFT distills its trajectories into the policy, and Meta-Harness^[6] has it rewrite the harness.

Table 1. Resolve rate (%) averaged over three runs, with standard deviations.

Method	Harness	Policy	SWE-Bench Verified	SWE-Bench Multilingual	SWE-Bench Pro	Avg.
Base (Qwen3.5-35B-A3B)	✗	✗	61.0 ±0.2	54.2 ±0.3	58.3 ±1.2	57.8 ±0.4
Self-improving setup (no external model)
RLVR	✗	✓	67.5 ±0.8	56.3 ±1.0	61.8 ±1.3	61.9 ±0.6
Self-Harness	✓	✗	65.2 ±1.5	54.3 ±0.7	60.7 ±0.9	60.1 ±0.6
Self-HPO (ours)	✓	✓	69.5 ±0.9	59.7 ±0.8	65.2 ±1.4	64.8 ±0.6
External-model setup (Opus 4.8)
SFT	✗	✓	65.0 ±1.1	49.6 ±0.8	49.1 ±0.6	54.6 ±0.5
Meta-Harness	✓	✗	69.1 ±1.9	56.5 ±0.9	64.8 ±1.4	63.5 ±0.8
HPO (ours)	✓	✓	71.6 ±0.9	61.4 ±0.7	66.0 ±0.5	66.3 ±0.4

Three things stand out.

Self-HPO consistently outperforms policy-only and harness-only optimization. Among self-improving methods, Self-HPO beats RLVR by 2.0 points and Self-Harness by 4.3 points on SWE-Bench Verified, and RLVR by 2.9 points on average. Swapping in Opus 4.8 as the harness writer (HPO) adds only 2.1 more points on SWE-Bench Verified, so most of the gain does not need a stronger model.
The gains carry over to benchmarks the loop never saw. Both the harness search and RL use only Python tasks. Still, Self-HPO adds 5.5 points on SWE-Bench Multilingual and 6.9 on SWE-Bench Pro over the base agent. Harness search alone barely moves SWE-Bench Multilingual (+0.1).
If you have a strong model, put its knowledge into the harness rather than the weights. Distilling Opus 4.8 trajectories with SFT helps in domain but falls below the base agent on both out-of-domain benchmarks. Letting Opus 4.8 write the harness does better everywhere: 8.9 points higher than SFT on average.

Resolve rate along Base, H1, P1, H2, P2 for Self-HPO, Self-Harness and RLVR. — **Figure 3. Step by step.** Self-HPO and Self-Harness share the first H-step and split once Self-HPO trains its policy. After that, the second H-step helps the trained policy on all three benchmarks, and the next P-step adds more. A second harness-only round does not reliably help: Self-Harness drops from 55.0% to 54.3% on SWE-Bench Multilingual. Dashed markers are steps where a method makes no update.

Case study: what the agent wrote into its harness

What does the policy change when it is allowed to edit its own harness? Almost everything it added happens at one moment: when it says it is done. The seed harness ends the run right there and sends whatever was submitted to the grader. The harness the policy wrote first checks the patch (the code change it is about to submit), tells the policy what looks wrong, and gives it one more try before accepting. Figure 4 shows how this grew over two rounds. The code is simplified for reading, but every check and every message is the policy's own:

Coverage check: did every file it looked at and worked on end up in the patch?
Decorator reminder: does the patch use a decorator, such as @cache, that it forgot to import?
PR intent reminder: does the patch mention the key terms of the issue it is meant to fix?

The coverage message in Figure 4 comes from a real case, sphinx-doc__sphinx-9658. The final policy P₂ had already fixed the source, but it finished by printing a summary instead of its patch, so the submission contained no files. The seed harness would grade that empty submission as a failure. H₂ noticed the empty patch, handed control back once, and P₂ then submitted a valid patch that resolved the task.

61.0%

class AgentHarness(ProgressTrackingAgent):

def execute_actions(self, message):

self.touched |= files_read_or_written(message)

try:

return super().execute_actions(message)

except Submitted:

raise # seed: the run ends here

gap = self.touched - files_in_patch()

self.add_messages(coverage_check(gap))

if decorators_without_import(patch):

self.add_messages(decorator_reminder)

if not mentions(patch, key_terms(issue)):

self.add_messages(pr_intent_reminder)

if first_submit:

return [] # hold once, try again

raise

What the policy sees when it submits

Nothing. The run ends.

[Full Files Auditor] You touched 1 files during exploration: ['sphinx/util/typing.py'] Your submitted patch has 0 files: (none) FOUND GAP: 1 touched files NOT in your patch

[Decorator Reminder] Your patch uses these decorator(s) which may need importing: - @<decorator> Make sure these are imported before submitting.

[PR Intent Reminder] Your patch may not address the core PR intent. Key terms from PR: - <key terms from the issue> Consider: Does your edit actually address the issue described?

The seed harness accepts whatever the policy submits.

Figure 4. Left: Self-HPO's harness as the policy rewrote it over two rounds (simplified). Right: what the policy sees when it submits. The score is SWE-Bench Verified at that step of the loop (H₀ and H₁ with P₀, H₂ with P₁), as in Figure 3.

Why it works

Harness and policy gains compound beyond simple addition

To see what the harness and the policy each contribute, we pair every harness from one Self-HPO run with every policy from that run and evaluate all nine combinations on SWE-Bench Verified (Table 2).

Table 2. Every harness from one Self-HPO run paired with every policy from that run, on SWE-Bench Verified.

Harness \ Policy	\(P_0\)	\(P_1\)	\(P_2\)
\(H_0\) (seed)	61.0	61.7	64.4
\(H_1\)	64.3	67.8	68.1
\(H_2\)	64.5	68.6	69.5

The final harness helps even the untrained base policy (61.0 → 64.5). The final policy helps even under the seed harness (61.0 → 64.4). So neither gain depends on the other being present. Put together they reach 69.5, an 8.5-point gain over the base agent. That is 1.6 points more than the two individual gains added up (3.5 + 3.4 = 6.9).

Self-HPO improves both reliability and the capability boundary

Pass@k of Base and Self-HPO. — **Figure 5.** **Left:** pass@k on SWE-Bench Verified with eight attempts per task. Self-HPO stays above the base agent for every k, and pass@8 rises from 80.0% to 84.8%. **Middle:** pass@1 grouped by how often the base agent solves each task. The biggest gains are on tasks the base agent solves only sometimes (+17.2 on Medium, +12.9 on Easy). Hard tasks, which the base agent never solves in eight attempts, gain +6.1, and tasks the base agent always solves drop by 2.2. **Right:** share of RL training groups in which no attempt succeeds.

Pass@1 by task difficulty. — **Figure 5.** **Left:** pass@k on SWE-Bench Verified with eight attempts per task. Self-HPO stays above the base agent for every k, and pass@8 rises from 80.0% to 84.8%. **Middle:** pass@1 grouped by how often the base agent solves each task. The biggest gains are on tasks the base agent solves only sometimes (+17.2 on Medium, +12.9 on Easy). Hard tasks, which the base agent never solves in eight attempts, gain +6.1, and tasks the base agent always solves drop by 2.2. **Right:** share of RL training groups in which no attempt succeeds.

Most of the gain is reliability: tasks the base agent could already solve some of the time now get solved most of the time. But the gains on hard tasks and the higher pass@8 show that the agent also solves tasks it could not solve before. The improvement is not uniform, though: tasks the base agent always solved drop slightly.

The right panel of Figure 5 shows one reason the loop helps training too. With pass/fail rewards, RL learns nothing from a group of attempts where every one fails. In Self-HPO's final P-step, such groups fall to 6.5% over the last ten updates. In RLVR they stay mostly between 11% and 17% over its last 50 updates. Fewer such groups means more of each batch carries a learning signal.

What matters in the recipe

Table 3. Ablations on SWE-Bench Verified. Each row changes one part of HPO.

Configuration (SWE-Bench Verified)	Resolve rate
HPO	71.6
P-step uses SFT on Opus 4.8 trajectories	67.2 (−4.4)
P-step uses RFT on its own successes	70.5 (−1.1)
H-step reuses the first search set	70.1 (−1.5)
H-step uses the policy itself as proposer (= Self-HPO)	69.5 (−2.1)

RL is the right update. Fine-tuning on the policy's own successful trajectories (RFT) is close, but RL is still 1.1 points better. Distilling Opus 4.8 trajectories is 4.4 points worse.
The search set should move with the policy. Searching on the same initial tasks every round costs 1.5 points. The failures that mattered for an earlier policy are not always the useful ones for the next.
A stronger proposer helps, but you do not need one. Swapping Opus 4.8 for the policy itself costs 2.1 points, and the result still beats every self-improving single-component baseline.

Citation

@article{wei2026selfhpo,
  title   = {Self-Evolving Coding Agents via Harness-Policy Optimization},
  author  = {Wei, Zhepei and Yao, Wenlin and Peng, Baolin and Wang, Danqing and Wu, Qianhui and Cheng, Hao and Meng, Yu and Gao, Jianfeng},
  journal = {arXiv preprint arXiv:2610.22791},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.22791}
}

References

H. Kim, Y. Lee, G. Lee, C. Finn, K. Lee. WHALE: A Simple Recipe for Joint Harness-Weight Optimization. arXiv:2609.00196, 2026. [link]
M. Chen, C. Lv, G. Zhang, H. Chang, S. Zhou. HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems. arXiv:2606.01779, 2026. [link]
B. Peng, W. Yao, Q. Wu, H. Cheng, et al. Orchard: An Open-Source Agentic Modeling Framework. arXiv:2605.15040, 2026. [link]
H. Zhang, S. Zhang, K. Li, et al. Self-Harness: Harnesses That Improve Themselves. arXiv:2606.09498, 2026. [link]
J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, O. Press. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. NeurIPS 2024. [link]
Y. Lee, R. Nair, Q. Zhang, K. Lee, O. Khattab, C. Finn. Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv:2603.28052, 2026. [link]