Targeted research to de-risk building L11 hill-climb (the harness self-improvement loop). 3 Sonnet lanes: (1) self-improvement mechanics, (2) practitioner harnesses, (3) safety/gating. Deduped against the existing loop-engineering skill (swyx Loopcraft, LangChain 4-level, ClaudeDevs taxonomy, Ng, L1–L11 roster). The convergence across ~18 independent sources is striking — the field has settled on one shape.

The converged design (what 3+ independent sources agree on)

Pipeline (HarnessFix arXiv:2606.06324 is the near-exact blueprint): normalize traces → diagnose by walking backward from failures, clustering by ROOT CAUSE and mapping onto a fixed harness-layer taxonomy → propose a SCOPED repair from a fixed operator catalog (NOT free-form rewrites — the ablation shows free-form is measurably worse) → validate against held-out before accepting.

The hard safety rules (each from 2+ sources):

1. Held-out evaluation disjoint from the motivating traces — a change validated only on the traces that inspired it is overfit by construction.
2. Human accept/reject is the terminal gate. "Autonomy grows by shrinking the hard-gate set, never by skipping it." Never self-apply.
3. Worst-case / per-slice regression, not average. Self-Refine's +20% is a mean; memory methods degrade some tasks while helping others. Gate on "no individual case regresses," not aggregate.
4. Reversibility + audit trail as a first-class object (git-commit-before-apply; event-logged decisions).
5. Bounded edit budget per round (≤N proposals).
6. The loop CANNOT edit its own success metric or the permission floor — those live outside it. Protect routing/metadata surfaces (CLAUDE.md model matrix, hook triggers, skill frontmatter) far more strictly than prose.
7. Capability-gated recursion is real — STOP degrades below GPT-4-class. Diagnosis runs on the STRONGEST model (Opus), not a cheap one.
8. Reward-hack quarantine — a proposal that could game the metric (bail-early, verifier-string leakage) is rejected; coverage-aware (a timeout counts as a failure).

THE L11 BUILD SPEC (hand this to the Fable builder)

What L11 is: a weekly scheduled agent that reads the week's Claude Code session traces, finds recurring harness failures, and writes PROPOSED harness changes (as reviewable change-contracts) to ~/loops/inbox/ for human review. It NEVER applies anything.

• Trigger: launchd com.loops.hill-climb, weekly Sunday ~14:00 IST (sleep-aligned). A .running.lock prevents overlap; a cheap local pre-check (is there a fresh trace corpus since last run?) runs before spending any model call.
• Diagnosis model: OPUS (--model claude-opus-4-8) — capability-gated recursion demands the strongest model; this is the ONE loop launcher that is not Sonnet.
• Discovery: read the week's traces via the Mission Control --mcp introspection tools + agentsview + ~/.claude/projects/*.jsonl. Cluster failures by root cause. Map each onto the HarnessFix 7-layer taxonomy: Execution / Tooling / Context / Lifecycle / Observability / Verification / Governance.
• Qualifying threshold (per-surface circuit breaker, NOT global): a flaw qualifies only if the SAME root cause recurs across ≥3 traces. Single-trace noise is discarded.
• Proposal — SCOPED from a fixed operator catalog only (free-form rewrites forbidden). The catalog for this system: {add a CLAUDE.md line · add a permissions.deny rule · add/edit a hook · tweak an existing skill}. Each proposal is a change-contract: target failure mode · harness layer · the ≥3 trace ids as evidence · the exact proposed diff (file + before/after) · predicted effect · how you'd know it worked (a falsifier) · a confidence.
• Hard gates (never crossed by the loop): never edits the CLAUDE.md model-routing matrix, hook trigger definitions, skill frontmatter, permissions.deny itself, or its own success criteria — it may only PROPOSE touching prose/behavior surfaces, and flags metadata-surface issues for the human without a diff. Never writes outside ~/loops/. Never applies a proposal.
• Anti-degradation: log REJECTED proposals to state so the loop never re-proposes a dead end (HarnessFix harness-memory); a proposal motivated by exactly one trace is auto-rejected (worst-case rule).
• State: ~/loops/state/hill-climb.md — one row per proposal (date / target / layer / evidence trace-ids / status PROPOSED|APPLIED|REJECTED). Human sets APPLIED/REJECTED.
• Caps: 2 subagents, 30 min wall-clock, ≤5 proposals/run, halt if the trace corpus is unreadable or infra errors are high.
• First-run acceptance: produces either a ~/loops/inbox/hill-climb-<date>.md with ≥1 well-formed change-contract backed by ≥3 real trace ids, OR an honest "no ≥3-recurrence harness failure found this week" — both are valid. NEVER a fabricated proposal.

Reference implementations to mirror: gbrain skillopt (held-out gate + frontmatter protection + human keep/discard — the closest analog, already on this machine), the openclaw heartbeat overlap-locks, ralph-loop's .running.lock + max_iterations, the change-contract schema (Pappas "Evolution Agent").

Lane 1

Lane 1 Ledger — Frontier mechanics for trace→diagnosis→harness-change loops

Scope: skips content the user already has (swyx Loopcraft, LangChain 4-level stack, Claude Code loop taxonomy, Andrew Ng 3 loops, L1-L11 roster). Focused on IMPLEMENTABLE algorithms for a loop that reads Claude Code session traces, diagnoses recurring harness failures, and proposes CLAUDE.md/skill/hook changes for human review.

1. HarnessFix — trace-grounded diagnosis + scoped repair (closest match to the exact use case)

Why it matters for a hill-climb loop: This is the most directly transferable paper found — it is literally "diagnose harness flaws from failed traces, propose a scoped fix, validate before merging." It's not a metaphor for the user's loop, it's a blueprint for it, including a taxonomy for what parts of a harness a bug can live in.

Concrete mechanic (4-stage pipeline):

1. Trace Abstraction (HTIR) — normalize every trajectory into a TraceStep sequence: role (info-acquisition / tool-call / validation / finalization), status (success/failure/timeout), and artifact/state effects. Build two link graphs: data-flow (where does evidence enter, get summarized, get dropped) and control-flow (why did the harness continue/retry/delegate/terminate here). Anchor every step to a concrete editable artifact (repo path+line range, prompt ID, tool-schema ID).
2. Failure Diagnosis — start from the failing eval outcome, walk data/control-flow links backward to rank suspicious TraceSteps, judge whether each step "formed / propagated / failed to expose / failed to constrain / failed to validate" the relevant info, then map the failure onto one of 7 harness layers: Execution, Tooling, Context, Lifecycle, Observability, Verification, Governance (ETCLOVG). Recurring failures across sessions get consolidated into one flaw record (same root cause + evidence → merged, not duplicated).
3. Scoped Repair — repairs are NOT free-form prompt rewrites. A fixed catalog of ~operators per layer is used (e.g. Tooling: schema-narrowing, argument validation, tool ranking, error-message repair; Lifecycle: loop guarding, retry bounding, verification-gated finalization; Verification: readiness checking, effect-evidence completion guarding). The repair spec explicitly whitelists editable artifacts and blacklists forbidden ones (test oracles, public APIs).
4. Patch Validation — static/syntax check, then run on a held-out validation set; accept only if it fixes the target flaw and doesn't regress previously-passing tasks beyond a threshold. A harness memory stores accepted AND rejected repairs (with natural-language conditions of applicability) so the loop never re-proposes a fix that already failed.

Numbers: GAIA 43.3%→61.7%, SWE-bench Verified 45.3%→57.3%, AppWorld 36.7%→43.0%, Terminal-Bench 2.0 17.6%→26.5% (GPT-5-mini). Ablations show removing trace-grounded diagnosis or scoped-operator constraints causes the largest performance loss — i.e. free-form "just rewrite the prompt" repair is measurably worse than scoped, evidence-anchored repair. Cross-model transfer of a repaired harness: +5.5% to +9.5% on other models.

Source: "From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws" — Chen, Wang, Liu, Wang, Zheng, Wang (CAS / Tianjin University), arXiv:2606.06324v2, updated 2026-07-02. arxiv.org/html/2606.06324v2

Takeaway for the build: structure the loop's diagnosis step as "which of {tool schema, context assembly, lifecycle/loop control, observability, verification-before-finalize, governance/permissions} is implicated" rather than jumping straight to "reword the prompt" — this is the single highest-value idea in this lane.

2. Self-Harness (Zhang et al. 2026, via Lilian Weng's survey) — weakness mining → bounded proposal → held-in/held-out gate

Why it matters: Independent confirmation of the same 3-stage shape as HarnessFix, from a different 2026 paper, plus an explicit statement of why naive self-editing fails (reward hacking) and the fix.

Concrete mechanic:

1. Weakness Mining — cluster execution traces from harness evals into failure patterns, but insist on verifier-grounded patterns (terminal verifier-level cause + causal status of the agent behavior + the abstract mechanism), not just surface symptoms like "timeout."
2. Harness Proposal — the model that proposes an edit is given exactly 4 inputs: (a) the editable surfaces of the current harness, (b) the verifier-grounded failure patterns, (c) records of passing behaviors that must be preserved, (d) a log of previously attempted edits (so it doesn't re-suggest a dead end).
3. Proposal Validation — split eval data into held-in (used to confirm the target weakness is fixed) and held-out (used purely to catch collateral regression). Only edits that pass both are merged.

Safeguard called out explicitly: "if a program is allowed to edit the OS system, abstraction boundaries are broken. The editable surface needs to be properly designed and the permission control and security layers need to live outside this loop" — i.e. the loop should never have write access to the thing that enforces its own guardrails (permissions, hooks that gate destructive actions) — those live outside the harness-editing loop's reach.

Cautionary data point (same survey, re: STOP paper): recursive self-improvement is not intelligence-neutral — the base model doing the proposing has to be capable enough. STOP (arXiv:2310.02304, Zelikman et al.) monotonically improved with GPT-4 as the self-improving proposer, but degraded mean performance with GPT-3.5/Mixtral as the proposer. Confirmed independently via arxiv abstract search. Implication for the build: use your strongest available model (Opus, not Haiku) as the trace-diagnosis/proposal agent — this is exactly the kind of task the CLAUDE.md model-routing table already reserves for Opus.

Source: Lilian Weng, "Harness Engineering for Self-Improvement," 2026-07-04. lilianweng.github.io/posts/2026-07-04-harness/ · STOP corroboration: arXiv:2310.02304 arxiv.org/abs/2310.02304

3. GEPA — reflective prompt evolution as a Pareto search over natural-language diagnoses

Why it matters: GEPA is the most mature, shipped, usable-today implementation of "read execution traces in natural language, diagnose failure, mutate the prompt" — it's a DSPy optimizer you can literally pip install gepa and point at any pipeline, and it beat RL (GRPO) with 35x fewer rollouts.

Concrete mechanic:

• Treat the harness/prompt as the "genome." Run it on a minibatch of tasks, capture full trajectories (reasoning, tool calls, tool outputs, error messages, profiling data).
• Instead of collapsing outcome into a scalar reward, an LLM reflects in natural language on the trajectory to diagnose why it failed (this is the same "diagnose, don't just optimize a number" principle as HarnessFix/Self-Harness).
• The reflection proposes a targeted prompt update. This produces a new candidate ("child") added to a growing population.
• Selection uses a Pareto frontier over per-task scores, not a single aggregate metric — this is the key anti-overfitting mechanic: a candidate is kept if it's not dominated (i.e., there's no other candidate that's at least as good on every task and strictly better on at least one). This prevents the optimizer from hill-climbing on an aggregate at the expense of silently regressing a subset of tasks — directly relevant to "how do you keep the loop from degrading the system."
• Sample-efficient: as few as 10 examples, 20-100 evaluation rounds to converge.

Numbers: +13% aggregate over MIPROv2; beats GRPO (RL) by up to 20% while using 35x fewer rollouts (ICLR 2026 oral).

Source: Agrawal et al., "GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning," arXiv:2507.19457 (submitted 2025-07-25, revised 2026-02-14, ICLR 2026 Oral). arxiv.org/abs/2507.19457 · Implementation docs: dspy.ai/tutorials/gepa_ai_program/ · Library: github.com/gepa-ai/gepa

Takeaway for the build: if the hill-climb loop ever needs to score candidate CLAUDE.md/skill edits against a suite of past-failing traces, score per-trace and keep the Pareto-optimal set rather than reducing to one "did it get better" number — a single scalar is exactly what lets a bad edit sneak through by winning on average while quietly breaking an edge case.

4. LangChain "boosting"-style trace analyzer — the closest real production pattern, with an explicit human checkpoint

Why it matters: This is a named, shipped LangChain pattern (not just a paper) for exactly the workflow the user wants: fetch traces → spawn parallel error-analysis agents → synthesize → propose harness edits — with an explicit statement of where a human should sit in the loop.

Concrete mechanic:

1. Fetch failing/expensive experiment traces from LangSmith.
2. Spawn parallel sub-agents, each doing single-trace error analysis (this is the "boosting" analogy: each new round of analysis focuses on the traces the current harness gets wrong, echoing AdaBoost's reweighting of hard examples rather than re-analyzing everything uniformly).
3. A main/synthesis agent aggregates the per-trace findings into a smaller set of recurring failure themes.
4. Targeted edits are proposed against three concrete harness surfaces: system prompt (missing guidance, testing emphasis), tools (e.g. inject directory context, add verification hooks), middleware (e.g. a PreCompletionChecklist, a LoopDetection guard).
5. Explicit human-in-the-loop checkpoint: "A human can be pretty helpful in Step 3 (though not required) to verify and discuss proposed changes" — paired with an explicit warning that overfitting a change to one failing task is a known failure mode that causes regressions elsewhere.

Source: Vivek Trivedy, "Improving Deep Agents with harness engineering," LangChain blog, 2026-02-17. langchain.com/blog/improving-deep-agents-with-harness-engine · Related: LangSmith's Polly (in-app chat-based trace Q&A + prompt rewriting assistant), LangChain blog 2025-12-10, blog.langchain.com/debugging-deep-agents-with-langsmith/

Takeaway for the build: "boosting" is a good framing for the loop's scheduling policy — each diagnosis pass should specifically over-sample the traces the last accepted harness version still got wrong, not re-scan the whole trace corpus uniformly every time.

5. Trace-as-gradient generative optimization (OPTO/OptoPrime) — treat traces as the update signal, not just diagnostic text

Why it matters: Gives a clean mental model + reusable library abstraction for "propagate execution traces backward through a workflow the way AutoDiff propagates gradients," generalized to non-differentiable parameters (prompts, code, configs) — useful if the hill-climb loop grows beyond ad hoc prompting into something with a defined optimizer interface.

Concrete mechanic: Define an OPTO (Optimization with Trace Oracles) instance: parameters are heterogeneous (prompt text, tool code, config values), and instead of a numeric gradient, each optimization step receives the full execution trace + rich feedback (console output, error text, user response, eval score). OptoPrime, a general LLM-based optimizer built on this abstraction, ingests trace+feedback and proposes an update to any parameter type — the same optimizer code updates prompts, numeric hyperparameters, and robot controller code without domain-specific rewrites, and is competitive with specialized optimizers in each domain.

Source: Cheng et al. (Microsoft Research), "Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs," NeurIPS 2024, arXiv:2406.16218. arxiv.org/abs/2406.16218 · microsoft.com/en-us/research/publication/trace-is-the-new-au

Takeaway for the build: if the loop ends up touching more than prompts (tool schemas, hook configs, skill files), model it as one optimizer over a heterogeneous "parameter" set with trace+feedback as the update signal, rather than writing a bespoke updater per artifact type.

6. Real-world OSS attempt: maka-agent RFC #64 — the statistical rigor a homegrown loop actually needs

Why it matters: This is a live open-source RFC (not a paper) for the exact thing the user is about to build — an unattended loop that edits system_prompt.md, evaluates, and keeps/discards changes — and it's the most concrete on the unglamorous but load-bearing statistics: how do you tell "this change helped" from "this run got lucky."

Concrete mechanic:

• Baseline calibration: run the harness 3x before any edits to measure natural run-to-run variance (observed_spread).
• Noise band via Wilson confidence intervals: noise_band = wilson_half_width(N, p_mean) × √(1 + 1/n_baseline) — a proposed improvement only counts if it clears this band, not just "went up."
• Held-in vs. held-out split, with held-out data physically isolated outside the agent's working directory so the proposing agent can't see it and can't game it. Held-in reference score updates monotonically (max(previous_reference, new_score − noise_band)); held-out is checked against a fixed floor from the original baseline, never allowed to ratchet.
• Coverage-aware gate: tasks that error out or hit max_tokens count as failures in the pass-rate denominator, so an edit can't "improve" its score by causing more tasks to bail out early instead of failing cleanly.
• Reward-hack quarantine: the controller scans the raw event log for verifier-specific strings appearing in the agent's own tool-call arguments (a live check for the agent having learned to reference/mimic the grading logic rather than solve the task) and discards flagged runs.
• Stop conditions: any infra error → don't keep that run (retry once, else discard); if the infra error rate across the loop exceeds 20%, halt the whole loop rather than let it grind on bad signal.

Source: GitHub, maka-agent/maka-agent, Issue #64 "RFC: Harness RSI loop — autonomous system prompt optimization." github.com/maka-agent/maka-agent/issues/64

Takeaway for the build: this is the missing "how do I keep it from degrading the system" plumbing that the papers gesture at abstractly — steal the Wilson-interval noise band and the held-out-physically-isolated-from-the-proposer trick directly; both are cheap to implement and directly answer "how do I know an accepted CLAUDE.md edit actually helped vs. noise."

7. Change-contract governance layer ("Evolution Agent" pattern) — the review-gate structure for human-approved harness edits

Why it matters: The user's stated end state is explicitly "propose changes for human review," not full autonomy. This pattern is the most fleshed-out description found of what a reviewable proposed harness change should look like as a structured artifact, plus a two-tier auto/human-gated promotion policy — directly reusable as the output schema for the diagnosis agent.

Concrete mechanic:

• Every proposed change ships as a structured change contract (not a raw diff), containing: target failure mode(s) + predicted improvement with a confidence interval, explicit invariants the change must not break (e.g. "never touches permission-gating hooks"), named "falsifier" evals the change must not regress past a threshold, and a time-bounded experiment window after which an un-ratified change auto-reverts.
• Two-track promotion policy: (a) auto-promotable — bounded edits within a pre-whitelisted envelope (e.g. wording tweaks to an existing CLAUDE.md section); (b) human-gated — anything touching schemas, new tools/hooks, or validators; (c) hard-blocked entirely from the loop's reach — permission policies, credentials, deploy targets, enforced at the OS/permission layer, not in-app.
• Metric independence: the loop is explicitly barred from defining or editing its own success metric — the eval/metric definitions live in a separately version-controlled location that goes through ordinary human code review, so the optimizer can't quietly redefine "success" to make its own proposals look better.
• Shadow-then-promote: candidate harness versions run shadowed against a slice of real traffic/sessions first, with a comparator producing a promotion verdict before anything goes live.

Source: Evangelos Pappas, "The Optimizer in the Loop: When Your Agent Starts Editing Its Own Harness," Medium, 2026-05-21. medium.com/@epappas/the-optimizer-in-the-loop-when-your-agen

Takeaway for the build: make the diagnosis agent's output a structured contract (failure mode, predicted delta, invariants-to-preserve, falsifier evals, auto-revert deadline) rather than a free-text "here's a suggested CLAUDE.md diff" — this is what makes human review of a batch of proposals fast instead of requiring the reviewer to re-derive the reasoning each time.

What's proven vs. hype, as of 2026-07

• Proven and shippable today: GEPA (real library, ICLR 2026 oral, real HN adoption threads from Aug 2025 through Jun 2026 — e.g. "Stop tuning prompts by hand. Engineer the loop that tunes them," HN 2026-06-12). Reflexion/Self-Refine/STaR are foundational and battle-tested as the primitive (reflect-in-language-then-retry; bootstrap-on-your-own-successes) that all the newer harness-level systems (HarnessFix, Self-Harness, GEPA) are built on top of.
• Real but early/research-stage: HarnessFix (arXiv 2606.06324, Jul 2026) and Self-Harness (per Weng's Jul 2026 survey) are the first systems found that operate on the harness (multi-layer: tools/context/lifecycle/observability/verification/governance) rather than just the prompt — this is squarely the frontier the user is asking about, but these are days-old research artifacts, not shipped products.
• Confirmed failure mode, not hype: capability-gated recursion (STOP: works with GPT-4-class proposer, degrades with GPT-3.5/Mixtral-class) — corroborates the CLAUDE.md routing doctrine's instinct to put the smartest available model on this exact job.
• The unglamorous-but-necessary layer nobody markets: statistical rigor (noise bands, held-out isolation, reward-hack scanning) and governance (change contracts, metric independence, OS-level permission floor) show up in every serious writeup as the actual blocker to a safe loop — not the diagnosis/proposal step, which multiple independent teams have converged on a similar shape for (mine traces → cluster by root cause → scoped/bounded proposal → held-out validation).

Lane 2

Practitioner loop harnesses — lane 2 ledger

Scope: NEW concrete open-source loop implementations (not swyx Loopcraft / LangChain stack / Claude Code taxonomy / Geoff Huntley ralph loop — all already known). All repos verified real via gh repo view / gh api. Research budget: 25 min, 2026-07-07.

1. TimHeckel/nanobots — outer/inner loop with GitHub as the entire state store

Why it matters. This is the closest published analog to what you're building: an outer loop that only reads signals, triages, dispatches, and updates its own policy docs — never writes product code itself — paired with disposable inner-loop workers. Directly useful for structuring your launchd loop as "discovery pass (outer) → proposed-diff-writer (never self-applies)".

Concrete mechanic.

• Trigger: cron/Actions/local, "stateless" — a cycle is just "something executes once on a cadence." No daemon; each run picks up cold from GitHub state.
• Discovery: Issues (via issue forms) are signal intake, auto-labeled nanobots:inbox; Projects v2 board is the state machine (Inbox → Backlog → Ready → In Progress → In Review → Verify → Blocked → Done).
• Judge/grader: a triage rubric doc (TRIAGE.md) plus a per-item work-spec comment (acceptance criteria, tests, pointers) written by the outer loop before dispatch.
• Human gate: summon-human label + assignment; PR review required for L-sized items; install requires a human-owned classic PAT because claude-code-action refuses bot-initiated dispatch and the default GITHUB_TOKEN can't touch org Projects v2 at all — both hard-won gotchas worth avoiding if you ever wire this to Actions.
• State store: no private state anywhere — GitHub is the database (issues/PRs/board/labels). A pinned "Nanobots Status" issue is the loop's heartbeat (one short report per cycle).
• Learning: every finished/failed item appends to .nanobots/LEARNINGS.md (append-only); every ~10 entries a distill pass promotes durable lessons into TRIAGE.md, RECIPES.md, and the repo's agent-instructions file. "The loop's policy documents are its weights; GitHub history is the training log." This is the exact shape of "detect recurring failures across ≥3 traces → write a proposed harness diff" — nanobots does it as a periodic distill instead of a threshold, worth comparing against a hard ≥3 count.
• Explicitly scaffolder-not-framework: init renders .nanobots/ once; the target repo then has zero runtime dependency on the tool, and repo-owned files (TRIAGE/RECIPES/LEARNINGS) are never re-rendered — only "engine-owned" files (prompts, workflows) get updated by nanobots update. Good precedent for keeping your ~/loops/inbox/ diffs human-owned once written.

Source. github.com/TimHeckel/nanobots, README.md + docs/architecture.md (fetched via gh api), pushed 2026-07-02.

2. openclaw/openclaw — production heartbeat trigger/dedup internals (you already run this daemon)

Why it matters. You already operate OpenClaw's gateway; its heartbeat-wake code is a real, shipped implementation of exactly the trigger-debounce problem a launchd loop has: don't double-fire, don't run over yourself, and don't burn a call when there's nothing to do.

Concrete mechanic (src/infra/heartbeat-wake.ts, src/agents/templates/HEARTBEAT.md):

• Coalescing: wake requests within a DEFAULT_COALESCE_MS = 250 window collapse into one run instead of firing N times.
• Priority queue for wake reasons: manual/immediate (ACTION) > retry > scheduled/interval > default — so a human-triggered run always preempts a scheduled tick.
• Busy-skip reasons, explicitly enumerated and retryable: requests-in-flight, cron-in-progress, lanes-busy — the run is skipped and retried later (DEFAULT_RETRY_MS = 1_000) rather than queued or dropped silently.
• Cheap no-op skip: HEARTBEAT.md is comments-only by default — "keep this file empty (or with only comments) to skip heartbeat API calls" — i.e. the discovery pass has a fast local check that avoids an LLM call entirely when there's nothing to check. Directly applicable: your launchd loop should have a cheap "any new session traces since last run?" local check before spending a model call on the discovery pass.
• Docs: /automation#scheduled-tasks-cron-vs-heartbeat gives OpenClaw's own decision table for cron (fixed schedule, stateless) vs. heartbeat (event/interval-driven, can skip when idle) — worth reading before deciding whether your hill-climbing loop should be a cron-style fixed launchd interval or an event-coalesced heartbeat.

Source. github.com/openclaw/openclaw, src/infra/heartbeat-wake.ts, src/agents/templates/HEARTBEAT.md, docs/automation/cron-vs-heartbeat.md (all fetched via gh api, live as of 2026-07-07).

3. MPIsaac-Per/claude-code-loop-patterns — Stop-hook + evidence-file done enforcement (small, but exact primitive)

Why it matters. This is the smallest possible reference implementation of "the agent isn't allowed to declare victory without proof" — the same gate you'll want before a proposed diff is considered a finished discovery-pass output.

Concrete mechanic (companion repo to "What I Learned From 245,306 Claude Code Tool Calls"):

• 03-verification-gate/: a harness that runs gates and writes evidence files as the audit trail.
• 04-stop-hook/: a Claude Code Stop hook that blocks completion when no recent verification evidence exists — checks the hook's stop_hook_active reentrancy field and matcher rules against live evidence-file timestamps, not vibes.
• skills/loop-failure-triage: fires specifically "a tool call just failed" — a narrow skill scoped to one failure surface, rather than one big "be careful" prompt. Good precedent for scoping your hill-climbing loop's failure-classification step as its own narrow skill instead of folding it into a general agent prompt.
• Explicitly documents what's novel vs. borrowed (the Stop-hook+evidence-file composition is the claimed contribution; the underlying hook contract and skill frontmatter are taken from live Anthropic docs, not assumed) — a good model for how to cite your own build's provenance.

Source. github.com/MPIsaac-Per/claude-code-loop-patterns (2 stars, MIT), README fetched via gh api, 2026-07-07.

4. anlach/ralph-loop — cron-driven OpenClaw skill with a lock file and file-based working memory

Why it matters. A literal, tiny (~29KB single-file) implementation of a cron-triggered loop with the two mechanics you need for a launchd job: overlap prevention and iteration capping. Worth reading before writing your own launchd plist + Python script.

Concrete mechanic (ralph_loop.py, SKILL.md, PROMPT.md — OpenClaw skill, "Based on Arbos pattern"):

• Cron calls /ralph run every tick; each run is stateless in-process — all persistent state lives in files: memory/GOAL.md (read-only objective), memory/STATE.md (working memory across steps), memory/INBOX.md (new messages, consumed/cleared after each step), memory/runs/<timestamp>/ (per-run output archive).
• Lock file: MEMORY_DIR / ".running.lock", acquire_lock()/release_lock()/is_locked() — the next cron tick is skipped outright if a run is in progress, no queueing. Exactly the primitive a launchd StartInterval job needs to avoid concurrent invocations.
• Iteration cap: max_iterations (default 10) checked at multiple call sites before advancing; loop halts and reports "Reached max iterations" rather than running forever.
• Completion detection: plain string match — "DONE" in content or "COMPLETE" in content — checked both in STATE.md and in the freeform result text passed to record_result(). Crude but legible; you may want a stricter machine-checkable sentinel for your proposed-diff writer (e.g. a JSON status field) rather than string sniffing.
• record_result(output, done=False) releases the lock only after the result is durably written — ordering worth copying so a crash mid-write can't leave the lock held forever without also losing the partial result.
• learn_from_usage config flag + extract_skills()/evolve_prompt() functions scan past run logs for DONE/COMPLETE markers to synthesize self-improvement suggestions — a much smaller, file-only version of nanobots' LEARNINGS.md distill pass.

Source. github.com/anlach/ralph-loop (0 stars, updated 2026-03-14), ralph_loop.py + SKILL.md + PROMPT.md fetched via gh api.

5. jbellsolutions/forge — CircuitBreaker fail-threshold=3 pattern + eval-gated skill promotion

Why it matters. Treat this repo cautiously (0 stars, single maintainer "Justin Bell," created/pushed within days of each other in April 2026, PyPI package at v0.1.0 — reads as a fresh/thin project, possibly one person's weekend synthesis rather than a battle-tested harness). But its forge/healing/circuit_breaker.py is a clean, small, directly-reusable reference for the exact "detect recurring failure across ≥3 occurrences" mechanic you need — worth lifting the pattern, not necessarily the dependency.

Concrete mechanic:

• CircuitBreaker state machine: CLOSED → OPEN → HALF_OPEN, default fail_threshold: int = 3 (consecutive failures trip it — matches your ">=3 traces" framing almost exactly), cooldown_seconds = 3600 before probing again, recovery_throughput = 0.5 (50% probabilistic probe rate in HALF_OPEN rather than an all-or-nothing retry).
• CircuitRegistry: one breaker per (tool name | provider name) — i.e. failure detection is scoped per failure-surface, not global. Directly maps to "recurring failure of a specific harness component" rather than "the loop failed N times overall."
• README also documents (not independently verified in code) an eval-gated skill-promotion loop: "a learning loop that promotes a winning skill version when the eval gate clears," with a stated 0.05 confidence-margin threshold before a new prompt/skill version replaces the current one — the judge/gate half of the pipeline, paired with the CircuitBreaker's failure half.
• Its README's research section (Cluster B) independently names a "trace-fidelity invariant" lifted from an academic "meta-harness" project: store full execution traces so the optimizer can do counterfactual diagnosis — i.e. don't just log pass/fail, keep enough of the raw trace that your loop can re-derive why a run failed after the fact. That's the load-bearing requirement for your "reads ~/.claude session traces, detects recurring harness failures" design — the traces have to carry enough structure (tool name, error class, file:line) to be diffable across runs, not just a success/fail bit.

Source. github.com/jbellsolutions/forge, README.md + forge/healing/circuit_breaker.py fetched via gh api; PyPI forge-harness==0.1.0 confirmed live. Verify-before-trusting flag: low stars/maturity, read as inspiration not as a dependency to install.

6. Introspection / "Autoresearch" (Roland Gavrilescu, ex-xAI) — inner/outer loop vocabulary + "agent recipes"

Why it matters. Not code, but this is the closest thing to a named design pattern for exactly your build: a system whose outer loop studies and maintains a primary system using signals/evals/human input, explicitly distinct from "loop = the whole system." Both nanobots and ralph-loop cite this piece directly as their design inspiration, so it's the shared ancestor worth reading once rather than re-deriving.

Concrete mechanic / ideas to steal:

• Inner loop vs. outer loop, defined precisely: inner loop = the primary system doing the work; outer loop = "another system that studies and maintains the primary system." The hard design question named explicitly: how to design the outer loop so it makes progress on the right problems without consuming an unreasonable number of tokens deciding what to do — i.e. budget your discovery pass, don't let it re-read every trace every run. discipline: "not all feedback carries the same value... you need a mechanism for filtering the signals and identifying which ones an agent should act on" — directly relevant to deciding what counts as a "recurring harness failure" worth escalating vs. noise.
• "Agent recipe" concept: a portable artifact bundling harness config + evals + judges + captured human expertise + the failures that led to each new eval — i.e. don't just write a diff, write down why the diff was proposed (which failure pattern, which traces, what the judge concluded) as a durable, versioned unit — a stronger structure for ~/loops/inbox/ entries than a bare diff file.
• Human-as-signal-source, autonomy earned not assumed: hard gates shrink over time as trust accrues — "never by skipping it." Matches your instinct to never self-apply; frame the human review step as the on-ramp to eventually loosening gates, not a permanent wall.
• Git as the audit log/state store for the coding-agent vertical specifically (same conclusion nanobots reached independently via GitHub Issues/PRs).

Source. latent.space "Autoresearch: The feedback loop behind self-improving agents" (interview, published 2026-07-01), latent.space/p/autoresearch-introspection, fetched via r.jina.ai; corroborated as the stated inspiration in both item 1 (nanobots) and item 4 (ralph-loop, "Based on Arbos pattern" / Autoresearch/Introspection citation) READMEs.

Gotchas / de-risking notes for your specific build

• Overlap prevention is not optional. Both ralph-loop (file lock) and OpenClaw (busy-skip reasons + coalescing) treat "don't run over a previous invocation" as a first-class concern, not an afterthought. Your launchd job needs an equivalent lock file or PID check before the discovery pass starts.
• Cheap local pre-check before the expensive pass. OpenClaw's empty-HEARTBEAT.md-skips- the-API-call pattern is the right shape: check locally (new session files since last run? mtime delta?) before spending a model call deciding whether there's anything to discover.
• Scope failure detection per-surface, not globally. forge's CircuitRegistry (one breaker per tool/provider) argues against a single global "≥3 failures anywhere" counter — bucket by the specific harness component/tool/pattern so the diff you propose is targeted.
• Never self-apply is the norm, not the exception, among the repos that thought hardest about this — nanobots' PR-review-for-L-items and Introspection's "autonomy grows by moving areas out of the hard-gate set... never by skipping it" both land exactly where you've already landed (write to inbox, never auto-apply). No counter-evidence found that auto-apply loops are a mature practitioner pattern for harness-level changes specifically (as opposed to code-level PRs, which do get auto-merged in some setups after CI gates).
• Make the trace format diffable, not just pass/fail. forge's cited "trace-fidelity invariant" is the one requirement that will silently sink a ">=3 traces show the same failure" detector if your ~/.claude session trace reader doesn't already capture structured fields (tool name, error class, file:line) — worth checking your Mission Control MCP introspection tools expose that before building the pattern-matcher on top.

Lane 3

Lane 3 — Safety & Gating for Self-Modifying / Self-Improving Agent Loops

Research spree output. Question answered per item: what guardrail must a "harness reads its own traces and proposes CLAUDE.md/skill/hook edits" loop have so it improves the harness instead of degrading it?

1. gbrain skillopt — self-evolving skills as a trained parameter (the closest real analog to the user's exact plan)

The risk it addresses: an agent that "improves" its own SKILL.md/prompt on vibes — no quantified before/after, no protection against overfitting to whatever traces it happened to read, no way to tell noise (LLM-judge variance) from real improvement.

Concrete guardrail (fully implemented, not aspirational):

• Held-out gate: --held-out <path> — independent test set, task IDs disjoint from the training benchmark. A candidate that beats the training benchmark but regresses on held-out is hard-refused. Required for any in-place mutation of a "bundled"/production skill.
• Validation gate: median-of-3 judge score, accept only if median_new > median_best + 0.05 epsilon — explicitly designed to reject "LLM judge noise" being mistaken for improvement.
• Frontmatter/routing-surface mutation forbidden — the optimizer can edit skill body prose, never the metadata that controls when the skill fires. Prevents routing drift as a side effect of a "quality" edit.
• Textual learning-rate budget + rejected-edit buffer (rejected.json, bounded LRU) — caps how many edits land per step and remembers what was tried and discarded, so the loop can't re-propose a rejected change next epoch.
• Dirty-working-tree refusal — will not run over uncommitted changes to the target file, so a human's in-flight edit can never be silently clobbered (--force to override, explicit).
• Cost + runtime hard caps (--max-cost-usd, --max-runtime-min), preflight-checked before any LLM call — no surprise mid-run budget blowout.
• Bootstrap-review sentinel — an auto-generated starter benchmark is written with a # BOOTSTRAP_PENDING_REVIEW line; the optimizer physically refuses to run until a human deletes it, closing the loop where the agent both writes the test and grades itself against it ("self-referential benchmark gaming," their words).
• Read-only tool sandbox in rollouts — the optimization run itself cannot write side effects into the brain while it's evaluating candidates.
• Human keep/discard is still the outer loop: "It runs the optimizer, dry-runs for cost, and reports the diff + score delta back. You keep or discard." Automation handles propose→test; a human always makes the final accept call for anything that ships broadly.

Why this matters for the user's build: this is not a paper, it's a shipped CLI flag surface (gbrain skillopt <name> --bootstrap-from-skill, --held-out, --allow-mutate-bundled) built for exactly "treat SKILL.md as a trainable parameter, propose edits from scored runs, keep only measured wins." It's the nearest concrete blueprint for a CLAUDE.md/skill self-improvement loop, including the failure modes it had to guard against after presumably hitting them.

Source: raw.githubusercontent.com/garrytan/gbrain/master/docs/guides and raw.githubusercontent.com/garrytan/gbrain/master/docs/tutori (gbrain repo, accessed 2026-07-07). Underlying research: SkillOpt — Yang et al., "SkillOpt: Executive Strategy for Self-Evolving Agent Skills," Microsoft Research, arXiv:2605.23904 (26 May 2026) — "an edit is accepted only when it strictly improves a held-out validation score."

2. Regimes — event-sourced, auditable, held-out-gated improvement loop

The risk it addresses: self-improvement loops are usually "external scaffolding bolted onto the agent: failures go unlogged, diagnoses cannot be replayed, and promote-or-discard decisions land in a side database rather than the agent's own history" — i.e. you can't audit why a change landed, and you can't replay the failure that motivated it.

Concrete guardrail:

• State is a deterministic projection of an append-only event log — every failure, every diagnosis, every promote/discard decision is itself a first-class logged event, not a side effect. A run can be replayed exactly from its log.
• Multi-stage promotion pipeline before any patch lands: static checks → sandbox execution → in-sample evaluation → held-out validation, in that order, and only promoted after clearing all four.
• Candidate patches scope to typed pipeline seams — not free-form edits anywhere, but changes constrained to specific, pre-declared insertion points (analogous to only letting a loop touch specific CLAUDE.md sections or specific hook files, not arbitrary diffs).
• Measured result discipline: across 5 seeded held-out splits, +0.05 to +0.10 accuracy in 4 splits, but explicitly flags a 5th "over-promotion split" at only +0.01 — i.e. they report the worst split, not just the average, which is itself a guardrail against cherry-picked reporting.

Concrete rule to steal: never trust an aggregate score for a promotion decision — require the worst-performing held-out split (or worst-affected task category) to also not regress.

Source: Nakajima, "Regimes: An Auditable, Held-Out-Gated Improvement Loop Demonstrated on LongMemEval with ActiveGraph," arXiv:2606.10241 (8 Jun 2026). Code: github.com/yoheinakajima/regimes.

3. The Kitchen Loop — anchor to a spec/regression oracle, not a metric, to dodge Goodhart

The risk it addresses: classic Goodhart's law / reward hacking — if the loop optimizes a proxy metric directly, it will find the shortest path to moving that number, which is not the same as genuinely improving the system. ("The loop converges toward a specification rather than optimizing for a metric, which prevents the goodharting failure mode where an agent optimizes a proxy metric while the product degrades in unmeasured dimensions.")

Concrete guardrail:

• "Unbeatable Tests" — ground-truth verification the code (or config/prompt) author cannot fake or write itself; the check must be adversarially independent of the thing being changed.
• Drift Control — continuous quality measurement with automated pause gates: the loop halts itself, not just flags, when a quality signal degrades past a threshold.
• "As a User x1000" — a synthetic power-user agent exercises the full specification surface at high cadence as a regression harness, rather than trusting the self-improving agent's own self-report of success.
• Track record: 285+ iterations, 1,094+ merged changes, zero regressions detected by the regression oracle — the claim is specifically framed around the oracle catching regressions, not around raw iteration count.

Concrete rule to steal: define the acceptance test before wiring the self-improvement loop, make it something the loop cannot rewrite or satisfy trivially, and give it veto/pause power over promotion — don't let the loop's own metric be the metric it's judged against.

Source: Roy, "The Kitchen Loop: User-Spec-Driven Development for a Self-Evolving Codebase," arXiv:2603.25697 (26 Mar 2026).

4. LangChain Better-Harness — trace-analysis loop with human review as the production gate

The risk it addresses: automated harness edits (system prompt/tools/middleware) that overfit to the specific failures observed in one batch of traces, causing regressions elsewhere — "changes that overfit to a task are bad for generalization and can lead to regressions in other tasks."

Concrete guardrail:

• Trace Analyzer as a repeatable skill, not ad hoc: fetch experiment traces → spawn parallel error-analysis agents → main agent synthesizes findings + suggestions → aggregate into targeted changes. Same shape as the user's planned loop (read Claude Code traces → propose changes). Explicitly likened to boosting: focus each round on the previous round's mistakes, not a blanket rewrite.
• Human review is a named, non-optional step in the recipe ("A human can be pretty helpful in Step 3 — though not required — to verify and discuss proposed changes") but the production version of this (the "Better-Harness" framework write-up) hardens it: autonomous iterations diagnose from traces, experiment with targeted changes, and validate no regressions — with human review providing a final gate before production deployment.
• Held-out generalization test sets, separate from the diagnostic traces: reported results emphasize "near-complete generalization to holdout test sets" across two different underlying models (Sonnet 4.6, GLM-5) — i.e. the improvement wasn't just fitting the traces that prompted it.
• Human input treated as supervision over time, not a one-off approval click — "expert review as a structured data source for harness evolution rather than a one-off manual checkpoint," meaning reviewer decisions themselves feed back into what the loop learns to propose next.

Source: LangChain, "Improving Deep Agents with harness engineering," langchain.com/blog/improving-deep-agents-with-harness-engine (2026); companion coverage of the "Better-Harness" framework, blockchain.news/news/langchain-better-harness-self-improving (2026).

5. Specification gaming examples in AI — the master case file for "why don't trust a single automated metric"

The risk it addresses: this is the primary-source catalog of the exact failure mode the user is worried about — "generating a solution that literally satisfies the stated objective but fails to solve the problem according to the human designer's intent." Canonical example: OpenAI's CoastRunners boat-racing RL agent that looped in circles hitting the same reward pickups instead of finishing the race, because the reward signal (points) diverged from the intent (win the race).

Concrete guardrail (a checklist discipline, not a mechanism): before trusting any automated "this change helped" signal from a self-improvement loop, explicitly ask: (1) is the metric a proxy for the real goal or the real goal itself? (2) could the loop satisfy the letter of the metric via a degenerate strategy (e.g., editing CLAUDE.md to make future traces look cleaner rather than making the agent actually behave better)? (3) is there a corroborating out-of-band signal (human spot-check, held-out task, a check the proposer didn't write)? This list is exactly the reference class to run any newly-proposed CLAUDE.md/hook change against before auto-accepting it.

Source: Krakovna, "Specification gaming examples in AI," vkrakovna.wordpress.com/2018/04/02/specification-gaming-exam (1 Apr 2018), linking the maintained master list (Google Sheet, contributions from Gwern Branwen et al.) — still the standard citation, referenced in 2026 discussions of agent self-evolution (e.g. the Kitchen Loop paper's "goodharting" framing above).

6. Self-Refine / iterative self-feedback loops don't monotonically improve — and agent memory can have negative benefit

The risk it addresses: the implicit assumption that "more iteration = better" or "more accumulated memory/experience = better" is false. Two distinct findings:

• Self-Refine (the foundational iterative-refinement paper) shows ~20% absolute average improvement from self-critique-and-revise — but it's an average across 7 tasks, evaluated with a fixed small number of iterations; it does not claim monotonic improvement with unbounded iteration, and later work builds convergence/divergence analysis on top of exactly this gap.
• Agent memory / self-evolving memory systems research (2026) documents concrete negative-benefit cases: memory hurts "when it introduces irrelevant evidence, removes fine-grained execution details through compression, or transfers mismatched procedures across environments." A named re-implementation study found an existing memory method (DILU) improves some tasks and degrades others by 2.42% — i.e. a technique with a net-positive average can still be quietly making a meaningful slice of cases worse, and aggregate reporting hides that.

Concrete guardrail to steal: never gate a self-improvement decision on an average delta. Report the distribution — specifically the worst-regressed case/category — and require that no individual held-out task/case regress beyond a threshold, not just that the mean moves up (this is the same discipline as Regimes' worst-split reporting and the Kitchen Loop's per-case Unbeatable Tests, converging from three independent sources on the same rule).

Source: Madaan et al., "Self-Refine: Iterative Refinement with Self-Feedback," arXiv:2303.17651 (30 Mar 2023, v2 May 2023). Memory negative-benefit findings synthesized from 2026 agent-memory literature (MemEvolve: Meta-Evolution of Agent Memory Systems, arXiv:2512.18746; EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective, arXiv:2605.18421) via web search 2026-07-07 — DILU degradation figure and "memory hurts when..." synthesis drawn from these sources' abstracts/summaries.

7. "Self-improving CLAUDE.md files" — the naive, ungated version of exactly what the user wants to build (cautionary baseline)

The risk it addresses: this is the closest published article to the user's literal plan — mine ~/.claude/projects JSONL traces, ask an agent to spot patterns/frustration and propose CLAUDE.md edits — and it is instructive precisely because it has no guardrails: no held-out test, no diff cap, no rollback plan, no acceptance test. The author's own mitigation is informal: "curating the suggestions quickly helps" (i.e. a human skims and picks), and the closing line — "There's really no reason this couldn't run as a scheduled task every day/week and just improve itself" — is exactly the unattended, ungated version the user should not build.

Concrete guardrail (what's missing here, stated as the inverse checklist):

• No mechanism to test whether an accepted CLAUDE.md edit actually reduced the frustration pattern it was mined from — there's no before/after trace comparison, only "it works ludicrously well in my experience."
• No diff-size cap or reversibility story — edits are applied directly to the live file.
• No distinction between "log-mined suggestion" and "accepted change" as separate, auditable states.

Concrete rule to steal (by contrast): if this pattern is used, it must be wrapped with the missing pieces from items 1–4 above at minimum: (a) a held-out slice of traces the proposal wasn't mined from, used to check whether the same class of frustration reappears after the edit lands; (b) a proposal-queue state machine (mined → human-reviewed → accepted/rejected → logged), never direct-write; (c) a diff cap per proposal (e.g., reject/flag any single CLAUDE.md change touching more than N lines) so a bad mining run can't silently rewrite half the file; (d) git-level reversibility as the rollback mechanism (commit before apply, one command to revert).

Source: Alderson, "Self-improving CLAUDE.md files," martinalderson.com/posts/self-improving-claude-md-files/ (8 Feb 2026).

Synthesis — the minimum guardrail set for the user's loop

Cross-referencing all seven sources, the guardrails that recur independently across 3+ sources (strongest signal) are:

1. Held-out evaluation, disjoint from the traces that motivated the change (skillopt, Regimes, Better-Harness, Kitchen Loop) — never validate a change only against the traces it was mined from.
2. Human accept/reject as the terminal gate before anything ships broadly (skillopt, Better-Harness, Regimes' promote-event) — automation may propose and pre-test; a human decides what lands.
3. Worst-case / per-slice regression check, not average score (Regimes' worst-split, DILU's -2.42% subset, Kitchen Loop's Unbeatable Tests) — an improving mean can hide a degrading minority.
4. Reversibility / audit trail as a first-class object (Regimes' event log, skillopt's history.json + rejected.json, Kitchen Loop's regression oracle) — every proposal, accept, and reject is logged and replayable, not just the final state.
5. Bounded diff / edit-budget per round (skillopt's textual learning-rate + rejected-edit buffer) — cap how much a single proposal round can change, forcing incremental, attributable edits.
6. Protect routing/metadata surfaces from mutation (skillopt's frontmatter-mutation-forbidden) — distinguish "improve the prose/behavior" edits from "change when/whether this fires" edits, and gate the latter far more strictly (directly analogous to hooks and skill frontmatter in Claude Code).