Full codebase review (multi-agent, adversarially verified) surfaced several documented-but-dead mechanisms and doc/code drift. Fixes: - adam-observe: struggle signals now emit `active_skills`, so silent_drift's primary cluster key AND §5b skill-attribution sub-clustering (+1 rubric bonus) actually fire — both were silently dead (no struggle signal carried the field). - adam-cooldown: new `--compute` CLI deterministically derives proposal_fingerprint. The exported computeProposalFingerprint() was never called and the analyst was told to hand-compute a djb2 hash it cannot reproduce. Spec now mandates a *stable* cluster id so fingerprints reproduce across /reflect runs. Removed one dead normalization line. - spec: reinforcement proposals excluded from A/B tracking — agents/adam.md contradicted itself (:376 included, :476 excluded); SKILL.md aligned. - adam-nudge: PENDING_CHECK_PATHS now mirrors the full install set (adam-utils / adam-batch / adam-rollback were missing). - adam-explain: synthesized clustering summary carries `regressions: 0` (structural consistency with parsed summaries). - docs: test-count drift (87/94 -> 126) and "350-line hook" (-> ~600) fixed; adam-score header documents severity_sum/severity_by_type; adam-batch §4 reference corrected. Tests: +12 assertions (114 -> 126), all green. New regression tests cover the active_skills fix and --compute, plus boundary gaps the review flagged: retry_loop/weak_agent thresholds, A/B exact +/-25% deltas, cooldown 30d blacklist edge.
21 KiB
name, description
| name | description |
|---|---|
| adam-self-improvement | Use when the user types /reflect, asks "what has adam learned", asks to "review proposals", or wants to inspect the self-improvement queue. Dispatches the adam subagent to analyse the observation journal and presents proposals for approve/reject/edit. |
adam-self-improvement
When to invoke
- User types
/reflect - User types
/reflect --explain(same flow, but the analyst's clustering trace is shown to the user — see §2b below) - User asks: "what has adam learned", "any proposals", "review the queue"
- SessionStart nudge said proposals are pending and user wants to act on it
Protocol
0. Parse flags
Check the slash-command argument string for the literal token --explain. Set explain=true when present; otherwise explain=false. Unknown flags: print one-line warning, continue with explain=false. This single flag is the only argument /reflect currently accepts.
1. Pre-filter the journal (window + exclusion) + score
Before dispatching the analyst, run the windowed-journal filter:
node ~/.claude/adam/scripts/adam-window.mjs --home ~/.claude > /tmp/adam-windowed-journal.jsonl 2> /tmp/adam-windowed-journal.log
The script reads the active journal plus all rotated journal files (new
journal/YYYY-Www.jsonl weekly format AND legacy
journal/YYYY-MM-DD-<ts>.jsonl size-rotated format are both supported), applies
per-signal-type sliding windows (see SIGNAL_WINDOWS_DAYS in
adam-window.mjs), and drops entries already actioned via
applied/*.md / rejected/*.md frontmatter source_entries.
If adam-window.mjs exits non-zero: log the stderr file to the user, fall
through to passing the raw ~/.claude/adam/journal.jsonl path to the agent
(graceful degradation — the agent's manual excluded-timestamps logic still
filters actioned entries; only the freshness window is lost).
Then run the scoring pre-step on the same windowed journal:
node ~/.claude/adam/scripts/adam-score.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-scores.json 2> /tmp/adam-scores.log
This produces a per-session dampener (0.5 / 0.75 / 1.0 based on
task_completed_count) and a reinforcement_candidates list (skills cited by
≥3 clean task_completed events). The analyst uses both — see
agents/adam.md §"Scoring: task_completed dampener". If the score step fails,
log stderr to the user and pass an empty {"sessions":[],"reinforcement_candidates":[]}
to the analyst (dampener defaults to 1.0).
Finally, run the A/B measurement pre-step on any previously auto-applied proposals (see §3 ab-tracking write):
node ~/.claude/adam/scripts/adam-ab-measure.mjs --home ~/.claude --format json > /tmp/adam-ab-regressions.json 2> /tmp/adam-ab-regressions.log
The JSON output is an array of A/B delta objects (pre_count, post_count,
delta_pct, status ∈ {improved,neutral,regressed,no_baseline,pending}).
Filter to status == "regressed" before passing to the analyst as
ab_regressions. The analyst is required (see agents/adam.md §"A/B
effectiveness") to surface a ## Regressions section at the top of its output
when this list is non-empty. If the script fails: log stderr, pass [].
Auto-rollback (MOSS §3.5): if any entries have status == "regressed", run the rollback script to auto-revert them before analyst dispatch:
node ~/.claude/adam/scripts/adam-rollback.mjs --auto --home ~/.claude > /tmp/adam-rollback-results.json 2> /tmp/adam-rollback.log
For each rolled-back proposal, print to user: adam: rolled back "<proposal_id>" — regression detected (delta: <delta_pct>%). The rollback script moves the proposal from applied/ back to proposals/ with rolled_back: true and creates a regression nudge. If the script fails: log stderr, continue (rollback is best-effort).
Evidence batching (MOSS §3.1): pre-cluster the windowed journal into coherent failure batches:
node ~/.claude/adam/scripts/adam-batch.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-batches.json 2> /tmp/adam-batch.log
This groups entries by (signal_type, cluster_key) and reports per-batch metadata including has_context_window (whether transcript evidence is attached). If the script fails: log stderr, pass null to the analyst (graceful degradation — analyst falls back to raw journal clustering).
2. Dispatch the analyst (two-stage pipeline)
MOSS §3.3: "A single prompt asked to diagnose, plan, implement, verify, and decide overloads context and produces lower-quality output than a sequenced flow." The analyst is dispatched in two stages with a validation gate between them.
Stage 1 — Diagnose + Plan: Use the Agent tool with subagent_type: "adam" and prompt:
stage=diagnose
Read the batched journal entries, cluster by signal type, diagnose root causes,
plan fix types, and score the keypoint matrix. Write diagnoses to /tmp/adam-diagnoses.json.
Do NOT draft proposal files.
Inputs:
- windowed_journal_path: /tmp/adam-windowed-journal.jsonl
- batches_path: /tmp/adam-batches.json # pre-clustered evidence batches
- scores_path: /tmp/adam-scores.json
- ab_regressions_path: /tmp/adam-ab-regressions.json
- journal_path: ~/.claude/adam/journal.jsonl # raw — fallback only
- state_path: ~/.claude/adam/state.json
- usage_path: ~/.claude/adam/usage.json
- applied_dir: ~/.claude/adam/applied/
- rejected_dir: ~/.claude/adam/rejected/
- transcripts_root: ~/.claude/projects/
- skills_root: ~/.claude/skills/
Use batches_path for pre-clustered evidence when available. Prefer context_window
fields in journal entries over transcript file lookups. Write /tmp/adam-diagnoses.json
per the "Diagnose-stage output format" in your system prompt.
Wait for return.
Inter-stage validation (§2a): after stage 1 returns, read /tmp/adam-diagnoses.json and validate each diagnosis:
- Every
source_entriestimestamp exists in the windowed journal (read/tmp/adam-windowed-journal.jsonl, check timestamps match). - Every diagnosis has all four fields (
trigger,action,mismatch,outcome). - The planned
typeis a valid proposal type. - Remove diagnoses that fail validation — log a one-line warning per removal.
If all diagnoses are removed or the file is missing/empty, print "adam: no valid diagnoses — nothing to implement" and skip to §6.
Stage 2 — Implement: Use the Agent tool with subagent_type: "adam" and prompt:
stage=implement
Read the validated diagnoses and draft full proposal files.
Inputs:
- diagnoses_path: /tmp/adam-diagnoses.json # validated stage-1 output
- windowed_journal_path: /tmp/adam-windowed-journal.jsonl
- scores_path: /tmp/adam-scores.json
- ab_regressions_path: /tmp/adam-ab-regressions.json
- state_path: ~/.claude/adam/state.json
- usage_path: ~/.claude/adam/usage.json
- proposals_dir: ~/.claude/adam/proposals/
- applied_dir: ~/.claude/adam/applied/
- rejected_dir: ~/.claude/adam/rejected/
- transcripts_root: ~/.claude/projects/
- skills_root: ~/.claude/skills/
Draft proposal files to proposals_dir/ for each diagnosis. Score against the
confidence rubric. Emit the clustering trace and punch list as your final message.
Wait for return.
2b. Persist and render the clustering trace
The analyst's final message always contains a fenced ```trace block (per agents/adam.md §"Clustering trace (always emit)") immediately before its punch-list JSON line.
- Extract the trace block. If it is missing, print a one-line warning to the user (
adam: trace block missing from agent output — proceeding without observability) and continue; do not block on this. - ALWAYS write the trace verbatim (without the surrounding fences) to
~/.claude/adam/last-trace.txt(overwrite each run). This persists for retrospection vianode ~/.claude/adam/scripts/adam-explain.mjs. - Extract the
SUMMARY:line from the trace. ALWAYS display it as a one-line status to the user BEFORE the proposals are listed, e.g.clustering: <SUMMARY line>. This single-line status is shown in both--explainand default modes. - If
explain=true(from §0): ALSO render the full trace block back to the user as a fenced code block (```text…```) under a headerClustering trace:. Ifexplain=false: SUPPRESS the cluster-line body from the user-visible output (the SUMMARY line is already shown in step 3).
The user can re-render any past trace at any time via:
node ~/.claude/adam/scripts/adam-explain.mjs --mode summary # SUMMARY + per-decision counts
node ~/.claude/adam/scripts/adam-explain.mjs --mode full # verbatim trace + rejection histogram
node ~/.claude/adam/scripts/adam-explain.mjs --mode json # machine-readable
3. Pre-apply verification gate (MOSS §3.4)
MOSS §3.4: "Verification must therefore be runtime, on a production-equivalent environment, and against the same prompts that produced the failure evidence." Before auto-applying, verify each proposal deterministically:
For each id in high_confidence:
- Read the proposal file from
~/.claude/adam/proposals/<id>-*.md. - Verification checks (all must pass for auto-apply to proceed):
- Source entries exist: every timestamp in
source_entriesfrontmatter must appear in/tmp/adam-windowed-journal.jsonl. If any are missing, the evidence is stale or was already actioned — demote toqueued. - Diagnosis grounded: the
# Diagnosissection must have all four fields (Trigger, Action, Mismatch, Outcome) with ≥1 backtick-wrapped quote. If malformed, demote toqueued. - Type-evidence match: the proposal
typemust match what the evidence supports:correctionsignals →memory,skill_new,skill_edit(notnudge)dead_endsignals →nudge,skill_new,skill_edit(notmemory)tool_error_loopsignals →memory,skill_new,skill_editharness_edit→ must cite harness-level evidence (false negative, scoring bias, window miscalibration) If mismatch, demote toqueued.
- No conflicting applied proposal: grep
~/.claude/adam/applied/for any proposal with the sametargetapplied in the last 7 days. If found, demote toqueued(prevents stacking rapid edits).
- Source entries exist: every timestamp in
- Print verification result:
verified: <id> (4/4 checks passed)ordemoted: <id> (failed: <check_name>). - Demoted proposals are moved from
high_confidencetoqueuedfor manual review.
3a. Apply verified high-confidence items
For each id that passed verification:
-
Print
id,target,confidence,blast_radius,cross_session_evidence,auto_apply_eligible. -
Apply the change:
-
For
skill_new:mkdir -p ~/.claude/skills/<slug>/, thenWritethe proposal's# Proposed changebody to~/.claude/skills/<slug>/SKILL.md. After write, print: "skill<slug>written to~/.claude/skills/<slug>/SKILL.md— activates immediately — Claude Code v2.1.0+ auto-hot-reloads user-level skills, no restart needed." -
For
memory:Writethe proposal's# Proposed changebody (which MUST include the auto-memory frontmatter — see "Memory drafting protocol" inagents/adam.md) to the path intarget. Then updateMEMORY.mdindex with a one-line pointer. -
For
nudge: low-blast auto-apply path. Single-session evidence is sufficient — skip the cross-session gate. Append a new entry to~/.claude/adam/active-nudges.json(create the file with[]if absent) with shape{kind, message, created_at: <now_ms>, expires_at_ts: <now_ms + 7*86400000>, max_displays: 3, displays_used: 0, source_session: <session_id from proposal>}. Do NOT modify any skill, memory, agent, or CLAUDE.md. Tell user: "nudge queued — surfaces on next SessionStart in a different session (expires in 7 days)." -
For
reinforcement: gated byconfidence >= 4 AND blast_radius == low(same as memory). Apply by invoking the helper:node ~/.claude/adam/scripts/adam-apply-reinforcement.mjs ~/.claude/adam/proposals/<id>-*.md --home ~/.claudeThe helper reads the proposal frontmatter (
skill_slug,count,source_session) and appends one JSON line to~/.claude/adam/reinforcements.jsonl. No code/memory/skill modifications. Output:{"status":"applied"|"gated", ...}— ongatedleave proposal inproposals/(helper failed its own re-check), onappliedcontinue to the archive step. Tell user: "reinforcement logged for<skill_slug>(count=) — appended to reinforcements.jsonl." -
For
skill_edit: enforce the apply-time gate before writing.- Verify proposal frontmatter has
auto_apply_eligible: true. If not, abort and queue for review. - Read
targetSKILL.md, capturecurrent_bytesfrom a fresh stat — do NOT trust frontmatterbytes_before. - Verify diff in
# Proposed change:- Unified-diff format.
- Zero
-lines on existing SKILL.md content (additions only). - Total
+lines ≤ 30. If any check fails, print one-line refusal reason, leave proposal inproposals/, continue.
- Cooldown re-check: run
node ~/.claude/adam/scripts/adam-cooldown.mjs --skill <target_skill> --fingerprint <proposal_fingerprint>(both fields come from proposal frontmatter; missing fingerprint → "legacy"). Refuse if the script returnsstatus: cooldownORstatus: blacklisted. This per-(skill, fingerprint) gate replaces the previous coarse per-skill scan — proposals for the same skill with a different fingerprint are NOT blocked by an older entry. - (covered by step 4 — blacklisted status is returned by
adam-cooldown.mjswhenauto_apply_blacklist: trueis found inrejected/within 30 days for the same (skill, fingerprint)) - Apply via
Edittool (append the new section per the diff). Never useWriteon existing SKILL.md. - Re-stat target. If new size exceeds
2 * current_bytes(captured in step 2), revert viaEdit(remove the just-appended section) and refuse — print refusal reason. - Add
last_auto_edit: <iso8601 utc now>to the proposal frontmatter before moving it. - Tell user: "skill
<slug>extended (added lines) — auto-applied via win-evidence gate."
- Verify proposal frontmatter has
-
-
Move proposal to
~/.claude/adam/applied/<UTC-ts>-<id>.md. -
A/B tracking append (skip for
reinforcement— positive-only ledger, intentionally not A/B-tracked peragents/adam.md§"reinforcementproposals"): as a separate atomic step right after the move, append one JSON line to~/.claude/adam/ab-tracking.jsonl(create with empty contents if absent). Read fields from the proposal's frontmatter (proposal_fingerprint,originating_signals— both populated peragents/adam.md;originating_signalsis a list of{type, count, session_ids}objects). Schema:{ "applied_at": <unix_ms now>, "proposal_id": "<id>", "proposal_type": "skill_edit|skill_new|memory|nudge", "target_skill": "<slug or target basename>", "proposal_fingerprint": "<hash>", "originating_signals": [{"type":"<signal>","count":<N>,"session_ids":[...]}], "pre_window_days": 7 }This entry is consumed by
adam-ab-measure.mjson subsequent/reflectruns to compute pre/post signal-count deltas. Seeagents/adam.md§"A/B effectiveness". If the append fails (disk-full etc.) log a warning but do NOT abort the apply path — A/B is observability, not a gate. -
Archive consumed journal entries:
node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<UTC-ts>-<id>.md— moves entries listed in proposal'ssource_entriesfromjournal.jsonltojournal/actioned-<id>.jsonlso subsequent/reflectruns do not re-cluster them.
Print: auto-applied N proposals: [ids].
4. Walk the queue
For each id in queued:
a. Read and display the proposal in full (frontmatter + body). b. Ask the user: approve / reject / edit. c. On approve:
- For
claude_md_edit: backupcp ~/.claude/CLAUDE.md ~/.claude/adam/applied/<ts>-claude-md-backup.mdfirst. - For
deletion:mkdir -p ~/.claude/adam/trash/<ts>thenmvthe artifact into it. Print restoration command. - For
skill_new:mkdir -p ~/.claude/skills/<slug>/, then write# Proposed changebody to<slug>/SKILL.md. Tell user: "skill<slug>written — activates immediately (CC v2.1.0+ auto-hot-reload)." - For
skill_edit: apply the unified diff in# Proposed changeto the existing SKILL.md attarget(append-only — never replace existing content). - For
memory: write# Proposed changebody (must include auto-memory frontmatter) totargetand updateMEMORY.mdindex with a one-line pointer. - For
harness_edit(MOSS §1): apply the unified diff to the target harness file. Before applying:- Run
bash ~/.claude/adam/tests/run-tests.sh— capture pass count. - Apply the diff via
Edit. - Run
bash ~/.claude/adam/tests/run-tests.shagain — verify pass count is equal or higher and 0 failures. - If test regression: revert the edit, print "harness_edit reverted — test regression detected", leave proposal in
proposals/. - If tests pass: tell user "harness edit applied to
<target>— tests pass ( passed)."
- Run
- For all others: apply via Write/Edit per the proposal's
# Proposed change. - Move proposal to
~/.claude/adam/applied/<ts>-<id>.md. - Archive:
node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<ts>-<id>.md. d. On reject: ask for reason in one line. Append# Reason\n<reason>to proposal body. If the proposaltypeisskill_edit, ALSO addauto_apply_blacklist: trueto its frontmatter (so future reflects skip auto-apply on this target for 30 days). Move to~/.claude/adam/rejected/<id>.md. Archive:node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/rejected/<id>.md. e. On edit: ask the user for the change, edit the proposal in place, then loop back to step 4a for that same id.
5. Handle failures
If apply fails (file write error, target missing): leave proposal in proposals/, append # Apply error\n<error> to its body. Tell the user. Do not move it.
6. Summary
End with one block:
adam reflect summary:
observations processed: <new>
batches formed: <N>
diagnoses validated: <N>/<total>
rolled back (regression): <N>
verification passed: <N>/<total high_confidence>
auto-applied: <N>
approved: <N>
rejected: <N>
edited+approved: <N>
failed: <N>
Keypoint history: after all proposals are processed, append one JSON line to ~/.claude/adam/keypoint-history.jsonl with the aggregate keypoint scores from the diagnose stage:
{"ts":"<iso>","session":"<session_id>","keypoints":{"tool_selection":N,"scope_discipline":N,"error_recovery":N,"first_attempt":N,"build_reliability":N},"proposals_emitted":N,"proposals_applied":N}
This builds a longitudinal record of which capabilities are improving across /reflect runs.
Karpathy constraints (you must enforce on each apply)
Before writing any proposal:
- Confirm
# Assumptionssection is non-empty. - Confirm
# Diagnosissection exists and contains all four labelled lines (Trigger:,Action:,Mismatch:,Outcome:) AND at least one backtick-wrapped quote ≤80 chars in the Outcome line. Refuse if missing or malformed — agent must redraft per the "Diagnosis drafting protocol" inagents/adam.md. - Confirm
# Success criterionsection is non-empty and runnable. - Confirm change is ≤50 LOC for non-
skill_new, or ≤80 LOC forskill_newbody. If larger, ask the user once: "this proposal is N LOC — proceed?" - For
claude_md_edit: confirm 3+ distinct cwds in the# Whysection. - For
deletion: confirm both criteria (a) and (b) from the agent's special handling are documented in the proposal. - For
skill_new: confirm the slug doesn't collide with any existing skill in~/.claude/skills/. If it does, refuse and ask user to rename. - For
skill_edit: confirm the diff is append-only (no-lines that remove existing content) and that target SKILL.md exists. When auto-applying, ALSO re-verify the eligibility gate steps in §3 (cooldown, blacklist, byte cap) before anyEditcall — never trust frontmatter alone. - For
skill_editwithauto_apply_eligible: true: confirmcontradiction_flagis absent or null in frontmatter. Refuse auto-apply ifcontradiction_flagis set with any non-empty value (treat the agent's flag as a hard veto on auto-apply; user can still manually approve in walk-the-queue if they disagree with the heuristic). - For
memory: confirm# Proposed changebody starts with---frontmatter containing required fieldsname,description,type,originSessionId. Refuse if frontmatter missing — agent must redraft per the Memory drafting protocol. - For
harness_edit: confirmauto_apply_eligible: false(never auto-apply). Confirmconfidence ≥ 5. Confirm# Test verificationsection names the test command. Confirm diff is ≤30 LOC and targets a single allowed harness file (seeagents/adam.md§"Harness self-modification"). Run test suite before AND after applying — revert on any regression. - Confirm
source_entriesis present in proposal frontmatter as a non-empty list (used for archive). Warn (do not refuse) if missing — legacy proposals from before v0.2.0 won't have it.
If any check fails, refuse to apply and ask the user how to proceed.
Things you MUST NOT do
- Do not auto-apply anything not in
high_confidence. - Do not invoke other skills during a
/reflectrun. - Do not modify
settings.jsonwithout explicit user yes. - Do not hard-delete anything. Use
mvto~/.claude/adam/trash/<ts>/. - Do not bypass the rubric (
auto_apply_eligible: falsemeans queue, full stop).