mirror of
https://github.com/lukaszraczylo/claude-adam.git
synced 2026-10-30 07:17:12 +00:00
Compare commits
12
Commits
v0.6.1
...
7ccb07a412
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7ccb07a412 | ||
|
|
f418afa275 | ||
|
|
e3fe691b45 | ||
|
|
b2fff8f271 | ||
|
|
f1a186af7e | ||
|
|
67e8eede85 | ||
|
|
0ec17517d9 | ||
|
|
2b6e8da885 | ||
|
|
4d1276a73f | ||
|
|
c23b09cc09 | ||
|
|
fcddb6bf79 | ||
|
|
d929101af4 |
@@ -13,7 +13,7 @@ Watches the friction in your coding sessions, clusters the signals via an LLM an
|
|||||||
|
|
||||||
[](LICENSE)
|
[](LICENSE)
|
||||||
[](https://github.com/lukaszraczylo/claude-adam/releases)
|
[](https://github.com/lukaszraczylo/claude-adam/releases)
|
||||||
[](./adam/tests/run-tests.sh)
|
[](./adam/tests/run-tests.sh)
|
||||||
[](https://nodejs.org)
|
[](https://nodejs.org)
|
||||||
[]()
|
[]()
|
||||||
|
|
||||||
@@ -54,7 +54,7 @@ The installer copies files into `~/.claude/`, offers to merge ADAM's hook entrie
|
|||||||
Then:
|
Then:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
bash ~/.claude/adam/tests/run-tests.sh # expect: 132 passed, 0 failed
|
bash ~/.claude/adam/tests/run-tests.sh # expect: 150 passed, 0 failed
|
||||||
# … start a fresh Claude Code session …
|
# … start a fresh Claude Code session …
|
||||||
/reflect # walks the proposal queue
|
/reflect # walks the proposal queue
|
||||||
/reflect --explain # also shows the analyst's clustering trace
|
/reflect --explain # also shows the analyst's clustering trace
|
||||||
@@ -63,10 +63,27 @@ bash ~/.claude/adam/tests/run-tests.sh # expect: 132 passed, 0 failed
|
|||||||
Pin a release for reproducibility:
|
Pin a release for reproducibility:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
curl -fsSL https://raw.githubusercontent.com/lukaszraczylo/claude-adam/v0.6.0/install.sh \
|
curl -fsSL https://raw.githubusercontent.com/lukaszraczylo/claude-adam/v0.6.3/install.sh \
|
||||||
| VERSION=v0.6.0 bash
|
| VERSION=v0.6.3 bash
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Staying up to date
|
||||||
|
|
||||||
|
`install.sh` records the installed release in `~/.claude/adam/.version`. The
|
||||||
|
SessionStart hook (`adam-nudge.mjs`) then checks the latest GitHub release **at
|
||||||
|
most once a day** (cached in `~/.claude/adam/.update-check.json`, network call
|
||||||
|
hard-capped at 1.5 s, fully best-effort — it never blocks or slows session
|
||||||
|
start). When a newer release exists it prints a one-line, **notify-only** prompt:
|
||||||
|
|
||||||
|
```
|
||||||
|
[adam] update available: v0.6.3 → v0.6.4. Apply: curl -fsSL …/install.sh | bash
|
||||||
|
(re-runs install.sh — resets ADAM's own /reflect-applied skill edits; apply when you're ready)
|
||||||
|
```
|
||||||
|
|
||||||
|
It is deliberately **not** auto-applied: re-running `install.sh` overwrites
|
||||||
|
ADAM's own `/reflect`-applied skill edits, so you decide when to take an update.
|
||||||
|
Disable the check entirely with `ADAM_NO_UPDATE_CHECK=1` in your environment.
|
||||||
|
|
||||||
## How it works
|
## How it works
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
@@ -218,11 +235,12 @@ Or pass `--explain` to `/reflect` to render the full trace inline.
|
|||||||
```
|
```
|
||||||
~/.claude/
|
~/.claude/
|
||||||
├── hooks/
|
├── hooks/
|
||||||
│ ├── adam-observe.mjs # signal collector (UserPromptSubmit / PreToolUse / PostToolUse)
|
│ ├── adam-observe.mjs # signal collector (UserPromptSubmit / PreToolUse / PostToolUse / PostToolUseFailure)
|
||||||
│ └── adam-nudge.mjs # SessionStart reminder + pending-upgrade warning
|
│ └── adam-nudge.mjs # SessionStart reminder + pending-upgrade warning
|
||||||
├── agents/adam.md # analyst subagent (system prompt + rubric)
|
├── agents/adam.md # analyst subagent (system prompt + rubric)
|
||||||
├── skills/adam-self-improvement/
|
├── skills/adam-self-improvement/
|
||||||
│ └── SKILL.md # /reflect protocol
|
│ ├── SKILL.md # /reflect protocol
|
||||||
|
│ └── apply-by-type.md # per-type apply steps, read before any apply
|
||||||
├── commands/reflect.md # /reflect slash command
|
├── commands/reflect.md # /reflect slash command
|
||||||
└── adam/
|
└── adam/
|
||||||
├── journal.jsonl # active observations
|
├── journal.jsonl # active observations
|
||||||
@@ -245,14 +263,23 @@ Or pass `--explain` to `/reflect` to render the full trace inline.
|
|||||||
│ ├── adam-cooldown.mjs # per-(skill, fingerprint) cooldown gate
|
│ ├── adam-cooldown.mjs # per-(skill, fingerprint) cooldown gate
|
||||||
│ ├── adam-nudge-eligibility.mjs # dead_end session-count check
|
│ ├── adam-nudge-eligibility.mjs # dead_end session-count check
|
||||||
│ ├── adam-explain.mjs # clustering trace parser/renderer
|
│ ├── adam-explain.mjs # clustering trace parser/renderer
|
||||||
|
│ ├── adam-batch.mjs # evidence batching (optional output cap)
|
||||||
|
│ ├── adam-rollback.mjs # auto-rollback of regressed proposals
|
||||||
|
│ ├── adam-skill-utility.mjs # per-skill good:bad utility report
|
||||||
│ ├── adam-apply-reinforcement.mjs # reinforcement proposal apply
|
│ ├── adam-apply-reinforcement.mjs # reinforcement proposal apply
|
||||||
|
│ ├── adam-apply-nudge.mjs # nudge proposal apply (message + cue)
|
||||||
│ ├── adam-upgrade.mjs # .adam-new file UX (list/diff/accept)
|
│ ├── adam-upgrade.mjs # .adam-new file UX (list/diff/accept)
|
||||||
│ └── adam-archive.mjs # post-apply journal cleanup
|
│ └── adam-archive.mjs # post-apply journal cleanup
|
||||||
└── tests/run-tests.sh # 132 isolated tests; never touches live state
|
└── tests/run-tests.sh # 150 isolated tests of the tree it sits in; never touches live state
|
||||||
```
|
```
|
||||||
|
|
||||||
## What's new
|
## What's new
|
||||||
|
|
||||||
|
- **v0.7.1** — ADAM now sees real tool failures. Claude Code sends failed tool calls to the `PostToolUseFailure` hook event, which ADAM never registered, so every `tool_error_loop` came from `ERROR_RE` matching successful output. `adam-observe.mjs` now handles `PostToolUseFailure` through the same detectors as `PostToolUse` (interrupts excluded), and `settings.json.example` registers it; re-run `install.sh` or add the entry by hand. Successful Read, Edit and Write responses are no longer fingerprinted: source files containing words such as `error` produced 151 false `tool_error_loop` entries across 9 sessions. Found by ADAM's own `/reflect`. 150 tests (up from 145).
|
||||||
|
- **v0.7.0** — query-conditioned nudges and a lighter `/reflect` skill. A nudge can carry `cue.cwd`: `adam-nudge.mjs` replays it at SessionStart only in that codebase, and `adam-observe.mjs` now also surfaces it once mid-session on a matching `UserPromptSubmit` (opt out with `ADAM_NO_MIDSESSION_NUDGE=1`). New `adam-apply-nudge.mjs` builds the nudge message and cue from proposal frontmatter, so the analyst no longer hand-writes JSON. New `adam-skill-utility.mjs` ranks skills by good:bad outcome co-occurrence (Wilson lower bound, lift against baseline) and feeds a utility-lift rubric penalty; the analyst re-diagnoses unclear or top-severity clusters once per run. `adam-batch.mjs` gains an optional `--max-batches` / `--top-fraction` cap (off by default) and reports `capped`. The per-type apply steps move from `SKILL.md` to `apply-by-type.md`, so `/reflect` loads less context until it applies a proposal. The test harness now tests the tree it sits in, not `~/.claude`, so a repo checkout no longer tests an older install. 145 tests (up from 140).
|
||||||
|
- **v0.6.4** — rollback now keeps its promise. `adam-rollback.mjs`'s docstring always claimed it "removes the ab-tracking entry (so it doesn't re-trigger)," but `executeRollback()` never did — so a rolled-back proposal kept flagging as `regressed` on every subsequent `/reflect`, triggering endless `not_found` rollback attempts. It now deletes the matching `ab-tracking.jsonl` row by `proposal_id` (preserving unrelated rows). Surfaced by running ADAM's own loop twice. 140 tests (up from 138).
|
||||||
|
- **v0.6.3** — release-update notifier. `install.sh` now writes a `~/.claude/adam/.version` marker; `adam-nudge.mjs` (SessionStart) compares it against the latest GitHub release at most once/day (cached, 1.5 s network cap, best-effort — never blocks) and prints a **notify-only** one-line update prompt. Deliberately not auto-applied: re-running the installer resets ADAM's own `/reflect`-applied skill edits, so you choose when to update. Opt out with `ADAM_NO_UPDATE_CHECK=1`. See "Staying up to date". 138 tests (up from 134).
|
||||||
|
- **v0.6.2** — two fixes surfaced by running ADAM's loop on a large real journal. **(1) A/B volume normalization** (`adam-ab-measure.mjs`): regressions are now measured on the signal's *share* of total activity (rate = count / window-total), not raw count — so a generally busier journal after an apply no longer masquerades as a regression. Falls back to raw delta when the signal is the only activity in the window (preserves prior behavior + tests); output adds `raw_delta_pct`, `pre_total`, `post_total`, `normalized` for transparency. **(2) Memory frontmatter schema** (`agents/adam.md`, `SKILL.md`): the drafting protocol now emits the live auto-memory shape — `name` = slug + a `metadata: {node_type, type, originSessionId}` block — instead of flat `type:`/`originSessionId:`, so auto-applied memories load and categorize correctly. 134 tests (up from 132).
|
||||||
- **v0.6.1** — new `file_reread` signal (MOSS §1 harness self-modification, proposed and approved through ADAM's own `/reflect` loop). Consecutive Reads of the same file at different `offset`/`limit` escaped `retry_loop`'s arg-hash dedup and leaked into `tool_error_loop`; `file_reread` now catches them (same file ≥3× in the 10-event window, offset-agnostic, guarded against double-counting byte-identical reads). Fully wired: detection (`adam-observe.mjs`), 14-day window (`adam-window.mjs`), severity divisor 3 (`adam-score.mjs`), file-basename clustering (`adam-batch.mjs`), and the analyst rubric/spec. 132 tests (up from 126).
|
- **v0.6.1** — new `file_reread` signal (MOSS §1 harness self-modification, proposed and approved through ADAM's own `/reflect` loop). Consecutive Reads of the same file at different `offset`/`limit` escaped `retry_loop`'s arg-hash dedup and leaked into `tool_error_loop`; `file_reread` now catches them (same file ≥3× in the 10-event window, offset-agnostic, guarded against double-counting byte-identical reads). Fully wired: detection (`adam-observe.mjs`), 14-day window (`adam-window.mjs`), severity divisor 3 (`adam-score.mjs`), file-basename clustering (`adam-batch.mjs`), and the analyst rubric/spec. 132 tests (up from 126).
|
||||||
- **v0.6.0** — review hardening. Struggle signals now emit `active_skills`, so `silent_drift`'s primary cluster key and the §5b skill-attribution sub-clustering (+1 rubric bonus) actually fire (both were silently dead). `proposal_fingerprint` is now deterministically computable via `adam-cooldown.mjs --compute` instead of asking the LLM analyst to hand-compute a djb2 hash; spec now mandates a *stable* cluster id so fingerprints reproduce across runs. `reinforcement` proposals are correctly excluded from A/B tracking (the spec previously contradicted itself). `adam-nudge.mjs` pending-upgrade check now mirrors the full install set (`adam-utils`/`adam-batch`/`adam-rollback` were missing). Doc/test-count drift corrected. 126 tests (up from 114).
|
- **v0.6.0** — review hardening. Struggle signals now emit `active_skills`, so `silent_drift`'s primary cluster key and the §5b skill-attribution sub-clustering (+1 rubric bonus) actually fire (both were silently dead). `proposal_fingerprint` is now deterministically computable via `adam-cooldown.mjs --compute` instead of asking the LLM analyst to hand-compute a djb2 hash; spec now mandates a *stable* cluster id so fingerprints reproduce across runs. `reinforcement` proposals are correctly excluded from A/B tracking (the spec previously contradicted itself). `adam-nudge.mjs` pending-upgrade check now mirrors the full install set (`adam-utils`/`adam-batch`/`adam-rollback` were missing). Doc/test-count drift corrected. 126 tests (up from 114).
|
||||||
- **v0.5.0** — MOSS-grounded self-evolution (arXiv 2605.22794). Transcript capture: `context_window` field on struggle signals captures 8 surrounding events for evidence-based diagnosis. Two-stage analysis pipeline: diagnose+plan → inter-stage validation → implement (§3.3). Evidence batching via `adam-batch.mjs`: pre-clusters journal into coherent failure batches (§3.1). Pre-apply verification: 4-check deterministic gate before auto-apply (§3.4). Auto-rollback via `adam-rollback.mjs`: reverts regressed proposals detected by A/B measurement, creates regression nudges (§3.5). Harness self-modification: new `harness_edit` proposal type lets ADAM propose edits to its own scripts with test-suite-gated apply (§1 Table 1). Keypoint matrix: 5 capability dimensions scored per batch for structured evaluation (§4.2). 114 tests (up from 94).
|
- **v0.5.0** — MOSS-grounded self-evolution (arXiv 2605.22794). Transcript capture: `context_window` field on struggle signals captures 8 surrounding events for evidence-based diagnosis. Two-stage analysis pipeline: diagnose+plan → inter-stage validation → implement (§3.3). Evidence batching via `adam-batch.mjs`: pre-clusters journal into coherent failure batches (§3.1). Pre-apply verification: 4-check deterministic gate before auto-apply (§3.4). Auto-rollback via `adam-rollback.mjs`: reverts regressed proposals detected by A/B measurement, creates regression nudges (§3.5). Harness self-modification: new `harness_edit` proposal type lets ADAM propose edits to its own scripts with test-suite-gated apply (§1 Table 1). Keypoint matrix: 5 capability dimensions scored per batch for structured evaluation (§4.2). 114 tests (up from 94).
|
||||||
|
|||||||
@@ -3,11 +3,19 @@
|
|||||||
//
|
//
|
||||||
// Reads ~/.claude/adam/ab-tracking.jsonl (one line per auto-apply event,
|
// Reads ~/.claude/adam/ab-tracking.jsonl (one line per auto-apply event,
|
||||||
// written by adam-self-improvement/SKILL.md), then for each entry old enough
|
// written by adam-self-improvement/SKILL.md), then for each entry old enough
|
||||||
// (>= --min-age-days; default 7) compares signal counts in the 7-day window
|
// (>= --min-age-days; default 7) compares the originating signal in the 7-day
|
||||||
// BEFORE applied_at against the 7-day window AFTER applied_at across the
|
// window BEFORE applied_at against the 7-day window AFTER applied_at across the
|
||||||
// full journal corpus (active + rotated). Surfaces regressions so /reflect
|
// full journal corpus (active + rotated). Surfaces regressions so /reflect
|
||||||
// can flag proposals that made things worse.
|
// can flag proposals that made things worse.
|
||||||
//
|
//
|
||||||
|
// Volume normalization: when the windows contain other (non-originating)
|
||||||
|
// activity, the delta is computed on the signal's SHARE of total activity
|
||||||
|
// (rate = count / total), not its raw count — so a generally busier journal
|
||||||
|
// after apply does not masquerade as a regression. When the signal is the only
|
||||||
|
// activity in the windows, it falls back to the raw-count delta. Output carries
|
||||||
|
// both `delta_pct` (drives status) and `raw_delta_pct` + `normalized` for
|
||||||
|
// transparency.
|
||||||
|
//
|
||||||
// CLI:
|
// CLI:
|
||||||
// adam-ab-measure.mjs [--home <path>] [--format json|table] [--min-age-days N]
|
// adam-ab-measure.mjs [--home <path>] [--format json|table] [--min-age-days N]
|
||||||
//
|
//
|
||||||
@@ -92,31 +100,60 @@ export function computeDeltas(entries, journal, opts = {}) {
|
|||||||
|
|
||||||
const preStart = appliedAt - windowDays * DAY_MS;
|
const preStart = appliedAt - windowDays * DAY_MS;
|
||||||
const postEnd = appliedAt + windowDays * DAY_MS;
|
const postEnd = appliedAt + windowDays * DAY_MS;
|
||||||
|
// preCount/postCount = originating-signal occurrences; preTotal/postTotal =
|
||||||
|
// ALL journal entries in the window (the activity denominator).
|
||||||
let preCount = 0;
|
let preCount = 0;
|
||||||
let postCount = 0;
|
let postCount = 0;
|
||||||
|
let preTotal = 0;
|
||||||
|
let postTotal = 0;
|
||||||
for (const je of journal || []) {
|
for (const je of journal || []) {
|
||||||
if (!je || typeof je !== "object") continue;
|
if (!je || typeof je !== "object") continue;
|
||||||
if (!sigSet.has(je.type)) continue;
|
|
||||||
const t = tsMs(je);
|
const t = tsMs(je);
|
||||||
if (Number.isNaN(t)) continue;
|
if (Number.isNaN(t)) continue;
|
||||||
if (t >= preStart && t < appliedAt) preCount++;
|
const inPre = t >= preStart && t < appliedAt;
|
||||||
else if (t >= appliedAt && t < postEnd) postCount++;
|
const inPost = t >= appliedAt && t < postEnd;
|
||||||
|
if (!inPre && !inPost) continue;
|
||||||
|
if (inPre) preTotal++; else postTotal++;
|
||||||
|
if (!sigSet.has(je.type)) continue;
|
||||||
|
if (inPre) preCount++; else postCount++;
|
||||||
}
|
}
|
||||||
|
|
||||||
let status;
|
let status;
|
||||||
let deltaPct;
|
let deltaPct;
|
||||||
|
let rawDeltaPct = null;
|
||||||
|
let normalized = false;
|
||||||
if (preCount === 0) {
|
if (preCount === 0) {
|
||||||
status = "no_baseline";
|
status = "no_baseline";
|
||||||
deltaPct = null;
|
deltaPct = null;
|
||||||
|
} else {
|
||||||
|
rawDeltaPct = Math.round(((postCount - preCount) / preCount) * 10000) / 100;
|
||||||
|
// Volume normalization: when the windows contain non-originating activity,
|
||||||
|
// compare the signal's SHARE of activity (rate), not its absolute count —
|
||||||
|
// otherwise a generally busier post-window masquerades as a regression.
|
||||||
|
// No background (signal IS the only activity) → fall back to raw delta,
|
||||||
|
// preserving prior behavior.
|
||||||
|
const hasBackground = (preTotal - preCount) + (postTotal - postCount) > 0;
|
||||||
|
if (hasBackground && postTotal > 0) {
|
||||||
|
const preRate = preCount / preTotal; // preTotal >= preCount > 0
|
||||||
|
const postRate = postCount / postTotal;
|
||||||
|
deltaPct = ((postRate - preRate) / preRate) * 100;
|
||||||
|
normalized = true;
|
||||||
} else {
|
} else {
|
||||||
deltaPct = ((postCount - preCount) / preCount) * 100;
|
deltaPct = ((postCount - preCount) / preCount) * 100;
|
||||||
|
}
|
||||||
// Round to 2 dp for stable comparison + presentation.
|
// Round to 2 dp for stable comparison + presentation.
|
||||||
deltaPct = Math.round(deltaPct * 100) / 100;
|
deltaPct = Math.round(deltaPct * 100) / 100;
|
||||||
if (deltaPct <= IMPROVED_PCT) status = "improved";
|
if (deltaPct <= IMPROVED_PCT) status = "improved";
|
||||||
else if (deltaPct >= REGRESSED_PCT) status = "regressed";
|
else if (deltaPct >= REGRESSED_PCT) status = "regressed";
|
||||||
else status = "neutral";
|
else status = "neutral";
|
||||||
}
|
}
|
||||||
out.push({ ...base, pre_count: preCount, post_count: postCount, delta_pct: deltaPct, status });
|
out.push({
|
||||||
|
...base,
|
||||||
|
pre_count: preCount, post_count: postCount,
|
||||||
|
pre_total: preTotal, post_total: postTotal,
|
||||||
|
raw_delta_pct: rawDeltaPct, normalized,
|
||||||
|
delta_pct: deltaPct, status,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,115 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
// adam-apply-nudge.mjs — apply-path for `nudge` proposals.
|
||||||
|
//
|
||||||
|
// Reads a nudge proposal markdown file, validates the apply gate
|
||||||
|
// (type == "nudge" AND source_session present), and on success appends one
|
||||||
|
// entry to ~/.claude/adam/active-nudges.json. The message and the `cue` are
|
||||||
|
// built DETERMINISTICALLY in code from the proposal frontmatter — the analyst
|
||||||
|
// does not hand-author the JSON, so the cue (ReContext-style query-conditioning)
|
||||||
|
// is always populated when a cwd is known, not left to chance.
|
||||||
|
//
|
||||||
|
// Frontmatter consumed: type, source_session, cwd (optional — becomes cue.cwd),
|
||||||
|
// dead_end_count (optional — shapes the message), max_displays (optional, ≥1).
|
||||||
|
//
|
||||||
|
// CLI: adam-apply-nudge.mjs <proposal-path> [--home <path>]
|
||||||
|
// Output: JSON one-liner on stdout: {"status":"applied"|"gated", "reason"|"path"}
|
||||||
|
// Exit: 0 on apply, 0 on gated, 1 on I/O or parse error.
|
||||||
|
//
|
||||||
|
// SKILL.md invokes this in the nudge auto-apply path. No code/memory/skill edits.
|
||||||
|
|
||||||
|
import { readFileSync, writeFileSync, existsSync, mkdirSync } from "node:fs";
|
||||||
|
import { join, dirname } from "node:path";
|
||||||
|
import { homedir } from "node:os";
|
||||||
|
import { parseFrontmatter } from "./adam-utils.mjs";
|
||||||
|
|
||||||
|
export { parseFrontmatter };
|
||||||
|
|
||||||
|
const SEVEN_DAYS_MS = 7 * 86400000;
|
||||||
|
const DEFAULT_MAX_DISPLAYS = 3;
|
||||||
|
|
||||||
|
function parseArgs(argv) {
|
||||||
|
const args = { home: null, path: null, help: false };
|
||||||
|
for (let i = 0; i < argv.length; i++) {
|
||||||
|
const a = argv[i];
|
||||||
|
if (a === "--home" && i + 1 < argv.length) args.home = argv[++i];
|
||||||
|
else if (a === "--help" || a === "-h") args.help = true;
|
||||||
|
else if (!args.path && !a.startsWith("--")) args.path = a;
|
||||||
|
}
|
||||||
|
return args;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function checkGate(fm) {
|
||||||
|
if ((fm.type || "") !== "nudge") {
|
||||||
|
return { ok: false, reason: `type != nudge (got: ${fm.type || "<none>"})` };
|
||||||
|
}
|
||||||
|
if (!fm.source_session) {
|
||||||
|
return { ok: false, reason: "source_session missing in frontmatter" };
|
||||||
|
}
|
||||||
|
return { ok: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function buildEntry(fm, now = Date.now()) {
|
||||||
|
const cwd = fm.cwd ? String(fm.cwd) : "";
|
||||||
|
const base = cwd ? cwd.split("/").filter(Boolean).pop() : "";
|
||||||
|
const count = Number(fm.dead_end_count) > 0 ? Number(fm.dead_end_count) : 3;
|
||||||
|
const maxDisplays = Number(fm.max_displays) > 0 ? Number(fm.max_displays) : DEFAULT_MAX_DISPLAYS;
|
||||||
|
const message = `adam: a previous session hit ${count} dead-ends${base ? ` in ${base}` : ""} — consider a checkpoint before continuing.`;
|
||||||
|
const entry = {
|
||||||
|
kind: "dead_end_reminder",
|
||||||
|
message,
|
||||||
|
created_at: now,
|
||||||
|
expires_at_ts: now + SEVEN_DAYS_MS,
|
||||||
|
max_displays: maxDisplays,
|
||||||
|
displays_used: 0,
|
||||||
|
source_session: String(fm.source_session || ""),
|
||||||
|
};
|
||||||
|
// The cue is what makes replay query-conditioned: bind the reminder to the
|
||||||
|
// project it came from so it only surfaces there (SessionStart + mid-session).
|
||||||
|
if (cwd) entry.cue = { cwd };
|
||||||
|
return entry;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function appendNudge(nudgesPath, entry) {
|
||||||
|
let arr = [];
|
||||||
|
if (existsSync(nudgesPath)) {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(readFileSync(nudgesPath, "utf8"));
|
||||||
|
if (Array.isArray(parsed)) arr = parsed;
|
||||||
|
} catch { arr = []; }
|
||||||
|
}
|
||||||
|
arr.push(entry);
|
||||||
|
const dir = dirname(nudgesPath);
|
||||||
|
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||||
|
writeFileSync(nudgesPath, JSON.stringify(arr, null, 2));
|
||||||
|
return arr.length;
|
||||||
|
}
|
||||||
|
|
||||||
|
function main() {
|
||||||
|
const args = parseArgs(process.argv.slice(2));
|
||||||
|
if (args.help || !args.path) {
|
||||||
|
process.stdout.write("usage: adam-apply-nudge.mjs <proposal-path> [--home <path>]\n");
|
||||||
|
process.exit(args.help ? 0 : 1);
|
||||||
|
}
|
||||||
|
const claudeHome = args.home || join(homedir(), ".claude");
|
||||||
|
const nudgesPath = join(claudeHome, "adam", "active-nudges.json");
|
||||||
|
try {
|
||||||
|
const content = readFileSync(args.path, "utf8");
|
||||||
|
const fm = parseFrontmatter(content);
|
||||||
|
const gate = checkGate(fm);
|
||||||
|
if (!gate.ok) {
|
||||||
|
process.stdout.write(JSON.stringify({ status: "gated", reason: gate.reason }) + "\n");
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
const entry = buildEntry(fm);
|
||||||
|
const count = appendNudge(nudgesPath, entry);
|
||||||
|
process.stdout.write(JSON.stringify({ status: "applied", path: nudgesPath, total: count, cue: entry.cue || null }) + "\n");
|
||||||
|
process.exit(0);
|
||||||
|
} catch (e) {
|
||||||
|
process.stderr.write(`adam-apply-nudge error: ${e.message}\n`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||||
|
main();
|
||||||
|
}
|
||||||
@@ -83,7 +83,7 @@ function clusterKey(entry) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function parseArgs(argv) {
|
function parseArgs(argv) {
|
||||||
const args = { input: null, minEntries: DEFAULT_MIN_ENTRIES, minSessions: DEFAULT_MIN_SESSIONS, help: false };
|
const args = { input: null, minEntries: DEFAULT_MIN_ENTRIES, minSessions: DEFAULT_MIN_SESSIONS, maxBatches: null, topFraction: null, help: false };
|
||||||
for (let i = 0; i < argv.length; i++) {
|
for (let i = 0; i < argv.length; i++) {
|
||||||
const a = argv[i];
|
const a = argv[i];
|
||||||
if (a === "--input" && i + 1 < argv.length) args.input = argv[++i];
|
if (a === "--input" && i + 1 < argv.length) args.input = argv[++i];
|
||||||
@@ -95,6 +95,14 @@ function parseArgs(argv) {
|
|||||||
const n = Number(argv[++i]);
|
const n = Number(argv[++i]);
|
||||||
if (!Number.isNaN(n) && n > 0) args.minSessions = n;
|
if (!Number.isNaN(n) && n > 0) args.minSessions = n;
|
||||||
}
|
}
|
||||||
|
else if (a === "--max-batches" && i + 1 < argv.length) {
|
||||||
|
const n = Number(argv[++i]);
|
||||||
|
if (!Number.isNaN(n) && n > 0) args.maxBatches = n;
|
||||||
|
}
|
||||||
|
else if (a === "--top-fraction" && i + 1 < argv.length) {
|
||||||
|
const n = Number(argv[++i]);
|
||||||
|
if (!Number.isNaN(n) && n > 0 && n < 1) args.topFraction = n;
|
||||||
|
}
|
||||||
else if (a === "--help" || a === "-h") args.help = true;
|
else if (a === "--help" || a === "-h") args.help = true;
|
||||||
}
|
}
|
||||||
return args;
|
return args;
|
||||||
@@ -146,13 +154,28 @@ export function buildBatches(entries, opts = {}) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
batches.sort((a, b) => b.entry_count - a.entry_count);
|
batches.sort((a, b) => b.entry_count - a.entry_count);
|
||||||
return { batches, unbatched, total: (entries || []).length };
|
|
||||||
|
// Optional output cap (default: no cap — the full ranked list is returned).
|
||||||
|
// ReContext P2: relevance is concentrated, so surfacing only the top head keeps
|
||||||
|
// reviewer attention on the highest-value clusters. `topFraction` keeps the top
|
||||||
|
// ceil(f * N) after ranking (so the survivor count tracks corpus volume);
|
||||||
|
// `maxBatches` is a hard ceiling. The more restrictive of the two applies.
|
||||||
|
// Never silent: `capped` reports how many ranked batches were dropped by the cap.
|
||||||
|
let keepN = batches.length;
|
||||||
|
const frac = Number(opts.topFraction);
|
||||||
|
if (frac > 0 && frac < 1) keepN = Math.min(keepN, Math.max(1, Math.ceil(frac * batches.length)));
|
||||||
|
const maxB = Number(opts.maxBatches);
|
||||||
|
if (Number.isFinite(maxB) && maxB > 0) keepN = Math.min(keepN, maxB);
|
||||||
|
const capped = batches.length - keepN;
|
||||||
|
const kept = capped > 0 ? batches.slice(0, keepN) : batches;
|
||||||
|
|
||||||
|
return { batches: kept, unbatched, capped, total: (entries || []).length };
|
||||||
}
|
}
|
||||||
|
|
||||||
function main() {
|
function main() {
|
||||||
const args = parseArgs(process.argv.slice(2));
|
const args = parseArgs(process.argv.slice(2));
|
||||||
if (args.help) {
|
if (args.help) {
|
||||||
process.stdout.write("usage: adam-batch.mjs [--input <jsonl-path>] [--min-entries N] [--min-sessions N]\n");
|
process.stdout.write("usage: adam-batch.mjs [--input <jsonl-path>] [--min-entries N] [--min-sessions N] [--max-batches N] [--top-fraction F]\n");
|
||||||
process.exit(0);
|
process.exit(0);
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
@@ -170,7 +193,7 @@ function main() {
|
|||||||
process.stderr.write("adam-batch: no input (use --input or pipe)\n");
|
process.stderr.write("adam-batch: no input (use --input or pipe)\n");
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
}
|
}
|
||||||
const result = buildBatches(entries, { minEntries: args.minEntries, minSessions: args.minSessions });
|
const result = buildBatches(entries, { minEntries: args.minEntries, minSessions: args.minSessions, maxBatches: args.maxBatches, topFraction: args.topFraction });
|
||||||
process.stdout.write(JSON.stringify(result) + "\n");
|
process.stdout.write(JSON.stringify(result) + "\n");
|
||||||
process.exit(0);
|
process.exit(0);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
|
|||||||
@@ -167,6 +167,23 @@ export function executeRollback(plan, adamRoot, opts = {}) {
|
|||||||
result.actions.push(`nudge failed: ${e.message}`);
|
result.actions.push(`nudge failed: ${e.message}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Remove the ab-tracking entry for this proposal so it stops re-flagging as a
|
||||||
|
// regression on every future /reflect (which would trigger endless not_found
|
||||||
|
// rollback attempts). This is the documented contract for rollback.
|
||||||
|
try {
|
||||||
|
const abPath = join(adamRoot, "ab-tracking.jsonl");
|
||||||
|
if (existsSync(abPath)) {
|
||||||
|
const before = readJsonlSafe(abPath);
|
||||||
|
const kept = before.filter((e) => !(e && e.proposal_id === plan.proposal_id));
|
||||||
|
if (kept.length !== before.length) {
|
||||||
|
writeFileSync(abPath, kept.length ? kept.map((e) => JSON.stringify(e)).join("\n") + "\n" : "");
|
||||||
|
result.actions.push(`ab-tracking entry removed (${before.length - kept.length})`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
result.actions.push(`ab-tracking cleanup failed: ${e.message}`);
|
||||||
|
}
|
||||||
|
|
||||||
result.status = "rolled_back";
|
result.status = "rolled_back";
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|||||||
Executable
+272
@@ -0,0 +1,272 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
// adam-skill-utility.mjs — execution-grounded per-skill utility report.
|
||||||
|
//
|
||||||
|
// Inspired by SkillsInjector (arXiv 2605.29794v1), which shows skill injection
|
||||||
|
// should be driven by execution-grounded *utility* Δ(t,s), not surface keyword
|
||||||
|
// match — and that some topically-relevant skills actively *lower* success.
|
||||||
|
// The paper learns Δ(t,s) from rollout outcomes. We don't train anything: the
|
||||||
|
// adam journal already attaches `active_skills` to both positive outcome events
|
||||||
|
// (task_completed, clean_recovery, correction_free_streak) and negative ones
|
||||||
|
// (dead_end, tool_error_loop, …). So we approximate Δ(s) as a co-occurrence
|
||||||
|
// ratio over the data we already collect.
|
||||||
|
//
|
||||||
|
// CAVEAT (honest): this is CO-OCCURRENCE, not causation. A skill active during
|
||||||
|
// a dead_end did not necessarily cause it. Read the report as "which skills
|
||||||
|
// correlate with friction", a prompt for review — never as proof.
|
||||||
|
//
|
||||||
|
// Metric, per skill active during scored events:
|
||||||
|
// pos / neg — count of positive / negative outcome events it co-occurred with
|
||||||
|
// share — pos / (pos+neg)
|
||||||
|
// lift — share − global_baseline (>0 above baseline, <0 below)
|
||||||
|
// wLB — Wilson 95% lower bound of the positive proportion; ranks
|
||||||
|
// *reliably* below-baseline skills to the top (sample-aware)
|
||||||
|
// sevNeg — severity-weighted negative sum (adam SEVERITY_DIVISORS)
|
||||||
|
// topNeg — dominant negative event type
|
||||||
|
// Rows sorted worst-first (lowest wLB) so harmful/over-eager skills surface.
|
||||||
|
//
|
||||||
|
// CLI:
|
||||||
|
// adam-skill-utility.mjs [--home <path>] [--input <jsonl-path>]
|
||||||
|
// [--min <n>] [--days <n>] [--json]
|
||||||
|
// --min min event count (n) to treat a skill's signal as confident (default 8)
|
||||||
|
// --days only consider events within the last <n> days (default: all)
|
||||||
|
// --json emit machine-readable JSON instead of the text table
|
||||||
|
//
|
||||||
|
// Reuses adam-utils (jsonl IO) and adam-score (canonical NEGATIVE set +
|
||||||
|
// severity), so the positive/negative taxonomy stays single-sourced.
|
||||||
|
|
||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { homedir } from "node:os";
|
||||||
|
import { readJsonlSafe, listJsonlFiles } from "./adam-utils.mjs";
|
||||||
|
import { NEGATIVE_SIGNAL_TYPES, entrySeverity } from "./adam-score.mjs";
|
||||||
|
|
||||||
|
// Positive outcome signals (mirror adam's vocabulary; task_completed is adam's
|
||||||
|
// canonical "clean task", the same one adam-score uses for reinforcement).
|
||||||
|
export const POSITIVE_SIGNAL_TYPES = new Set([
|
||||||
|
"task_completed",
|
||||||
|
"clean_recovery",
|
||||||
|
"correction_free_streak",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export const DEFAULT_MIN_SAMPLE = 8;
|
||||||
|
|
||||||
|
function round(x) {
|
||||||
|
return Math.round(x * 1000) / 1000;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wilson score interval lower bound for a binomial proportion. Sample-aware:
|
||||||
|
// a skill with 1 pos / 0 neg does NOT outrank one with 40 pos / 2 neg.
|
||||||
|
export function wilsonLower(pos, n, z = 1.96) {
|
||||||
|
if (n <= 0) return 0;
|
||||||
|
const p = pos / n;
|
||||||
|
const z2 = z * z;
|
||||||
|
const denom = 1 + z2 / n;
|
||||||
|
const center = p + z2 / (2 * n);
|
||||||
|
const margin = z * Math.sqrt((p * (1 - p) + z2 / (4 * n)) / n);
|
||||||
|
return (center - margin) / denom;
|
||||||
|
}
|
||||||
|
|
||||||
|
// computeSkillUtility: pure. entries → { baseline, totalPos, totalNeg, min, skills[] }.
|
||||||
|
export function computeSkillUtility(entries, opts = {}) {
|
||||||
|
const min = Number.isFinite(opts.min) ? opts.min : DEFAULT_MIN_SAMPLE;
|
||||||
|
const per = new Map();
|
||||||
|
let totalPos = 0;
|
||||||
|
let totalNeg = 0;
|
||||||
|
|
||||||
|
for (const e of entries || []) {
|
||||||
|
if (!e || typeof e !== "object") continue;
|
||||||
|
const isPos = POSITIVE_SIGNAL_TYPES.has(e.type);
|
||||||
|
const isNeg = NEGATIVE_SIGNAL_TYPES.has(e.type);
|
||||||
|
if (!isPos && !isNeg) continue;
|
||||||
|
|
||||||
|
if (isPos) totalPos++;
|
||||||
|
else totalNeg++;
|
||||||
|
const sev = isNeg ? entrySeverity(e) : 0;
|
||||||
|
|
||||||
|
const skills = Array.isArray(e.active_skills) ? e.active_skills : [];
|
||||||
|
for (const slug of skills) {
|
||||||
|
if (!slug || typeof slug !== "string") continue;
|
||||||
|
if (!per.has(slug)) {
|
||||||
|
per.set(slug, { pos: 0, neg: 0, sevNeg: 0, negTypes: {}, recent_ts: null });
|
||||||
|
}
|
||||||
|
const s = per.get(slug);
|
||||||
|
if (isPos) {
|
||||||
|
s.pos++;
|
||||||
|
} else {
|
||||||
|
s.neg++;
|
||||||
|
s.sevNeg += sev;
|
||||||
|
s.negTypes[e.type] = (s.negTypes[e.type] || 0) + 1;
|
||||||
|
}
|
||||||
|
const ts = typeof e.ts === "string" ? e.ts : null;
|
||||||
|
if (ts && (!s.recent_ts || ts > s.recent_ts)) s.recent_ts = ts;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const scored = totalPos + totalNeg;
|
||||||
|
const baseline = scored ? totalPos / scored : 0;
|
||||||
|
|
||||||
|
const skills = [];
|
||||||
|
for (const [slug, s] of per.entries()) {
|
||||||
|
const n = s.pos + s.neg;
|
||||||
|
const share = n ? s.pos / n : 0;
|
||||||
|
const topNeg = Object.entries(s.negTypes).sort((a, b) => b[1] - a[1])[0];
|
||||||
|
skills.push({
|
||||||
|
skill: slug,
|
||||||
|
n,
|
||||||
|
pos: s.pos,
|
||||||
|
neg: s.neg,
|
||||||
|
share: round(share),
|
||||||
|
lift: round(share - baseline),
|
||||||
|
wLB: round(wilsonLower(s.pos, n)),
|
||||||
|
sevNeg: s.sevNeg,
|
||||||
|
topNeg: topNeg ? topNeg[0] : null,
|
||||||
|
lowSample: n < min,
|
||||||
|
recent_ts: s.recent_ts,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Worst-first: lowest Wilson lower bound, then most negatives.
|
||||||
|
skills.sort(
|
||||||
|
(a, b) =>
|
||||||
|
a.wLB - b.wLB ||
|
||||||
|
b.neg - a.neg ||
|
||||||
|
(a.skill < b.skill ? -1 : a.skill > b.skill ? 1 : 0),
|
||||||
|
);
|
||||||
|
|
||||||
|
return { baseline: round(baseline), totalPos, totalNeg, min, skills };
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseArgs(argv) {
|
||||||
|
const args = { home: null, input: null, min: DEFAULT_MIN_SAMPLE, days: null, json: false, help: false };
|
||||||
|
for (let i = 0; i < argv.length; i++) {
|
||||||
|
const a = argv[i];
|
||||||
|
if (a === "--home" && i + 1 < argv.length) args.home = argv[++i];
|
||||||
|
else if (a === "--input" && i + 1 < argv.length) args.input = argv[++i];
|
||||||
|
else if (a === "--min" && i + 1 < argv.length) args.min = Number(argv[++i]);
|
||||||
|
else if (a === "--days" && i + 1 < argv.length) args.days = Number(argv[++i]);
|
||||||
|
else if (a === "--json") args.json = true;
|
||||||
|
else if (a === "--help" || a === "-h") args.help = true;
|
||||||
|
}
|
||||||
|
return args;
|
||||||
|
}
|
||||||
|
|
||||||
|
function readAllStdin() {
|
||||||
|
try { return readFileSync(0, "utf8"); } catch { return ""; }
|
||||||
|
}
|
||||||
|
|
||||||
|
function entriesFromText(text) {
|
||||||
|
const out = [];
|
||||||
|
for (const line of (text || "").split("\n")) {
|
||||||
|
if (!line) continue;
|
||||||
|
try { out.push(JSON.parse(line)); } catch { /* skip */ }
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Same gathering strategy as adam-score.mjs: explicit --input, else piped
|
||||||
|
// stdin (e.g. from adam-window.mjs), else the active journal + rotated files.
|
||||||
|
function gatherInputEntries(args) {
|
||||||
|
if (args.input) return readJsonlSafe(args.input);
|
||||||
|
if (!process.stdin.isTTY) {
|
||||||
|
const piped = readAllStdin();
|
||||||
|
if (piped && piped.trim()) return entriesFromText(piped);
|
||||||
|
}
|
||||||
|
const home = args.home || join(homedir(), ".claude");
|
||||||
|
const adamRoot = join(home, "adam");
|
||||||
|
const sources = [join(adamRoot, "journal.jsonl"), ...listJsonlFiles(join(adamRoot, "journal"))];
|
||||||
|
const all = [];
|
||||||
|
for (const p of sources) {
|
||||||
|
for (const e of readJsonlSafe(p)) all.push(e);
|
||||||
|
}
|
||||||
|
return all;
|
||||||
|
}
|
||||||
|
|
||||||
|
function filterByDays(entries, days) {
|
||||||
|
if (!Number.isFinite(days) || days <= 0) return entries;
|
||||||
|
// Anchor the window to the newest ts in the data (avoids Date.now()
|
||||||
|
// nondeterminism and works on historical exports).
|
||||||
|
let maxMs = 0;
|
||||||
|
for (const e of entries) {
|
||||||
|
const ms = e && typeof e.ts === "string" ? Date.parse(e.ts) : NaN;
|
||||||
|
if (Number.isFinite(ms) && ms > maxMs) maxMs = ms;
|
||||||
|
}
|
||||||
|
if (!maxMs) return entries;
|
||||||
|
const cutoff = maxMs - days * 86400000;
|
||||||
|
return entries.filter((e) => {
|
||||||
|
const ms = e && typeof e.ts === "string" ? Date.parse(e.ts) : NaN;
|
||||||
|
return Number.isFinite(ms) ? ms >= cutoff : false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function pad(s, w) {
|
||||||
|
s = String(s);
|
||||||
|
return s.length >= w ? s : s + " ".repeat(w - s.length);
|
||||||
|
}
|
||||||
|
function padL(s, w) {
|
||||||
|
s = String(s);
|
||||||
|
return s.length >= w ? s : " ".repeat(w - s.length) + s;
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderText(report) {
|
||||||
|
const { baseline, totalPos, totalNeg, min, skills } = report;
|
||||||
|
const lines = [];
|
||||||
|
lines.push("adam skill-utility report — execution-grounded Δ(skill) proxy");
|
||||||
|
lines.push(
|
||||||
|
`baseline positive-rate ${(baseline * 100).toFixed(1)}% ` +
|
||||||
|
`(${totalPos} positive / ${totalNeg} negative outcome events) min-sample n≥${min}`,
|
||||||
|
);
|
||||||
|
lines.push("CAVEAT: co-occurrence, not causation. Worst-first. ⚠ = below baseline with n≥min.");
|
||||||
|
lines.push("");
|
||||||
|
const head =
|
||||||
|
pad("skill", 44) + padL("n", 5) + padL("pos", 6) + padL("neg", 6) +
|
||||||
|
padL("share", 8) + padL("lift", 8) + padL("wLB", 7) + padL("sevNeg", 8) +
|
||||||
|
" " + pad("topNeg", 18) + "flag";
|
||||||
|
lines.push(head);
|
||||||
|
lines.push("-".repeat(head.length));
|
||||||
|
for (const s of skills) {
|
||||||
|
const below = s.lift < 0 && !s.lowSample;
|
||||||
|
const flag = below ? "⚠" : s.lowSample ? "·(low n)" : "";
|
||||||
|
lines.push(
|
||||||
|
pad(s.skill, 44) +
|
||||||
|
padL(s.n, 5) +
|
||||||
|
padL(s.pos, 6) +
|
||||||
|
padL(s.neg, 6) +
|
||||||
|
padL((s.share * 100).toFixed(0) + "%", 8) +
|
||||||
|
padL((s.lift >= 0 ? "+" : "") + (s.lift * 100).toFixed(0) + "%", 8) +
|
||||||
|
padL(s.wLB.toFixed(2), 7) +
|
||||||
|
padL(s.sevNeg, 8) +
|
||||||
|
" " +
|
||||||
|
pad(s.topNeg || "-", 18) +
|
||||||
|
flag,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
function main() {
|
||||||
|
const args = parseArgs(process.argv.slice(2));
|
||||||
|
if (args.help) {
|
||||||
|
process.stdout.write(
|
||||||
|
"usage: adam-skill-utility.mjs [--home <path>] [--input <jsonl-path>] " +
|
||||||
|
"[--min <n>] [--days <n>] [--json]\n",
|
||||||
|
);
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
let entries = gatherInputEntries(args);
|
||||||
|
entries = filterByDays(entries, args.days);
|
||||||
|
const report = computeSkillUtility(entries, { min: args.min });
|
||||||
|
if (args.json) {
|
||||||
|
process.stdout.write(JSON.stringify(report) + "\n");
|
||||||
|
} else {
|
||||||
|
process.stdout.write(renderText(report) + "\n");
|
||||||
|
}
|
||||||
|
process.exit(0);
|
||||||
|
} catch (e) {
|
||||||
|
process.stderr.write(`adam-skill-utility error: ${e.message}\n`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||||
|
main();
|
||||||
|
}
|
||||||
+331
-15
@@ -5,19 +5,24 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REAL_HOME="$HOME"
|
REAL_HOME="$HOME"
|
||||||
HOOK="$REAL_HOME/.claude/hooks/adam-observe.mjs"
|
# Test the tree this script lives in: <base>/adam/tests/run-tests.sh. The repo
|
||||||
NUDGE="$REAL_HOME/.claude/hooks/adam-nudge.mjs"
|
# and an install (~/.claude) share the layout <base>/hooks, <base>/adam/scripts.
|
||||||
ARCHIVE="$REAL_HOME/.claude/adam/scripts/adam-archive.mjs"
|
BASE="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||||
WINDOW="$REAL_HOME/.claude/adam/scripts/adam-window.mjs"
|
HOOK="$BASE/hooks/adam-observe.mjs"
|
||||||
EXPLAIN="$REAL_HOME/.claude/adam/scripts/adam-explain.mjs"
|
NUDGE="$BASE/hooks/adam-nudge.mjs"
|
||||||
ELIGIBILITY="$REAL_HOME/.claude/adam/scripts/adam-nudge-eligibility.mjs"
|
ARCHIVE="$BASE/adam/scripts/adam-archive.mjs"
|
||||||
COOLDOWN="$REAL_HOME/.claude/adam/scripts/adam-cooldown.mjs"
|
WINDOW="$BASE/adam/scripts/adam-window.mjs"
|
||||||
SCORE="$REAL_HOME/.claude/adam/scripts/adam-score.mjs"
|
EXPLAIN="$BASE/adam/scripts/adam-explain.mjs"
|
||||||
ABMEASURE="$REAL_HOME/.claude/adam/scripts/adam-ab-measure.mjs"
|
ELIGIBILITY="$BASE/adam/scripts/adam-nudge-eligibility.mjs"
|
||||||
APPLYREIN="$REAL_HOME/.claude/adam/scripts/adam-apply-reinforcement.mjs"
|
COOLDOWN="$BASE/adam/scripts/adam-cooldown.mjs"
|
||||||
UPGRADE="$REAL_HOME/.claude/adam/scripts/adam-upgrade.mjs"
|
SCORE="$BASE/adam/scripts/adam-score.mjs"
|
||||||
BATCH="$REAL_HOME/.claude/adam/scripts/adam-batch.mjs"
|
ABMEASURE="$BASE/adam/scripts/adam-ab-measure.mjs"
|
||||||
ROLLBACK="$REAL_HOME/.claude/adam/scripts/adam-rollback.mjs"
|
APPLYREIN="$BASE/adam/scripts/adam-apply-reinforcement.mjs"
|
||||||
|
UPGRADE="$BASE/adam/scripts/adam-upgrade.mjs"
|
||||||
|
BATCH="$BASE/adam/scripts/adam-batch.mjs"
|
||||||
|
ROLLBACK="$BASE/adam/scripts/adam-rollback.mjs"
|
||||||
|
SKILLUTIL="$BASE/adam/scripts/adam-skill-utility.mjs"
|
||||||
|
APPLYNUDGE="$BASE/adam/scripts/adam-apply-nudge.mjs"
|
||||||
|
|
||||||
TMP_HOME="$(mktemp -d -t adam-test.XXXXXX)"
|
TMP_HOME="$(mktemp -d -t adam-test.XXXXXX)"
|
||||||
trap 'rm -rf "$TMP_HOME"' EXIT INT TERM
|
trap 'rm -rf "$TMP_HOME"' EXIT INT TERM
|
||||||
@@ -37,6 +42,8 @@ APPLYREIN_RUN(){ HOME="$TMP_HOME" node "$APPLYREIN" "$@" --home "$TMP_HOME/.clau
|
|||||||
UPGRADE_RUN() { HOME="$TMP_HOME" node "$UPGRADE" "$@"; }
|
UPGRADE_RUN() { HOME="$TMP_HOME" node "$UPGRADE" "$@"; }
|
||||||
BATCH_RUN() { HOME="$TMP_HOME" node "$BATCH" "$@"; }
|
BATCH_RUN() { HOME="$TMP_HOME" node "$BATCH" "$@"; }
|
||||||
ROLLBACK_RUN(){ HOME="$TMP_HOME" node "$ROLLBACK" "$@"; }
|
ROLLBACK_RUN(){ HOME="$TMP_HOME" node "$ROLLBACK" "$@"; }
|
||||||
|
SKILLUTIL_RUN(){ HOME="$TMP_HOME" node "$SKILLUTIL" "$@"; }
|
||||||
|
APPLYNUDGE_RUN(){ HOME="$TMP_HOME" node "$APPLYNUDGE" "$@" --home "$TMP_HOME/.claude"; }
|
||||||
|
|
||||||
PASS=0
|
PASS=0
|
||||||
FAIL=0
|
FAIL=0
|
||||||
@@ -180,6 +187,25 @@ for i in 1 2 3; do
|
|||||||
done
|
done
|
||||||
assert_grep "$ROOT/journal.jsonl" '"type":"tool_error_loop"' "3x same error logs tool_error_loop"
|
assert_grep "$ROOT/journal.jsonl" '"type":"tool_error_loop"' "3x same error logs tool_error_loop"
|
||||||
|
|
||||||
|
# --- Test 9b: PostToolUseFailure feeds the same detectors ---
|
||||||
|
echo "Test 9b: 3x PostToolUseFailure with the same error logs tool_error_loop"
|
||||||
|
reset_state
|
||||||
|
for i in 1 2 3; do
|
||||||
|
echo '{"hook_event_name":"PostToolUseFailure","tool_name":"Bash","tool_input":{"command":"npm test"},"error":"Exit code 1\nError: Cannot find module express","is_interrupt":false,"session_id":"s9b","cwd":"/tmp/x"}' \
|
||||||
|
| HOOK_RUN >/dev/null 2>&1 || true
|
||||||
|
done
|
||||||
|
assert_grep "$ROOT/journal.jsonl" '"type":"tool_error_loop"' "3x PostToolUseFailure logs tool_error_loop"
|
||||||
|
|
||||||
|
# --- Test 9c: interrupted tool calls are not errors ---
|
||||||
|
echo "Test 9c: 8x PostToolUseFailure with is_interrupt:true counts as tool use but logs no tool_error_loop"
|
||||||
|
reset_state
|
||||||
|
for i in 1 2 3 4 5 6 7 8; do
|
||||||
|
echo '{"hook_event_name":"PostToolUseFailure","tool_name":"Bash","tool_input":{"command":"sleep 99"},"error":"Error: aborted","is_interrupt":true,"session_id":"s9c","cwd":"/tmp/x"}' \
|
||||||
|
| HOOK_RUN >/dev/null 2>&1 || true
|
||||||
|
done
|
||||||
|
assert_grep "$ROOT/journal.jsonl" '"type":"dead_end"' "interrupted calls still count as tool events"
|
||||||
|
assert_no_grep "$ROOT/journal.jsonl" '"type":"tool_error_loop"' "interrupts do not log tool_error_loop"
|
||||||
|
|
||||||
# --- Test 10: dead_end on long autonomous run ---
|
# --- Test 10: dead_end on long autonomous run ---
|
||||||
echo "Test 10: dead_end after 8 tools without UserPromptSubmit"
|
echo "Test 10: dead_end after 8 tools without UserPromptSubmit"
|
||||||
reset_state
|
reset_state
|
||||||
@@ -624,6 +650,27 @@ else
|
|||||||
echo " FAIL: hex/epoch fingerprint mismatch (a=$fp_a b=$fp_b)"; FAIL=$((FAIL+1))
|
echo " FAIL: hex/epoch fingerprint mismatch (a=$fp_a b=$fp_b)"; FAIL=$((FAIL+1))
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# --- Test 36b: fingerprint — file-tool success objects are not errors ---
|
||||||
|
echo "Test 36b: Read/Edit/Write success objects with error words return null"
|
||||||
|
fp_r=$(node -e "import('$HOOK').then(m => console.log(m.errorFingerprint({type:'text',file:{filePath:'/x/cache.go',content:'return fmt.Errorf(\"failed: not found\")'}})))")
|
||||||
|
fp_e=$(node -e "import('$HOOK').then(m => console.log(m.errorFingerprint({filePath:'/x/a.ts',oldString:'throw new Error(1)',newString:'x',originalFile:'undefined error'})))")
|
||||||
|
fp_w=$(node -e "import('$HOOK').then(m => console.log(m.errorFingerprint({type:'create',filePath:'/x/b.ts',content:'const errorRateTier = \"failed\"'})))")
|
||||||
|
if [ "$fp_r" = "null" ] && [ "$fp_e" = "null" ] && [ "$fp_w" = "null" ]; then
|
||||||
|
echo " PASS: file-tool success shapes not fingerprinted"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: success shape fingerprinted (r=$fp_r e=$fp_e w=$fp_w)"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- Test 36c: fingerprint — real error shapes still fingerprinted ---
|
||||||
|
echo "Test 36c: is_error:true with filePath and plain error strings still fingerprint"
|
||||||
|
fp_x=$(node -e "import('$HOOK').then(m => console.log(m.errorFingerprint({is_error:true,filePath:'/x/a.ts',content:'String to replace not found in file.'})))")
|
||||||
|
fp_s=$(node -e "import('$HOOK').then(m => console.log(m.errorFingerprint('Error: result (70,310 characters) exceeds maximum allowed tokens')))")
|
||||||
|
if [ "$fp_x" != "null" ] && [ "$fp_s" != "null" ]; then
|
||||||
|
echo " PASS: error shapes keep fingerprints ($fp_x, $fp_s)"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: error shape lost fingerprint (x=$fp_x s=$fp_s)"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
|
||||||
# --- Test 37: correction corpus — strong tokens fire ---
|
# --- Test 37: correction corpus — strong tokens fire ---
|
||||||
echo "Test 37: strong-correction tokens each emit correction signal"
|
echo "Test 37: strong-correction tokens each emit correction signal"
|
||||||
strong_ok=1
|
strong_ok=1
|
||||||
@@ -1379,9 +1426,10 @@ rm -f "$ROOT/applied/2026-05-11-legacy-target.md"
|
|||||||
|
|
||||||
# --- Test 77: install.sh covers every adam-*.mjs script in scripts/ ---
|
# --- Test 77: install.sh covers every adam-*.mjs script in scripts/ ---
|
||||||
echo "Test 77: install.sh references every adam-*.mjs file"
|
echo "Test 77: install.sh references every adam-*.mjs file"
|
||||||
SCRIPTS_DIR="$REAL_HOME/.claude/adam/scripts"
|
SCRIPTS_DIR="$BASE/adam/scripts"
|
||||||
INSTALL_SH=""
|
INSTALL_SH=""
|
||||||
for cand in "$REAL_HOME/Documents/projects/private/claude-adam/install.sh" \
|
for cand in "$BASE/install.sh" \
|
||||||
|
"$REAL_HOME/Documents/projects/private/claude-adam/install.sh" \
|
||||||
"$REAL_HOME/Documents/projects/private/adam/install.sh"; do
|
"$REAL_HOME/Documents/projects/private/adam/install.sh"; do
|
||||||
[ -f "$cand" ] && INSTALL_SH="$cand" && break
|
[ -f "$cand" ] && INSTALL_SH="$cand" && break
|
||||||
done
|
done
|
||||||
@@ -2014,6 +2062,274 @@ done
|
|||||||
assert_grep "$ROOT/journal.jsonl" '"type":"retry_loop"' "3x byte-identical reads emit retry_loop"
|
assert_grep "$ROOT/journal.jsonl" '"type":"retry_loop"' "3x byte-identical reads emit retry_loop"
|
||||||
assert_no_grep "$ROOT/journal.jsonl" '"type":"file_reread"' "byte-identical reads NOT double-counted as file_reread (sameToolArgs>=RETRY guard)"
|
assert_no_grep "$ROOT/journal.jsonl" '"type":"file_reread"' "byte-identical reads NOT double-counted as file_reread (sameToolArgs>=RETRY guard)"
|
||||||
|
|
||||||
|
# --- Test 112: A/B volume normalization — busier journal does NOT fake a regression ---
|
||||||
|
echo "Test 112: A/B volume-normalized (raw +200% but flat share → neutral)"
|
||||||
|
reset_state
|
||||||
|
applied_at_ms=$(node -e 'console.log(Date.now() - 14*86400000)')
|
||||||
|
cat > "$ROOT/ab-tracking.jsonl" <<EOF
|
||||||
|
{"applied_at":$applied_at_ms,"proposal_id":"ab-vol-001","proposal_type":"memory","target_skill":"vol","proposal_fingerprint":"fpV","originating_signals":[{"type":"correction","count":2,"session_ids":["sV"]}],"pre_window_days":7}
|
||||||
|
EOF
|
||||||
|
> "$ROOT/journal.jsonl"
|
||||||
|
# pre window: 2 correction + 8 dead_end (rate 0.2)
|
||||||
|
for i in 1 2; do ts=$(node -e "console.log(new Date(Date.now()-(15+$i*0.2)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV\",\"type\":\"correction\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
for i in 1 2 3 4 5 6 7 8; do ts=$(node -e "console.log(new Date(Date.now()-(15+$i*0.1)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV\",\"type\":\"dead_end\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
# post window: 6 correction + 24 dead_end (rate 0.2 — share unchanged, raw count +200%)
|
||||||
|
for i in $(seq 1 6); do ts=$(node -e "console.log(new Date(Date.now()-(8+$i*0.1)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV\",\"type\":\"correction\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
for i in $(seq 1 24); do ts=$(node -e "console.log(new Date(Date.now()-(8+$i*0.05)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV\",\"type\":\"dead_end\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
out=$(ABMEASURE_RUN --format json 2>/dev/null)
|
||||||
|
if echo "$out" | node -e 'let b="";process.stdin.on("data",d=>b+=d).on("end",()=>{const a=JSON.parse(b);const e=a.find(x=>x.proposal_id==="ab-vol-001");process.exit(e&&e.normalized===true&&e.raw_delta_pct===200&&e.status==="neutral"?0:1)})'; then
|
||||||
|
echo " PASS: volume growth normalized → neutral (raw +200%)"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: volume normalization wrong (got: $out)"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/ab-tracking.jsonl"
|
||||||
|
|
||||||
|
# --- Test 113: A/B genuine rate regression still flagged ---
|
||||||
|
echo "Test 113: A/B genuine share increase → regressed"
|
||||||
|
reset_state
|
||||||
|
applied_at_ms=$(node -e 'console.log(Date.now() - 14*86400000)')
|
||||||
|
cat > "$ROOT/ab-tracking.jsonl" <<EOF
|
||||||
|
{"applied_at":$applied_at_ms,"proposal_id":"ab-vol-002","proposal_type":"memory","target_skill":"vol2","proposal_fingerprint":"fpV2","originating_signals":[{"type":"correction","count":2,"session_ids":["sV2"]}],"pre_window_days":7}
|
||||||
|
EOF
|
||||||
|
> "$ROOT/journal.jsonl"
|
||||||
|
# pre: 2 correction + 8 dead_end (rate 0.2)
|
||||||
|
for i in 1 2; do ts=$(node -e "console.log(new Date(Date.now()-(15+$i*0.2)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV2\",\"type\":\"correction\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
for i in 1 2 3 4 5 6 7 8; do ts=$(node -e "console.log(new Date(Date.now()-(15+$i*0.1)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV2\",\"type\":\"dead_end\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
# post: 6 correction + 6 dead_end (rate 0.5 — share up → genuine regression)
|
||||||
|
for i in $(seq 1 6); do ts=$(node -e "console.log(new Date(Date.now()-(8+$i*0.1)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV2\",\"type\":\"correction\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
for i in 1 2 3 4 5 6; do ts=$(node -e "console.log(new Date(Date.now()-(8+$i*0.07)*86400000).toISOString())"); echo "{\"ts\":\"$ts\",\"session\":\"sV2\",\"type\":\"dead_end\"}" >> "$ROOT/journal.jsonl"; done
|
||||||
|
out=$(ABMEASURE_RUN --format json 2>/dev/null)
|
||||||
|
if echo "$out" | node -e 'let b="";process.stdin.on("data",d=>b+=d).on("end",()=>{const a=JSON.parse(b);const e=a.find(x=>x.proposal_id==="ab-vol-002");process.exit(e&&e.normalized===true&&e.status==="regressed"?0:1)})'; then
|
||||||
|
echo " PASS: genuine share increase → regressed"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: genuine regression missed (got: $out)"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/ab-tracking.jsonl"
|
||||||
|
|
||||||
|
# --- Test 114: update notifier nudges from cache when a newer release exists (no network) ---
|
||||||
|
echo "Test 114: update notifier — cached newer release prints nudge"
|
||||||
|
reset_state
|
||||||
|
printf 'v0.6.2\n' > "$ROOT/.version"
|
||||||
|
node -e "require('fs').writeFileSync('$ROOT/.update-check.json', JSON.stringify({last_check: Date.now(), latest: 'v9.9.9'}))"
|
||||||
|
out=$(echo '{"hook_event_name":"SessionStart","session_id":"sUP"}' | NUDGE_RUN 2>/dev/null)
|
||||||
|
if echo "$out" | grep -q "update available: v0.6.2 → v9.9.9"; then
|
||||||
|
echo " PASS: update nudge printed from cache (offline)"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: expected update nudge (got: $out)"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/.version" "$ROOT/.update-check.json"
|
||||||
|
|
||||||
|
# --- Test 115: update notifier silent when installed is current ---
|
||||||
|
echo "Test 115: update notifier — up-to-date is silent"
|
||||||
|
reset_state
|
||||||
|
printf 'v9.9.9\n' > "$ROOT/.version"
|
||||||
|
node -e "require('fs').writeFileSync('$ROOT/.update-check.json', JSON.stringify({last_check: Date.now(), latest: 'v9.9.9'}))"
|
||||||
|
out=$(echo '{"hook_event_name":"SessionStart","session_id":"sUP2"}' | NUDGE_RUN 2>/dev/null)
|
||||||
|
if echo "$out" | grep -q "update available"; then
|
||||||
|
echo " FAIL: nudged despite being current (got: $out)"; FAIL=$((FAIL+1))
|
||||||
|
else
|
||||||
|
echo " PASS: no nudge when up-to-date"; PASS=$((PASS+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/.version" "$ROOT/.update-check.json"
|
||||||
|
|
||||||
|
# --- Test 116: ADAM_NO_UPDATE_CHECK disables the notifier ---
|
||||||
|
echo "Test 116: ADAM_NO_UPDATE_CHECK opt-out"
|
||||||
|
reset_state
|
||||||
|
printf 'v0.6.2\n' > "$ROOT/.version"
|
||||||
|
node -e "require('fs').writeFileSync('$ROOT/.update-check.json', JSON.stringify({last_check: Date.now(), latest: 'v9.9.9'}))"
|
||||||
|
out=$(echo '{"hook_event_name":"SessionStart","session_id":"sUP3"}' | HOME="$TMP_HOME" ADAM_NO_UPDATE_CHECK=1 node "$NUDGE" 2>/dev/null)
|
||||||
|
if echo "$out" | grep -q "update available"; then
|
||||||
|
echo " FAIL: notifier ran despite opt-out (got: $out)"; FAIL=$((FAIL+1))
|
||||||
|
else
|
||||||
|
echo " PASS: ADAM_NO_UPDATE_CHECK suppressed the check"; PASS=$((PASS+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/.version" "$ROOT/.update-check.json"
|
||||||
|
|
||||||
|
# --- Test 117: no .version marker → notifier no-op (no crash) ---
|
||||||
|
echo "Test 117: missing .version marker → notifier silent, hook still runs"
|
||||||
|
reset_state
|
||||||
|
node -e "require('fs').writeFileSync('$ROOT/.update-check.json', JSON.stringify({last_check: Date.now(), latest: 'v9.9.9'}))"
|
||||||
|
out=$(echo '{"hook_event_name":"SessionStart","session_id":"sUP4"}' | NUDGE_RUN 2>/dev/null)
|
||||||
|
if echo "$out" | grep -q "update available"; then
|
||||||
|
echo " FAIL: nudged without a .version marker (got: $out)"; FAIL=$((FAIL+1))
|
||||||
|
else
|
||||||
|
echo " PASS: no marker → no update nudge"; PASS=$((PASS+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/.update-check.json"
|
||||||
|
|
||||||
|
# --- Test 118: rollback removes the proposal's ab-tracking entry (stops re-flagging) ---
|
||||||
|
echo "Test 118: rollback purges ab-tracking entry by proposal_id"
|
||||||
|
reset_state
|
||||||
|
rm -f "$ROOT/proposals/"*rollback* "$ROOT/active-nudges.json"
|
||||||
|
cat > "$ROOT/applied/2026-05-20T00-00-00Z-rb-ab-001.md" <<'EOF'
|
||||||
|
---
|
||||||
|
id: rb-ab-001
|
||||||
|
type: memory
|
||||||
|
target: ~/.claude/projects/-Users-nvm/memory/x.md
|
||||||
|
confidence: 5
|
||||||
|
blast_radius: low
|
||||||
|
status: applied
|
||||||
|
source_entries:
|
||||||
|
- "2026-05-18T10:00:00Z"
|
||||||
|
---
|
||||||
|
# Why
|
||||||
|
test
|
||||||
|
# Rollback
|
||||||
|
```bash
|
||||||
|
rm -f x
|
||||||
|
```
|
||||||
|
EOF
|
||||||
|
cat > "$ROOT/ab-tracking.jsonl" <<'EOF'
|
||||||
|
{"applied_at":1,"proposal_id":"rb-ab-001","proposal_type":"memory","target_skill":"x","proposal_fingerprint":"f1","originating_signals":[{"type":"correction","count":3}],"pre_window_days":7}
|
||||||
|
{"applied_at":2,"proposal_id":"keep-me-002","proposal_type":"memory","target_skill":"y","proposal_fingerprint":"f2","originating_signals":[{"type":"correction","count":3}],"pre_window_days":7}
|
||||||
|
EOF
|
||||||
|
ROLLBACK_RUN --proposal-id rb-ab-001 --home "$TMP_HOME/.claude" >/dev/null 2>&1 || true
|
||||||
|
if grep -q '"proposal_id":"rb-ab-001"' "$ROOT/ab-tracking.jsonl"; then
|
||||||
|
echo " FAIL: rolled-back proposal still in ab-tracking.jsonl"; FAIL=$((FAIL+1))
|
||||||
|
else
|
||||||
|
echo " PASS: rolled-back proposal removed from ab-tracking.jsonl"; PASS=$((PASS+1))
|
||||||
|
fi
|
||||||
|
if grep -q '"proposal_id":"keep-me-002"' "$ROOT/ab-tracking.jsonl"; then
|
||||||
|
echo " PASS: unrelated ab-tracking entry preserved"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: rollback clobbered an unrelated ab-tracking entry"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/proposals/"*rb-ab-001* "$ROOT/applied/"*rb-ab-001* "$ROOT/ab-tracking.jsonl" "$ROOT/active-nudges.json"
|
||||||
|
|
||||||
|
# --- Test 119: adam-skill-utility ranks friction-correlated skills below baseline ---
|
||||||
|
echo "Test 119: adam-skill-utility computes per-skill good:bad utility (execution-grounded Δ)"
|
||||||
|
reset_state
|
||||||
|
SU_INPUT="$TMP_HOME/su-input.jsonl"
|
||||||
|
{
|
||||||
|
for i in 1 2 3 4 5; do echo "{\"ts\":\"2026-05-20T0$i:00:00Z\",\"session\":\"sSU\",\"type\":\"task_completed\",\"active_skills\":[\"goodskill\"]}"; done
|
||||||
|
for i in 1 2 3 4 5; do echo "{\"ts\":\"2026-05-20T1$i:00:00Z\",\"session\":\"sSU\",\"type\":\"dead_end\",\"count\":8,\"active_skills\":[\"badskill\"]}"; done
|
||||||
|
} > "$SU_INPUT"
|
||||||
|
su_out=$(SKILLUTIL_RUN --input "$SU_INPUT" --json --min 3 2>/dev/null)
|
||||||
|
su_check=$(echo "$su_out" | node -e '
|
||||||
|
let buf=""; process.stdin.on("data",d=>buf+=d).on("end",()=>{
|
||||||
|
try {
|
||||||
|
const p=JSON.parse(buf);
|
||||||
|
const bad=p.skills.find(s=>s.skill==="badskill");
|
||||||
|
const good=p.skills.find(s=>s.skill==="goodskill");
|
||||||
|
const ok = bad && good && bad.lift<0 && good.lift>0 && p.skills[0].skill==="badskill" && bad.neg===5 && good.pos===5;
|
||||||
|
console.log(ok?"ok":"bad:"+JSON.stringify({bad,good,first:p.skills[0]&&p.skills[0].skill}));
|
||||||
|
} catch(e){ console.log("parse-error:"+e.message); }
|
||||||
|
});')
|
||||||
|
if [ "$su_check" = "ok" ]; then
|
||||||
|
echo " PASS: badskill below baseline + ranked worst-first, goodskill above"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: skill-utility ranking wrong ($su_check)"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$SU_INPUT"
|
||||||
|
|
||||||
|
# --- Test 120: nudge cue.cwd gates surfacing on the current session's cwd ---
|
||||||
|
echo "Test 120: nudge cue.cwd — matching cwd prints, mismatch deferred (kept, not consumed)"
|
||||||
|
reset_state
|
||||||
|
cue_now=$(node -e 'console.log(Date.now())')
|
||||||
|
cue_future=$(node -e 'console.log(Date.now() + 7*86400000)')
|
||||||
|
cat > "$ROOT/active-nudges.json" <<EOF
|
||||||
|
[{"kind":"dead_end_reminder","message":"adam: cue-match checkpoint","created_at":$cue_now,"expires_at_ts":$cue_future,"max_displays":3,"displays_used":0,"source_session":"sOLD","cue":{"cwd":"/proj/a"}}]
|
||||||
|
EOF
|
||||||
|
out_match=$(echo '{"hook_event_name":"SessionStart","session_id":"sNEW","cwd":"/proj/a"}' | NUDGE_RUN 2>&1 || true)
|
||||||
|
cat > "$ROOT/active-nudges.json" <<EOF
|
||||||
|
[{"kind":"dead_end_reminder","message":"adam: cue-miss checkpoint","created_at":$cue_now,"expires_at_ts":$cue_future,"max_displays":3,"displays_used":0,"source_session":"sOLD","cue":{"cwd":"/proj/a"}}]
|
||||||
|
EOF
|
||||||
|
out_miss=$(echo '{"hook_event_name":"SessionStart","session_id":"sNEW","cwd":"/proj/b"}' | NUDGE_RUN 2>&1 || true)
|
||||||
|
cue_remaining=$(node -e 'const j=require("'"$ROOT"'/active-nudges.json");console.log(j.length)')
|
||||||
|
cue_used=$(node -e 'const j=require("'"$ROOT"'/active-nudges.json");console.log((j[0]||{}).displays_used)')
|
||||||
|
if echo "$out_match" | grep -q "cue-match checkpoint" \
|
||||||
|
&& ! echo "$out_miss" | grep -q "cue-miss checkpoint" \
|
||||||
|
&& [ "$cue_remaining" = "1" ] && [ "$cue_used" = "0" ]; then
|
||||||
|
echo " PASS: cue.cwd match prints; mismatch deferred without consuming a display"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: match=$out_match miss=$out_miss remaining=$cue_remaining used=$cue_used"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/active-nudges.json"
|
||||||
|
|
||||||
|
# --- Test 121: adam-batch output cap (--max-batches / --top-fraction), default off ---
|
||||||
|
echo "Test 121: adam-batch caps ranked output and reports capped; no flag = no cap"
|
||||||
|
cap_input=$(cat <<'EOF'
|
||||||
|
{"ts":"2026-05-20T10:00:00Z","type":"correction","session":"s1","cwd":"/a","phrase":"alpha alpha"}
|
||||||
|
{"ts":"2026-05-20T11:00:00Z","type":"correction","session":"s2","cwd":"/a","phrase":"alpha alpha"}
|
||||||
|
{"ts":"2026-05-20T12:00:00Z","type":"retry_loop","session":"s1","cwd":"/a","tool":"Bash"}
|
||||||
|
{"ts":"2026-05-21T10:00:00Z","type":"tool_error_loop","session":"s1","cwd":"/a","fp":"ENOENT:x","count":3}
|
||||||
|
EOF
|
||||||
|
)
|
||||||
|
GET() { node -e 'let b="";process.stdin.on("data",d=>b+=d).on("end",()=>{const j=JSON.parse(b);console.log(eval("j."+process.argv[1]))})' "$1"; }
|
||||||
|
mb_out=$(echo "$cap_input" | BATCH_RUN --max-batches 2 2>/dev/null)
|
||||||
|
mb_bc=$(echo "$mb_out" | GET "batches.length"); mb_cap=$(echo "$mb_out" | GET "capped"); mb_top=$(echo "$mb_out" | GET "batches[0].signal_type")
|
||||||
|
none_out=$(echo "$cap_input" | BATCH_RUN 2>/dev/null)
|
||||||
|
none_bc=$(echo "$none_out" | GET "batches.length"); none_cap=$(echo "$none_out" | GET "capped")
|
||||||
|
tf_out=$(echo "$cap_input" | BATCH_RUN --top-fraction 0.5 2>/dev/null)
|
||||||
|
tf_bc=$(echo "$tf_out" | GET "batches.length"); tf_cap=$(echo "$tf_out" | GET "capped")
|
||||||
|
if [ "$mb_bc" = "2" ] && [ "$mb_cap" = "1" ] && [ "$mb_top" = "correction" ] \
|
||||||
|
&& [ "$none_bc" = "3" ] && [ "$none_cap" = "0" ] \
|
||||||
|
&& [ "$tf_bc" = "2" ] && [ "$tf_cap" = "1" ]; then
|
||||||
|
echo " PASS: --max-batches 2 → 2 batches (capped=1, head=correction); no flag → 3/capped=0; --top-fraction 0.5 → 2/capped=1"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: mb($mb_bc,$mb_cap,$mb_top) none($none_bc,$none_cap) tf($tf_bc,$tf_cap)"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- Test 122: adam-apply-nudge builds a cue'd entry deterministically; gates non-nudge ---
|
||||||
|
echo "Test 122: adam-apply-nudge populates cue.cwd + generated message from frontmatter"
|
||||||
|
reset_state
|
||||||
|
rm -f "$ROOT/active-nudges.json"
|
||||||
|
cat > "$ROOT/proposals/nudge-prop-001.md" <<'EOF'
|
||||||
|
---
|
||||||
|
type: nudge
|
||||||
|
source_session: sNUDGE
|
||||||
|
cwd: /Users/me/proj-alpha
|
||||||
|
dead_end_count: 4
|
||||||
|
confidence: 4
|
||||||
|
blast_radius: low
|
||||||
|
---
|
||||||
|
# Nudge
|
||||||
|
checkpoint reminder
|
||||||
|
EOF
|
||||||
|
an_out=$(APPLYNUDGE_RUN "$ROOT/proposals/nudge-prop-001.md" 2>&1 || true)
|
||||||
|
an_status=$(echo "$an_out" | node -e 'let b="";process.stdin.on("data",d=>b+=d).on("end",()=>{try{console.log(JSON.parse(b).status)}catch{console.log("parse-error")}})')
|
||||||
|
an_cue=$(node -e 'const j=require("'"$ROOT"'/active-nudges.json");console.log((j[0]&&j[0].cue&&j[0].cue.cwd)||"")')
|
||||||
|
an_src=$(node -e 'const j=require("'"$ROOT"'/active-nudges.json");console.log((j[0]||{}).source_session||"")')
|
||||||
|
an_msg=$(node -e 'const j=require("'"$ROOT"'/active-nudges.json");console.log((j[0]||{}).message||"")')
|
||||||
|
cat > "$ROOT/proposals/nudge-prop-002.md" <<'EOF'
|
||||||
|
---
|
||||||
|
type: memory
|
||||||
|
source_session: sX
|
||||||
|
---
|
||||||
|
EOF
|
||||||
|
gate_status=$(APPLYNUDGE_RUN "$ROOT/proposals/nudge-prop-002.md" 2>&1 | node -e 'let b="";process.stdin.on("data",d=>b+=d).on("end",()=>{try{console.log(JSON.parse(b).status)}catch{console.log("parse-error")}})')
|
||||||
|
if [ "$an_status" = "applied" ] && [ "$an_cue" = "/Users/me/proj-alpha" ] && [ "$an_src" = "sNUDGE" ] \
|
||||||
|
&& echo "$an_msg" | grep -q "4 dead-ends in proj-alpha" && [ "$gate_status" = "gated" ]; then
|
||||||
|
echo " PASS: applied with cue.cwd + generated message; non-nudge proposal gated"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: status=$an_status cue=$an_cue src=$an_src msg=$an_msg gate=$gate_status"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/proposals/nudge-prop-00"*.md "$ROOT/active-nudges.json"
|
||||||
|
|
||||||
|
# --- Test 123: mid-session nudge reactivation on UserPromptSubmit (cue.cwd gated) ---
|
||||||
|
echo "Test 123: adam-observe surfaces a cue-matched nudge mid-session — once/session, cwd-gated, opt-out"
|
||||||
|
ms_future=$(node -e 'console.log(Date.now() + 7*86400000)')
|
||||||
|
ms_nudge() { cat > "$ROOT/active-nudges.json" <<EOF
|
||||||
|
[{"kind":"dead_end_reminder","message":"adam: midsession checkpoint alpha","created_at":111,"expires_at_ts":$ms_future,"max_displays":3,"displays_used":0,"source_session":"sOLD","cue":{"cwd":"/proj/x"}}]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
reset_state; ms_nudge
|
||||||
|
ms1=$(echo '{"hook_event_name":"UserPromptSubmit","session_id":"sMS","cwd":"/proj/x","prompt":"hello"}' | HOOK_RUN 2>&1 || true)
|
||||||
|
ms2=$(echo '{"hook_event_name":"UserPromptSubmit","session_id":"sMS","cwd":"/proj/x","prompt":"again"}' | HOOK_RUN 2>&1 || true)
|
||||||
|
reset_state; ms_nudge
|
||||||
|
ms_miss=$(echo '{"hook_event_name":"UserPromptSubmit","session_id":"sMS2","cwd":"/proj/y","prompt":"hello"}' | HOOK_RUN 2>&1 || true)
|
||||||
|
reset_state; ms_nudge
|
||||||
|
ms_off=$(echo '{"hook_event_name":"UserPromptSubmit","session_id":"sMS3","cwd":"/proj/x","prompt":"hello"}' | HOME="$TMP_HOME" ADAM_NO_MIDSESSION_NUDGE=1 node "$HOOK" 2>&1 || true)
|
||||||
|
if echo "$ms1" | grep -q "midsession checkpoint alpha" \
|
||||||
|
&& ! echo "$ms2" | grep -q "midsession checkpoint alpha" \
|
||||||
|
&& ! echo "$ms_miss" | grep -q "midsession checkpoint alpha" \
|
||||||
|
&& ! echo "$ms_off" | grep -q "midsession checkpoint alpha"; then
|
||||||
|
echo " PASS: surfaced once on cwd match; deduped 2nd prompt; silent on cwd mismatch + opt-out"; PASS=$((PASS+1))
|
||||||
|
else
|
||||||
|
echo " FAIL: ms1=$ms1 || ms2=$ms2 || miss=$ms_miss || off=$ms_off"; FAIL=$((FAIL+1))
|
||||||
|
fi
|
||||||
|
rm -f "$ROOT/active-nudges.json"
|
||||||
|
|
||||||
echo
|
echo
|
||||||
echo "Results: $PASS passed, $FAIL failed"
|
echo "Results: $PASS passed, $FAIL failed"
|
||||||
[ "$FAIL" = "0" ]
|
[ "$FAIL" = "0" ]
|
||||||
|
|||||||
+32
-21
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: adam
|
name: adam
|
||||||
description: Self-improvement analyst. Reads adam journal + transcript context, clusters observations, scores against a deterministic rubric, and emits proposal files for new skills, memory entries, agent edits, hook changes, CLAUDE.md edits, and soft deletions. Invoked only via the adam-self-improvement skill.
|
description: "Self-improvement analyst. Reads the adam journal and transcript context, clusters observations, scores them against a deterministic rubric, and writes proposal files for new skills, memory entries, agent edits, hook changes, CLAUDE.md edits and soft deletions. Invoked only via the adam-self-improvement skill (/reflect). Not for: editing skills, agents or memory directly (the main thread applies approved proposals), or reviewing code (use code-reviewer)."
|
||||||
tools: Read, Write, Edit, Grep, Glob, Bash
|
tools: Read, Write, Edit, Grep, Glob, Bash
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -17,6 +17,8 @@ The skill dispatches you in one of two stages (MOSS-inspired multi-stage pipelin
|
|||||||
|
|
||||||
If no `stage` is specified in the dispatch prompt, run **both stages sequentially** within a single pass (backward-compatible with pre-MOSS flow).
|
If no `stage` is specified in the dispatch prompt, run **both stages sequentially** within a single pass (backward-compatible with pre-MOSS flow).
|
||||||
|
|
||||||
|
**Adaptive re-diagnose** (ReContext P5 — depth is task-dependent, not uniformly larger): a `stage=diagnose` dispatch may carry `redigest: <cluster_id[,…]>` and `prior_diagnoses_path`. When it does, re-diagnose ONLY those clusters: re-read each cluster's `context_window` (and up to ~30 extra transcript messages) *conditioned on the prior diagnosis*, and overwrite the trigger/action/mismatch/outcome for those `cluster_id`s in `/tmp/adam-diagnoses.json`. This is how an ambiguous cluster earns a second, deeper pass while clear clusters stay at one. Never re-diagnose a cluster more than once per `/reflect`.
|
||||||
|
|
||||||
### Diagnose-stage output format
|
### Diagnose-stage output format
|
||||||
|
|
||||||
When `stage=diagnose`, write `/tmp/adam-diagnoses.json` containing:
|
When `stage=diagnose`, write `/tmp/adam-diagnoses.json` containing:
|
||||||
@@ -161,12 +163,14 @@ The hook emits these `type` values into the journal:
|
|||||||
- `correction_free_streak`: cluster by `active_skills[0]`. Treat ≥3 streaks across ≥2 sessions naming the same skill as cross-session evidence.
|
- `correction_free_streak`: cluster by `active_skills[0]`. Treat ≥3 streaks across ≥2 sessions naming the same skill as cross-session evidence.
|
||||||
- `clean_recovery`: cluster by (`recovered_from`, `active_skills[0]`). A win cluster qualifies for `skill_edit` only when the named skill exists in `skills_root`.
|
- `clean_recovery`: cluster by (`recovered_from`, `active_skills[0]`). A win cluster qualifies for `skill_edit` only when the named skill exists in `skills_root`.
|
||||||
- `task_completed`: cluster by sorted `tool_kinds` tuple (the multi-tool recipe). Single entry qualifies for `skill_new` proposal (drafting protocol applies). Cross-session evidence requires ≥2 entries from distinct sessions with same tuple — without it, proposal queues, never auto-applies. Run the existing skill-overlap rule before drafting: if the recipe matches an existing skill's name/description tokens, route to `skill_edit` instead.
|
- `task_completed`: cluster by sorted `tool_kinds` tuple (the multi-tool recipe). Single entry qualifies for `skill_new` proposal (drafting protocol applies). Cross-session evidence requires ≥2 entries from distinct sessions with same tuple — without it, proposal queues, never auto-applies. Run the existing skill-overlap rule before drafting: if the recipe matches an existing skill's name/description tokens, route to `skill_edit` instead.
|
||||||
|
4b. **Cross-type associative pass** (ReContext P5 — surface evidence related to already-selected spans): after primary clustering, for each cluster gather active observations from *other* signal types that share the cluster's `session` or file basename (e.g. a `dead_end` and the `edit_churn` on the same file/session that led to it). Attach them to the cluster as `associated_entries` — do NOT merge cluster keys or re-count them for the rubric — so the diagnosis can reconstruct causation across signal types instead of from one isolated type. Cap at ~8 associated entries per cluster, nearest-in-time first.
|
||||||
5. **Multi-axis correlation**: for each session that produced ≥2 distinct struggle types (`tool_error_loop`, `dead_end`, `weak_agent`, `retry_loop`, `edit_churn`, `build_loop`, `silent_drift`, `file_reread`, `error_after_recovery`), tag clusters from that session as `multi_axis: true`. This grants +1 confidence at scoring.
|
5. **Multi-axis correlation**: for each session that produced ≥2 distinct struggle types (`tool_error_loop`, `dead_end`, `weak_agent`, `retry_loop`, `edit_churn`, `build_loop`, `silent_drift`, `file_reread`, `error_after_recovery`), tag clusters from that session as `multi_axis: true`. This grants +1 confidence at scoring.
|
||||||
|
|
||||||
5b. **Skill-attribution sub-clustering**: after primary clustering (step 4), for every struggle cluster (`tool_error_loop`, `dead_end`, `weak_agent`, `retry_loop`, `edit_churn`, `build_loop`, `silent_drift`, `file_reread`, `error_after_recovery`) that contains entries with non-empty `active_skills[0]`:
|
5b. **Skill-attribution sub-clustering**: after primary clustering (step 4), for every struggle cluster (`tool_error_loop`, `dead_end`, `weak_agent`, `retry_loop`, `edit_churn`, `build_loop`, `silent_drift`, `file_reread`, `error_after_recovery`) that contains entries with non-empty `active_skills[0]`:
|
||||||
- Split into per-skill sub-clusters keyed on `active_skills[0]`. Entries with empty `active_skills` stay in the original cluster.
|
- Split into per-skill sub-clusters keyed on `active_skills[0]`. Entries with empty `active_skills` stay in the original cluster.
|
||||||
- If a sub-cluster has ≥3 entries AND names a skill that exists in `skills_root`, mark it as a candidate for `skill_edit` (struggle-driven variant; see "Struggle-driven `skill_edit` eligibility"). Otherwise treat the parent cluster normally.
|
- If a sub-cluster has ≥3 entries AND names a skill that exists in `skills_root`, mark it as a candidate for `skill_edit` (struggle-driven variant; see "Struggle-driven `skill_edit` eligibility"). Otherwise treat the parent cluster normally.
|
||||||
- The umbrella cluster (cross-skill) still emits its usual proposal type (memory, etc.) — sub-clusters do NOT replace it, they supplement it.
|
- The umbrella cluster (cross-skill) still emits its usual proposal type (memory, etc.) — sub-clusters do NOT replace it, they supplement it.
|
||||||
|
- **Utility cross-check**: for every `skill_edit`/`skill_new` candidate, look up the named skill in `skill_utility_path` (the pre-computed `adam-skill-utility.mjs --json` report). If the skill is reliably below baseline (negative `lift` + worst-tier `wLB` at ≥ its `--min` events), apply the utility-lift penalty (see rubric) — do NOT pile more text onto a skill that already correlates with friction; prefer a `memory` or a scoped new skill instead.
|
||||||
6. For each cluster qualifying under the rubric — ≥3 occurrences across ≥2 sessions, OR (for struggle types) ≥1 entry within a single session, OR (for `correction`) ≥3 occurrences across ≥2 cwds:
|
6. For each cluster qualifying under the rubric — ≥3 occurrences across ≥2 sessions, OR (for struggle types) ≥1 entry within a single session, OR (for `correction`) ≥3 occurrences across ≥2 cwds:
|
||||||
a. If cluster topic matches a rejected idea via the rejected-ideas fuzzy set (≥2 token overlap with rejection's `# Why`), skip with reason `"rejected-similar"`.
|
a. If cluster topic matches a rejected idea via the rejected-ideas fuzzy set (≥2 token overlap with rejection's `# Why`), skip with reason `"rejected-similar"`.
|
||||||
b. Pull ~20 messages of transcript context from `transcripts_root` to enrich. Never read full transcripts.
|
b. Pull ~20 messages of transcript context from `transcripts_root` to enrich. Never read full transcripts.
|
||||||
@@ -250,8 +254,10 @@ Required structure:
|
|||||||
|
|
||||||
```markdown
|
```markdown
|
||||||
---
|
---
|
||||||
name: <human-readable name, ≤80 chars>
|
name: <slug — snake_case, MUST equal the target filename without `.md`, e.g. feedback_go_test_cache>
|
||||||
description: <one-line description used to decide future relevance — be specific, ≤200 chars>
|
description: "<one-line used to decide future relevance — be specific, ≤200 chars>"
|
||||||
|
metadata:
|
||||||
|
node_type: memory
|
||||||
type: user | feedback | project | reference
|
type: user | feedback | project | reference
|
||||||
originSessionId: <session_id from journal entries that fed this cluster>
|
originSessionId: <session_id from journal entries that fed this cluster>
|
||||||
---
|
---
|
||||||
@@ -263,12 +269,17 @@ originSessionId: <session_id from journal entries that fed this cluster>
|
|||||||
- reference: pointer to external system + what's there.>
|
- reference: pointer to external system + what's there.>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The frontmatter MUST match the live auto-memory schema exactly: `name` is the
|
||||||
|
slug (NOT a prose title), and `node_type`, `type`, `originSessionId` live under
|
||||||
|
a `metadata:` block (verify against an existing file in the target memory dir
|
||||||
|
before drafting — match its shape).
|
||||||
|
|
||||||
Constraints:
|
Constraints:
|
||||||
- Frontmatter fields `name`, `description`, `type` are **required**. Skill enforces this at apply time.
|
- Top-level `name` + `description` and nested `metadata.node_type` (always `memory`) + `metadata.type` are **required**. Skill enforces this at apply time.
|
||||||
- `originSessionId` is required — must be a `session` value from one of the cluster's journal entries.
|
- `metadata.originSessionId` is required — must be a `session` value from one of the cluster's journal entries.
|
||||||
- ≤50 LOC of body content. Surgical.
|
- ≤50 LOC of body content. Surgical.
|
||||||
- Slug (used in `target` path filename) must not collide with any existing memory file.
|
- `name`/slug (also the `target` path filename) must not collide with any existing memory file.
|
||||||
- For `type=feedback` and `type=project`, body MUST contain `**Why:**` and `**How to apply:**` lines (CLAUDE.md memory schema).
|
- For `type: feedback` and `type: project`, body MUST contain `**Why:**` and `**How to apply:**` lines (CLAUDE.md memory schema).
|
||||||
|
|
||||||
## Diagnosis drafting protocol (required for every proposal)
|
## Diagnosis drafting protocol (required for every proposal)
|
||||||
|
|
||||||
@@ -384,7 +395,7 @@ The skill (`adam-self-improvement/SKILL.md` §1) runs `adam-score.mjs` immediate
|
|||||||
|
|
||||||
## A/B effectiveness
|
## A/B effectiveness
|
||||||
|
|
||||||
Every auto-applied edit (`skill_edit`, `skill_new`, `memory`, `nudge`) gets a one-line tracking entry written to `~/.claude/adam/ab-tracking.jsonl` by `adam-self-improvement/SKILL.md` immediately after the proposal is moved to `applied/`. **`reinforcement` is the one exception — it is a positive-only ledger and is intentionally NOT A/B-tracked (see §"`reinforcement` proposals"), to avoid skewing regression detection.** Schema:
|
Every auto-applied edit (`skill_edit`, `skill_new`, `memory`, `nudge`) gets a one-line tracking entry written to `~/.claude/adam/ab-tracking.jsonl` by `adam-self-improvement/apply-by-type.md` (loaded from SKILL.md) immediately after the proposal is moved to `applied/`. **`reinforcement` is the one exception — it is a positive-only ledger and is intentionally NOT A/B-tracked (see §"`reinforcement` proposals"), to avoid skewing regression detection.** Schema:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{"applied_at":<ms>,"proposal_id":"<id>","proposal_type":"...","target_skill":"<slug>","proposal_fingerprint":"<hash>","originating_signals":[{"type":"<signal>","count":<N>,"session_ids":[...]}],"pre_window_days":7}
|
{"applied_at":<ms>,"proposal_id":"<id>","proposal_type":"...","target_skill":"<slug>","proposal_fingerprint":"<hash>","originating_signals":[{"type":"<signal>","count":<N>,"session_ids":[...]}],"pre_window_days":7}
|
||||||
@@ -435,6 +446,7 @@ Sum:
|
|||||||
- Cluster severity-sum ≥ 32: **+1** *(additive — a severity-sum of 32 gets +1 from the previous bullet AND +1 here, total +2.)*
|
- Cluster severity-sum ≥ 32: **+1** *(additive — a severity-sum of 32 gets +1 from the previous bullet AND +1 here, total +2.)*
|
||||||
- Skill-attributed sub-cluster (≥3 entries naming the same `active_skills[0]` that exists in `skills_root`): **+1**
|
- Skill-attributed sub-cluster (≥3 entries naming the same `active_skills[0]` that exists in `skills_root`): **+1**
|
||||||
- Type-bias penalty from feedback loop (≥3 rejections, applied:rejected ratio <1:2 for this `type`): **-1**
|
- Type-bias penalty from feedback loop (≥3 rejections, applied:rejected ratio <1:2 for this `type`): **-1**
|
||||||
|
- Utility-lift penalty: proposal is a `skill_edit` (or a `skill_new` whose recipe duplicates an existing skill's scope) whose target skill is ranked reliably below baseline in `skill_utility_path` — negative `lift` AND worst-tier `wLB`, with event count ≥ its `--min`: **-2** *(adding text to a skill that already correlates with friction raises access while likely lowering utilization — SkillsInjector, cited in `adam-skill-utility.mjs`. Read the skill's row as a review prompt, not proof of causation.)*
|
||||||
- Diagnosis flags `Mismatch: unclear` (causation could not be reconstructed from transcript context): **-1**
|
- Diagnosis flags `Mismatch: unclear` (causation could not be reconstructed from transcript context): **-1**
|
||||||
- Blast radius: low **+1**, medium **0**, high **-1** (default per type — see Proposal types table)
|
- Blast radius: low **+1**, medium **0**, high **-1** (default per type — see Proposal types table)
|
||||||
- Surgical (one file, ≤50 LOC for non-skill_new; ≤80 LOC for skill_new): **+1**
|
- Surgical (one file, ≤50 LOC for non-skill_new; ≤80 LOC for skill_new): **+1**
|
||||||
@@ -464,24 +476,23 @@ Sum:
|
|||||||
|
|
||||||
### `nudge` proposals
|
### `nudge` proposals
|
||||||
|
|
||||||
A `nudge` proposal does NOT modify any persistent rubric/skill/memory artifact. Its sole side-effect is to append an entry to `~/.claude/adam/active-nudges.json` so the next SessionStart hook surfaces a one-line reminder to the user in a *different* session.
|
A `nudge` proposal does NOT modify any persistent rubric/skill/memory artifact. Its sole side-effect is to append an entry to `~/.claude/adam/active-nudges.json` so a future session surfaces a one-line checkpoint reminder — at SessionStart (`adam-nudge.mjs`) and mid-session on the matching cwd (`adam-observe.mjs` on UserPromptSubmit).
|
||||||
|
|
||||||
Trigger: `adam-nudge-eligibility.mjs --session <id>` returns `eligible: true` (i.e. ≥3 `dead_end` entries inside a single session). Distinguished from `skill_edit` precisely because there is no learning artifact to mutate — the action surfaces a checkpoint reminder, not a behavior change.
|
Trigger: `adam-nudge-eligibility.mjs --session <id>` returns `eligible: true` (i.e. ≥3 `dead_end` entries inside a single session). Distinguished from `skill_edit` precisely because there is no learning artifact to mutate — the action surfaces a checkpoint reminder, not a behavior change.
|
||||||
|
|
||||||
`active-nudges.json` entry shape (created by the skill at apply time):
|
Required proposal frontmatter (the entry is built deterministically by `adam-apply-nudge.mjs` — do NOT hand-author the JSON message/cue):
|
||||||
|
|
||||||
```json
|
```yaml
|
||||||
{
|
type: nudge
|
||||||
"kind": "dead_end_reminder",
|
source_session: <originating session_id>
|
||||||
"message": "adam: previous session hit 3 dead_ends — consider a checkpoint before continuing.",
|
cwd: <cluster's dominant cwd — becomes cue.cwd; adam-batch.mjs aggregates `cwds` per batch>
|
||||||
"created_at": <ms>,
|
dead_end_count: <N ≥ 3 from adam-nudge-eligibility.mjs>
|
||||||
"expires_at_ts": <ms now + 7 days>,
|
confidence: <int>
|
||||||
"max_displays": 3,
|
blast_radius: low
|
||||||
"displays_used": 0,
|
|
||||||
"source_session": "<originating session_id>"
|
|
||||||
}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The helper writes `{kind, message, created_at, expires_at_ts: now+7d, max_displays: 3, displays_used: 0, source_session, cue: {cwd}}`. The `cue.cwd` binds the reminder to the project it came from: both surfaces (SessionStart + mid-session) fire the nudge only when the session's cwd matches, deferring it elsewhere without consuming a display (ReContext-style query-conditioned replay). Omit `cwd` from frontmatter only for a deliberately project-agnostic reminder (then the nudge has no cue and surfaces unconditionally at SessionStart, as legacy).
|
||||||
|
|
||||||
### `reinforcement` proposals
|
### `reinforcement` proposals
|
||||||
|
|
||||||
A `reinforcement` proposal is logged when `adam-score.mjs` reports `count >= 3` clean `task_completed` events citing the same `active_skills[0]` slug. Frontmatter MUST include `skill_slug`, `count`, `source_session`, `confidence`, `blast_radius: low`. Apply gate (`confidence >= 4 AND blast_radius == low`) is identical to the `memory` gate — when both hold, the skill invokes `~/.claude/adam/scripts/adam-apply-reinforcement.mjs <proposal-path>` which appends one JSON line to `~/.claude/adam/reinforcements.jsonl` of shape `{ts, skill_slug, count, source_session}`. No code/memory/skill modifications either side of the gate — reinforcements are a positive-only ledger, separate from `ab-tracking.jsonl` (A/B intentionally does NOT measure positive signals to avoid skewing regression detection).
|
A `reinforcement` proposal is logged when `adam-score.mjs` reports `count >= 3` clean `task_completed` events citing the same `active_skills[0]` slug. Frontmatter MUST include `skill_slug`, `count`, `source_session`, `confidence`, `blast_radius: low`. Apply gate (`confidence >= 4 AND blast_radius == low`) is identical to the `memory` gate — when both hold, the skill invokes `~/.claude/adam/scripts/adam-apply-reinforcement.mjs <proposal-path>` which appends one JSON line to `~/.claude/adam/reinforcements.jsonl` of shape `{ts, skill_slug, count, source_session}`. No code/memory/skill modifications either side of the gate — reinforcements are a positive-only ledger, separate from `ab-tracking.jsonl` (A/B intentionally does NOT measure positive signals to avoid skewing regression detection).
|
||||||
@@ -509,7 +520,7 @@ MOSS's core thesis: "routing, hook ordering, state invariants, and dispatch live
|
|||||||
2. `cross_session_evidence == true` (≥5 occurrences across ≥3 sessions)
|
2. `cross_session_evidence == true` (≥5 occurrences across ≥3 sessions)
|
||||||
3. `auto_apply_eligible: false` — **always**. Harness edits are never auto-applied.
|
3. `auto_apply_eligible: false` — **always**. Harness edits are never auto-applied.
|
||||||
4. `blast_radius: high`
|
4. `blast_radius: high`
|
||||||
5. Proposal includes a `# Test verification` section with the command `bash ~/.claude/adam/tests/run-tests.sh` and the expected result "132 passed, 0 failed" (or current pass count). The skill runs this test before applying.
|
5. Proposal includes a `# Test verification` section with the command `bash ~/.claude/adam/tests/run-tests.sh` and the expected result "140 passed, 0 failed" (or current pass count). The skill runs this test before applying.
|
||||||
6. Change is surgical: ≤30 LOC diff, single file.
|
6. Change is surgical: ≤30 LOC diff, single file.
|
||||||
7. `# Diagnosis` reconstructs the causal chain from harness-level behavior (not from text-artifact behavior). The mismatch must name a specific code path (function, regex, threshold) in the target file.
|
7. `# Diagnosis` reconstructs the causal chain from harness-level behavior (not from text-artifact behavior). The mismatch must name a specific code path (function, regex, threshold) in the target file.
|
||||||
|
|
||||||
|
|||||||
+110
-11
@@ -1,9 +1,19 @@
|
|||||||
#!/usr/bin/env node
|
#!/usr/bin/env node
|
||||||
// adam-nudge.mjs — SessionStart hook. Prints two kinds of reminders:
|
// adam-nudge.mjs — SessionStart hook. Prints reminders:
|
||||||
// 1. Pending proposals (≥3 queued in adam/proposals/).
|
// 1. Pending proposals (≥3 queued in adam/proposals/).
|
||||||
// 2. Cross-session nudges (entries in adam/active-nudges.json whose
|
// 2. Cross-session nudges (entries in adam/active-nudges.json whose
|
||||||
// source_session differs from the current session and that haven't
|
// source_session differs from the current session and that haven't
|
||||||
// expired or exhausted their max_displays).
|
// expired or exhausted their max_displays). An entry may carry an optional
|
||||||
|
// `cue.cwd`; when set, it only surfaces in a session whose cwd matches
|
||||||
|
// (otherwise it is deferred, not consumed).
|
||||||
|
// 3. Pending local-edit upgrades (`.adam-new` sidecars).
|
||||||
|
// 4. New-release notice: if a newer GitHub release exists than the installed
|
||||||
|
// `.version`, print a notify-only one-line update prompt. Cached + checked
|
||||||
|
// at most once/day, network call hard-capped at 1.5s, fully best-effort —
|
||||||
|
// never blocks SessionStart. Opt out with ADAM_NO_UPDATE_CHECK=1.
|
||||||
|
// NOTE: notify-only by design — applying an update re-runs install.sh,
|
||||||
|
// which resets ADAM's own /reflect-applied skill edits. The user chooses
|
||||||
|
// when to accept that, so we never auto-install.
|
||||||
import { readdirSync, readFileSync, writeFileSync, existsSync } from "node:fs";
|
import { readdirSync, readFileSync, writeFileSync, existsSync } from "node:fs";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { homedir } from "node:os";
|
import { homedir } from "node:os";
|
||||||
@@ -14,7 +24,13 @@ const ADAM_ROOT = join(CLAUDE_ROOT, "adam");
|
|||||||
const PROPOSALS = join(ADAM_ROOT, "proposals");
|
const PROPOSALS = join(ADAM_ROOT, "proposals");
|
||||||
const NUDGES_FILE = join(ADAM_ROOT, "active-nudges.json");
|
const NUDGES_FILE = join(ADAM_ROOT, "active-nudges.json");
|
||||||
const STATE_FILE = join(ADAM_ROOT, "state.json");
|
const STATE_FILE = join(ADAM_ROOT, "state.json");
|
||||||
|
const VERSION_FILE = join(ADAM_ROOT, ".version");
|
||||||
|
const UPDATE_CHECK_FILE = join(ADAM_ROOT, ".update-check.json");
|
||||||
const THRESHOLD = 3;
|
const THRESHOLD = 3;
|
||||||
|
const UPDATE_CHECK_INTERVAL_MS = 24 * 60 * 60 * 1000;
|
||||||
|
const UPDATE_FETCH_TIMEOUT_MS = 1500;
|
||||||
|
const RELEASES_API = "https://api.github.com/repos/lukaszraczylo/claude-adam/releases/latest";
|
||||||
|
const INSTALL_ONELINER = "curl -fsSL https://raw.githubusercontent.com/lukaszraczylo/claude-adam/main/install.sh | bash";
|
||||||
|
|
||||||
// Known installable paths (mirrors install.sh copy_file list). Checking a
|
// Known installable paths (mirrors install.sh copy_file list). Checking a
|
||||||
// fixed shortlist keeps SessionStart latency under control vs full FS walk.
|
// fixed shortlist keeps SessionStart latency under control vs full FS walk.
|
||||||
@@ -23,6 +39,7 @@ const PENDING_CHECK_PATHS = [
|
|||||||
"hooks/adam-nudge.mjs",
|
"hooks/adam-nudge.mjs",
|
||||||
"agents/adam.md",
|
"agents/adam.md",
|
||||||
"skills/adam-self-improvement/SKILL.md",
|
"skills/adam-self-improvement/SKILL.md",
|
||||||
|
"skills/adam-self-improvement/apply-by-type.md",
|
||||||
"commands/reflect.md",
|
"commands/reflect.md",
|
||||||
"adam/scripts/adam-archive.mjs",
|
"adam/scripts/adam-archive.mjs",
|
||||||
"adam/scripts/adam-upgrade.mjs",
|
"adam/scripts/adam-upgrade.mjs",
|
||||||
@@ -32,7 +49,9 @@ const PENDING_CHECK_PATHS = [
|
|||||||
"adam/scripts/adam-cooldown.mjs",
|
"adam/scripts/adam-cooldown.mjs",
|
||||||
"adam/scripts/adam-score.mjs",
|
"adam/scripts/adam-score.mjs",
|
||||||
"adam/scripts/adam-ab-measure.mjs",
|
"adam/scripts/adam-ab-measure.mjs",
|
||||||
|
"adam/scripts/adam-skill-utility.mjs",
|
||||||
"adam/scripts/adam-apply-reinforcement.mjs",
|
"adam/scripts/adam-apply-reinforcement.mjs",
|
||||||
|
"adam/scripts/adam-apply-nudge.mjs",
|
||||||
"adam/scripts/adam-utils.mjs",
|
"adam/scripts/adam-utils.mjs",
|
||||||
"adam/scripts/adam-batch.mjs",
|
"adam/scripts/adam-batch.mjs",
|
||||||
"adam/scripts/adam-rollback.mjs",
|
"adam/scripts/adam-rollback.mjs",
|
||||||
@@ -45,13 +64,13 @@ function readJson(path, fallback) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function readSessionInput() {
|
function readSessionInput() {
|
||||||
// SessionStart payload arrives on stdin; capture session_id if present.
|
// SessionStart payload arrives on stdin; return the parsed object so callers
|
||||||
// We don't block on stdin — best-effort, non-blocking.
|
// can read both session_id and cwd. We don't block on stdin — best-effort.
|
||||||
try {
|
try {
|
||||||
const buf = readFileSync(0, "utf8");
|
const buf = readFileSync(0, "utf8");
|
||||||
if (!buf) return null;
|
if (!buf) return null;
|
||||||
const parsed = JSON.parse(buf);
|
const parsed = JSON.parse(buf);
|
||||||
return parsed && typeof parsed.session_id === "string" ? parsed.session_id : null;
|
return parsed && typeof parsed === "object" ? parsed : null;
|
||||||
} catch { return null; }
|
} catch { return null; }
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -65,7 +84,7 @@ function emitProposalReminder() {
|
|||||||
} catch { /* proposals dir absent → silent */ }
|
} catch { /* proposals dir absent → silent */ }
|
||||||
}
|
}
|
||||||
|
|
||||||
function emitActiveNudges(currentSession) {
|
function emitActiveNudges(currentSession, currentCwd) {
|
||||||
if (!existsSync(NUDGES_FILE)) return;
|
if (!existsSync(NUDGES_FILE)) return;
|
||||||
const raw = readJson(NUDGES_FILE, null);
|
const raw = readJson(NUDGES_FILE, null);
|
||||||
if (!Array.isArray(raw)) return;
|
if (!Array.isArray(raw)) return;
|
||||||
@@ -85,6 +104,16 @@ function emitActiveNudges(currentSession) {
|
|||||||
kept.push(entry);
|
kept.push(entry);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
// Cue gate (optional, additive): a nudge may carry `cue.cwd` binding it to
|
||||||
|
// the project it originated in. When set and the current session's cwd does
|
||||||
|
// not match, defer it — keep the entry, do NOT consume a display — instead of
|
||||||
|
// surfacing an irrelevant reminder. Nudges without a cue print unconditionally,
|
||||||
|
// exactly as before (ReContext P5/P7: replay should be conditioned on the cue).
|
||||||
|
if (entry.cue && typeof entry.cue === "object" && entry.cue.cwd &&
|
||||||
|
currentCwd && entry.cue.cwd !== currentCwd) {
|
||||||
|
kept.push(entry);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
if (typeof entry.message === "string" && entry.message) {
|
if (typeof entry.message === "string" && entry.message) {
|
||||||
process.stdout.write(entry.message + "\n");
|
process.stdout.write(entry.message + "\n");
|
||||||
const nextUsed = used + 1;
|
const nextUsed = used + 1;
|
||||||
@@ -118,17 +147,87 @@ function emitPendingUpgrades() {
|
|||||||
} catch { /* never break SessionStart */ }
|
} catch { /* never break SessionStart */ }
|
||||||
}
|
}
|
||||||
|
|
||||||
function main() {
|
// --- update notifier (notify-only; see header note) ---
|
||||||
const stdinSession = readSessionInput();
|
|
||||||
|
function readVersion() {
|
||||||
|
try { return readFileSync(VERSION_FILE, "utf8").trim() || null; } catch { return null; }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse "vX.Y.Z" (leading v optional; pre-release/build suffix ignored).
|
||||||
|
function parseSemver(s) {
|
||||||
|
if (typeof s !== "string") return null;
|
||||||
|
const m = s.trim().replace(/^v/i, "").match(/^(\d+)\.(\d+)\.(\d+)/);
|
||||||
|
return m ? [Number(m[1]), Number(m[2]), Number(m[3])] : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// isNewer(a, b): true iff version a is strictly newer than b. Unparseable → false.
|
||||||
|
function isNewer(a, b) {
|
||||||
|
const pa = parseSemver(a), pb = parseSemver(b);
|
||||||
|
if (!pa || !pb) return false;
|
||||||
|
for (let i = 0; i < 3; i++) { if (pa[i] !== pb[i]) return pa[i] > pb[i]; }
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchLatestTag() {
|
||||||
|
// Best-effort, hard-capped. Any failure (offline / timeout / rate-limit /
|
||||||
|
// parse / fetch-unavailable) returns null and the caller silently skips.
|
||||||
|
try {
|
||||||
|
if (typeof fetch !== "function") return null;
|
||||||
|
const ctrl = new AbortController();
|
||||||
|
const timer = setTimeout(() => ctrl.abort(), UPDATE_FETCH_TIMEOUT_MS);
|
||||||
|
let tag = null;
|
||||||
|
try {
|
||||||
|
const res = await fetch(RELEASES_API, {
|
||||||
|
signal: ctrl.signal,
|
||||||
|
headers: { "User-Agent": "claude-adam-nudge", "Accept": "application/vnd.github+json" },
|
||||||
|
});
|
||||||
|
if (res && res.ok) {
|
||||||
|
const j = await res.json();
|
||||||
|
if (j && typeof j.tag_name === "string") tag = j.tag_name;
|
||||||
|
}
|
||||||
|
} finally { clearTimeout(timer); }
|
||||||
|
return tag;
|
||||||
|
} catch { return null; }
|
||||||
|
}
|
||||||
|
|
||||||
|
function printUpdateNudge(latest, installed) {
|
||||||
|
process.stdout.write(
|
||||||
|
`[adam] update available: ${installed} → ${latest}. Apply: ${INSTALL_ONELINER}\n` +
|
||||||
|
` (re-runs install.sh — resets ADAM's own /reflect-applied skill edits; apply when you're ready)\n`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function emitUpdateCheck() {
|
||||||
|
if (process.env.ADAM_NO_UPDATE_CHECK) return; // explicit opt-out
|
||||||
|
const installed = readVersion();
|
||||||
|
if (!installed) return; // no marker → nothing to compare
|
||||||
|
const cache = readJson(UPDATE_CHECK_FILE, {}) || {};
|
||||||
|
const now = Date.now();
|
||||||
|
let nudged = false;
|
||||||
|
// Instant nudge from cache (no network).
|
||||||
|
if (cache.latest && isNewer(cache.latest, installed)) { printUpdateNudge(cache.latest, installed); nudged = true; }
|
||||||
|
// Refresh cache at most once/day, best-effort — drives the nudge on the NEXT run.
|
||||||
|
if (!cache.last_check || (now - Number(cache.last_check)) > UPDATE_CHECK_INTERVAL_MS) {
|
||||||
|
const latest = await fetchLatestTag();
|
||||||
|
const next = { last_check: now, latest: latest || cache.latest || null };
|
||||||
|
try { writeFileSync(UPDATE_CHECK_FILE, JSON.stringify(next)); } catch { /* swallow */ }
|
||||||
|
if (latest && !nudged && isNewer(latest, installed)) printUpdateNudge(latest, installed);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const payload = readSessionInput();
|
||||||
|
const stdinSession = payload && typeof payload.session_id === "string" ? payload.session_id : null;
|
||||||
const stateSession = (() => {
|
const stateSession = (() => {
|
||||||
const st = readJson(STATE_FILE, null);
|
const st = readJson(STATE_FILE, null);
|
||||||
return st && typeof st.session_id === "string" ? st.session_id : null;
|
return st && typeof st.session_id === "string" ? st.session_id : null;
|
||||||
})();
|
})();
|
||||||
const currentSession = stdinSession || stateSession || "";
|
const currentSession = stdinSession || stateSession || "";
|
||||||
|
const currentCwd = (payload && typeof payload.cwd === "string" && payload.cwd) || process.cwd();
|
||||||
emitProposalReminder();
|
emitProposalReminder();
|
||||||
emitActiveNudges(currentSession);
|
emitActiveNudges(currentSession, currentCwd);
|
||||||
emitPendingUpgrades();
|
emitPendingUpgrades();
|
||||||
|
await emitUpdateCheck();
|
||||||
}
|
}
|
||||||
|
|
||||||
try { main(); } catch { /* never block SessionStart */ }
|
main().catch(() => { /* never block SessionStart */ }).finally(() => process.exit(0));
|
||||||
process.exit(0);
|
|
||||||
|
|||||||
+50
-5
@@ -249,6 +249,12 @@ function snapshotContext(state) {
|
|||||||
|
|
||||||
function errorFingerprint(toolResponse) {
|
function errorFingerprint(toolResponse) {
|
||||||
if (!toolResponse) return null;
|
if (!toolResponse) return null;
|
||||||
|
// Read/Edit/Write success objects carry `file`/`filePath` and no `is_error`.
|
||||||
|
// PostToolUse never receives failed calls (CC routes them to
|
||||||
|
// PostToolUseFailure), so ERROR_RE here would only match source-file text.
|
||||||
|
if (typeof toolResponse === "object" && !Array.isArray(toolResponse) &&
|
||||||
|
toolResponse.is_error !== true &&
|
||||||
|
("file" in toolResponse || "filePath" in toolResponse)) return null;
|
||||||
let text = "";
|
let text = "";
|
||||||
if (typeof toolResponse === "string") text = toolResponse;
|
if (typeof toolResponse === "string") text = toolResponse;
|
||||||
else if (toolResponse.content !== undefined) {
|
else if (toolResponse.content !== undefined) {
|
||||||
@@ -289,6 +295,37 @@ function errorFingerprint(toolResponse) {
|
|||||||
if (!normalized) return null;
|
if (!normalized) return null;
|
||||||
return `raw:${djb2(normalized)}`;
|
return `raw:${djb2(normalized)}`;
|
||||||
}
|
}
|
||||||
|
// Mid-session nudge reactivation (ReContext P5/P7): on UserPromptSubmit, surface a
|
||||||
|
// stored active-nudge whose `cue.cwd` matches the current directory, so a deferred
|
||||||
|
// reminder lands when the user is actually working in that project instead of only
|
||||||
|
// at SessionStart. Read-only — SessionStart (adam-nudge.mjs) owns display accounting;
|
||||||
|
// this path surfaces each matching nudge at most once per session (state.surfaced_nudges)
|
||||||
|
// and respects expiry + max_displays. Opt out with ADAM_NO_MIDSESSION_NUDGE.
|
||||||
|
// UserPromptSubmit hook stdout is injected into the session context, so this is the
|
||||||
|
// reliable surface (unlike PostToolUse). Any failure is swallowed — never block a prompt.
|
||||||
|
function surfaceMatchingNudges(state, cwd) {
|
||||||
|
if (process.env.ADAM_NO_MIDSESSION_NUDGE || !cwd) return;
|
||||||
|
const nudgesPath = join(ROOT, "active-nudges.json");
|
||||||
|
let raw;
|
||||||
|
try { raw = JSON.parse(readFileSync(nudgesPath, "utf8")); } catch { return; }
|
||||||
|
if (!Array.isArray(raw)) return;
|
||||||
|
const now = Date.now();
|
||||||
|
if (!Array.isArray(state.surfaced_nudges)) state.surfaced_nudges = [];
|
||||||
|
for (const n of raw) {
|
||||||
|
if (!n || typeof n !== "object" || typeof n.message !== "string" || !n.message) continue;
|
||||||
|
if (Number(n.expires_at_ts || 0) <= now) continue;
|
||||||
|
const max = Number(n.max_displays || 0);
|
||||||
|
if (max > 0 && Number(n.displays_used || 0) >= max) continue;
|
||||||
|
const cue = n.cue && typeof n.cue === "object" ? n.cue : null;
|
||||||
|
if (!cue || !cue.cwd || cue.cwd !== cwd) continue; // cue-conditioned on cwd
|
||||||
|
const id = String(n.created_at || n.message);
|
||||||
|
if (state.surfaced_nudges.includes(id)) continue;
|
||||||
|
process.stdout.write(n.message + "\n");
|
||||||
|
state.surfaced_nudges.push(id);
|
||||||
|
if (state.surfaced_nudges.length > 50) state.surfaced_nudges.shift();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
function resetFrictionCounters(state) {
|
function resetFrictionCounters(state) {
|
||||||
state.tools_since_user = 0;
|
state.tools_since_user = 0;
|
||||||
@@ -315,6 +352,7 @@ function resetSessionLocal(state) {
|
|||||||
state.task_tool_kinds = {};
|
state.task_tool_kinds = {};
|
||||||
state.task_tool_count = 0;
|
state.task_tool_count = 0;
|
||||||
state.task_corrections = 0;
|
state.task_corrections = 0;
|
||||||
|
state.surfaced_nudges = [];
|
||||||
}
|
}
|
||||||
|
|
||||||
function ensureStateDefaults(state) {
|
function ensureStateDefaults(state) {
|
||||||
@@ -339,6 +377,7 @@ function ensureStateDefaults(state) {
|
|||||||
if (!Array.isArray(state.recentRecoveries)) state.recentRecoveries = [];
|
if (!Array.isArray(state.recentRecoveries)) state.recentRecoveries = [];
|
||||||
if (typeof state.session_post_count !== "number") state.session_post_count = 0;
|
if (typeof state.session_post_count !== "number") state.session_post_count = 0;
|
||||||
if (!Array.isArray(state.context_ring)) state.context_ring = [];
|
if (!Array.isArray(state.context_ring)) state.context_ring = [];
|
||||||
|
if (!Array.isArray(state.surfaced_nudges)) state.surfaced_nudges = [];
|
||||||
}
|
}
|
||||||
|
|
||||||
function main() {
|
function main() {
|
||||||
@@ -364,6 +403,7 @@ function main() {
|
|||||||
if (event === "UserPromptSubmit") {
|
if (event === "UserPromptSubmit") {
|
||||||
const prompt = (input.prompt || "").slice(0, 200);
|
const prompt = (input.prompt || "").slice(0, 200);
|
||||||
pushContext(state, { event: "user", prompt: excerpt(prompt, CONTEXT_EXCERPT_LEN), ts });
|
pushContext(state, { event: "user", prompt: excerpt(prompt, CONTEXT_EXCERPT_LEN), ts });
|
||||||
|
surfaceMatchingNudges(state, cwd);
|
||||||
if (isCorrection(prompt)) {
|
if (isCorrection(prompt)) {
|
||||||
const last = state.tool_window[state.tool_window.length - 1] || {};
|
const last = state.tool_window[state.tool_window.length - 1] || {};
|
||||||
appendJournal({
|
appendJournal({
|
||||||
@@ -425,12 +465,17 @@ function main() {
|
|||||||
state.subagent_dispatch_emitted[name] = true;
|
state.subagent_dispatch_emitted[name] = true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} else if (event === "PostToolUse") {
|
} else if (event === "PostToolUse" || event === "PostToolUseFailure") {
|
||||||
const tool = input.tool_name || "unknown";
|
const tool = input.tool_name || "unknown";
|
||||||
const argsHash = djb2(JSON.stringify(input.tool_input || {}));
|
const argsHash = djb2(JSON.stringify(input.tool_input || {}));
|
||||||
const file = (input.tool_input && (input.tool_input.file_path || input.tool_input.path)) || null;
|
const file = (input.tool_input && (input.tool_input.file_path || input.tool_input.path)) || null;
|
||||||
|
|
||||||
const toolResponse = input.tool_response;
|
// Claude Code sends failed tool calls to PostToolUseFailure with the error
|
||||||
|
// text in `error`. Shape it like an is_error response so every PostToolUse
|
||||||
|
// detector sees real failures. An interrupt is an abort, not a tool error.
|
||||||
|
const toolResponse = event === "PostToolUseFailure"
|
||||||
|
? { is_error: input.is_interrupt !== true, content: String(input.error || "") }
|
||||||
|
: input.tool_response;
|
||||||
const respExcerpt = (() => {
|
const respExcerpt = (() => {
|
||||||
if (!toolResponse) return null;
|
if (!toolResponse) return null;
|
||||||
const text = typeof toolResponse === "string" ? toolResponse
|
const text = typeof toolResponse === "string" ? toolResponse
|
||||||
@@ -501,11 +546,11 @@ function main() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (input.tool_response && typeof input.tool_response === "object") {
|
if (toolResponse && typeof toolResponse === "object") {
|
||||||
bumpUsage("payload:tool_response_seen");
|
bumpUsage("payload:tool_response_seen");
|
||||||
}
|
}
|
||||||
|
|
||||||
const fp = errorFingerprint(input.tool_response);
|
const fp = errorFingerprint(toolResponse);
|
||||||
if (fp) {
|
if (fp) {
|
||||||
bumpUsage("payload:tool_response_error_seen");
|
bumpUsage("payload:tool_response_error_seen");
|
||||||
if (state.recentRecoveries.length) {
|
if (state.recentRecoveries.length) {
|
||||||
@@ -550,7 +595,7 @@ function main() {
|
|||||||
if (tool === "Bash") {
|
if (tool === "Bash") {
|
||||||
const cmd = (input.tool_input && input.tool_input.command) || "";
|
const cmd = (input.tool_input && input.tool_input.command) || "";
|
||||||
const isBuildCmd = BUILD_RE.test(cmd);
|
const isBuildCmd = BUILD_RE.test(cmd);
|
||||||
const hasError = (input.tool_response && input.tool_response.is_error === true) || fp !== null;
|
const hasError = (toolResponse && toolResponse.is_error === true) || fp !== null;
|
||||||
if (isBuildCmd && hasError) {
|
if (isBuildCmd && hasError) {
|
||||||
state.build_failure_count += 1;
|
state.build_failure_count += 1;
|
||||||
if (state.build_failure_count >= BUILD_LOOP_THRESHOLD && !state.build_loop_emitted) {
|
if (state.build_failure_count >= BUILD_LOOP_THRESHOLD && !state.build_loop_emitted) {
|
||||||
|
|||||||
+18
-2
@@ -120,13 +120,15 @@ copy_file "$SRC/hooks/adam-nudge.mjs" "$DEST/hook
|
|||||||
# Agent / skill / command
|
# Agent / skill / command
|
||||||
copy_file "$SRC/agents/adam.md" "$DEST/agents/adam.md"
|
copy_file "$SRC/agents/adam.md" "$DEST/agents/adam.md"
|
||||||
copy_file "$SRC/skills/adam-self-improvement/SKILL.md" "$DEST/skills/adam-self-improvement/SKILL.md"
|
copy_file "$SRC/skills/adam-self-improvement/SKILL.md" "$DEST/skills/adam-self-improvement/SKILL.md"
|
||||||
|
copy_file "$SRC/skills/adam-self-improvement/apply-by-type.md" "$DEST/skills/adam-self-improvement/apply-by-type.md"
|
||||||
copy_file "$SRC/commands/reflect.md" "$DEST/commands/reflect.md"
|
copy_file "$SRC/commands/reflect.md" "$DEST/commands/reflect.md"
|
||||||
# Adam internals
|
# Adam internals
|
||||||
copy_file "$SRC/adam/scripts/adam-archive.mjs" "$DEST/adam/scripts/adam-archive.mjs"
|
copy_file "$SRC/adam/scripts/adam-archive.mjs" "$DEST/adam/scripts/adam-archive.mjs"
|
||||||
copy_file "$SRC/adam/scripts/adam-upgrade.mjs" "$DEST/adam/scripts/adam-upgrade.mjs"
|
copy_file "$SRC/adam/scripts/adam-upgrade.mjs" "$DEST/adam/scripts/adam-upgrade.mjs"
|
||||||
# v0.3.3 helper scripts — invoked from SKILL.md / hooks / analyst flow
|
# v0.3.3 helper scripts — invoked from SKILL.md / hooks / analyst flow
|
||||||
for _adam_script in adam-utils adam-window adam-explain adam-nudge-eligibility adam-cooldown \
|
for _adam_script in adam-utils adam-window adam-explain adam-nudge-eligibility adam-cooldown \
|
||||||
adam-score adam-ab-measure adam-apply-reinforcement adam-batch adam-rollback; do
|
adam-score adam-ab-measure adam-apply-reinforcement adam-apply-nudge adam-batch adam-rollback \
|
||||||
|
adam-skill-utility; do
|
||||||
copy_file "$SRC/adam/scripts/${_adam_script}.mjs" \
|
copy_file "$SRC/adam/scripts/${_adam_script}.mjs" \
|
||||||
"$DEST/adam/scripts/${_adam_script}.mjs"
|
"$DEST/adam/scripts/${_adam_script}.mjs"
|
||||||
run "chmod +x \"$DEST/adam/scripts/${_adam_script}.mjs\""
|
run "chmod +x \"$DEST/adam/scripts/${_adam_script}.mjs\""
|
||||||
@@ -143,6 +145,20 @@ copy_file "$SRC/adam/tests/fixtures/seed-corrections.jsonl" "$DEST/adam
|
|||||||
# install marker — used by future runs to detect local mtime drift
|
# install marker — used by future runs to detect local mtime drift
|
||||||
run "touch \"$DEST/adam/.install-marker\""
|
run "touch \"$DEST/adam/.install-marker\""
|
||||||
|
|
||||||
|
# version marker — records the installed release tag for the update notifier
|
||||||
|
# (adam-nudge.mjs compares it against the latest GitHub release).
|
||||||
|
ADAM_VERSION=""
|
||||||
|
if [ -n "$VERSION" ]; then
|
||||||
|
ADAM_VERSION="$VERSION"
|
||||||
|
elif [ "$PIPED" = 1 ] && [ -n "${REF:-}" ]; then
|
||||||
|
ADAM_VERSION="$REF"
|
||||||
|
else
|
||||||
|
ADAM_VERSION="$(git -C "$SRC" describe --tags --abbrev=0 2>/dev/null || true)"
|
||||||
|
fi
|
||||||
|
[ -z "$ADAM_VERSION" ] && ADAM_VERSION="unknown"
|
||||||
|
run "printf '%s\\n' \"$ADAM_VERSION\" > \"$DEST/adam/.version\""
|
||||||
|
log " version marker: $ADAM_VERSION"
|
||||||
|
|
||||||
# --------------------------------------------------------------------- settings.json
|
# --------------------------------------------------------------------- settings.json
|
||||||
SETTINGS="$DEST/settings.json"
|
SETTINGS="$DEST/settings.json"
|
||||||
EXAMPLE="$SRC/settings.json.example"
|
EXAMPLE="$SRC/settings.json.example"
|
||||||
@@ -207,7 +223,7 @@ log ""
|
|||||||
log "installed:"
|
log "installed:"
|
||||||
log " hooks/adam-observe.mjs, hooks/adam-nudge.mjs"
|
log " hooks/adam-observe.mjs, hooks/adam-nudge.mjs"
|
||||||
log " agents/adam.md"
|
log " agents/adam.md"
|
||||||
log " skills/adam-self-improvement/SKILL.md"
|
log " skills/adam-self-improvement/SKILL.md, apply-by-type.md"
|
||||||
log " commands/reflect.md"
|
log " commands/reflect.md"
|
||||||
log " adam/scripts/adam-archive.mjs"
|
log " adam/scripts/adam-archive.mjs"
|
||||||
log " adam/scripts/adam-upgrade.mjs"
|
log " adam/scripts/adam-upgrade.mjs"
|
||||||
|
|||||||
@@ -34,6 +34,17 @@
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
|
"PostToolUseFailure": [
|
||||||
|
{
|
||||||
|
"matcher": "",
|
||||||
|
"hooks": [
|
||||||
|
{
|
||||||
|
"type": "command",
|
||||||
|
"command": "node \"$HOME/.claude/hooks/adam-observe.mjs\""
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
"SessionStart": [
|
"SessionStart": [
|
||||||
{
|
{
|
||||||
"hooks": [
|
"hooks": [
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: adam-self-improvement
|
name: adam-self-improvement
|
||||||
description: Use when the user types /reflect, asks "what has adam learned", asks to "review proposals", or wants to inspect the self-improvement queue. Dispatches the adam subagent to analyse the observation journal and presents proposals for approve/reject/edit.
|
description: Runs the ADAM /reflect review loop. Use when the user types /reflect or /reflect --explain, asks "what has adam learned", asks to "review proposals" or "review the queue", or acts on a SessionStart nudge about pending proposals. Pre-filters and scores the observation journal, dispatches the adam subagent in two stages, auto-applies verified high-confidence proposals, and walks the rest of the queue for approve, reject or edit.
|
||||||
---
|
---
|
||||||
|
|
||||||
# adam-self-improvement
|
# adam-self-improvement
|
||||||
@@ -8,82 +8,45 @@ description: Use when the user types /reflect, asks "what has adam learned", ask
|
|||||||
## When to invoke
|
## When to invoke
|
||||||
|
|
||||||
- User types `/reflect`
|
- User types `/reflect`
|
||||||
- User types `/reflect --explain` (same flow, but the analyst's clustering trace is shown to the user — see §2b below)
|
- User types `/reflect --explain` (same flow, but the analyst's clustering trace is shown to the user — see §2b)
|
||||||
- User asks: "what has adam learned", "any proposals", "review the queue"
|
- User asks: "what has adam learned", "any proposals", "review the queue"
|
||||||
- SessionStart nudge said proposals are pending and user wants to act on it
|
- SessionStart nudge said proposals are pending and user wants to act on it
|
||||||
|
|
||||||
|
## Supporting file
|
||||||
|
|
||||||
|
- [apply-by-type.md](apply-by-type.md): the Karpathy constraints and the per-type apply, move, A/B-tracking, archive, reject and edit procedures. **Read it before any apply in §3a or §4.** Its checks are mandatory. If no proposal reaches §3a or §4, you do not need it.
|
||||||
|
|
||||||
## Protocol
|
## Protocol
|
||||||
|
|
||||||
### 0. Parse flags
|
### 0. Parse flags
|
||||||
|
|
||||||
Check the slash-command argument string for the literal token `--explain`. Set `explain=true` when present; otherwise `explain=false`. Unknown flags: print one-line warning, continue with `explain=false`. This single flag is the only argument `/reflect` currently accepts.
|
Check the slash-command argument string for the literal token `--explain`. Set `explain=true` when present, otherwise `explain=false`. Unknown flags: print a one-line warning, continue with `explain=false`. `--explain` is the only argument `/reflect` accepts.
|
||||||
|
|
||||||
### 1. Pre-filter the journal (window + exclusion) + score
|
### 1. Pre-filter the journal (window + exclusion) + score
|
||||||
|
|
||||||
Before dispatching the analyst, run the windowed-journal filter:
|
Run these steps in order before dispatching the analyst. Each step writes stdout to the output file and stderr to the log file. On failure, show the stderr log to the user and apply the fallback. No failure stops the run.
|
||||||
|
|
||||||
```bash
|
| # | Step | Command | Fallback on non-zero exit |
|
||||||
node ~/.claude/adam/scripts/adam-window.mjs --home ~/.claude > /tmp/adam-windowed-journal.jsonl 2> /tmp/adam-windowed-journal.log
|
|---|------|---------|---------------------------|
|
||||||
```
|
| 1 | Window filter | `node ~/.claude/adam/scripts/adam-window.mjs --home ~/.claude > /tmp/adam-windowed-journal.jsonl 2> /tmp/adam-windowed-journal.log` | Pass the raw `~/.claude/adam/journal.jsonl` path to the agent. Its manual excluded-timestamps logic still filters actioned entries. Only the freshness window is lost. |
|
||||||
|
| 2 | Score | `node ~/.claude/adam/scripts/adam-score.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-scores.json 2> /tmp/adam-scores.log` | Pass `{"sessions":[],"reinforcement_candidates":[]}` (dampener defaults to 1.0). |
|
||||||
|
| 3 | A/B measure | `node ~/.claude/adam/scripts/adam-ab-measure.mjs --home ~/.claude --format json > /tmp/adam-ab-regressions.json 2> /tmp/adam-ab-regressions.log` | Pass `[]`. |
|
||||||
|
| 4 | Auto-rollback (only if step 3 found `regressed`) | `node ~/.claude/adam/scripts/adam-rollback.mjs --auto --home ~/.claude > /tmp/adam-rollback-results.json 2> /tmp/adam-rollback.log` | Continue (best-effort). |
|
||||||
|
| 5 | Evidence batching | `node ~/.claude/adam/scripts/adam-batch.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-batches.json 2> /tmp/adam-batch.log` | Pass `null`. The analyst falls back to raw journal clustering. |
|
||||||
|
| 6 | Skill utility | `node ~/.claude/adam/scripts/adam-skill-utility.mjs --input /tmp/adam-windowed-journal.jsonl --json > /tmp/adam-skill-utility.json 2> /tmp/adam-skill-utility.log` | Skip (best-effort). |
|
||||||
|
|
||||||
The script reads the active journal plus all rotated journal files (new
|
What each step does and how to use its output:
|
||||||
`journal/YYYY-Www.jsonl` weekly format AND legacy
|
|
||||||
`journal/YYYY-MM-DD-<ts>.jsonl` size-rotated format are both supported), applies
|
|
||||||
per-signal-type sliding windows (see `SIGNAL_WINDOWS_DAYS` in
|
|
||||||
`adam-window.mjs`), and drops entries already actioned via
|
|
||||||
`applied/*.md` / `rejected/*.md` frontmatter `source_entries`.
|
|
||||||
|
|
||||||
If `adam-window.mjs` exits non-zero: log the stderr file to the user, fall
|
1. **Window filter.** Reads the active journal and all rotated journals (weekly `journal/YYYY-Www.jsonl` and legacy size-rotated `journal/YYYY-MM-DD-<ts>.jsonl`). Applies per-signal-type sliding windows (`SIGNAL_WINDOWS_DAYS` in `adam-window.mjs`). Drops entries already actioned through the `source_entries` frontmatter of `applied/*.md` and `rejected/*.md`.
|
||||||
through to passing the raw `~/.claude/adam/journal.jsonl` path to the agent
|
2. **Score.** Produces a per-session `dampener` (0.5 / 0.75 / 1.0, from `task_completed_count`) and a `reinforcement_candidates` list (skills cited by ≥3 clean `task_completed` events). The analyst uses both (`agents/adam.md` §"Scoring: task_completed dampener").
|
||||||
(graceful degradation — the agent's manual excluded-timestamps logic still
|
3. **A/B measure.** Runs on previously auto-applied proposals (see the A/B tracking append in apply-by-type.md). Output is an array of delta objects (`pre_count`, `post_count`, `delta_pct`, `status` ∈ {`improved`,`neutral`,`regressed`,`no_baseline`,`pending`}). Filter to `status == "regressed"` and pass that as `ab_regressions`. When the list is non-empty, the analyst must put a `## Regressions` section at the top of its output (`agents/adam.md` §"A/B effectiveness").
|
||||||
filters actioned entries; only the freshness window is lost).
|
4. **Auto-rollback** (MOSS §3.5). Reverts regressed proposals before analyst dispatch. The script moves each one from `applied/` back to `proposals/` with `rolled_back: true` and creates a regression nudge. For each one, print: `adam: rolled back "<proposal_id>" — regression detected (delta: <delta_pct>%)`.
|
||||||
|
5. **Evidence batching** (MOSS §3.1). Groups entries by (signal_type, cluster_key) into coherent failure batches, with per-batch metadata such as `has_context_window` (transcript evidence attached).
|
||||||
Then run the scoring pre-step on the same windowed journal:
|
6. **Skill utility** (execution-grounded selection signal, after SkillsInjector arXiv 2605.29794: utility Δ(s), not surface match). Surfaces advisory candidates for description disambiguation or archival. Ranks skills by co-occurrence with positive (`task_completed`, `clean_recovery`, `correction_free_streak`) vs negative outcome events. **CO-OCCURRENCE, NOT CAUSATION.** Show the worst 3 below-baseline skills (`lift < 0`, not low-sample) to the *user* as a one-line advisory before listing proposals (e.g. `skill-utility: chezmoi 9% pos n=85, ghostty-config 14% pos n=50, …`). Do NOT feed this into the analyst's proposal machinery or auto-draft skill archival from it. The human decides.
|
||||||
|
|
||||||
```bash
|
|
||||||
node ~/.claude/adam/scripts/adam-score.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-scores.json 2> /tmp/adam-scores.log
|
|
||||||
```
|
|
||||||
|
|
||||||
This produces a per-session `dampener` (0.5 / 0.75 / 1.0 based on
|
|
||||||
`task_completed_count`) and a `reinforcement_candidates` list (skills cited by
|
|
||||||
≥3 clean `task_completed` events). The analyst uses both — see
|
|
||||||
`agents/adam.md` §"Scoring: task_completed dampener". If the score step fails,
|
|
||||||
log stderr to the user and pass an empty `{"sessions":[],"reinforcement_candidates":[]}`
|
|
||||||
to the analyst (dampener defaults to 1.0).
|
|
||||||
|
|
||||||
Finally, run the A/B measurement pre-step on any previously auto-applied
|
|
||||||
proposals (see §3 ab-tracking write):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
node ~/.claude/adam/scripts/adam-ab-measure.mjs --home ~/.claude --format json > /tmp/adam-ab-regressions.json 2> /tmp/adam-ab-regressions.log
|
|
||||||
```
|
|
||||||
|
|
||||||
The JSON output is an array of A/B delta objects (`pre_count`, `post_count`,
|
|
||||||
`delta_pct`, `status` ∈ {`improved`,`neutral`,`regressed`,`no_baseline`,`pending`}).
|
|
||||||
Filter to `status == "regressed"` before passing to the analyst as
|
|
||||||
`ab_regressions`. The analyst is required (see `agents/adam.md` §"A/B
|
|
||||||
effectiveness") to surface a `## Regressions` section at the top of its output
|
|
||||||
when this list is non-empty. If the script fails: log stderr, pass `[]`.
|
|
||||||
|
|
||||||
**Auto-rollback** (MOSS §3.5): if any entries have `status == "regressed"`, run the rollback script to auto-revert them before analyst dispatch:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
node ~/.claude/adam/scripts/adam-rollback.mjs --auto --home ~/.claude > /tmp/adam-rollback-results.json 2> /tmp/adam-rollback.log
|
|
||||||
```
|
|
||||||
|
|
||||||
For each rolled-back proposal, print to user: `adam: rolled back "<proposal_id>" — regression detected (delta: <delta_pct>%)`. The rollback script moves the proposal from `applied/` back to `proposals/` with `rolled_back: true` and creates a regression nudge. If the script fails: log stderr, continue (rollback is best-effort).
|
|
||||||
|
|
||||||
**Evidence batching** (MOSS §3.1): pre-cluster the windowed journal into coherent failure batches:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
node ~/.claude/adam/scripts/adam-batch.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-batches.json 2> /tmp/adam-batch.log
|
|
||||||
```
|
|
||||||
|
|
||||||
This groups entries by (signal_type, cluster_key) and reports per-batch metadata including `has_context_window` (whether transcript evidence is attached). If the script fails: log stderr, pass `null` to the analyst (graceful degradation — analyst falls back to raw journal clustering).
|
|
||||||
|
|
||||||
### 2. Dispatch the analyst (two-stage pipeline)
|
### 2. Dispatch the analyst (two-stage pipeline)
|
||||||
|
|
||||||
MOSS §3.3: "A single prompt asked to diagnose, plan, implement, verify, and decide overloads context and produces lower-quality output than a sequenced flow." The analyst is dispatched in two stages with a validation gate between them.
|
MOSS §3.3: "A single prompt asked to diagnose, plan, implement, verify, and decide overloads context and produces lower-quality output than a sequenced flow." Dispatch the analyst in two stages with a validation gate between them.
|
||||||
|
|
||||||
**Stage 1 — Diagnose + Plan**: Use the Agent tool with `subagent_type: "adam"` and prompt:
|
**Stage 1 — Diagnose + Plan**: Use the Agent tool with `subagent_type: "adam"` and prompt:
|
||||||
|
|
||||||
@@ -99,6 +62,7 @@ Inputs:
|
|||||||
- batches_path: /tmp/adam-batches.json # pre-clustered evidence batches
|
- batches_path: /tmp/adam-batches.json # pre-clustered evidence batches
|
||||||
- scores_path: /tmp/adam-scores.json
|
- scores_path: /tmp/adam-scores.json
|
||||||
- ab_regressions_path: /tmp/adam-ab-regressions.json
|
- ab_regressions_path: /tmp/adam-ab-regressions.json
|
||||||
|
- skill_utility_path: /tmp/adam-skill-utility.json # per-skill utility Δ (worst-first); already computed in pre-analysis
|
||||||
- journal_path: ~/.claude/adam/journal.jsonl # raw — fallback only
|
- journal_path: ~/.claude/adam/journal.jsonl # raw — fallback only
|
||||||
- state_path: ~/.claude/adam/state.json
|
- state_path: ~/.claude/adam/state.json
|
||||||
- usage_path: ~/.claude/adam/usage.json
|
- usage_path: ~/.claude/adam/usage.json
|
||||||
@@ -123,6 +87,8 @@ Wait for return.
|
|||||||
|
|
||||||
If all diagnoses are removed or the file is missing/empty, print "adam: no valid diagnoses — nothing to implement" and skip to §6.
|
If all diagnoses are removed or the file is missing/empty, print "adam: no valid diagnoses — nothing to implement" and skip to §6.
|
||||||
|
|
||||||
|
**Adaptive re-diagnose** (ReContext P5 — spend depth only where it is needed): after validation, collect validated diagnoses whose `mismatch` is `unclear` OR whose cluster severity-sum is top-tier (≥32, per `scores_path`); take at most the top 3 by severity. If any exist, re-dispatch **one** more `stage=diagnose` pass carrying `redigest: <their cluster_ids>` and `prior_diagnoses_path: /tmp/adam-diagnoses.json` (the analyst refines only those clusters in place), then re-run the inter-stage validation above on the refreshed file. Do this at most once per `/reflect`; clear clusters are never re-diagnosed. If none qualify, go straight to Stage 2.
|
||||||
|
|
||||||
**Stage 2 — Implement**: Use the Agent tool with `subagent_type: "adam"` and prompt:
|
**Stage 2 — Implement**: Use the Agent tool with `subagent_type: "adam"` and prompt:
|
||||||
|
|
||||||
```
|
```
|
||||||
@@ -135,6 +101,7 @@ Inputs:
|
|||||||
- windowed_journal_path: /tmp/adam-windowed-journal.jsonl
|
- windowed_journal_path: /tmp/adam-windowed-journal.jsonl
|
||||||
- scores_path: /tmp/adam-scores.json
|
- scores_path: /tmp/adam-scores.json
|
||||||
- ab_regressions_path: /tmp/adam-ab-regressions.json
|
- ab_regressions_path: /tmp/adam-ab-regressions.json
|
||||||
|
- skill_utility_path: /tmp/adam-skill-utility.json # per-skill utility Δ — drives the utility-lift rubric penalty
|
||||||
- state_path: ~/.claude/adam/state.json
|
- state_path: ~/.claude/adam/state.json
|
||||||
- usage_path: ~/.claude/adam/usage.json
|
- usage_path: ~/.claude/adam/usage.json
|
||||||
- proposals_dir: ~/.claude/adam/proposals/
|
- proposals_dir: ~/.claude/adam/proposals/
|
||||||
@@ -151,14 +118,14 @@ Wait for return.
|
|||||||
|
|
||||||
### 2b. Persist and render the clustering trace
|
### 2b. Persist and render the clustering trace
|
||||||
|
|
||||||
The analyst's final message always contains a fenced ` ```trace ` block (per `agents/adam.md` §"Clustering trace (always emit)") immediately before its punch-list JSON line.
|
The analyst's final message always contains a fenced ` ```trace ` block (`agents/adam.md` §"Clustering trace (always emit)") immediately before its punch-list JSON line.
|
||||||
|
|
||||||
1. Extract the trace block. If it is missing, print a one-line warning to the user (`adam: trace block missing from agent output — proceeding without observability`) and continue; do not block on this.
|
1. Extract the trace block. If it is missing, print `adam: trace block missing from agent output — proceeding without observability` and continue. Do not block on this.
|
||||||
2. ALWAYS write the trace verbatim (without the surrounding fences) to `~/.claude/adam/last-trace.txt` (overwrite each run). This persists for retrospection via `node ~/.claude/adam/scripts/adam-explain.mjs`.
|
2. ALWAYS write the trace verbatim (without the fences) to `~/.claude/adam/last-trace.txt`, overwriting each run.
|
||||||
3. Extract the `SUMMARY:` line from the trace. ALWAYS display it as a one-line status to the user BEFORE the proposals are listed, e.g. `clustering: <SUMMARY line>`. This single-line status is shown in both `--explain` and default modes.
|
3. ALWAYS show the trace's `SUMMARY:` line as a one-line status BEFORE the proposals are listed, e.g. `clustering: <SUMMARY line>`, in both modes.
|
||||||
4. If `explain=true` (from §0): ALSO render the full trace block back to the user as a fenced code block (` ```text ` … ` ``` `) under a header `Clustering trace:`. If `explain=false`: SUPPRESS the cluster-line body from the user-visible output (the SUMMARY line is already shown in step 3).
|
4. If `explain=true`: ALSO render the full trace as a fenced ` ```text ` block under the header `Clustering trace:`. If `explain=false`: SUPPRESS the cluster-line body (step 3 already showed the SUMMARY line).
|
||||||
|
|
||||||
The user can re-render any past trace at any time via:
|
The user can re-render any past trace:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
node ~/.claude/adam/scripts/adam-explain.mjs --mode summary # SUMMARY + per-decision counts
|
node ~/.claude/adam/scripts/adam-explain.mjs --mode summary # SUMMARY + per-decision counts
|
||||||
@@ -168,71 +135,24 @@ node ~/.claude/adam/scripts/adam-explain.mjs --mode json # machine-readabl
|
|||||||
|
|
||||||
### 3. Pre-apply verification gate (MOSS §3.4)
|
### 3. Pre-apply verification gate (MOSS §3.4)
|
||||||
|
|
||||||
MOSS §3.4: "Verification must therefore be runtime, on a production-equivalent environment, and against the same prompts that produced the failure evidence." Before auto-applying, verify each proposal deterministically:
|
MOSS §3.4: "Verification must therefore be runtime, on a production-equivalent environment, and against the same prompts that produced the failure evidence."
|
||||||
|
|
||||||
For each id in `high_confidence`:
|
For each id in `high_confidence`, read `~/.claude/adam/proposals/<id>-*.md` and run these checks. All four must pass for auto-apply. Any failure demotes the id to `queued` for manual review.
|
||||||
- Read the proposal file from `~/.claude/adam/proposals/<id>-*.md`.
|
|
||||||
- **Verification checks** (all must pass for auto-apply to proceed):
|
1. **Source entries exist**: every timestamp in `source_entries` frontmatter appears in `/tmp/adam-windowed-journal.jsonl`. A missing one means the evidence is stale or already actioned.
|
||||||
1. **Source entries exist**: every timestamp in `source_entries` frontmatter must appear in `/tmp/adam-windowed-journal.jsonl`. If any are missing, the evidence is stale or was already actioned — demote to `queued`.
|
2. **Diagnosis grounded**: the `# Diagnosis` section has all four fields (Trigger, Action, Mismatch, Outcome) with ≥1 backtick-wrapped quote.
|
||||||
2. **Diagnosis grounded**: the `# Diagnosis` section must have all four fields (Trigger, Action, Mismatch, Outcome) with ≥1 backtick-wrapped quote. If malformed, demote to `queued`.
|
3. **Type-evidence match**: the proposal `type` matches what the evidence supports:
|
||||||
3. **Type-evidence match**: the proposal `type` must match what the evidence supports:
|
|
||||||
- `correction` signals → `memory`, `skill_new`, `skill_edit` (not `nudge`)
|
- `correction` signals → `memory`, `skill_new`, `skill_edit` (not `nudge`)
|
||||||
- `dead_end` signals → `nudge`, `skill_new`, `skill_edit` (not `memory`)
|
- `dead_end` signals → `nudge`, `skill_new`, `skill_edit` (not `memory`)
|
||||||
- `tool_error_loop` signals → `memory`, `skill_new`, `skill_edit`
|
- `tool_error_loop` signals → `memory`, `skill_new`, `skill_edit`
|
||||||
- `harness_edit` → must cite harness-level evidence (false negative, scoring bias, window miscalibration)
|
- `harness_edit` → must cite harness-level evidence (false negative, scoring bias, window miscalibration)
|
||||||
If mismatch, demote to `queued`.
|
4. **No conflicting applied proposal**: grep `~/.claude/adam/applied/` for a proposal with the same `target` applied in the last 7 days. This prevents stacking rapid edits.
|
||||||
4. **No conflicting applied proposal**: grep `~/.claude/adam/applied/` for any proposal with the same `target` applied in the last 7 days. If found, demote to `queued` (prevents stacking rapid edits).
|
|
||||||
- Print verification result: `verified: <id> (4/4 checks passed)` or `demoted: <id> (failed: <check_name>)`.
|
Print `verified: <id> (4/4 checks passed)` or `demoted: <id> (failed: <check_name>)`.
|
||||||
- Demoted proposals are moved from `high_confidence` to `queued` for manual review.
|
|
||||||
|
|
||||||
### 3a. Apply verified high-confidence items
|
### 3a. Apply verified high-confidence items
|
||||||
|
|
||||||
For each id that passed verification:
|
Read [apply-by-type.md](apply-by-type.md). Enforce its Karpathy constraints, then follow its §3a sections for each verified id: the per-type apply, then move, A/B tracking append and archive. Print `auto-applied N proposals: [ids]`.
|
||||||
- Print `id`, `target`, `confidence`, `blast_radius`, `cross_session_evidence`, `auto_apply_eligible`.
|
|
||||||
- Apply the change:
|
|
||||||
- **For `skill_new`**: `mkdir -p ~/.claude/skills/<slug>/`, then `Write` the proposal's `# Proposed change` body to `~/.claude/skills/<slug>/SKILL.md`. After write, print: "skill `<slug>` written to `~/.claude/skills/<slug>/SKILL.md` — activates immediately — Claude Code v2.1.0+ auto-hot-reloads user-level skills, no restart needed."
|
|
||||||
- **For `memory`**: `Write` the proposal's `# Proposed change` body (which MUST include the auto-memory frontmatter — see "Memory drafting protocol" in `agents/adam.md`) to the path in `target`. Then update `MEMORY.md` index with a one-line pointer.
|
|
||||||
- **For `nudge`**: low-blast auto-apply path. Single-session evidence is sufficient — skip the cross-session gate. Append a new entry to `~/.claude/adam/active-nudges.json` (create the file with `[]` if absent) with shape `{kind, message, created_at: <now_ms>, expires_at_ts: <now_ms + 7*86400000>, max_displays: 3, displays_used: 0, source_session: <session_id from proposal>}`. Do NOT modify any skill, memory, agent, or CLAUDE.md. Tell user: "nudge queued — surfaces on next SessionStart in a different session (expires in 7 days)."
|
|
||||||
- **For `reinforcement`**: gated by `confidence >= 4 AND blast_radius == low` (same as memory). Apply by invoking the helper:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
node ~/.claude/adam/scripts/adam-apply-reinforcement.mjs ~/.claude/adam/proposals/<id>-*.md --home ~/.claude
|
|
||||||
```
|
|
||||||
|
|
||||||
The helper reads the proposal frontmatter (`skill_slug`, `count`, `source_session`) and appends one JSON line to `~/.claude/adam/reinforcements.jsonl`. No code/memory/skill modifications. Output: `{"status":"applied"|"gated", ...}` — on `gated` leave proposal in `proposals/` (helper failed its own re-check), on `applied` continue to the archive step. Tell user: "reinforcement logged for `<skill_slug>` (count=<N>) — appended to reinforcements.jsonl."
|
|
||||||
- **For `skill_edit`**: enforce the apply-time gate before writing.
|
|
||||||
1. Verify proposal frontmatter has `auto_apply_eligible: true`. If not, abort and queue for review.
|
|
||||||
2. Read `target` SKILL.md, capture `current_bytes` from a fresh stat — do NOT trust frontmatter `bytes_before`.
|
|
||||||
3. Verify diff in `# Proposed change`:
|
|
||||||
- Unified-diff format.
|
|
||||||
- Zero `-` lines on existing SKILL.md content (additions only).
|
|
||||||
- Total `+` lines ≤ 30.
|
|
||||||
If any check fails, print one-line refusal reason, leave proposal in `proposals/`, continue.
|
|
||||||
4. Cooldown re-check: run `node ~/.claude/adam/scripts/adam-cooldown.mjs --skill <target_skill> --fingerprint <proposal_fingerprint>` (both fields come from proposal frontmatter; missing fingerprint → "legacy"). Refuse if the script returns `status: cooldown` OR `status: blacklisted`. This per-(skill, fingerprint) gate replaces the previous coarse per-skill scan — proposals for the same skill with a different fingerprint are NOT blocked by an older entry.
|
|
||||||
5. (covered by step 4 — blacklisted status is returned by `adam-cooldown.mjs` when `auto_apply_blacklist: true` is found in `rejected/` within 30 days for the same (skill, fingerprint))
|
|
||||||
6. Apply via `Edit` tool (append the new section per the diff). Never use `Write` on existing SKILL.md.
|
|
||||||
7. Re-stat target. If new size exceeds `2 * current_bytes` (captured in step 2), revert via `Edit` (remove the just-appended section) and refuse — print refusal reason.
|
|
||||||
8. Add `last_auto_edit: <iso8601 utc now>` to the proposal frontmatter before moving it.
|
|
||||||
9. Tell user: "skill `<slug>` extended (added <N> lines) — auto-applied via win-evidence gate."
|
|
||||||
- Move proposal to `~/.claude/adam/applied/<UTC-ts>-<id>.md`.
|
|
||||||
- **A/B tracking append** (skip for `reinforcement` — positive-only ledger, intentionally not A/B-tracked per `agents/adam.md` §"`reinforcement` proposals"): as a separate atomic step right after the move, append one JSON line to `~/.claude/adam/ab-tracking.jsonl` (create with empty contents if absent). Read fields from the proposal's frontmatter (`proposal_fingerprint`, `originating_signals` — both populated per `agents/adam.md`; `originating_signals` is a list of `{type, count, session_ids}` objects). Schema:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"applied_at": <unix_ms now>,
|
|
||||||
"proposal_id": "<id>",
|
|
||||||
"proposal_type": "skill_edit|skill_new|memory|nudge",
|
|
||||||
"target_skill": "<slug or target basename>",
|
|
||||||
"proposal_fingerprint": "<hash>",
|
|
||||||
"originating_signals": [{"type":"<signal>","count":<N>,"session_ids":[...]}],
|
|
||||||
"pre_window_days": 7
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
This entry is consumed by `adam-ab-measure.mjs` on subsequent `/reflect` runs to compute pre/post signal-count deltas. See `agents/adam.md` §"A/B effectiveness". If the append fails (disk-full etc.) log a warning but do NOT abort the apply path — A/B is observability, not a gate.
|
|
||||||
- **Archive consumed journal entries**: `node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<UTC-ts>-<id>.md` — moves entries listed in proposal's `source_entries` from `journal.jsonl` to `journal/actioned-<id>.jsonl` so subsequent `/reflect` runs do not re-cluster them.
|
|
||||||
|
|
||||||
Print: `auto-applied N proposals: [ids]`.
|
|
||||||
|
|
||||||
### 4. Walk the queue
|
### 4. Walk the queue
|
||||||
|
|
||||||
@@ -240,27 +160,13 @@ For each id in `queued`:
|
|||||||
|
|
||||||
a. Read and display the proposal in full (frontmatter + body).
|
a. Read and display the proposal in full (frontmatter + body).
|
||||||
b. Ask the user: **approve** / **reject** / **edit**.
|
b. Ask the user: **approve** / **reject** / **edit**.
|
||||||
c. On **approve**:
|
c. On **approve**: enforce the Karpathy constraints, apply per the apply-by-type.md §4 approve section, move the proposal to `~/.claude/adam/applied/<ts>-<id>.md`, and archive it.
|
||||||
- For `claude_md_edit`: backup `cp ~/.claude/CLAUDE.md ~/.claude/adam/applied/<ts>-claude-md-backup.md` first.
|
d. On **reject**: follow apply-by-type.md §4 reject (reason, `auto_apply_blacklist` for `skill_edit`, move to `rejected/`, archive).
|
||||||
- For `deletion`: `mkdir -p ~/.claude/adam/trash/<ts>` then `mv` the artifact into it. Print restoration command.
|
e. On **edit**: ask the user for the change, edit the proposal in place, then loop back to step a for that same id.
|
||||||
- For `skill_new`: `mkdir -p ~/.claude/skills/<slug>/`, then write `# Proposed change` body to `<slug>/SKILL.md`. Tell user: "skill `<slug>` written — activates immediately (CC v2.1.0+ auto-hot-reload)."
|
|
||||||
- For `skill_edit`: apply the unified diff in `# Proposed change` to the existing SKILL.md at `target` (append-only — never replace existing content).
|
|
||||||
- For `memory`: write `# Proposed change` body (must include auto-memory frontmatter) to `target` and update `MEMORY.md` index with a one-line pointer.
|
|
||||||
- For `harness_edit` (MOSS §1): apply the unified diff to the target harness file. **Before applying**:
|
|
||||||
1. Run `bash ~/.claude/adam/tests/run-tests.sh` — capture pass count.
|
|
||||||
2. Apply the diff via `Edit`.
|
|
||||||
3. Run `bash ~/.claude/adam/tests/run-tests.sh` again — verify pass count is equal or higher and 0 failures.
|
|
||||||
4. If test regression: revert the edit, print "harness_edit reverted — test regression detected", leave proposal in `proposals/`.
|
|
||||||
5. If tests pass: tell user "harness edit applied to `<target>` — tests pass (<N> passed)."
|
|
||||||
- For all others: apply via Write/Edit per the proposal's `# Proposed change`.
|
|
||||||
- Move proposal to `~/.claude/adam/applied/<ts>-<id>.md`.
|
|
||||||
- Archive: `node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<ts>-<id>.md`.
|
|
||||||
d. On **reject**: ask for reason in one line. Append `# Reason\n<reason>` to proposal body. If the proposal `type` is `skill_edit`, ALSO add `auto_apply_blacklist: true` to its frontmatter (so future reflects skip auto-apply on this target for 30 days). Move to `~/.claude/adam/rejected/<id>.md`. Archive: `node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/rejected/<id>.md`.
|
|
||||||
e. On **edit**: ask the user for the change, edit the proposal in place, then loop back to step 4a for that same id.
|
|
||||||
|
|
||||||
### 5. Handle failures
|
### 5. Handle failures
|
||||||
|
|
||||||
If apply fails (file write error, target missing): leave proposal in `proposals/`, append `# Apply error\n<error>` to its body. Tell the user. Do not move it.
|
If apply fails (file write error, target missing): leave the proposal in `proposals/`, append `# Apply error\n<error>` to its body, and tell the user. Do not move it.
|
||||||
|
|
||||||
### 6. Summary
|
### 6. Summary
|
||||||
|
|
||||||
@@ -280,35 +186,16 @@ adam reflect summary:
|
|||||||
failed: <N>
|
failed: <N>
|
||||||
```
|
```
|
||||||
|
|
||||||
**Keypoint history**: after all proposals are processed, append one JSON line to `~/.claude/adam/keypoint-history.jsonl` with the aggregate keypoint scores from the diagnose stage:
|
**Keypoint history**: after all proposals are processed, append one JSON line to `~/.claude/adam/keypoint-history.jsonl` with the aggregate keypoint scores from the diagnose stage. This is the longitudinal record of which capabilities improve across `/reflect` runs.
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{"ts":"<iso>","session":"<session_id>","keypoints":{"tool_selection":N,"scope_discipline":N,"error_recovery":N,"first_attempt":N,"build_reliability":N},"proposals_emitted":N,"proposals_applied":N}
|
{"ts":"<iso>","session":"<session_id>","keypoints":{"tool_selection":N,"scope_discipline":N,"error_recovery":N,"first_attempt":N,"build_reliability":N},"proposals_emitted":N,"proposals_applied":N}
|
||||||
```
|
```
|
||||||
|
|
||||||
This builds a longitudinal record of which capabilities are improving across `/reflect` runs.
|
|
||||||
|
|
||||||
## Karpathy constraints (you must enforce on each apply)
|
|
||||||
|
|
||||||
Before writing any proposal:
|
|
||||||
- Confirm `# Assumptions` section is non-empty.
|
|
||||||
- Confirm `# Diagnosis` section exists and contains all four labelled lines (`Trigger:`, `Action:`, `Mismatch:`, `Outcome:`) AND at least one backtick-wrapped quote ≤80 chars in the Outcome line. Refuse if missing or malformed — agent must redraft per the "Diagnosis drafting protocol" in `agents/adam.md`.
|
|
||||||
- Confirm `# Success criterion` section is non-empty and runnable.
|
|
||||||
- Confirm change is ≤50 LOC for non-`skill_new`, or ≤80 LOC for `skill_new` body. If larger, ask the user once: "this proposal is N LOC — proceed?"
|
|
||||||
- For `claude_md_edit`: confirm 3+ distinct cwds in the `# Why` section.
|
|
||||||
- For `deletion`: confirm both criteria (a) and (b) from the agent's special handling are documented in the proposal.
|
|
||||||
- For `skill_new`: confirm the slug doesn't collide with any existing skill in `~/.claude/skills/`. If it does, refuse and ask user to rename.
|
|
||||||
- For `skill_edit`: confirm the diff is append-only (no `-` lines that remove existing content) and that target SKILL.md exists. When auto-applying, ALSO re-verify the eligibility gate steps in §3 (cooldown, blacklist, byte cap) before any `Edit` call — never trust frontmatter alone.
|
|
||||||
- For `skill_edit` with `auto_apply_eligible: true`: confirm `contradiction_flag` is absent or null in frontmatter. Refuse auto-apply if `contradiction_flag` is set with any non-empty value (treat the agent's flag as a hard veto on auto-apply; user can still manually approve in walk-the-queue if they disagree with the heuristic).
|
|
||||||
- For `memory`: confirm `# Proposed change` body starts with `---` frontmatter containing required fields `name`, `description`, `type`, `originSessionId`. Refuse if frontmatter missing — agent must redraft per the Memory drafting protocol.
|
|
||||||
- For `harness_edit`: confirm `auto_apply_eligible: false` (never auto-apply). Confirm `confidence ≥ 5`. Confirm `# Test verification` section names the test command. Confirm diff is ≤30 LOC and targets a single allowed harness file (see `agents/adam.md` §"Harness self-modification"). Run test suite before AND after applying — revert on any regression.
|
|
||||||
- Confirm `source_entries` is present in proposal frontmatter as a non-empty list (used for archive). Warn (do not refuse) if missing — legacy proposals from before v0.2.0 won't have it.
|
|
||||||
|
|
||||||
If any check fails, refuse to apply and ask the user how to proceed.
|
|
||||||
|
|
||||||
## Things you MUST NOT do
|
## Things you MUST NOT do
|
||||||
|
|
||||||
- Do not auto-apply anything not in `high_confidence`.
|
- Do not auto-apply anything not in `high_confidence`.
|
||||||
|
- Do not apply any proposal before you read apply-by-type.md and enforce its Karpathy constraints.
|
||||||
- Do not invoke other skills during a `/reflect` run.
|
- Do not invoke other skills during a `/reflect` run.
|
||||||
- Do not modify `settings.json` without explicit user yes.
|
- Do not modify `settings.json` without explicit user yes.
|
||||||
- Do not hard-delete anything. Use `mv` to `~/.claude/adam/trash/<ts>/`.
|
- Do not hard-delete anything. Use `mv` to `~/.claude/adam/trash/<ts>/`.
|
||||||
|
|||||||
@@ -0,0 +1,108 @@
|
|||||||
|
# Apply procedures by proposal type
|
||||||
|
|
||||||
|
Read this file before any apply in SKILL.md §3a (auto-apply) or §4 (walk the queue).
|
||||||
|
Every check here is mandatory. Section numbers (§N) refer to SKILL.md.
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
|
||||||
|
- Karpathy constraints (every apply)
|
||||||
|
- §3a auto-apply, per type
|
||||||
|
- §3a after each auto-apply: move, A/B tracking, archive
|
||||||
|
- §4 approve, per type
|
||||||
|
- §4 reject and edit
|
||||||
|
|
||||||
|
## Karpathy constraints (you must enforce on each apply)
|
||||||
|
|
||||||
|
Before writing any proposal:
|
||||||
|
|
||||||
|
- Confirm `# Assumptions` section is non-empty.
|
||||||
|
- Confirm `# Diagnosis` section exists and contains all four labelled lines (`Trigger:`, `Action:`, `Mismatch:`, `Outcome:`) AND at least one backtick-wrapped quote ≤80 chars in the Outcome line. Refuse if missing or malformed — agent must redraft per the "Diagnosis drafting protocol" in `agents/adam.md`.
|
||||||
|
- Confirm `# Success criterion` section is non-empty and runnable.
|
||||||
|
- Confirm change is ≤50 LOC for non-`skill_new`, or ≤80 LOC for `skill_new` body. If larger, ask the user once: "this proposal is N LOC — proceed?"
|
||||||
|
- For `claude_md_edit`: confirm 3+ distinct cwds in the `# Why` section.
|
||||||
|
- For `deletion`: confirm both criteria (a) and (b) from the agent's special handling are documented in the proposal.
|
||||||
|
- For `skill_new`: confirm the slug doesn't collide with any existing skill in `~/.claude/skills/`. If it does, refuse and ask user to rename.
|
||||||
|
- For `skill_edit`: confirm the diff is append-only (no `-` lines that remove existing content) and that target SKILL.md exists. When auto-applying, ALSO re-verify the eligibility gate steps in §3 (cooldown, blacklist, byte cap) before any `Edit` call — never trust frontmatter alone.
|
||||||
|
- For `skill_edit` with `auto_apply_eligible: true`: confirm `contradiction_flag` is absent or null in frontmatter. Refuse auto-apply if `contradiction_flag` is set with any non-empty value (treat the agent's flag as a hard veto on auto-apply; user can still manually approve in walk-the-queue if they disagree with the heuristic).
|
||||||
|
- For `memory`: confirm `# Proposed change` body starts with `---` frontmatter matching the live auto-memory schema — top-level `name` (the slug) + `description`, plus a `metadata:` block with `node_type: memory`, `type`, and `originSessionId`. Cross-check the shape against an existing file in the target memory dir. Refuse if frontmatter is flat (`type:`/`originSessionId:` at top level) or missing the `metadata:` block — agent must redraft per the Memory drafting protocol.
|
||||||
|
- For `harness_edit`: confirm `auto_apply_eligible: false` (never auto-apply). Confirm `confidence ≥ 5`. Confirm `# Test verification` section names the test command. Confirm diff is ≤30 LOC and targets a single allowed harness file (see `agents/adam.md` §"Harness self-modification"). Run test suite before AND after applying — revert on any regression.
|
||||||
|
- Confirm `source_entries` is present in proposal frontmatter as a non-empty list (used for archive). Warn (do not refuse) if missing — legacy proposals from before v0.2.0 won't have it.
|
||||||
|
|
||||||
|
If any check fails, refuse to apply and ask the user how to proceed.
|
||||||
|
|
||||||
|
## §3a auto-apply, per type
|
||||||
|
|
||||||
|
For each id that passed §3 verification, print `id`, `target`, `confidence`, `blast_radius`, `cross_session_evidence`, `auto_apply_eligible`. Then apply:
|
||||||
|
|
||||||
|
- **`skill_new`**: `mkdir -p ~/.claude/skills/<slug>/`, then `Write` the proposal's `# Proposed change` body to `~/.claude/skills/<slug>/SKILL.md`. After write, print: "skill `<slug>` written to `~/.claude/skills/<slug>/SKILL.md` — activates immediately — Claude Code v2.1.0+ auto-hot-reloads user-level skills, no restart needed."
|
||||||
|
- **`memory`**: `Write` the proposal's `# Proposed change` body (which MUST include the auto-memory frontmatter — see "Memory drafting protocol" in `agents/adam.md`) to the path in `target`. Then update `MEMORY.md` index with a one-line pointer.
|
||||||
|
- **`nudge`**: low-blast auto-apply path. Single-session evidence is sufficient — skip the cross-session gate. Apply by invoking the helper (do NOT hand-write the JSON — the helper builds the message and the `cue` deterministically from frontmatter):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node ~/.claude/adam/scripts/adam-apply-nudge.mjs ~/.claude/adam/proposals/<id>-*.md --home ~/.claude
|
||||||
|
```
|
||||||
|
|
||||||
|
The proposal frontmatter MUST carry `type: nudge`, `source_session`, and `cwd` (the cluster's dominant `cwd`, from `adam-batch.mjs`'s per-batch `cwds`), plus `dead_end_count`. The helper appends one entry to `~/.claude/adam/active-nudges.json` (creating it if absent) with `cue: {cwd}` populated — so the reminder is query-conditioned: it surfaces at SessionStart *and* mid-session (via `adam-observe.mjs`) only when the user is working in that codebase. Output: `{"status":"applied"|"gated", ...}` — on `gated` leave the proposal in `proposals/`, on `applied` continue to archive. Do NOT modify any skill, memory, agent, or CLAUDE.md. Tell user: "nudge queued — surfaces when you return to `<cwd basename>` (expires in 7 days)."
|
||||||
|
- **`reinforcement`**: gated by `confidence >= 4 AND blast_radius == low` (same as memory). Apply by invoking the helper:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node ~/.claude/adam/scripts/adam-apply-reinforcement.mjs ~/.claude/adam/proposals/<id>-*.md --home ~/.claude
|
||||||
|
```
|
||||||
|
|
||||||
|
The helper reads the proposal frontmatter (`skill_slug`, `count`, `source_session`) and appends one JSON line to `~/.claude/adam/reinforcements.jsonl`. No code/memory/skill modifications. Output: `{"status":"applied"|"gated", ...}`. On `gated`, leave the proposal in `proposals/` (the helper failed its own re-check). On `applied`, continue to the archive step. Tell user: "reinforcement logged for `<skill_slug>` (count=<N>) — appended to reinforcements.jsonl."
|
||||||
|
- **`skill_edit`**: enforce the apply-time gate before writing.
|
||||||
|
1. Verify proposal frontmatter has `auto_apply_eligible: true`. If not, abort and queue for review.
|
||||||
|
2. Read `target` SKILL.md, capture `current_bytes` from a fresh stat — do NOT trust frontmatter `bytes_before`.
|
||||||
|
3. Verify diff in `# Proposed change`: unified-diff format, zero `-` lines on existing SKILL.md content (additions only), total `+` lines ≤ 30. If any check fails, print one-line refusal reason, leave proposal in `proposals/`, continue.
|
||||||
|
4. Cooldown re-check: run `node ~/.claude/adam/scripts/adam-cooldown.mjs --skill <target_skill> --fingerprint <proposal_fingerprint>` (both fields come from proposal frontmatter; missing fingerprint → "legacy"). Refuse if the script returns `status: cooldown` OR `status: blacklisted`. This per-(skill, fingerprint) gate replaces the previous coarse per-skill scan — proposals for the same skill with a different fingerprint are NOT blocked by an older entry. `adam-cooldown.mjs` returns `blacklisted` when `auto_apply_blacklist: true` is found in `rejected/` within 30 days for the same (skill, fingerprint).
|
||||||
|
5. Apply via `Edit` tool (append the new section per the diff). Never use `Write` on existing SKILL.md.
|
||||||
|
6. Re-stat target. If new size exceeds `2 * current_bytes` (captured in step 2), revert via `Edit` (remove the just-appended section) and refuse — print refusal reason.
|
||||||
|
7. Add `last_auto_edit: <iso8601 utc now>` to the proposal frontmatter before moving it.
|
||||||
|
8. Tell user: "skill `<slug>` extended (added <N> lines) — auto-applied via win-evidence gate."
|
||||||
|
|
||||||
|
## §3a after each auto-apply: move, A/B tracking, archive
|
||||||
|
|
||||||
|
1. Move proposal to `~/.claude/adam/applied/<UTC-ts>-<id>.md`.
|
||||||
|
2. **A/B tracking append** (skip for `reinforcement` — positive-only ledger, intentionally not A/B-tracked per `agents/adam.md` §"`reinforcement` proposals"): as a separate atomic step right after the move, append one JSON line to `~/.claude/adam/ab-tracking.jsonl` (create with empty contents if absent). Read fields from the proposal's frontmatter (`proposal_fingerprint`, `originating_signals` — both populated per `agents/adam.md`; `originating_signals` is a list of `{type, count, session_ids}` objects). Schema:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"applied_at": <unix_ms now>,
|
||||||
|
"proposal_id": "<id>",
|
||||||
|
"proposal_type": "skill_edit|skill_new|memory|nudge",
|
||||||
|
"target_skill": "<slug or target basename>",
|
||||||
|
"proposal_fingerprint": "<hash>",
|
||||||
|
"originating_signals": [{"type":"<signal>","count":<N>,"session_ids":[...]}],
|
||||||
|
"pre_window_days": 7
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`adam-ab-measure.mjs` consumes this entry on later `/reflect` runs to compute pre/post signal-count deltas. See `agents/adam.md` §"A/B effectiveness". If the append fails (disk-full etc.), log a warning but do NOT abort the apply path — A/B is observability, not a gate.
|
||||||
|
3. **Archive consumed journal entries**: `node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<UTC-ts>-<id>.md`. This moves the entries listed in the proposal's `source_entries` from `journal.jsonl` to `journal/actioned-<id>.jsonl`, so later `/reflect` runs do not re-cluster them.
|
||||||
|
|
||||||
|
After all ids, print: `auto-applied N proposals: [ids]`.
|
||||||
|
|
||||||
|
## §4 approve, per type
|
||||||
|
|
||||||
|
- **`claude_md_edit`**: back up first with `cp ~/.claude/CLAUDE.md ~/.claude/adam/applied/<ts>-claude-md-backup.md`.
|
||||||
|
- **`deletion`**: `mkdir -p ~/.claude/adam/trash/<ts>`, then `mv` the artifact into it. Print the restoration command.
|
||||||
|
- **`skill_new`**: `mkdir -p ~/.claude/skills/<slug>/`, then write the `# Proposed change` body to `<slug>/SKILL.md`. Tell user: "skill `<slug>` written — activates immediately (CC v2.1.0+ auto-hot-reload)."
|
||||||
|
- **`skill_edit`**: apply the unified diff in `# Proposed change` to the existing SKILL.md at `target` (append-only — never replace existing content).
|
||||||
|
- **`memory`**: write the `# Proposed change` body (must include auto-memory frontmatter) to `target` and update the `MEMORY.md` index with a one-line pointer.
|
||||||
|
- **`harness_edit`** (MOSS §1): apply the unified diff to the target harness file. **Before applying**:
|
||||||
|
1. Run `bash ~/.claude/adam/tests/run-tests.sh` — capture pass count.
|
||||||
|
2. Apply the diff via `Edit`.
|
||||||
|
3. Run `bash ~/.claude/adam/tests/run-tests.sh` again — verify pass count is equal or higher and 0 failures.
|
||||||
|
4. If test regression: revert the edit, print "harness_edit reverted — test regression detected", leave proposal in `proposals/`.
|
||||||
|
5. If tests pass: tell user "harness edit applied to `<target>` — tests pass (<N> passed)."
|
||||||
|
- **All other types**: apply via Write/Edit per the proposal's `# Proposed change`.
|
||||||
|
|
||||||
|
Then:
|
||||||
|
|
||||||
|
1. Move proposal to `~/.claude/adam/applied/<ts>-<id>.md`.
|
||||||
|
2. Archive: `node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<ts>-<id>.md`.
|
||||||
|
|
||||||
|
## §4 reject and edit
|
||||||
|
|
||||||
|
- **Reject**: ask for the reason in one line. Append `# Reason\n<reason>` to the proposal body. If the proposal `type` is `skill_edit`, ALSO add `auto_apply_blacklist: true` to its frontmatter (so future reflects skip auto-apply on this target for 30 days). Move to `~/.claude/adam/rejected/<id>.md`. Archive: `node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/rejected/<id>.md`.
|
||||||
|
- **Edit**: ask the user for the change, edit the proposal in place, then loop back to §4 step a for that same id.
|
||||||
Reference in New Issue
Block a user