mirror of
https://github.com/lukaszraczylo/claude-adam.git
synced 2026-07-10 03:53:03 +00:00
fix(v0.6.2): A/B volume normalization + memory frontmatter schema
Two issues surfaced by running ADAM's /reflect loop on a large real journal
(4015 entries, 119 sessions) — both caused false/broken auto-apply behavior.
1. A/B over-reported regressions (adam-ab-measure.mjs).
Regressions were measured on RAW originating-signal counts pre vs post. On a
busy, growing journal almost every signal count rises post-apply regardless
of whether the proposal helped — so the loop flagged 9 false "regressions"
(and would auto-roll-back good proposals). Now the delta is computed on the
signal's SHARE of total activity (rate = count / window-total). Falls back to
the raw-count delta when the signal is the only activity in the window
(preserves prior behavior + all existing A/B tests). Output adds
raw_delta_pct, pre_total, post_total, normalized for transparency.
2. Memory frontmatter drift (agents/adam.md, SKILL.md).
The drafting protocol emitted flat `type:`/`originSessionId:` with a prose
`name`, but the live auto-memory store uses `name` = slug plus a
`metadata: {node_type, type, originSessionId}` block. Auto-applied memories
could fail to load/categorize. Protocol + apply-time validation now require
the live metadata.* schema and cross-checking against an existing file.
Tests: 132 -> 134. New: volume growth (raw +200%) with flat activity-share
classifies neutral, not regressed; a genuine share increase still classifies
regressed.
This commit is contained in:
@@ -13,7 +13,7 @@ Watches the friction in your coding sessions, clusters the signals via an LLM an
|
||||
|
||||
[](LICENSE)
|
||||
[](https://github.com/lukaszraczylo/claude-adam/releases)
|
||||
[](./adam/tests/run-tests.sh)
|
||||
[](./adam/tests/run-tests.sh)
|
||||
[](https://nodejs.org)
|
||||
[]()
|
||||
|
||||
@@ -54,7 +54,7 @@ The installer copies files into `~/.claude/`, offers to merge ADAM's hook entrie
|
||||
Then:
|
||||
|
||||
```sh
|
||||
bash ~/.claude/adam/tests/run-tests.sh # expect: 132 passed, 0 failed
|
||||
bash ~/.claude/adam/tests/run-tests.sh # expect: 134 passed, 0 failed
|
||||
# … start a fresh Claude Code session …
|
||||
/reflect # walks the proposal queue
|
||||
/reflect --explain # also shows the analyst's clustering trace
|
||||
@@ -248,11 +248,12 @@ Or pass `--explain` to `/reflect` to render the full trace inline.
|
||||
│ ├── adam-apply-reinforcement.mjs # reinforcement proposal apply
|
||||
│ ├── adam-upgrade.mjs # .adam-new file UX (list/diff/accept)
|
||||
│ └── adam-archive.mjs # post-apply journal cleanup
|
||||
└── tests/run-tests.sh # 132 isolated tests; never touches live state
|
||||
└── tests/run-tests.sh # 134 isolated tests; never touches live state
|
||||
```
|
||||
|
||||
## What's new
|
||||
|
||||
- **v0.6.2** — two fixes surfaced by running ADAM's loop on a large real journal. **(1) A/B volume normalization** (`adam-ab-measure.mjs`): regressions are now measured on the signal's *share* of total activity (rate = count / window-total), not raw count — so a generally busier journal after an apply no longer masquerades as a regression. Falls back to raw delta when the signal is the only activity in the window (preserves prior behavior + tests); output adds `raw_delta_pct`, `pre_total`, `post_total`, `normalized` for transparency. **(2) Memory frontmatter schema** (`agents/adam.md`, `SKILL.md`): the drafting protocol now emits the live auto-memory shape — `name` = slug + a `metadata: {node_type, type, originSessionId}` block — instead of flat `type:`/`originSessionId:`, so auto-applied memories load and categorize correctly. 134 tests (up from 132).
|
||||
- **v0.6.1** — new `file_reread` signal (MOSS §1 harness self-modification, proposed and approved through ADAM's own `/reflect` loop). Consecutive Reads of the same file at different `offset`/`limit` escaped `retry_loop`'s arg-hash dedup and leaked into `tool_error_loop`; `file_reread` now catches them (same file ≥3× in the 10-event window, offset-agnostic, guarded against double-counting byte-identical reads). Fully wired: detection (`adam-observe.mjs`), 14-day window (`adam-window.mjs`), severity divisor 3 (`adam-score.mjs`), file-basename clustering (`adam-batch.mjs`), and the analyst rubric/spec. 132 tests (up from 126).
|
||||
- **v0.6.0** — review hardening. Struggle signals now emit `active_skills`, so `silent_drift`'s primary cluster key and the §5b skill-attribution sub-clustering (+1 rubric bonus) actually fire (both were silently dead). `proposal_fingerprint` is now deterministically computable via `adam-cooldown.mjs --compute` instead of asking the LLM analyst to hand-compute a djb2 hash; spec now mandates a *stable* cluster id so fingerprints reproduce across runs. `reinforcement` proposals are correctly excluded from A/B tracking (the spec previously contradicted itself). `adam-nudge.mjs` pending-upgrade check now mirrors the full install set (`adam-utils`/`adam-batch`/`adam-rollback` were missing). Doc/test-count drift corrected. 126 tests (up from 114).
|
||||
- **v0.5.0** — MOSS-grounded self-evolution (arXiv 2605.22794). Transcript capture: `context_window` field on struggle signals captures 8 surrounding events for evidence-based diagnosis. Two-stage analysis pipeline: diagnose+plan → inter-stage validation → implement (§3.3). Evidence batching via `adam-batch.mjs`: pre-clusters journal into coherent failure batches (§3.1). Pre-apply verification: 4-check deterministic gate before auto-apply (§3.4). Auto-rollback via `adam-rollback.mjs`: reverts regressed proposals detected by A/B measurement, creates regression nudges (§3.5). Harness self-modification: new `harness_edit` proposal type lets ADAM propose edits to its own scripts with test-suite-gated apply (§1 Table 1). Keypoint matrix: 5 capability dimensions scored per batch for structured evaluation (§4.2). 114 tests (up from 94).
|
||||
|
||||
Reference in New Issue
Block a user