2 Commits
Author SHA1 Message Date
lukaszraczylo 9d1d4d89e9 fix(catalog): unify negation guard and harden round-2 evaluators
A shared NoUnnegatedMention primitive replaces four per-category
implementations, with a bidirectional clause-scoped window and an
adjectival cue set. The docker layer fixture drops WORKDIR, which
creates a real layer and made the pinned answer wrong. The replica
failover test now forces a JSON verdict so a negated answer cannot
score. Remaining exact-answer tests migrate to ExactToken. Ordering
prompts shuffle their candidate lists so echoing scores zero.
2026-08-17 03:50:57 +01:00
lukaszraczylo e455eeecc7 test(catalog): add delivery category with 30 tests
Add a new delivery category with three subcategories: git, containers,
and release-engineering. Each subcategory holds 10 self-contained,
deterministically-scored tests, grounded in real git, Docker, and
release-engineering behaviour.

The git subcategory covers bisect step derivation, rebase versus merge,
conventional-commit classification, git worktree, force-with-lease
versus bare force, hook choice, detached-HEAD recovery, cherry-pick
versus revert, a .gitignore negation trace, and stash-pop conflict
behaviour. Two of these tests recompute their ground truth against a
real, throwaway git repository created in t.TempDir, and skip if git is
not installed.

The containers subcategory covers Dockerfile layer-cache busting,
ENTRYPOINT and CMD interaction, multi-stage image size, non-root users
and capabilities, HEALTHCHECK semantics, .dockerignore effects, COPY
versus ADD, build-arg versus runtime env, and layer counting.

The release-engineering subcategory covers semver bump derivation from
a changelog, the clean-tree requirement, rollback ordering, canary
versus blue-green, tag-to-release sequencing, commit-to-changelog
mapping, checksum and signing purpose, pipeline stage ordering, pin
versus range dependency policy, and hotfix flow ordering.

A shared negation-aware evaluator helper, local to this category's
files, scores a forbidden phrase only when it appears unnegated, so a
response correctly warning against bare --force or a silent
tracked-file mutation still scores full credit.

registerDeliveryTests wires the three subcategories together and is
exercised only by a new wiring test in this worktree, since catalog.go
and catalog_test.go stay untouched until the orchestrator integrates
this category centrally.
2026-08-17 02:28:01 +01:00