The 2026-08-30 suite-health review found 223/283 tests saturated. These
16 target the failure modes that still discriminated, one per existing
subcategory across all eight categories:
- multi-constraint reasoning: infeasible-plan detection, conflicting
delegation policy, adversarial tool descriptions
- forensic root-causing from noisy evidence: RAG stage attribution,
EXPLAIN ANALYZE with a red herring, interleaved multi-source incident
logs, README-vs-config conflict resolution
- composed gotchas: a Go 1.22 capture + named-return-defer trace, a
Python late-binding + shared-mutable-default trace
- constraint math: context-budget packing, capacity-fill projection,
MVCC anomaly classification, Docker layer-rebuild prediction, dual-key
rotation overlap, freeze-window scheduling, an open-redirect exploit
chain
Every answer is a structured JSON value scored by an exact evaluator,
never a prose heuristic (the source of every false positive in the
review). A ground-truth self-consistency test asserts each documented
correct answer scores 1.0 through its real evaluator.
lukaszraczylo
708622e2a8