lukaszraczylo 7264fe81b7 fix(catalog): harden evaluators against correct-answer phrasing variance ...

Both reviewers found evaluators that scored correct answers as zero:
hardcoded flag order, quote-style rigidity, missing multiline anchors,
comma-grouped numbers, ratio notation, and numeric strings in JSON.
A shared ExactToken evaluator now normalizes fences, quotes, and
trailing periods for exact-answer tests. The obsolete tmutil snapshot
reference now accepts the current localsnapshot verb.

2026-08-17 02:48:05 +01:00

..

agents_delegation_test.go

test(catalog): expand agents to 10 tests per subcategory

2026-08-17 01:47:36 +01:00

agents_delegation.go

test(catalog): expand agents to 10 tests per subcategory

2026-08-17 01:47:36 +01:00

agents_planning_test.go

test(catalog): expand agents to 10 tests per subcategory

2026-08-17 01:47:36 +01:00

agents_planning.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

agents_test.go

test(catalog): expand agents to 10 tests per subcategory

2026-08-17 01:47:36 +01:00

agents_toolrouting_test.go

test(catalog): expand agents to 10 tests per subcategory

2026-08-17 01:47:36 +01:00

agents_toolrouting.go

test(catalog): expand agents to 10 tests per subcategory

2026-08-17 01:47:36 +01:00

agents.go

test(catalog): expand agents to 10 tests per subcategory

2026-08-17 01:47:36 +01:00

c_test.go

test(catalog): expand programming to 10 tests per subcategory

2026-08-17 01:54:39 +01:00

c.go

test(catalog): expand programming to 10 tests per subcategory

2026-08-17 01:54:39 +01:00

catalog_test.go

test(catalog): assert at least 10 tests per subcategory

2026-08-17 01:58:04 +01:00

catalog.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

codebase_analysis_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

codebase_analysis.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

codebase_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

codebase_trace_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

codebase_trace.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

codebase.go

test(catalog): expand research to 10 tests per subcategory

2026-08-17 01:51:11 +01:00

golang_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

golang.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

kubernetes_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

kubernetes.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

linux_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

linux.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

macos_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

macos.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

mathutil.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

python_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

python.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

structlayout_test.go

fix: floor token budgets and surface truncation, add live progress

2026-08-17 01:55:14 +01:00

structlayout.go

fix: floor token budgets and surface truncation, add live progress

2026-08-17 01:55:14 +01:00

typescript_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

typescript.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

web_extract_test.go

test(catalog): expand research to 10 tests per subcategory

2026-08-17 01:51:11 +01:00

web_extract.go

test(catalog): expand research to 10 tests per subcategory

2026-08-17 01:51:11 +01:00

web_protocol_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

web_protocol.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

web_security_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

web_security.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

web_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

web.go

test(catalog): expand research to 10 tests per subcategory

2026-08-17 01:51:11 +01:00

whitepapers_compute.go

test(catalog): expand research to 10 tests per subcategory

2026-08-17 01:51:11 +01:00

whitepapers_systems_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

whitepapers_systems.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

whitepapers_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

whitepapers_vector_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

whitepapers_vector.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

whitepapers.go

test(catalog): expand research to 10 tests per subcategory

2026-08-17 01:51:11 +01:00