lukaszraczylo/llm-testbench
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/llm-testbench.git synced 2026-10-12 00:45:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
7264fe81b7b095bb8e706c193e08dbee1e7da3cf
llm-testbench/internal
T
History
lukaszraczylo 7264fe81b7 fix(catalog): harden evaluators against correct-answer phrasing variance 
Both reviewers found evaluators that scored correct answers as zero:
hardcoded flag order, quote-style rigidity, missing multiline anchors,
comma-grouped numbers, ratio notation, and numeric strings in JSON.
A shared ExactToken evaluator now normalizes fences, quotes, and
trailing periods for exact-answer tests. The obsolete tmutil snapshot
reference now accepts the current localsnapshot verb.
2026-08-17 02:48:05 +01:00
..
config
fix: floor token budgets and surface truncation, add live progress
2026-08-17 01:55:14 +01:00
eval
fix(catalog): harden evaluators against correct-answer phrasing variance
2026-08-17 02:48:05 +01:00
llm
fix: floor token budgets and surface truncation, add live progress
2026-08-17 01:55:14 +01:00
report
fix: floor token budgets and surface truncation, add live progress
2026-08-17 01:55:14 +01:00
runner
fix: floor token budgets and surface truncation, add live progress
2026-08-17 01:55:14 +01:00
testkit
feat: LLM accuracy testing framework with 17-test catalog
2026-08-17 01:29:45 +01:00
tests
fix(catalog): harden evaluators against correct-answer phrasing variance
2026-08-17 02:48:05 +01:00