llm-testbench

mirror of https://github.com/lukaszraczylo/llm-testbench.git synced 2026-10-12 00:45:19 +00:00

Files

T

lukaszraczylo 499956f8cc fix(tests): disambiguate two prompts, exempt --force-if-includes

git-detached-head-recovery: all three benchmarked models answered
["branch"], which the prompt's literal ask (save the commits) makes
correct while the evaluator requires ending attached. The prompt now
demands ending checked out on the new branch and names the bare-branch
gap explicitly.

scen-crashloop-order: the instruction excluded only acting steps, so
including the read-only journalctl step E followed the letter of the
prompt. The instruction now asks for the smallest sufficient set and
excludes redundant diagnostics.

git-force-with-lease-vs-force: exempt --force-if-includes, the
stricter safe variant, from the bare --force pattern.

2026-08-30 14:19:08 +01:00

config

fix: floor token budgets and surface truncation, add live progress

2026-08-17 01:55:14 +01:00

eval

fix(eval): remove four negation-detector false-positive classes

2026-08-30 14:18:53 +01:00

llm

fix: floor token budgets and surface truncation, add live progress

2026-08-17 01:55:14 +01:00

report

Update