fix: floor token budgets and surface truncation, add live progress

mirror of https://github.com/lukaszraczylo/llm-testbench.git synced 2026-10-12 00:45:19 +00:00

Reasoning models burn thousands of completion tokens before the
answer. Small per-test caps caused silent truncation and zero scores.
The runner now floors max_tokens at the config default (12000). The
client parses finish_reason. Reports mark truncated cells and the
JSON format carries response text for debugging. A stderr progress
reporter prints one line per completed request; --quiet disables it.

This commit is contained in:

lukaszraczylo

2026-08-17 01:55:14 +01:00

parent 764fe990fd

commit ace50757e4

30 changed files with 1063 additions and 184 deletions

.gitignore

View File

@@ -10,3 +10,4 @@
 # OS/editor cruft
 .DS_Store
 *.swp
 .claude/