llm-testbench

mirror of https://github.com/lukaszraczylo/llm-testbench.git synced 2026-10-12 00:45:19 +00:00

Files

T

lukaszraczylo 6934d4df84 fix(eval): bound exec evaluators and pin their environment

Limit concurrent compile and run subprocesses. Heavy load no longer pushes correct solutions past the timeout.
Build Go code before it runs, so a cold compile does not use the run timeout.
Cap captured output at 1 MiB. Set a wait delay so a child that holds a pipe cannot hang the run.
Report a timeout as "timed out after Ns".
Pin TZ, PYTHONHASHSEED, PYTHONUTF8 and GOTOOLCHAIN for repeatable output.

2026-10-02 01:01:00 +01:00

config

feat(config): raise max_tokens_default to 40000

2026-08-30 14:19:22 +01:00

eval

fix(eval): bound exec evaluators and pin their environment

2026-10-02 01:01:00 +01:00

llm

feat: function-calling (MCP-style) tests plus ultra-hard tasks

2026-08-30 20:22:20 +01:00

report

Update