lukaszraczylo 6934d4df84 fix(eval): bound exec evaluators and pin their environment ...

Limit concurrent compile and run subprocesses. Heavy load no longer pushes correct solutions past the timeout.
Build Go code before it runs, so a cold compile does not use the run timeout.
Cap captured output at 1 MiB. Set a wait delay so a child that holds a pipe cannot hang the run.
Report a timeout as "timed out after Ns".
Pin TZ, PYTHONHASHSEED, PYTHONUTF8 and GOTOOLCHAIN for repeatable output.

2026-10-02 01:01:00 +01:00

..

composite_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

composite.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

contains_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

contains.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

equals_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

equals.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

eval.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

exacttoken_test.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

exacttoken.go

fix(catalog): harden evaluators against correct-answer phrasing variance

2026-08-17 02:48:05 +01:00

exec_test.go

fix(eval): bound exec evaluators and pin their environment

2026-10-02 01:01:00 +01:00

exec.go

fix(eval): bound exec evaluators and pin their environment

2026-10-02 01:01:00 +01:00

extract_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

extract.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

jsonarray.go

fix: floor token budgets and surface truncation, add live progress

2026-08-17 01:55:14 +01:00

jsonfield_test.go

fix(catalog): unify negation guard and harden round-2 evaluators

2026-08-17 03:50:57 +01:00

jsonfield.go

fix(catalog): unify negation guard and harden round-2 evaluators

2026-08-17 03:50:57 +01:00

negation_test.go

fix(eval): remove four negation-detector false-positive classes

2026-08-30 14:18:53 +01:00

negation.go

fix(eval): remove four negation-detector false-positive classes

2026-08-30 14:18:53 +01:00

numeric_test.go

fix(catalog): unify negation guard and harden round-2 evaluators

2026-08-17 03:50:57 +01:00

numeric.go

fix(catalog): unify negation guard and harden round-2 evaluators

2026-08-17 03:50:57 +01:00

regex_test.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

regex.go

feat: LLM accuracy testing framework with 17-test catalog

2026-08-17 01:29:45 +01:00

toolcall_test.go

feat: function-calling (MCP-style) tests plus ultra-hard tasks

2026-08-30 20:22:20 +01:00

toolcall.go

feat: function-calling (MCP-style) tests plus ultra-hard tasks

2026-08-30 20:22:20 +01:00