llm-testbench/config.example.yaml at 764fe990fd2d055205b2f766d56ec422c9341eff - llm-testbench - Gitea — home-cluster

lukaszraczylo/llm-testbench

mirror of https://github.com/lukaszraczylo/llm-testbench.git synced 2026-10-12 00:45:19 +00:00

Files

T

lukaszraczylo 1d51d4cbbd feat: LLM accuracy testing framework with 17-test catalog

Framework runs a category/subcategory test catalog against multiple
LLMs through an OpenAI-compatible endpoint. Deterministic evaluators
score each response. Reports render as table, markdown, or JSON.

2026-08-17 01:29:45 +01:00

10 lines

238 B

YAML

Raw Blame History

 endpoint: https://llm-gateway.example.com/v1
 api_key: ""            # optional; env LLMTB_API_KEY wins
 models:
   - uni/deepseek-v4-flash-0731
   - uni/qwen3.6-27b
   - uni/btl-4
 concurrency: 4
 request_timeout: 120s
 max_tokens_default: 4000