mirror of
https://github.com/lukaszraczylo/llm-testbench.git
synced 2026-10-12 00:45:19 +00:00
feat: LLM accuracy testing framework with 17-test catalog
Framework runs a category/subcategory test catalog against multiple LLMs through an OpenAI-compatible endpoint. Deterministic evaluators score each response. Reports render as table, markdown, or JSON.
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
endpoint: https://llm-gateway.example.com/v1
|
||||
api_key: "" # optional; env LLMTB_API_KEY wins
|
||||
models:
|
||||
- uni/deepseek-v4-flash-0731
|
||||
- uni/qwen3.6-27b
|
||||
- uni/btl-4
|
||||
concurrency: 4
|
||||
request_timeout: 120s
|
||||
max_tokens_default: 4000
|
||||
Reference in New Issue
Block a user