mirror of
https://github.com/lukaszraczylo/llm-testbench.git
synced 2026-10-12 00:45:19 +00:00
test(catalog): add ai category with 30 tests
Add a new ai test category with three subcategories: vector-search, llm-integration, and rag. Each subcategory registers 10 tests, for 30 tests total. Vector-search tests cover cosine-vs-dot equivalence, recall@k, HNSW efSearch tradeoffs, product-quantization memory math, ANN pre/post filtering, Reciprocal Rank Fusion, distance-to-similarity conversion, near-duplicate thresholds, and index/embedding-dimension tradeoffs. LLM-integration tests cover OpenAI-compatible field semantics, token budget math for reasoning models, 429 retry/backoff, context-overflow strategy, tool-call handling, role placement, the temperature=0 determinism caveat, SSE stream parsing, embedding batch math, and stop-sequence behavior. RAG tests cover chunk-size tradeoffs, reranker placement, retrieval failure modes, citation grounding, context selection versus stuffing, index staleness, evaluation metric choice, hallucination mitigation ordering, multi-hop decomposition, and pre-assembly deduplication. Every numeric test derives its expected value from an ai-prefixed helper or an existing mathutil/whitepapers helper, and each unit test independently recomputes the same value with plain arithmetic. New files only: ai.go, ai_vectorsearch.go, ai_llmintegration.go, ai_rag.go, and their _test.go siblings, plus a wiring test that calls registerAITests directly. catalog.go and catalog_test.go are untouched; the orchestrator wires registration in at integration.
This commit is contained in:
@@ -0,0 +1,14 @@
|
||||
package tests
|
||||
|
||||
import "github.com/lukaszraczylo/llm-testbench/internal/testkit"
|
||||
|
||||
// registerAITests registers every ai-category test. Each subcategory -
|
||||
// vector-search, llm-integration, and rag - has its own register function
|
||||
// and source file (ai_vectorsearch.go, ai_llmintegration.go, ai_rag.go) to
|
||||
// keep any one file from growing past a few hundred lines, mirroring
|
||||
// agents.go's split across its three subcategories.
|
||||
func registerAITests(r *testkit.Registry) {
|
||||
registerAIVectorSearchTests(r)
|
||||
registerAILLMIntegrationTests(r)
|
||||
registerAIRAGTests(r)
|
||||
}
|
||||
Reference in New Issue
Block a user