mirror of
https://github.com/lukaszraczylo/llm-testbench.git
synced 2026-10-12 00:45:19 +00:00
lukaszraczylo
d4b3d4601f
Add a new ai test category with three subcategories: vector-search, llm-integration, and rag. Each subcategory registers 10 tests, for 30 tests total. Vector-search tests cover cosine-vs-dot equivalence, recall@k, HNSW efSearch tradeoffs, product-quantization memory math, ANN pre/post filtering, Reciprocal Rank Fusion, distance-to-similarity conversion, near-duplicate thresholds, and index/embedding-dimension tradeoffs. LLM-integration tests cover OpenAI-compatible field semantics, token budget math for reasoning models, 429 retry/backoff, context-overflow strategy, tool-call handling, role placement, the temperature=0 determinism caveat, SSE stream parsing, embedding batch math, and stop-sequence behavior. RAG tests cover chunk-size tradeoffs, reranker placement, retrieval failure modes, citation grounding, context selection versus stuffing, index staleness, evaluation metric choice, hallucination mitigation ordering, multi-hop decomposition, and pre-assembly deduplication. Every numeric test derives its expected value from an ai-prefixed helper or an existing mathutil/whitepapers helper, and each unit test independently recomputes the same value with plain arithmetic. New files only: ai.go, ai_vectorsearch.go, ai_llmintegration.go, ai_rag.go, and their _test.go siblings, plus a wiring test that calls registerAITests directly. catalog.go and catalog_test.go are untouched; the orchestrator wires registration in at integration.
15 lines
562 B
Go
15 lines
562 B
Go
package tests
|
|
|
|
import "github.com/lukaszraczylo/llm-testbench/internal/testkit"
|
|
|
|
// registerAITests registers every ai-category test. Each subcategory -
|
|
// vector-search, llm-integration, and rag - has its own register function
|
|
// and source file (ai_vectorsearch.go, ai_llmintegration.go, ai_rag.go) to
|
|
// keep any one file from growing past a few hundred lines, mirroring
|
|
// agents.go's split across its three subcategories.
|
|
func registerAITests(r *testkit.Registry) {
|
|
registerAIVectorSearchTests(r)
|
|
registerAILLMIntegrationTests(r)
|
|
registerAIRAGTests(r)
|
|
}
|