test(catalog): add ai category with 30 tests

Add a new ai test category with three subcategories: vector-search,
llm-integration, and rag. Each subcategory registers 10 tests, for
30 tests total.

Vector-search tests cover cosine-vs-dot equivalence, recall@k,
HNSW efSearch tradeoffs, product-quantization memory math, ANN
pre/post filtering, Reciprocal Rank Fusion, distance-to-similarity
conversion, near-duplicate thresholds, and index/embedding-dimension
tradeoffs.

LLM-integration tests cover OpenAI-compatible field semantics, token
budget math for reasoning models, 429 retry/backoff, context-overflow
strategy, tool-call handling, role placement, the temperature=0
determinism caveat, SSE stream parsing, embedding batch math, and
stop-sequence behavior.

RAG tests cover chunk-size tradeoffs, reranker placement, retrieval
failure modes, citation grounding, context selection versus stuffing,
index staleness, evaluation metric choice, hallucination mitigation
ordering, multi-hop decomposition, and pre-assembly deduplication.

Every numeric test derives its expected value from an ai-prefixed
helper or an existing mathutil/whitepapers helper, and each unit test
independently recomputes the same value with plain arithmetic. New
files only: ai.go, ai_vectorsearch.go, ai_llmintegration.go, ai_rag.go,
and their _test.go siblings, plus a wiring test that calls
registerAITests directly. catalog.go and catalog_test.go are
untouched; the orchestrator wires registration in at integration.
This commit is contained in:
2026-08-17 02:16:57 +01:00
parent 8ab83a57ca
commit d4b3d4601f
8 changed files with 2181 additions and 0 deletions
+14
View File
@@ -0,0 +1,14 @@
package tests
import "github.com/lukaszraczylo/llm-testbench/internal/testkit"
// registerAITests registers every ai-category test. Each subcategory -
// vector-search, llm-integration, and rag - has its own register function
// and source file (ai_vectorsearch.go, ai_llmintegration.go, ai_rag.go) to
// keep any one file from growing past a few hundred lines, mirroring
// agents.go's split across its three subcategories.
func registerAITests(r *testkit.Registry) {
registerAIVectorSearchTests(r)
registerAILLMIntegrationTests(r)
registerAIRAGTests(r)
}