mirror of
https://github.com/lukaszraczylo/llm-testbench.git
synced 2026-10-12 00:45:19 +00:00
lukaszraczylo
d4b3d4601f
Add a new ai test category with three subcategories: vector-search, llm-integration, and rag. Each subcategory registers 10 tests, for 30 tests total. Vector-search tests cover cosine-vs-dot equivalence, recall@k, HNSW efSearch tradeoffs, product-quantization memory math, ANN pre/post filtering, Reciprocal Rank Fusion, distance-to-similarity conversion, near-duplicate thresholds, and index/embedding-dimension tradeoffs. LLM-integration tests cover OpenAI-compatible field semantics, token budget math for reasoning models, 429 retry/backoff, context-overflow strategy, tool-call handling, role placement, the temperature=0 determinism caveat, SSE stream parsing, embedding batch math, and stop-sequence behavior. RAG tests cover chunk-size tradeoffs, reranker placement, retrieval failure modes, citation grounding, context selection versus stuffing, index staleness, evaluation metric choice, hallucination mitigation ordering, multi-hop decomposition, and pre-assembly deduplication. Every numeric test derives its expected value from an ai-prefixed helper or an existing mathutil/whitepapers helper, and each unit test independently recomputes the same value with plain arithmetic. New files only: ai.go, ai_vectorsearch.go, ai_llmintegration.go, ai_rag.go, and their _test.go siblings, plus a wiring test that calls registerAITests directly. catalog.go and catalog_test.go are untouched; the orchestrator wires registration in at integration.