3 Commits

Author	SHA1	Message	Date
lukaszraczylo	5c2685c7b6	feat(leann-phase2): implement hybrid vector storage and graph-based search (#20) ... * feat(leann-phase2): implement hybrid vector storage and graph-based search - [x] Add AST-aware code chunking for Go, Python, and TypeScript using tree-sitter - [x] Implement LEANN-inspired hybrid vector storage with hub detection and selective embedding storage (60-80% savings) - [x] Add observation relationship graph with CSR format and edge detection (file overlap, semantic similarity, temporal, concept) - [x] Implement graph-aware search with two-level traversal and relationship-based ranking - [x] Add auto-tuning system for dynamic hub threshold adjustment based on query performance - [x] Add comprehensive metrics tracking for vector storage, queries, latency, and graph traversals - [x] Update configuration system with graph and hybrid storage settings - [x] Add graph stats and vector metrics endpoints to worker service - [x] Enhance UI sidebar with advanced metrics display and graph visualization - [x] Optimize struct field alignment throughout codebase for memory efficiency - [x] Update documentation with LEANN Phase 2 features and performance benefits - [x] Add tree-sitter dependency for AST parsing * fix: add fts5 build tag to CI workflow Pass build-tags: "fts5" to shared workflow to properly compile sqlite-vec-go-bindings with SQLite FTS5 support. This fixes test failures in hybrid vector storage tests that require CGO and FTS5 build tags. Requires shared-actions@8f7f235 or later. * docs: add testing documentation and macOS ARM64 known issue Document the macOS ARM64 CGO linking issue with sqlite-vec-go-bindings that prevents hybrid package tests from compiling locally. Added: - .github/TESTING.md: Comprehensive testing guide with platform-specific issues, workarounds, and CI configuration details - internal/vector/hybrid/README.md: Package-specific documentation explaining the macOS limitation - .github/CI_FIX_SUMMARY.md: Technical details of the CI fix Key points: - 41 out of 42 packages test successfully on all platforms - hybrid package tests fail only on macOS ARM64 (local dev issue) - Linux CI tests pass with proper build-tags: "fts5" configuration - Production builds and runtime functionality unaffected This is a known limitation of sqlite-vec-go-bindings on macOS ARM64 and does not impact CI/CD or production deployments. * fix: add SQLite busy_timeout to prevent database locked errors Set PRAGMA busy_timeout=5000 (5 seconds) to allow SQLite to retry when the database is locked instead of failing immediately. This fixes race conditions when multiple goroutines try to write simultaneously, particularly in tests where StoreObservation spawns async cleanup goroutines. Root cause: - StoreObservation launches goroutine -> CleanupOldObservations - Multiple concurrent cleanups caused "database is locked" errors - Without busy_timeout, SQLite fails immediately on lock contention Solution: - Add 5-second busy timeout for automatic retry on lock - Standard practice for concurrent SQLite usage - Works with existing WAL mode configuration Fixes TestObservationStore_CleanupOldObservations in CI. * docs: complete summary of all CI test fixes Comprehensive documentation of all fixes applied: 1. Missing build tags (fts5) 2. Database locked errors (busy_timeout) All 41/42 packages now pass tests. The hybrid package has a known macOS ARM64 limitation that doesn't affect CI or production. No functionality was removed - all fixes are additive only. * fix: add SQLite driver import to hybrid tests for CGO linking Add blank import of mattn/go-sqlite3 to hybrid test files to ensure the SQLite driver is linked into the test binary. This provides the SQLite symbols that sqlite-vec-go-bindings requires. Root cause: - hybrid package imports sqlitevec (transitively depends on sqlite-vec CGO) - Test binary needs SQLite symbols for linking - sqlitevec tests already had this import, but hybrid tests didn't - Without the driver import, linker fails with "undefined symbols" This fix enables hybrid tests to run with -race flag on all platforms. Before: 41/42 packages pass (hybrid failed to link) After: 42/42 packages pass ✅ Fixes hybrid test compilation on macOS ARM64, Linux, and Windows. * docs: remove outdated macOS limitation documentation The hybrid test linking issue has been fixed by adding the SQLite driver import. All tests now pass on all platforms including macOS. Removed: - internal/vector/hybrid/README.md (documented workaround no longer needed) - .github/TESTING.md (macOS limitation section obsolete) All 42/42 packages now test successfully with -race flag. * docs: final comprehensive summary of all CI fixes All three issues now resolved: 1. Missing fts5 build tags 2. Database busy_timeout for concurrent writes 3. Missing SQLite driver import in hybrid tests Result: 42/42 packages pass with -race on all platforms. Credit to reviewer for identifying the race detector concern.	2026-01-07 22:03:59 +00:00
lukaszraczylo	7a061c85eb	general improvements (#17) ... * refactor(hooks): simplify hook execution with shared context - [x] Extract BaseInput struct to eliminate duplicate fields across hooks - [x] Create RunHook handler pattern for session-start and user-prompt - [x] Create RunStatuslineHook for fast statusline rendering without worker startup - [x] Add HookContext struct to pass port, project, CWD, SessionID to handlers - [x] Add db/interface.go with ObservationReader/Writer interfaces - [x] Add comprehensive conflict management tests in sqlite/conflict_test.go - [x] Add vector client tests for Count, ModelVersion, NeedsRebuild, GetStaleVectors - [x] Add FilterByThreshold helper tests for query result filtering - [x] Make handlers_test more robust for network-dependent update checks - [x] Update package versions in UI * Move to GORM + general cleanup * feat(mcp): add observation relations discovery and scoring integration - [x] Add find_related_observations MCP tool for discovering related observations by confidence - [x] Integrate scoring calculator and recalculator into MCP server initialization - [x] Add pattern, relation, and session stores to MCP server dependencies - [x] Register MCP server in Claude Code settings during plugin installation - [x] Update install scripts (bash, PowerShell) to configure MCP server settings - [x] Switch plugin manifest files to template-based versioning (plugin.json.tpl, marketplace.json.tpl) - [x] Update all MCP server tests to pass new dependency parameters	2026-01-07 00:26:20 +00:00
lukaszraczylo	f79782a008	Release dec 2025 (#15) ... * Resolves issue #13 - Switched model to bge-small-en-v1.5 - Added lazy re-embedding - Added model version tracking per vector - Added conversion of vectors to the new model * Add lfs support to the workflow. * Implements importance scoring with decay + voting #6 * Resolves issue #5 by marking observations as superseeded and scheduled for deletion * Implement pattern detection #7 * Improve injections and observations accuracy - Session start: Recent observations for project context (recency-based) - User prompt: Semantically relevant observations (similarity-based with threshold) * Added two stage retrieval with bi and cross encoder #8 * Implement query expansion and reformulation #9 * Knowledge graph and relationships ( resolves #4 ) - File Overlap Detection: Detects relationships when observations modify/read the same files - Concept Overlap Detection: Detects relationships based on shared semantic concepts - Type Progression Detection: Infers relationships from natural observation type progressions (e.g., discovery → bugfix = "fixes") - Temporal Proximity Detection: Detects relationships between observations in the same session within 5 minutes - Narrative Mention Detection: Detects explicit relationship language in narratives (e.g., "fixes", "depends on", "supersedes") * Add visualisation of the relations to the dashboard. * fixup! Add visualisation of the relations to the dashboard. * Update documentation with new settings and screenshots.	2025-12-19 17:57:11 +00:00