Lukasz Raczylo 5c2685c7b6 feat(leann-phase2): implement hybrid vector storage and graph-based search (#20)
* feat(leann-phase2): implement hybrid vector storage and graph-based search

- [x] Add AST-aware code chunking for Go, Python, and TypeScript using tree-sitter
- [x] Implement LEANN-inspired hybrid vector storage with hub detection and selective embedding storage (60-80% savings)
- [x] Add observation relationship graph with CSR format and edge detection (file overlap, semantic similarity, temporal, concept)
- [x] Implement graph-aware search with two-level traversal and relationship-based ranking
- [x] Add auto-tuning system for dynamic hub threshold adjustment based on query performance
- [x] Add comprehensive metrics tracking for vector storage, queries, latency, and graph traversals
- [x] Update configuration system with graph and hybrid storage settings
- [x] Add graph stats and vector metrics endpoints to worker service
- [x] Enhance UI sidebar with advanced metrics display and graph visualization
- [x] Optimize struct field alignment throughout codebase for memory efficiency
- [x] Update documentation with LEANN Phase 2 features and performance benefits
- [x] Add tree-sitter dependency for AST parsing

* fix: add fts5 build tag to CI workflow

Pass build-tags: "fts5" to shared workflow to properly compile
sqlite-vec-go-bindings with SQLite FTS5 support.

This fixes test failures in hybrid vector storage tests that require
CGO and FTS5 build tags.

Requires shared-actions@8f7f235 or later.

* docs: add testing documentation and macOS ARM64 known issue

Document the macOS ARM64 CGO linking issue with sqlite-vec-go-bindings
that prevents hybrid package tests from compiling locally.

Added:
- .github/TESTING.md: Comprehensive testing guide with platform-specific
  issues, workarounds, and CI configuration details
- internal/vector/hybrid/README.md: Package-specific documentation
  explaining the macOS limitation
- .github/CI_FIX_SUMMARY.md: Technical details of the CI fix

Key points:
- 41 out of 42 packages test successfully on all platforms
- hybrid package tests fail only on macOS ARM64 (local dev issue)
- Linux CI tests pass with proper build-tags: "fts5" configuration
- Production builds and runtime functionality unaffected

This is a known limitation of sqlite-vec-go-bindings on macOS ARM64
and does not impact CI/CD or production deployments.

* fix: add SQLite busy_timeout to prevent database locked errors

Set PRAGMA busy_timeout=5000 (5 seconds) to allow SQLite to retry
when the database is locked instead of failing immediately.

This fixes race conditions when multiple goroutines try to write
simultaneously, particularly in tests where StoreObservation spawns
async cleanup goroutines.

Root cause:
- StoreObservation launches goroutine -> CleanupOldObservations
- Multiple concurrent cleanups caused "database is locked" errors
- Without busy_timeout, SQLite fails immediately on lock contention

Solution:
- Add 5-second busy timeout for automatic retry on lock
- Standard practice for concurrent SQLite usage
- Works with existing WAL mode configuration

Fixes TestObservationStore_CleanupOldObservations in CI.

* docs: complete summary of all CI test fixes

Comprehensive documentation of all fixes applied:
1. Missing build tags (fts5)
2. Database locked errors (busy_timeout)

All 41/42 packages now pass tests. The hybrid package has a known
macOS ARM64 limitation that doesn't affect CI or production.

No functionality was removed - all fixes are additive only.

* fix: add SQLite driver import to hybrid tests for CGO linking

Add blank import of mattn/go-sqlite3 to hybrid test files to ensure
the SQLite driver is linked into the test binary. This provides the
SQLite symbols that sqlite-vec-go-bindings requires.

Root cause:
- hybrid package imports sqlitevec (transitively depends on sqlite-vec CGO)
- Test binary needs SQLite symbols for linking
- sqlitevec tests already had this import, but hybrid tests didn't
- Without the driver import, linker fails with "undefined symbols"

This fix enables hybrid tests to run with -race flag on all platforms.

Before: 41/42 packages pass (hybrid failed to link)
After:  42/42 packages pass 

Fixes hybrid test compilation on macOS ARM64, Linux, and Windows.

* docs: remove outdated macOS limitation documentation

The hybrid test linking issue has been fixed by adding the SQLite
driver import. All tests now pass on all platforms including macOS.

Removed:
- internal/vector/hybrid/README.md (documented workaround no longer needed)
- .github/TESTING.md (macOS limitation section obsolete)

All 42/42 packages now test successfully with -race flag.

* docs: final comprehensive summary of all CI fixes

All three issues now resolved:
1. Missing fts5 build tags
2. Database busy_timeout for concurrent writes
3. Missing SQLite driver import in hybrid tests

Result: 42/42 packages pass with -race on all platforms.

Credit to reviewer for identifying the race detector concern.
2026-01-07 22:03:59 +00:00
2026-01-07 00:26:20 +00:00
2026-01-07 00:26:20 +00:00
2025-12-19 02:17:02 +00:00
2026-01-07 00:26:20 +00:00
2025-12-19 02:17:02 +00:00
2026-01-07 00:26:20 +00:00
2025-12-19 17:57:11 +00:00

Claude Mnemonic

Give Claude Code a memory that actually remembers.

Release License Go


Claude Code forgets everything when your session ends. Claude Mnemonic fixes that.

It captures what Claude learns during your coding sessions - bug fixes, architecture decisions, patterns that work - and brings that knowledge back in future conversations. No more re-explaining your codebase.

Claude Mnemonic Dashboard

What's New in v0.7

  • Two-Stage Retrieval - Cross-encoder reranking for dramatically improved search relevance
  • Knowledge Graph - Automatic relationship detection between observations with visual graph in dashboard Knowledge Graph
  • Pattern Detection - Identifies recurring patterns across sessions and projects
  • Importance Scoring - Time decay and voting system to surface the most valuable memories
  • Query Expansion - Reformulates searches to find semantically related content
  • Conflict Detection - Identifies and resolves contradictory observations
  • Observation Lifecycle - Memories can be superseded when better information arrives
Previous: v0.6
  • Auto-Updates - Automatically stays up-to-date with the latest version
  • Slash Command: /restart - Restart the worker directly from Claude Code
  • Local Embeddings - All semantic search runs locally via ONNX Runtime (no external API calls)
  • Async Queue Processing - Non-blocking observation capture for faster sessions
  • Smarter Storage - Filters out system/agent summaries to keep knowledge relevant
  • Improved Reliability - Better handling of connectivity issues and dead connections

Requirements

Dependency Required Purpose
Claude Code CLI Yes Host application (this is a plugin)
jq Yes JSON processing during installation

That's it. No Python. No external services. Everything runs locally.

No API keys needed! Claude Mnemonic uses Claude Code CLI, which works with your existing Claude Pro or Max subscription. No separate API costs.

Install

One command. That's it.

curl -sSL https://raw.githubusercontent.com/lukaszraczylo/claude-mnemonic/main/scripts/install.sh | bash
Windows (PowerShell)
irm https://raw.githubusercontent.com/lukaszraczylo/claude-mnemonic/main/scripts/install.ps1 | iex
Build from source
git clone https://github.com/lukaszraczylo/claude-mnemonic.git
cd claude-mnemonic
make build && make install

Requires: Go 1.24+, Node.js 18+, CGO-compatible compiler

After install, open http://localhost:37777 to see the dashboard. Start a new Claude Code session - memory is now active.

Verifying Release Signatures

All release checksums are signed with cosign using keyless signing. To verify:

# Download the checksum file and its sigstore bundle from the release
cosign verify-blob \
  --certificate-identity-regexp "https://github.com/lukaszraczylo/claude-mnemonic/.*" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  --bundle "checksums.txt.sigstore.json" \
  checksums.txt

What it does

Feature Description
Persistent Memory Observations survive across sessions and restarts
Project Isolation Each project has its own knowledge base
Global Patterns Best practices are shared across all projects
Semantic Search Find relevant context with natural language (local embeddings)
Live Statusline Real-time metrics in Claude Code: `[mnemonic] ● served:42
Web Dashboard Browse and manage memories at localhost:37777
Auto-Updates Automatically downloads and applies new versions
Slash Commands Control the worker directly from Claude Code

How knowledge flows

You code with Claude
        ↓
Claude learns something useful
        ↓
Mnemonic captures it automatically
        ↓
Next session: Claude remembers

Behind the scenes: hooks capture Claude's observations → SQLite stores with full-text search → sqlite-vec enables semantic search with local embeddings (all-MiniLM-L6-v2) → relevant context is injected at session start.

Configuration

Config file: ~/.claude-mnemonic/settings.json

{
  "CLAUDE_MNEMONIC_WORKER_PORT": 37777,
  "CLAUDE_MNEMONIC_CONTEXT_OBSERVATIONS": 100,
  "CLAUDE_MNEMONIC_CONTEXT_FULL_COUNT": 25,
  "CLAUDE_MNEMONIC_RERANKING_ENABLED": true
}

Core Settings

Variable Default What it does
WORKER_PORT 37777 Dashboard & API port
CONTEXT_OBSERVATIONS 100 Max memories per session
CONTEXT_FULL_COUNT 25 Full detail memories (rest are condensed)
CONTEXT_SESSION_COUNT 10 Recent sessions to reference
CONTEXT_RELEVANCE_THRESHOLD 0.3 Minimum similarity score (0.0-1.0) for inclusion
CONTEXT_MAX_PROMPT_RESULTS 10 Max results per prompt search

Reranking Settings (Two-Stage Retrieval)

Variable Default What it does
RERANKING_ENABLED true Enable cross-encoder reranking
RERANKING_CANDIDATES 100 Candidates to retrieve before reranking
RERANKING_RESULTS 10 Final results after reranking
RERANKING_ALPHA 0.7 Score blend: alpha×rerank + (1-alpha)×original
RERANKING_PURE_MODE false Use pure cross-encoder scores only

Embedding Settings

Variable Default What it does
EMBEDDING_MODEL bge-v1.5 Embedding model for semantic search

All variables are prefixed with CLAUDE_MNEMONIC_ in the config file.

Project vs Global scope

Observations are automatically scoped:

  • Project scope (default) - stays within the project directory
  • Global scope - shared everywhere

Global scope triggers on tags like: best-practice, security, architecture, pattern, performance

Example: A bug fix in your auth module stays local. "Always validate JWT server-side" goes global.

MCP Tools

These search tools are available via MCP:

  • search - semantic search across all memories
  • timeline - browse by time
  • decisions - find architecture decisions
  • changes - find code modifications
  • how_it_works - system understanding queries

Slash Commands

Available commands within Claude Code:

Command Description
/restart Restart the worker process when experiencing issues

Auto-Updates

Claude Mnemonic automatically checks for updates and applies them. Updates are downloaded in the background and applied on restart.

  • Automatic update checks on startup
  • Background downloads (up to 250MB)
  • Seamless restart after update
  • Manual trigger: curl -X POST http://127.0.0.1:37777/api/update/apply

Check update status: curl http://127.0.0.1:37777/api/update/status

Troubleshooting

Worker won't start?

lsof -i :37777              # check if port is in use
cat /tmp/claude-mnemonic-worker.log  # view logs

Database locked?

rm -f ~/.claude-mnemonic/*.db-wal ~/.claude-mnemonic/*.db-shm

Worker unresponsive?

# Restart via API
curl -X POST http://127.0.0.1:37777/api/restart

# Or use the slash command in Claude Code
/restart

Check health status:

curl http://127.0.0.1:37777/api/selfcheck

Uninstall

# Remove everything
curl -sSL https://raw.githubusercontent.com/lukaszraczylo/claude-mnemonic/main/scripts/uninstall.sh | bash

# Keep your data
curl -sSL https://raw.githubusercontent.com/lukaszraczylo/claude-mnemonic/main/scripts/uninstall.sh | bash -s -- --keep-data

Architecture

  • SQLite + FTS5 - Full-text search for exact matches
  • sqlite-vec - Vector database embedded in SQLite
  • Two-Stage Retrieval - Bi-encoder (embedding) + cross-encoder (reranking) for high accuracy
  • Local Models - all-MiniLM-L6-v2 for embeddings, BGE reranker for relevance scoring
  • Go - Single binary, no external dependencies

Everything runs locally. No Python. No external vector database. No API calls.

Platform support

Platform Status
macOS Intel Supported
macOS Apple Silicon Supported
Linux amd64 Supported
Linux arm64 Supported
Windows amd64 Supported

Development

make build          # build all
make test           # run tests
make dev            # dev mode with hot reload
make install        # install to Claude plugins

License

MIT


Links: Releases · Issues

S
Description
Mirror of github.com/lukaszraczylo/claude-mnemonic
Readme MIT 316 MiB
Languages
Go 88.4%
Vue 5.2%
Shell 2.6%
TypeScript 2.1%
PowerShell 0.9%
Other 0.6%