refactor: replace Git LFS with runtime model download from Hugging Face

Remove ~170MB of model files from the repository (LFS + committed).
Models are now downloaded at runtime from Hugging Face on first use
and cached to the OS cache directory with progress reporting and retries.

- Add internal/models/download.go: runtime downloader with retry, progress bar, checksums
- Remove go:embed for ONNX models (keep tokenizers embedded)
- Use file-based ONNX session loading instead of byte-slice
- Add scripts/download-models.sh for dev/CI model setup
- Update Makefile with setup-models target
- Update workflow-prepare.sh to download models in CI
- Set lfs: false in all CI workflows
- SHA256: bge=828e14..., cross-encoder=5d3e70...
This commit is contained in:
2026-05-26 17:52:55 +01:00
parent c8b462aaec
commit 1a4fea5c17
15 changed files with 431 additions and 53 deletions
+1 -1
View File
@@ -17,5 +17,5 @@ jobs:
with:
go-version: ">=1.24"
release-workflow: "release.yaml"
lfs: true
lfs: false
secrets: inherit
+1 -1
View File
@@ -20,6 +20,6 @@ jobs:
uses: lukaszraczylo/shared-actions/.github/workflows/go-pr.yaml@main
with:
go-version: ">=1.24"
lfs: true
lfs: false
build-tags: "fts5"
secrets: inherit
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
node-cache-dependency-path: "ui/package-lock.json"
node-output-path: "ui/dist"
node-embed-path: "internal/worker/static"
lfs: true
lfs: false
secrets: inherit
commit-marketplace: