lukaszraczylo/gohoarder
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/gohoarder.git synced 2026-07-13 04:57:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
151 Commits 4 Branches 131 Tags
e39d6a0f0d64ad37f7cb07de5a8c554221b31de1
Commit Graph

151 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
lukaszraczylo e39d6a0f0d feat: comprehensive audit + Tier 3 wiring (security/correctness/features) 
Multi-agent audit + fix pass covering bugs, security, dead config wiring,
and missing features. All quality gates green: build/vet/test -race/govulncheck/
golangci-lint. Frontend tests 47/47.

SECURITY & CORRECTNESS

- Scanner pipeline fail-closed: cache.go scan timeout now 503 (was goto servePkg);
  scanner.go all-scanners-fail saves ScanStatusError; CheckVulnerabilities blocks
  on missing/error result instead of allowing.
- Scanner argv corrections: govulncheck source-mode + go.mod discovery;
  npm-audit generates lockfile via npm install --package-lock-only --ignore-scripts;
  pip-audit per-extension dispatch (wheel direct / sdist extract+pyproject).
- GHSA version-range filtering implemented (was always-include); url.QueryEscape
  on package name.
- pypi SSRF closed via host allowlist on original_url; url.QueryEscape on
  rewriteURL output.
- Path traversal: cache temp file uses os.CreateTemp; smb keyToPath sanitized;
  filesystem keyToPath returns error + filepath.Abs prefix-verify.
- Goroutine leaks plugged: cache.cleanupWorker stop channel; auth.ValidationCache
  Stop() with sync.Once; ws.unregister buffered with non-blocking send.
- WebSocket double-close panic fixed via sync.Once Client.closeSend().
- Race conditions: gormstore.registryCache sync.RWMutex (was concurrent map
  panic); auth.LastUsedAt no longer mutated under RLock; analytics strict
  lock-ordering invariant (statsMu before downloadsMu).
- Auth validator fails CLOSED on transport/unknown-status errors (was returning
  true,err 'allow cache fallback').
- Credential cache hash full SHA256 (was 8-byte truncate; eliminates 2^32
  collision risk).
- filesystem.fs.used incremented after successful rename (no quota inflation
  race).
- gormstore aggregation events deleted in same tx as insert (no double-counting).
- gormstore.SavePackage uses Updates(map) so zero-value security flags
  (RequiresAuth=false) can be cleared.
- partition_manager validates partition name regex before DROP TABLE.
- vcs/git: version regex validation rejects --option-injection; checkout uses
  --detach -- separator; removed TrimPrefix(repo,'v') that corrupted vault/
  vitess/vim-go.
- WebSocket CheckOrigin allowlist (was return true; CSWSH closed); same-origin
  default + ServerConfig.AllowedOrigins.
- fiber CVE GO-2026-4543 patched (v2.52.10 -> v2.52.12).

FUNCTIONAL FEATURES

- API key DB persistence: APIKeyModel + migration 202604280002, full CRUD,
  async LastUsedAt updates with WaitGroup-tracked goroutines drained on Close.
- Admin bootstrap via GOHOARDER_BOOTSTRAP_ADMIN_KEY env var; idempotent
  (skipped when non-revoked admin already exists).
- Auth middleware factory in pkg/app: RequireAuth, RequireRole, RequirePermission,
  OptionalAuth. Mounted on proxy + read APIs when Auth.Enabled=true.
  DELETE /api/packages/* requires admin role. /health public for k8s probes.
- NFS storage backend: pkg/storage/nfs wraps filesystem with /proc/mounts
  detection (Linux), post-write fsync, read-after-write health probe.
- WebSocket real-time pipeline: pkg/events Broadcaster interface; cache + scanner
  managers emit EventPackageCached / EventPackageDownloaded / EventScanComplete;
  app.go runs 30s EventStatsUpdate ticker. Frontend has full WS client (reconnect
  with exponential backoff 1s->30s, 25s heartbeat, subscriber pattern), Pinia
  realtime store, Dashboard live indicator + activity feed + reactive stats
  override.
- TLS termination: fiber.ListenTLS when Server.TLS.Enabled.
- Pre-warming: Prewarming.Enabled flag wired (was hardcoded false); Interval,
  MaxConcurrent, TopPackages plumbed.
- NetworkConfig wired (timeouts, retry, rate-limit, circuit-breaker).
- AuthConfig.BcryptCost wired.
- Security.Scanners.Static.MaxPackageSize plumbed to cache.io.LimitReader.
- Handlers.{Go,NPM,PyPI}.Enabled gates route mounting.
- Graceful shutdown: authManager.Close drains async writes.

LINT/HYGIENE

- 80 golangci-lint issues resolved: errcheck (Close/Write/RemoveAll wrapped),
  gofmt, gosec G104/G306, govet shadow renames + fieldalignment reorders,
  staticcheck ST1000 pkg comments + QF1003 tagged switches + QF1008 embedded
  field selectors + QF1001 De Morgan's law.

BEHAVIOR CHANGES

- Scanner now fail-closed: deployments without scanner binaries
  (trivy/govulncheck/npm-audit/pip-audit/grype) BLOCK packages instead of
  serving unscanned. Add binaries or plan a future allow-on-scan-error flag.
- WS origin defaults to same-origin; cross-origin dashboards must set
  server.allowed_origins.
- Validator no longer fails open; private packages won't serve from cache when
  validation can't be performed.
- download_events retention dropped from 24h to ~5min (deleted in agg tx).
  Migration 202604280001 purges pre-upgrade events.
- DELETE /api/packages/* requires admin role when auth enabled.

NEW PACKAGES

- pkg/events  - Broadcaster interface
- pkg/storage/nfs  - NFS-aware filesystem wrapper

DEFERRED (out of scope this pass)

- Static scanner implementation (config defines AllowedLicenses/MaxPackageSize/
  BlockSuspicious but pkg/scanner/static/ does not exist).
- AuthConfig.AuditLog wiring (no audit log infra yet).
- CacheConfig.TTLOverrides passthrough (would touch cache.Config beyond
  integrator scope).
 2026-07-10 09:37:55 +01:00
lukaszraczylo 3c8b3eb46c fixup! fix: exclude Go module metadata files from package count stats 2026-07-10 09:35:45 +01:00
lukaszraczylo c8bc8eca8f fix: exclude Go module metadata files from package count stats 
The dashboard was showing 335 packages while the packages page showed 141
because GetStats was counting .mod and .info files as separate packages.

Changes:
- Filter out .mod and .info files in GetStats query
- Now only counts actual packages (.zip files)
- Dashboard and packages page counts will match

This fixes the discrepancy where:
- Dashboard (/api/stats): was 335, now will be ~141
- Packages page (/api/packages): already 141 (deduplicated)

Resolves: "Dashboard showing 335 packages but /packages showing 141"
 2026-07-10 09:35:45 +01:00
lukaszraczylo d5979a8b88 fix: skip scanning Go module metadata files (.mod, .info) 
Go module metadata files (.mod and .info) are not actual packages
and should not be scanned for vulnerabilities. Only .zip files
contain the actual module code.

Changes:
- Added .mod and .info suffix checks to isMetadataEntry filter
- These files are now excluded from database storage
- Scanner won't attempt to scan them during rescan cycles

This fixes OSV scanner 404 errors for metadata files like:
- github.com/mattn/go-isatty/@v/v0.0.20.mod
- github.com/clipperhouse/stringish/@v/v0.1.1.info

Resolves: OSV API errors "The current request is not defined by this API"
for Go module metadata files
 2026-07-10 09:35:45 +01:00
lukaszraczylo c38d6bec0d Update go.mod and go.sum (#95) v0.1.147 2026-07-09 05:23:50 +01:00
lukaszraczylo 1616a7d42d Update go.mod and go.sum (#94) v0.1.146 2026-07-07 05:25:04 +01:00
lukaszraczylo ec21194229 Update go.mod and go.sum (#93) v0.1.145 2026-07-05 05:27:13 +01:00
lukaszraczylo 8967d313e0 Update go.mod and go.sum (#92) v0.1.144 2026-07-02 05:30:08 +01:00
lukaszraczylo 75eda36638 Update go.mod and go.sum (#91) v0.1.143 2026-07-01 05:38:54 +01:00
lukaszraczylo 2a3666ae7c Update go.mod and go.sum (#90) v0.1.142 2026-06-30 05:31:08 +01:00
lukaszraczylo ab45e2f511 Update go.mod and go.sum (#89) v0.1.141 2026-06-28 05:37:58 +01:00
lukaszraczylo 6b8f8443fb Update go.mod and go.sum (#88) v0.1.140 2026-06-27 05:27:45 +01:00
lukaszraczylo e451ec5f1f Update go.mod and go.sum (#87) v0.1.139 2026-06-26 05:32:55 +01:00
lukaszraczylo 3898756bf6 Update go.mod and go.sum (#86) v0.1.138 2026-06-24 05:31:06 +01:00
lukaszraczylo 3c22e53c2e Update go.mod and go.sum (#85) v0.1.137 2026-06-23 05:30:05 +01:00
lukaszraczylo 12a75bf77d Update go.mod and go.sum (#84) v0.1.136 2026-06-22 05:38:46 +01:00
lukaszraczylo 1211320939 Update go.mod and go.sum (#83) v0.1.135 2026-06-19 05:39:39 +01:00
lukaszraczylo 86f7d974e7 Update go.mod and go.sum (#82) v0.1.134 2026-06-18 05:36:29 +01:00
lukaszraczylo f0eaf695ab Update go.mod and go.sum (#81) v0.1.133 2026-06-17 05:37:06 +01:00
lukaszraczylo cfceb063ee Update go.mod and go.sum (#80) v0.1.132 2026-06-11 05:36:25 +01:00
lukaszraczylo 76b38c5c10 Update go.mod and go.sum (#79) v0.1.131 2026-06-09 05:30:46 +01:00
lukaszraczylo ae85e74094 Update go.mod and go.sum (#78) v0.1.130 2026-06-06 05:29:56 +01:00
lukaszraczylo e08cab3311 Update go.mod and go.sum (#77) v0.1.129 2026-06-05 05:36:18 +01:00
lukaszraczylo c9a124e708 Update go.mod and go.sum (#76) v0.1.128 2026-06-04 05:37:15 +01:00
lukaszraczylo 640c54201f Update go.mod and go.sum (#75) v0.1.127 2026-06-03 05:37:50 +01:00
lukaszraczylo ba632c4c14 Update go.mod and go.sum (#74) v0.1.126 2026-05-30 05:26:59 +01:00
lukaszraczylo bd07e45c05 Update go.mod and go.sum (#73) v0.1.125 2026-05-29 05:34:39 +01:00
lukaszraczylo 69ef5a478a Update go.mod and go.sum (#72) v0.1.124 2026-05-28 05:33:24 +01:00
lukaszraczylo 2b425d4012 Update go.mod and go.sum (#71) v0.1.123 2026-05-27 05:35:38 +01:00
lukaszraczylo 875fd9025a Update go.mod and go.sum (#70) v0.1.122 2026-05-23 05:21:35 +01:00
lukaszraczylo 1fc61b1ec3 Update go.mod and go.sum (#69) v0.1.121 2026-05-22 05:30:56 +01:00
lukaszraczylo c38209d8d1 Update go.mod and go.sum (#68) v0.1.120 2026-05-09 05:14:14 +01:00
lukaszraczylo 4ffb17757c Update go.mod and go.sum (#67) 2026-05-07 05:21:07 +01:00
lukaszraczylo 5dcf1cf6b0 Update go.mod and go.sum (#66) v0.1.118 2026-05-05 05:08:46 +01:00
lukaszraczylo 923651d11a Update go.mod and go.sum (#65) v0.1.117 2026-05-02 05:11:11 +01:00
lukaszraczylo fedb40c826 Update go.mod and go.sum (#64) v0.1.116 2026-05-01 05:28:57 +01:00
lukaszraczylo b9cd641db3 Update go.mod and go.sum (#63) v0.1.115 2026-04-30 05:20:27 +01:00
lukaszraczylo ddad5883fe Update go.mod and go.sum (#62) v0.1.114 2026-04-27 05:19:04 +01:00
lukaszraczylo b25dfcd263 Update go.mod and go.sum (#61) v0.1.113 2026-04-26 05:15:01 +01:00
lukaszraczylo 8c85532c86 Update go.mod and go.sum (#60) v0.1.112 2026-04-24 05:10:16 +01:00
lukaszraczylo 7e9b0067f1 Update go.mod and go.sum (#59) v0.1.111 2026-04-23 05:07:32 +01:00
lukaszraczylo 7fb2ed6d8d Update go.mod and go.sum (#58) v0.1.110 2026-04-21 05:07:17 +01:00
lukaszraczylo 65446ddbe7 Update go.mod and go.sum (#57) v0.1.109 2026-04-19 05:08:17 +01:00
lukaszraczylo 205a8c0ee5 Update go.mod and go.sum (#56) v0.1.108 2026-04-18 05:01:42 +01:00
lukaszraczylo 9462c66edf Update go.mod and go.sum (#55) v0.1.107 2026-04-17 05:07:50 +01:00
lukaszraczylo 7c727df4f8 Update go.mod and go.sum (#54) v0.1.106 2026-04-16 05:08:18 +01:00
lukaszraczylo 26fd95cea9 Update go.mod and go.sum (#53) v0.1.105 2026-04-10 05:06:27 +01:00
lukaszraczylo 79c46f3b1e Update go.mod and go.sum (#52) v0.1.104 2026-04-09 05:01:23 +01:00
lukaszraczylo 33f371967f Update go.mod and go.sum (#51) v0.1.103 2026-04-08 05:02:55 +01:00
lukaszraczylo 7cc93d768d Update go.mod and go.sum (#50) v0.1.102 2026-04-07 05:02:05 +01:00