From e39d6a0f0d64ad37f7cb07de5a8c554221b31de1 Mon Sep 17 00:00:00 2001 From: Lukasz Raczylo Date: Wed, 29 Apr 2026 00:57:56 +0100 Subject: [PATCH] feat: comprehensive audit + Tier 3 wiring (security/correctness/features) Multi-agent audit + fix pass covering bugs, security, dead config wiring, and missing features. All quality gates green: build/vet/test -race/govulncheck/ golangci-lint. Frontend tests 47/47. SECURITY & CORRECTNESS - Scanner pipeline fail-closed: cache.go scan timeout now 503 (was goto servePkg); scanner.go all-scanners-fail saves ScanStatusError; CheckVulnerabilities blocks on missing/error result instead of allowing. - Scanner argv corrections: govulncheck source-mode + go.mod discovery; npm-audit generates lockfile via npm install --package-lock-only --ignore-scripts; pip-audit per-extension dispatch (wheel direct / sdist extract+pyproject). - GHSA version-range filtering implemented (was always-include); url.QueryEscape on package name. - pypi SSRF closed via host allowlist on original_url; url.QueryEscape on rewriteURL output. - Path traversal: cache temp file uses os.CreateTemp; smb keyToPath sanitized; filesystem keyToPath returns error + filepath.Abs prefix-verify. - Goroutine leaks plugged: cache.cleanupWorker stop channel; auth.ValidationCache Stop() with sync.Once; ws.unregister buffered with non-blocking send. - WebSocket double-close panic fixed via sync.Once Client.closeSend(). - Race conditions: gormstore.registryCache sync.RWMutex (was concurrent map panic); auth.LastUsedAt no longer mutated under RLock; analytics strict lock-ordering invariant (statsMu before downloadsMu). - Auth validator fails CLOSED on transport/unknown-status errors (was returning true,err 'allow cache fallback'). - Credential cache hash full SHA256 (was 8-byte truncate; eliminates 2^32 collision risk). - filesystem.fs.used incremented after successful rename (no quota inflation race). - gormstore aggregation events deleted in same tx as insert (no double-counting). - gormstore.SavePackage uses Updates(map) so zero-value security flags (RequiresAuth=false) can be cleared. - partition_manager validates partition name regex before DROP TABLE. - vcs/git: version regex validation rejects --option-injection; checkout uses --detach -- separator; removed TrimPrefix(repo,'v') that corrupted vault/ vitess/vim-go. - WebSocket CheckOrigin allowlist (was return true; CSWSH closed); same-origin default + ServerConfig.AllowedOrigins. - fiber CVE GO-2026-4543 patched (v2.52.10 -> v2.52.12). FUNCTIONAL FEATURES - API key DB persistence: APIKeyModel + migration 202604280002, full CRUD, async LastUsedAt updates with WaitGroup-tracked goroutines drained on Close. - Admin bootstrap via GOHOARDER_BOOTSTRAP_ADMIN_KEY env var; idempotent (skipped when non-revoked admin already exists). - Auth middleware factory in pkg/app: RequireAuth, RequireRole, RequirePermission, OptionalAuth. Mounted on proxy + read APIs when Auth.Enabled=true. DELETE /api/packages/* requires admin role. /health public for k8s probes. - NFS storage backend: pkg/storage/nfs wraps filesystem with /proc/mounts detection (Linux), post-write fsync, read-after-write health probe. - WebSocket real-time pipeline: pkg/events Broadcaster interface; cache + scanner managers emit EventPackageCached / EventPackageDownloaded / EventScanComplete; app.go runs 30s EventStatsUpdate ticker. Frontend has full WS client (reconnect with exponential backoff 1s->30s, 25s heartbeat, subscriber pattern), Pinia realtime store, Dashboard live indicator + activity feed + reactive stats override. - TLS termination: fiber.ListenTLS when Server.TLS.Enabled. - Pre-warming: Prewarming.Enabled flag wired (was hardcoded false); Interval, MaxConcurrent, TopPackages plumbed. - NetworkConfig wired (timeouts, retry, rate-limit, circuit-breaker). - AuthConfig.BcryptCost wired. - Security.Scanners.Static.MaxPackageSize plumbed to cache.io.LimitReader. - Handlers.{Go,NPM,PyPI}.Enabled gates route mounting. - Graceful shutdown: authManager.Close drains async writes. LINT/HYGIENE - 80 golangci-lint issues resolved: errcheck (Close/Write/RemoveAll wrapped), gofmt, gosec G104/G306, govet shadow renames + fieldalignment reorders, staticcheck ST1000 pkg comments + QF1003 tagged switches + QF1008 embedded field selectors + QF1001 De Morgan's law. BEHAVIOR CHANGES - Scanner now fail-closed: deployments without scanner binaries (trivy/govulncheck/npm-audit/pip-audit/grype) BLOCK packages instead of serving unscanned. Add binaries or plan a future allow-on-scan-error flag. - WS origin defaults to same-origin; cross-origin dashboards must set server.allowed_origins. - Validator no longer fails open; private packages won't serve from cache when validation can't be performed. - download_events retention dropped from 24h to ~5min (deleted in agg tx). Migration 202604280001 purges pre-upgrade events. - DELETE /api/packages/* requires admin role when auth enabled. NEW PACKAGES - pkg/events - Broadcaster interface - pkg/storage/nfs - NFS-aware filesystem wrapper DEFERRED (out of scope this pass) - Static scanner implementation (config defines AllowedLicenses/MaxPackageSize/ BlockSuspicious but pkg/scanner/static/ does not exist). - AuthConfig.AuditLog wiring (no audit log infra yet). - CacheConfig.TTLOverrides passthrough (would touch cache.Config beyond integrator scope). --- cmd/gohoarder/commands/serve.go | 7 +- cmd/gohoarder/commands/version.go | 14 +- cmd/migrate/main.go | 4 +- frontend/src/components/Dashboard.spec.ts | 31 ++ frontend/src/components/Dashboard.vue | 117 ++++- frontend/src/components/PackageList.spec.ts | 67 ++- frontend/src/components/Stats.spec.ts | 5 +- frontend/src/lib/ws.spec.ts | 223 ++++++++++ frontend/src/lib/ws.ts | 301 +++++++++++++ frontend/src/stores/realtime.spec.ts | 65 +++ frontend/src/stores/realtime.ts | 118 ++++++ internal/version/version.go | 2 + pkg/analytics/analytics.go | 98 +++-- pkg/app/app.go | 423 ++++++++++++++++--- pkg/app/handlers_analytics_test.go | 2 +- pkg/app/handlers_auth_test.go | 8 +- pkg/app/middleware.go | 193 +++++++++ pkg/app/middleware_test.go | 287 +++++++++++++ pkg/auth/auth.go | 308 ++++++++++++-- pkg/auth/auth_test.go | 372 ++++++++++++++++ pkg/auth/bootstrap.go | 99 +++++ pkg/auth/bootstrap_test.go | 142 +++++++ pkg/auth/hasher.go | 10 +- pkg/auth/validation_cache.go | 73 +++- pkg/auth/validator.go | 54 +-- pkg/cache/cache.go | 241 ++++++++--- pkg/cache/cache_test.go | 233 ++++++++++ pkg/cdn/cdn.go | 2 + pkg/cdn/cdn_test.go | 12 +- pkg/config/config.go | 94 +++-- pkg/config/config_test.go | 6 +- pkg/errors/errors.go | 2 + pkg/events/events.go | 24 ++ pkg/health/health.go | 7 +- pkg/logger/logger.go | 2 + pkg/metadata/file/file.go | 28 ++ pkg/metadata/gormstore/aggregation_worker.go | 22 +- pkg/metadata/gormstore/api_keys.go | 115 +++++ pkg/metadata/gormstore/api_keys_test.go | 231 ++++++++++ pkg/metadata/gormstore/config.go | 8 +- pkg/metadata/gormstore/gormstore_v2.go | 75 +++- pkg/metadata/gormstore/gormstore_v2_test.go | 12 +- pkg/metadata/gormstore/migrations.go | 52 ++- pkg/metadata/gormstore/models_v2.go | 231 +++++----- pkg/metadata/gormstore/partition_manager.go | 44 +- pkg/metadata/interface.go | 46 ++ pkg/metrics/metrics.go | 2 + pkg/network/client.go | 4 +- pkg/prewarming/worker.go | 4 +- pkg/proxy/goproxy/goproxy.go | 30 +- pkg/proxy/npm/npm.go | 74 +++- pkg/proxy/npm/npm_test.go | 81 ++++ pkg/proxy/pypi/pypi.go | 89 +++- pkg/proxy/pypi/pypi_test.go | 139 ++++++ pkg/scanner/ghsa/ghsa.go | 209 ++++++++- pkg/scanner/ghsa/ghsa_test.go | 160 +++++++ pkg/scanner/govulncheck/govulncheck.go | 81 +++- pkg/scanner/grype/grype.go | 2 + pkg/scanner/npmaudit/npmaudit.go | 53 ++- pkg/scanner/osv/osv.go | 5 +- pkg/scanner/pipaudit/pipaudit.go | 115 ++++- pkg/scanner/rescanner.go | 8 +- pkg/scanner/scanner.go | 127 +++++- pkg/scanner/scanner_test.go | 256 +++++++++++ pkg/scanner/trivy/trivy.go | 2 + pkg/storage/filesystem/filesystem.go | 128 ++++-- pkg/storage/filesystem/filesystem_test.go | 29 +- pkg/storage/interface.go | 2 + pkg/storage/nfs/nfs.go | 280 ++++++++++++ pkg/storage/nfs/nfs_test.go | 239 +++++++++++ pkg/storage/s3/s3.go | 8 +- pkg/storage/s3/s3_test.go | 4 +- pkg/storage/smb/smb.go | 84 ++-- pkg/storage/smb/smb_test.go | 3 +- pkg/uuid/uuid.go | 2 + pkg/vcs/credentials.go | 2 + pkg/vcs/git.go | 52 ++- pkg/vcs/module.go | 4 +- pkg/websocket/server.go | 72 +++- pkg/websocket/server_test.go | 184 ++++++++ 80 files changed, 6383 insertions(+), 661 deletions(-) create mode 100644 frontend/src/lib/ws.spec.ts create mode 100644 frontend/src/lib/ws.ts create mode 100644 frontend/src/stores/realtime.spec.ts create mode 100644 frontend/src/stores/realtime.ts create mode 100644 pkg/app/middleware.go create mode 100644 pkg/app/middleware_test.go create mode 100644 pkg/auth/auth_test.go create mode 100644 pkg/auth/bootstrap.go create mode 100644 pkg/auth/bootstrap_test.go create mode 100644 pkg/events/events.go create mode 100644 pkg/metadata/gormstore/api_keys.go create mode 100644 pkg/metadata/gormstore/api_keys_test.go create mode 100644 pkg/proxy/npm/npm_test.go create mode 100644 pkg/proxy/pypi/pypi_test.go create mode 100644 pkg/scanner/ghsa/ghsa_test.go create mode 100644 pkg/scanner/scanner_test.go create mode 100644 pkg/storage/nfs/nfs.go create mode 100644 pkg/storage/nfs/nfs_test.go create mode 100644 pkg/websocket/server_test.go diff --git a/cmd/gohoarder/commands/serve.go b/cmd/gohoarder/commands/serve.go index 7aa0525..b8bd33f 100644 --- a/cmd/gohoarder/commands/serve.go +++ b/cmd/gohoarder/commands/serve.go @@ -1,3 +1,4 @@ +// Package commands hosts the cobra commands wired into the gohoarder CLI. package commands import ( @@ -35,11 +36,11 @@ func runServe(cmd *cobra.Command, args []string) error { } // Initialize logger - if err := logger.Init(logger.Config{ + if logErr := logger.Init(logger.Config{ Level: cfg.Logging.Level, Format: cfg.Logging.Format, - }); err != nil { - return fmt.Errorf("failed to initialize logger: %w", err) + }); logErr != nil { + return fmt.Errorf("failed to initialize logger: %w", logErr) } log.Info(). diff --git a/cmd/gohoarder/commands/version.go b/cmd/gohoarder/commands/version.go index 52dffc7..d01f442 100644 --- a/cmd/gohoarder/commands/version.go +++ b/cmd/gohoarder/commands/version.go @@ -23,16 +23,16 @@ var VersionCmd = &cobra.Command{ if jsonOutput { data, err := json.MarshalIndent(info, "", " ") if err != nil { - fmt.Fprintf(cmd.OutOrStderr(), "Error: %v\n", err) + _, _ = fmt.Fprintf(cmd.OutOrStderr(), "Error: %v\n", err) return } - fmt.Fprintln(cmd.OutOrStdout(), string(data)) + _, _ = fmt.Fprintln(cmd.OutOrStdout(), string(data)) } else { - fmt.Fprintf(cmd.OutOrStdout(), "GoHoarder %s\n", info.Version) - fmt.Fprintf(cmd.OutOrStdout(), "Git Commit: %s\n", info.GitCommit) - fmt.Fprintf(cmd.OutOrStdout(), "Built: %s\n", info.BuildTime) - fmt.Fprintf(cmd.OutOrStdout(), "Go Version: %s\n", info.GoVersion) - fmt.Fprintf(cmd.OutOrStdout(), "Platform: %s\n", info.Platform) + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "GoHoarder %s\n", info.Version) + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "Git Commit: %s\n", info.GitCommit) + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "Built: %s\n", info.BuildTime) + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "Go Version: %s\n", info.GoVersion) + _, _ = fmt.Fprintf(cmd.OutOrStdout(), "Platform: %s\n", info.Platform) } }, } diff --git a/cmd/migrate/main.go b/cmd/migrate/main.go index f45ea7b..9502d51 100644 --- a/cmd/migrate/main.go +++ b/cmd/migrate/main.go @@ -25,10 +25,10 @@ import ( type MigrationConfig struct { Driver string DSN string - Timeout time.Duration Action string // migrate, rollback, rollback-to, list TargetID string // For rollback-to LogLevel string + Timeout time.Duration } func main() { @@ -97,7 +97,7 @@ func RunMigration(cfg MigrationConfig) error { if err != nil { return fmt.Errorf("failed to get sql.DB: %w", err) } - defer sqlDB.Close() + defer func() { _ = sqlDB.Close() }() // Wait for database to be ready if err := waitForDB(ctx, sqlDB, 60*time.Second); err != nil { diff --git a/frontend/src/components/Dashboard.spec.ts b/frontend/src/components/Dashboard.spec.ts index 73adafe..d9c2e09 100644 --- a/frontend/src/components/Dashboard.spec.ts +++ b/frontend/src/components/Dashboard.spec.ts @@ -3,10 +3,12 @@ import { mount } from '@vue/test-utils' import { createPinia, setActivePinia } from 'pinia' import Dashboard from './Dashboard.vue' import { usePackageStore } from '../stores/packages' +import { useRealtimeStore, __resetRealtimeSingleton } from '../stores/realtime' describe('Dashboard.vue', () => { beforeEach(() => { setActivePinia(createPinia()) + __resetRealtimeSingleton() // Mock the fetch functions to prevent actual API calls const store = usePackageStore() vi.spyOn(store, 'fetchStats').mockResolvedValue() @@ -213,4 +215,33 @@ describe('Dashboard.vue', () => { expect(wrapper.text()).toContain('0') expect(wrapper.text()).toContain('0 B') }) + + it('renders realtime lastStats over polled stats when present', async () => { + const wrapper = mount(Dashboard) + const store = usePackageStore() + const rt = useRealtimeStore() + + store.loading = false + store.stats = { + registry: '', + total_packages: 10, + total_size: 1024, + max_cache_size: 10737418240, + total_downloads: 5, + scanned_packages: 0, + vulnerable_packages: 0, + blocked_packages: 0, + } + rt.lastStats = { + total_packages: 999, + total_size: 2048, + total_downloads: 777, + scanned_packages: 0, + } + await wrapper.vm.$nextTick() + + // Realtime values should be visible (override polled values). + expect(wrapper.text()).toContain('999') + expect(wrapper.text()).toContain('777') + }) }) diff --git a/frontend/src/components/Dashboard.vue b/frontend/src/components/Dashboard.vue index 7281ecc..c51e3ae 100644 --- a/frontend/src/components/Dashboard.vue +++ b/frontend/src/components/Dashboard.vue @@ -1,6 +1,23 @@