mirror of
https://github.com/lukaszraczylo/gohoarder.git
synced 2026-07-15 05:25:45 +00:00
feat: comprehensive audit + Tier 3 wiring (security/correctness/features)
Multi-agent audit + fix pass covering bugs, security, dead config wiring,
and missing features. All quality gates green: build/vet/test -race/govulncheck/
golangci-lint. Frontend tests 47/47.
SECURITY & CORRECTNESS
- Scanner pipeline fail-closed: cache.go scan timeout now 503 (was goto servePkg);
scanner.go all-scanners-fail saves ScanStatusError; CheckVulnerabilities blocks
on missing/error result instead of allowing.
- Scanner argv corrections: govulncheck source-mode + go.mod discovery;
npm-audit generates lockfile via npm install --package-lock-only --ignore-scripts;
pip-audit per-extension dispatch (wheel direct / sdist extract+pyproject).
- GHSA version-range filtering implemented (was always-include); url.QueryEscape
on package name.
- pypi SSRF closed via host allowlist on original_url; url.QueryEscape on
rewriteURL output.
- Path traversal: cache temp file uses os.CreateTemp; smb keyToPath sanitized;
filesystem keyToPath returns error + filepath.Abs prefix-verify.
- Goroutine leaks plugged: cache.cleanupWorker stop channel; auth.ValidationCache
Stop() with sync.Once; ws.unregister buffered with non-blocking send.
- WebSocket double-close panic fixed via sync.Once Client.closeSend().
- Race conditions: gormstore.registryCache sync.RWMutex (was concurrent map
panic); auth.LastUsedAt no longer mutated under RLock; analytics strict
lock-ordering invariant (statsMu before downloadsMu).
- Auth validator fails CLOSED on transport/unknown-status errors (was returning
true,err 'allow cache fallback').
- Credential cache hash full SHA256 (was 8-byte truncate; eliminates 2^32
collision risk).
- filesystem.fs.used incremented after successful rename (no quota inflation
race).
- gormstore aggregation events deleted in same tx as insert (no double-counting).
- gormstore.SavePackage uses Updates(map) so zero-value security flags
(RequiresAuth=false) can be cleared.
- partition_manager validates partition name regex before DROP TABLE.
- vcs/git: version regex validation rejects --option-injection; checkout uses
--detach -- separator; removed TrimPrefix(repo,'v') that corrupted vault/
vitess/vim-go.
- WebSocket CheckOrigin allowlist (was return true; CSWSH closed); same-origin
default + ServerConfig.AllowedOrigins.
- fiber CVE GO-2026-4543 patched (v2.52.10 -> v2.52.12).
FUNCTIONAL FEATURES
- API key DB persistence: APIKeyModel + migration 202604280002, full CRUD,
async LastUsedAt updates with WaitGroup-tracked goroutines drained on Close.
- Admin bootstrap via GOHOARDER_BOOTSTRAP_ADMIN_KEY env var; idempotent
(skipped when non-revoked admin already exists).
- Auth middleware factory in pkg/app: RequireAuth, RequireRole, RequirePermission,
OptionalAuth. Mounted on proxy + read APIs when Auth.Enabled=true.
DELETE /api/packages/* requires admin role. /health public for k8s probes.
- NFS storage backend: pkg/storage/nfs wraps filesystem with /proc/mounts
detection (Linux), post-write fsync, read-after-write health probe.
- WebSocket real-time pipeline: pkg/events Broadcaster interface; cache + scanner
managers emit EventPackageCached / EventPackageDownloaded / EventScanComplete;
app.go runs 30s EventStatsUpdate ticker. Frontend has full WS client (reconnect
with exponential backoff 1s->30s, 25s heartbeat, subscriber pattern), Pinia
realtime store, Dashboard live indicator + activity feed + reactive stats
override.
- TLS termination: fiber.ListenTLS when Server.TLS.Enabled.
- Pre-warming: Prewarming.Enabled flag wired (was hardcoded false); Interval,
MaxConcurrent, TopPackages plumbed.
- NetworkConfig wired (timeouts, retry, rate-limit, circuit-breaker).
- AuthConfig.BcryptCost wired.
- Security.Scanners.Static.MaxPackageSize plumbed to cache.io.LimitReader.
- Handlers.{Go,NPM,PyPI}.Enabled gates route mounting.
- Graceful shutdown: authManager.Close drains async writes.
LINT/HYGIENE
- 80 golangci-lint issues resolved: errcheck (Close/Write/RemoveAll wrapped),
gofmt, gosec G104/G306, govet shadow renames + fieldalignment reorders,
staticcheck ST1000 pkg comments + QF1003 tagged switches + QF1008 embedded
field selectors + QF1001 De Morgan's law.
BEHAVIOR CHANGES
- Scanner now fail-closed: deployments without scanner binaries
(trivy/govulncheck/npm-audit/pip-audit/grype) BLOCK packages instead of
serving unscanned. Add binaries or plan a future allow-on-scan-error flag.
- WS origin defaults to same-origin; cross-origin dashboards must set
server.allowed_origins.
- Validator no longer fails open; private packages won't serve from cache when
validation can't be performed.
- download_events retention dropped from 24h to ~5min (deleted in agg tx).
Migration 202604280001 purges pre-upgrade events.
- DELETE /api/packages/* requires admin role when auth enabled.
NEW PACKAGES
- pkg/events - Broadcaster interface
- pkg/storage/nfs - NFS-aware filesystem wrapper
DEFERRED (out of scope this pass)
- Static scanner implementation (config defines AllowedLicenses/MaxPackageSize/
BlockSuspicious but pkg/scanner/static/ does not exist).
- AuthConfig.AuditLog wiring (no audit log infra yet).
- CacheConfig.TTLOverrides passthrough (would touch cache.Config beyond
integrator scope).
This commit is contained in:
@@ -1,3 +1,5 @@
|
||||
// Package file implements a JSON-on-disk metadata store used for local
|
||||
// development and tests.
|
||||
package file
|
||||
|
||||
import (
|
||||
@@ -544,3 +546,29 @@ func (s *Store) Close() error {
|
||||
// Nothing to close for file-based store
|
||||
return nil
|
||||
}
|
||||
|
||||
// SaveAPIKey is a stub: file-backed metadata does not persist API keys.
|
||||
// The auth.Manager treats ErrNotImplemented as "in-memory only" mode.
|
||||
func (s *Store) SaveAPIKey(ctx context.Context, key *metadata.APIKey) error {
|
||||
return metadata.ErrNotImplemented
|
||||
}
|
||||
|
||||
// GetAPIKey is a stub. See SaveAPIKey.
|
||||
func (s *Store) GetAPIKey(ctx context.Context, id string) (*metadata.APIKey, error) {
|
||||
return nil, metadata.ErrNotImplemented
|
||||
}
|
||||
|
||||
// ListAPIKeys is a stub. See SaveAPIKey.
|
||||
func (s *Store) ListAPIKeys(ctx context.Context) ([]*metadata.APIKey, error) {
|
||||
return nil, metadata.ErrNotImplemented
|
||||
}
|
||||
|
||||
// DeleteAPIKey is a stub. See SaveAPIKey.
|
||||
func (s *Store) DeleteAPIKey(ctx context.Context, id string) error {
|
||||
return metadata.ErrNotImplemented
|
||||
}
|
||||
|
||||
// UpdateAPIKeyLastUsed is a stub. See SaveAPIKey.
|
||||
func (s *Store) UpdateAPIKeyLastUsed(ctx context.Context, id string, t time.Time) error {
|
||||
return metadata.ErrNotImplemented
|
||||
}
|
||||
|
||||
@@ -66,7 +66,7 @@ func (w *AggregationWorker) AggregateHourly() error {
|
||||
log.Debug().Msg("Starting hourly aggregation")
|
||||
|
||||
// Get dialect name
|
||||
dialectName := w.db.Dialector.Name()
|
||||
dialectName := w.db.Name()
|
||||
|
||||
// Calculate cutoff time (aggregate events older than 5 minutes to avoid partial data)
|
||||
cutoff := time.Now().Add(-5 * time.Minute).Truncate(time.Hour)
|
||||
@@ -147,16 +147,22 @@ func (w *AggregationWorker) AggregateHourly() error {
|
||||
return err
|
||||
}
|
||||
|
||||
// Delete aggregated events (older than 24 hours to keep recent data for debugging)
|
||||
deleteOlder := time.Now().Add(-24 * time.Hour)
|
||||
deleteResult := tx.Exec("DELETE FROM download_events WHERE downloaded_at < ?", deleteOlder)
|
||||
// Delete aggregated events in the SAME transaction using the SAME cutoff
|
||||
// that drove the aggregation. This guarantees an event is counted exactly
|
||||
// once: prior to this commit it lives in download_events; after this commit
|
||||
// it's reflected only in download_stats_hourly. Any later run sees only
|
||||
// fresh events (downloaded_at >= cutoff at write time) and cannot
|
||||
// double-count rows that were already aggregated.
|
||||
// Trade-off: raw events older than `cutoff` are not retained for debugging.
|
||||
deleteResult := tx.Exec("DELETE FROM download_events WHERE downloaded_at < ?", cutoff)
|
||||
if deleteResult.Error != nil {
|
||||
return deleteResult.Error
|
||||
}
|
||||
|
||||
// Also update package-level stats (NULL package_id = registry totals)
|
||||
var registryAggSQL string
|
||||
if dialectName == "postgres" {
|
||||
switch dialectName {
|
||||
case "postgres":
|
||||
registryAggSQL = `
|
||||
INSERT INTO download_stats_hourly (registry_id, package_id, time_bucket, download_count, unique_ips, auth_downloads, created_at, updated_at)
|
||||
SELECT
|
||||
@@ -178,7 +184,7 @@ func (w *AggregationWorker) AggregateHourly() error {
|
||||
auth_downloads = EXCLUDED.auth_downloads,
|
||||
updated_at = NOW()
|
||||
`
|
||||
} else if dialectName == "mysql" {
|
||||
case "mysql":
|
||||
registryAggSQL = `
|
||||
INSERT INTO download_stats_hourly (registry_id, package_id, time_bucket, download_count, unique_ips, auth_downloads, created_at, updated_at)
|
||||
SELECT
|
||||
@@ -199,7 +205,7 @@ func (w *AggregationWorker) AggregateHourly() error {
|
||||
auth_downloads = VALUES(auth_downloads),
|
||||
updated_at = NOW()
|
||||
`
|
||||
} else {
|
||||
default:
|
||||
// SQLite
|
||||
registryAggSQL = `
|
||||
INSERT OR REPLACE INTO download_stats_hourly (registry_id, package_id, time_bucket, download_count, unique_ips, auth_downloads, created_at, updated_at)
|
||||
@@ -237,7 +243,7 @@ func (w *AggregationWorker) AggregateDaily() error {
|
||||
startTime := time.Now()
|
||||
log.Debug().Msg("Starting daily aggregation")
|
||||
|
||||
dialectName := w.db.Dialector.Name()
|
||||
dialectName := w.db.Name()
|
||||
|
||||
// Aggregate yesterday's data
|
||||
yesterday := time.Now().AddDate(0, 0, -1).Truncate(24 * time.Hour)
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
package gormstore
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/lukaszraczylo/gohoarder/pkg/errors"
|
||||
"github.com/lukaszraczylo/gohoarder/pkg/metadata"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
)
|
||||
|
||||
// SaveAPIKey persists an API key. Insert-or-update by primary key.
|
||||
func (s *GORMStoreV2) SaveAPIKey(ctx context.Context, key *metadata.APIKey) error {
|
||||
if key == nil || key.ID == "" {
|
||||
return errors.New(errors.ErrCodeBadRequest, "api key id required")
|
||||
}
|
||||
|
||||
model := &APIKeyModel{
|
||||
ID: key.ID,
|
||||
KeyHash: key.KeyHash,
|
||||
Project: key.Project,
|
||||
Role: key.Role,
|
||||
Permissions: key.Permissions,
|
||||
CreatedAt: key.CreatedAt,
|
||||
ExpiresAt: key.ExpiresAt,
|
||||
LastUsedAt: key.LastUsedAt,
|
||||
Revoked: key.Revoked,
|
||||
}
|
||||
if model.CreatedAt.IsZero() {
|
||||
model.CreatedAt = time.Now()
|
||||
}
|
||||
|
||||
// Upsert on primary key. Updates all columns on conflict — keeps the call
|
||||
// site simple (Generate then Revoke both flow through here).
|
||||
err := s.db.WithContext(ctx).
|
||||
Clauses(clause.OnConflict{
|
||||
Columns: []clause.Column{{Name: "id"}},
|
||||
UpdateAll: true,
|
||||
}).
|
||||
Create(model).Error
|
||||
if err != nil {
|
||||
return errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to save api key")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetAPIKey retrieves an API key by ID. Returns a not-found error if missing.
|
||||
func (s *GORMStoreV2) GetAPIKey(ctx context.Context, id string) (*metadata.APIKey, error) {
|
||||
var model APIKeyModel
|
||||
err := s.db.WithContext(ctx).Where("id = ?", id).First(&model).Error
|
||||
if err != nil {
|
||||
if err == gorm.ErrRecordNotFound {
|
||||
return nil, errors.New(errors.ErrCodeNotFound, "api key not found")
|
||||
}
|
||||
return nil, errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to get api key")
|
||||
}
|
||||
return apiKeyModelToMetadata(&model), nil
|
||||
}
|
||||
|
||||
// ListAPIKeys returns all API keys (revoked included). Caller filters as needed.
|
||||
func (s *GORMStoreV2) ListAPIKeys(ctx context.Context) ([]*metadata.APIKey, error) {
|
||||
var models []APIKeyModel
|
||||
if err := s.db.WithContext(ctx).Order("created_at DESC").Find(&models).Error; err != nil {
|
||||
return nil, errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to list api keys")
|
||||
}
|
||||
out := make([]*metadata.APIKey, len(models))
|
||||
for i := range models {
|
||||
out[i] = apiKeyModelToMetadata(&models[i])
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// DeleteAPIKey hard-deletes an API key by ID. Use SaveAPIKey with Revoked=true
|
||||
// for audit-friendly revocation; this method is for explicit purges.
|
||||
func (s *GORMStoreV2) DeleteAPIKey(ctx context.Context, id string) error {
|
||||
result := s.db.WithContext(ctx).Where("id = ?", id).Delete(&APIKeyModel{})
|
||||
if result.Error != nil {
|
||||
return errors.Wrap(result.Error, errors.ErrCodeStorageFailure, "failed to delete api key")
|
||||
}
|
||||
if result.RowsAffected == 0 {
|
||||
return errors.New(errors.ErrCodeNotFound, "api key not found")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// UpdateAPIKeyLastUsed updates only last_used_at. Cheap (single column UPDATE).
|
||||
// Auth manager calls this asynchronously per-request; do not perform extra
|
||||
// work here without measuring impact.
|
||||
func (s *GORMStoreV2) UpdateAPIKeyLastUsed(ctx context.Context, id string, t time.Time) error {
|
||||
result := s.db.WithContext(ctx).
|
||||
Model(&APIKeyModel{}).
|
||||
Where("id = ?", id).
|
||||
Update("last_used_at", t)
|
||||
if result.Error != nil {
|
||||
return errors.Wrap(result.Error, errors.ErrCodeStorageFailure, "failed to update api key last used")
|
||||
}
|
||||
// Missing rows are not fatal: caller validated the key against an
|
||||
// in-memory snapshot that may have lagged a concurrent revoke.
|
||||
return nil
|
||||
}
|
||||
|
||||
func apiKeyModelToMetadata(m *APIKeyModel) *metadata.APIKey {
|
||||
return &metadata.APIKey{
|
||||
ID: m.ID,
|
||||
KeyHash: m.KeyHash,
|
||||
Project: m.Project,
|
||||
Role: m.Role,
|
||||
Permissions: m.Permissions,
|
||||
CreatedAt: m.CreatedAt,
|
||||
ExpiresAt: m.ExpiresAt,
|
||||
LastUsedAt: m.LastUsedAt,
|
||||
Revoked: m.Revoked,
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,231 @@
|
||||
package gormstore
|
||||
|
||||
import (
|
||||
"context"
|
||||
stderrors "errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/lukaszraczylo/gohoarder/pkg/metadata"
|
||||
)
|
||||
|
||||
// newAPIKeyTestStore spins up an isolated SQLite store. Each test gets a
|
||||
// fresh DSN to avoid the shared-cache cross-test bleeding that the suite
|
||||
// uses elsewhere (we want a clean api_keys table).
|
||||
func newAPIKeyTestStore(t *testing.T) *GORMStoreV2 {
|
||||
t.Helper()
|
||||
// Use :memory: in the DSN so NewV2 skips the background aggregation
|
||||
// worker (it would otherwise contend on the shared cache and produce
|
||||
// "database table is locked" errors mid-test). MaxOpenConns=1 pins the
|
||||
// connection so the in-memory database survives across statements
|
||||
// (sqlite drops the DB when the only connection closes).
|
||||
cfg := Config{
|
||||
Driver: "sqlite",
|
||||
DSN: "file::memory:?_busy_timeout=5000",
|
||||
MaxOpenConns: 1,
|
||||
MaxIdleConns: 1,
|
||||
ConnMaxLifetime: time.Hour,
|
||||
LogLevel: "silent",
|
||||
}
|
||||
store, err := NewV2(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("NewV2: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
// Truncate first so cross-test isolation holds with shared cache.
|
||||
store.db.Exec("DELETE FROM api_keys")
|
||||
_ = store.Close()
|
||||
})
|
||||
return store
|
||||
}
|
||||
|
||||
func TestAPIKey_RoundTrip(t *testing.T) {
|
||||
store := newAPIKeyTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
expires := time.Now().Add(24 * time.Hour).UTC().Truncate(time.Second)
|
||||
key := &metadata.APIKey{
|
||||
ID: "test-id-1",
|
||||
KeyHash: "$2a$04$abcdefghijklmnopqrstuv",
|
||||
Project: "default",
|
||||
Role: "admin",
|
||||
CreatedAt: time.Now().UTC().Truncate(time.Second),
|
||||
ExpiresAt: &expires,
|
||||
}
|
||||
|
||||
if err := store.SaveAPIKey(ctx, key); err != nil {
|
||||
t.Fatalf("SaveAPIKey: %v", err)
|
||||
}
|
||||
|
||||
got, err := store.GetAPIKey(ctx, key.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetAPIKey: %v", err)
|
||||
}
|
||||
if got.ID != key.ID {
|
||||
t.Errorf("ID = %q, want %q", got.ID, key.ID)
|
||||
}
|
||||
if got.KeyHash != key.KeyHash {
|
||||
t.Errorf("KeyHash mismatch")
|
||||
}
|
||||
if got.Project != "default" {
|
||||
t.Errorf("Project = %q, want default", got.Project)
|
||||
}
|
||||
if got.Role != "admin" {
|
||||
t.Errorf("Role = %q, want admin", got.Role)
|
||||
}
|
||||
if got.Revoked {
|
||||
t.Errorf("Revoked = true, want false")
|
||||
}
|
||||
if got.ExpiresAt == nil || !got.ExpiresAt.Equal(expires) {
|
||||
t.Errorf("ExpiresAt = %v, want %v", got.ExpiresAt, expires)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIKey_Upsert(t *testing.T) {
|
||||
store := newAPIKeyTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
key := &metadata.APIKey{
|
||||
ID: "upsert-id",
|
||||
KeyHash: "hash1",
|
||||
Project: "p1",
|
||||
Role: "read_only",
|
||||
CreatedAt: time.Now().UTC().Truncate(time.Second),
|
||||
}
|
||||
if err := store.SaveAPIKey(ctx, key); err != nil {
|
||||
t.Fatalf("first SaveAPIKey: %v", err)
|
||||
}
|
||||
|
||||
// Update via Save (revoke).
|
||||
key.Revoked = true
|
||||
key.KeyHash = "hash2"
|
||||
if err := store.SaveAPIKey(ctx, key); err != nil {
|
||||
t.Fatalf("second SaveAPIKey: %v", err)
|
||||
}
|
||||
|
||||
got, err := store.GetAPIKey(ctx, key.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetAPIKey: %v", err)
|
||||
}
|
||||
if !got.Revoked {
|
||||
t.Errorf("Revoked = false after revoke")
|
||||
}
|
||||
if got.KeyHash != "hash2" {
|
||||
t.Errorf("KeyHash = %q, want hash2", got.KeyHash)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIKey_List_OrdersByCreatedDesc(t *testing.T) {
|
||||
store := newAPIKeyTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
now := time.Now().UTC().Truncate(time.Second)
|
||||
keys := []*metadata.APIKey{
|
||||
{ID: "k-old", KeyHash: "h", Project: "p", Role: "read_only", CreatedAt: now.Add(-2 * time.Hour)},
|
||||
{ID: "k-mid", KeyHash: "h", Project: "p", Role: "read_only", CreatedAt: now.Add(-1 * time.Hour)},
|
||||
{ID: "k-new", KeyHash: "h", Project: "p", Role: "read_only", CreatedAt: now},
|
||||
}
|
||||
for _, k := range keys {
|
||||
if err := store.SaveAPIKey(ctx, k); err != nil {
|
||||
t.Fatalf("save %s: %v", k.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
listed, err := store.ListAPIKeys(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListAPIKeys: %v", err)
|
||||
}
|
||||
if len(listed) != 3 {
|
||||
t.Fatalf("len = %d, want 3", len(listed))
|
||||
}
|
||||
wantOrder := []string{"k-new", "k-mid", "k-old"}
|
||||
for i, want := range wantOrder {
|
||||
if listed[i].ID != want {
|
||||
t.Errorf("listed[%d].ID = %q, want %q", i, listed[i].ID, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIKey_UpdateLastUsed(t *testing.T) {
|
||||
store := newAPIKeyTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
key := &metadata.APIKey{
|
||||
ID: "lu-id",
|
||||
KeyHash: "h",
|
||||
Project: "p",
|
||||
Role: "admin",
|
||||
CreatedAt: time.Now().UTC().Truncate(time.Second),
|
||||
}
|
||||
if err := store.SaveAPIKey(ctx, key); err != nil {
|
||||
t.Fatalf("SaveAPIKey: %v", err)
|
||||
}
|
||||
|
||||
used := time.Now().UTC().Truncate(time.Second).Add(time.Minute)
|
||||
if err := store.UpdateAPIKeyLastUsed(ctx, key.ID, used); err != nil {
|
||||
t.Fatalf("UpdateAPIKeyLastUsed: %v", err)
|
||||
}
|
||||
|
||||
got, err := store.GetAPIKey(ctx, key.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetAPIKey: %v", err)
|
||||
}
|
||||
if got.LastUsedAt == nil || !got.LastUsedAt.Equal(used) {
|
||||
t.Errorf("LastUsedAt = %v, want %v", got.LastUsedAt, used)
|
||||
}
|
||||
|
||||
// Updating a missing key must not error: validation can race revoke.
|
||||
if err := store.UpdateAPIKeyLastUsed(ctx, "no-such-id", used); err != nil {
|
||||
t.Errorf("UpdateAPIKeyLastUsed(missing) returned err: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIKey_Delete(t *testing.T) {
|
||||
store := newAPIKeyTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
key := &metadata.APIKey{
|
||||
ID: "del-id",
|
||||
KeyHash: "h",
|
||||
Project: "p",
|
||||
Role: "read_only",
|
||||
CreatedAt: time.Now().UTC().Truncate(time.Second),
|
||||
}
|
||||
if err := store.SaveAPIKey(ctx, key); err != nil {
|
||||
t.Fatalf("SaveAPIKey: %v", err)
|
||||
}
|
||||
if err := store.DeleteAPIKey(ctx, key.ID); err != nil {
|
||||
t.Fatalf("DeleteAPIKey: %v", err)
|
||||
}
|
||||
if _, err := store.GetAPIKey(ctx, key.ID); err == nil {
|
||||
t.Errorf("GetAPIKey after delete: want error, got nil")
|
||||
}
|
||||
// Second delete returns not-found.
|
||||
if err := store.DeleteAPIKey(ctx, key.ID); err == nil {
|
||||
t.Errorf("second DeleteAPIKey: want error, got nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIKey_SaveAPIKey_RejectsEmptyID(t *testing.T) {
|
||||
store := newAPIKeyTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
err := store.SaveAPIKey(ctx, &metadata.APIKey{KeyHash: "h"})
|
||||
if err == nil {
|
||||
t.Fatalf("want error for empty ID")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIKey_GetMissing_ReturnsNotFound(t *testing.T) {
|
||||
store := newAPIKeyTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
_, err := store.GetAPIKey(ctx, "missing")
|
||||
if err == nil {
|
||||
t.Fatalf("want error")
|
||||
}
|
||||
// Sanity: error chain doesn't accidentally surface ErrNotImplemented.
|
||||
if stderrors.Is(err, metadata.ErrNotImplemented) {
|
||||
t.Errorf("got ErrNotImplemented for missing key, expected not-found")
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,5 @@
|
||||
// Package gormstore implements the GORM-backed metadata store used for
|
||||
// production persistence (Postgres, MySQL, SQLite).
|
||||
package gormstore
|
||||
|
||||
import (
|
||||
@@ -13,13 +15,13 @@ type Config struct {
|
||||
Driver string // "sqlite", "postgres", "mysql"
|
||||
DSN string // Data Source Name
|
||||
|
||||
// GORM settings
|
||||
LogLevel string // "silent", "error", "warn", "info"
|
||||
|
||||
// Connection pool
|
||||
MaxOpenConns int
|
||||
MaxIdleConns int
|
||||
ConnMaxLifetime time.Duration
|
||||
|
||||
// GORM settings
|
||||
LogLevel string // "silent", "error", "warn", "info"
|
||||
}
|
||||
|
||||
// Validate validates the configuration
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/lukaszraczylo/gohoarder/pkg/errors"
|
||||
@@ -23,6 +24,7 @@ type GORMStoreV2 struct {
|
||||
registryCache map[string]int32 // Cache registry name -> ID mapping
|
||||
aggregationWorker *AggregationWorker
|
||||
partitionManager *PartitionManager
|
||||
registryCacheMu sync.RWMutex // Protects registryCache for concurrent access
|
||||
}
|
||||
|
||||
// NewV2 creates a new GORM-based metadata store with V2 schema
|
||||
@@ -102,7 +104,7 @@ func NewV2(cfg Config) (*GORMStoreV2, error) {
|
||||
}
|
||||
|
||||
// Seed default registries if empty
|
||||
if len(store.registryCache) == 0 {
|
||||
if store.registryCacheLen() == 0 {
|
||||
if err := store.seedDefaultRegistries(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -133,12 +135,21 @@ func (s *GORMStoreV2) loadRegistryCache() error {
|
||||
return errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to load registries")
|
||||
}
|
||||
|
||||
s.registryCacheMu.Lock()
|
||||
for _, r := range registries {
|
||||
s.registryCache[r.Name] = r.ID
|
||||
}
|
||||
s.registryCacheMu.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
// registryCacheLen returns current cache size (used after loading default seed data)
|
||||
func (s *GORMStoreV2) registryCacheLen() int {
|
||||
s.registryCacheMu.RLock()
|
||||
defer s.registryCacheMu.RUnlock()
|
||||
return len(s.registryCache)
|
||||
}
|
||||
|
||||
// seedDefaultRegistries creates default registry entries
|
||||
func (s *GORMStoreV2) seedDefaultRegistries() error {
|
||||
defaultRegistries := []RegistryModel{
|
||||
@@ -151,7 +162,9 @@ func (s *GORMStoreV2) seedDefaultRegistries() error {
|
||||
if err := s.db.Create(®).Error; err != nil {
|
||||
return errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to seed registry: "+reg.Name)
|
||||
}
|
||||
s.registryCacheMu.Lock()
|
||||
s.registryCache[reg.Name] = reg.ID
|
||||
s.registryCacheMu.Unlock()
|
||||
}
|
||||
|
||||
log.Info().Msg("Seeded default registries: npm, pypi, go")
|
||||
@@ -160,9 +173,13 @@ func (s *GORMStoreV2) seedDefaultRegistries() error {
|
||||
|
||||
// getRegistryID returns the registry ID from cache or database
|
||||
func (s *GORMStoreV2) getRegistryID(name string) (int32, error) {
|
||||
// Fast path: read lock for cache hit
|
||||
s.registryCacheMu.RLock()
|
||||
if id, ok := s.registryCache[name]; ok {
|
||||
s.registryCacheMu.RUnlock()
|
||||
return id, nil
|
||||
}
|
||||
s.registryCacheMu.RUnlock()
|
||||
|
||||
// Not in cache, try to load from database
|
||||
var reg RegistryModel
|
||||
@@ -173,7 +190,15 @@ func (s *GORMStoreV2) getRegistryID(name string) (int32, error) {
|
||||
return 0, errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to query registry")
|
||||
}
|
||||
|
||||
// Promote to write lock to populate cache
|
||||
s.registryCacheMu.Lock()
|
||||
// Re-check: another goroutine may have populated between RUnlock and Lock
|
||||
if existing, ok := s.registryCache[name]; ok {
|
||||
s.registryCacheMu.Unlock()
|
||||
return existing, nil
|
||||
}
|
||||
s.registryCache[name] = reg.ID
|
||||
s.registryCacheMu.Unlock()
|
||||
return reg.ID, nil
|
||||
}
|
||||
|
||||
@@ -212,10 +237,29 @@ func (s *GORMStoreV2) SavePackage(ctx context.Context, pkg *metadata.Package) er
|
||||
AuthProvider: pkg.AuthProvider,
|
||||
}
|
||||
|
||||
// Upsert package: first try to update, if no rows affected then create
|
||||
// Upsert package: first try to update, if no rows affected then create.
|
||||
// IMPORTANT: use map[string]interface{} so that zero values
|
||||
// (e.g. RequiresAuth=false, Size=0) are written explicitly. With Updates(struct)
|
||||
// GORM silently skips zero fields, which leaves stale security-relevant flags.
|
||||
updateFields := map[string]interface{}{
|
||||
"registry_id": registryID,
|
||||
"name": pkg.Name,
|
||||
"version": pkg.Version,
|
||||
"storage_key": pkg.StorageKey,
|
||||
"size": pkg.Size,
|
||||
"checksum_md5": pkg.ChecksumMD5,
|
||||
"checksum_sha256": pkg.ChecksumSHA256,
|
||||
"upstream_url": pkg.UpstreamURL,
|
||||
"cached_at": pkg.CachedAt,
|
||||
"last_accessed": pkg.LastAccessed,
|
||||
"expires_at": pkg.ExpiresAt,
|
||||
"requires_auth": pkg.RequiresAuth,
|
||||
"auth_provider": pkg.AuthProvider,
|
||||
}
|
||||
|
||||
result := tx.Model(&PackageModel{}).
|
||||
Where("registry_id = ? AND name = ? AND version = ?", registryID, pkg.Name, pkg.Version).
|
||||
Updates(model)
|
||||
Updates(updateFields)
|
||||
|
||||
if result.Error != nil {
|
||||
return errors.Wrap(result.Error, errors.ErrCodeStorageFailure, "failed to update package")
|
||||
@@ -226,6 +270,14 @@ func (s *GORMStoreV2) SavePackage(ctx context.Context, pkg *metadata.Package) er
|
||||
if err := tx.Create(model).Error; err != nil {
|
||||
return errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to create package")
|
||||
}
|
||||
} else {
|
||||
// On update, fetch the row's primary key for downstream metadata save
|
||||
if err := tx.Model(&PackageModel{}).
|
||||
Select("id").
|
||||
Where("registry_id = ? AND name = ? AND version = ?", registryID, pkg.Name, pkg.Version).
|
||||
First(model).Error; err != nil {
|
||||
return errors.Wrap(err, errors.ErrCodeStorageFailure, "failed to reload package id after update")
|
||||
}
|
||||
}
|
||||
|
||||
// Save metadata if present
|
||||
@@ -415,16 +467,15 @@ func (s *GORMStoreV2) ListPackages(ctx context.Context, opts *metadata.ListOptio
|
||||
|
||||
// Convert to metadata.Package
|
||||
packages := make([]*metadata.Package, len(models))
|
||||
// Snapshot id->name mapping under read lock once to avoid repeated locking
|
||||
s.registryCacheMu.RLock()
|
||||
idToName := make(map[int32]string, len(s.registryCache))
|
||||
for name, id := range s.registryCache {
|
||||
idToName[id] = name
|
||||
}
|
||||
s.registryCacheMu.RUnlock()
|
||||
for i, model := range models {
|
||||
// Get registry name from cache
|
||||
var regName string
|
||||
for name, id := range s.registryCache {
|
||||
if id == model.RegistryID {
|
||||
regName = name
|
||||
break
|
||||
}
|
||||
}
|
||||
packages[i] = s.modelToPackage(&model, regName)
|
||||
packages[i] = s.modelToPackage(&model, idToName[model.RegistryID])
|
||||
}
|
||||
|
||||
return packages, nil
|
||||
|
||||
@@ -73,7 +73,7 @@ func (s *GORMStoreV2TestSuite) TearDownTest() {
|
||||
s.store.db.Exec(fmt.Sprintf("DELETE FROM %s", table))
|
||||
}
|
||||
|
||||
s.store.Close()
|
||||
_ = s.store.Close()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -352,7 +352,7 @@ func (s *GORMStoreV2TestSuite) Test_V2_Count() {
|
||||
CachedAt: time.Now(),
|
||||
LastAccessed: time.Now(),
|
||||
}
|
||||
err := s.store.SavePackage(s.ctx, pkg)
|
||||
err = s.store.SavePackage(s.ctx, pkg)
|
||||
s.NoError(err)
|
||||
}
|
||||
|
||||
@@ -376,9 +376,9 @@ func (s *GORMStoreV2TestSuite) Test_V2_GetStats() {
|
||||
}
|
||||
|
||||
// Update download counts
|
||||
s.store.UpdateDownloadCount(s.ctx, "npm", "pkg1", "1.0.0")
|
||||
s.store.UpdateDownloadCount(s.ctx, "npm", "pkg1", "1.0.0")
|
||||
s.store.UpdateDownloadCount(s.ctx, "npm", "pkg2", "1.0.0")
|
||||
_ = s.store.UpdateDownloadCount(s.ctx, "npm", "pkg1", "1.0.0")
|
||||
_ = s.store.UpdateDownloadCount(s.ctx, "npm", "pkg1", "1.0.0")
|
||||
_ = s.store.UpdateDownloadCount(s.ctx, "npm", "pkg2", "1.0.0")
|
||||
|
||||
// Get stats for all registries
|
||||
statsAll, err := s.store.GetStats(s.ctx, "")
|
||||
@@ -486,7 +486,7 @@ func (s *GORMStoreV2TestSuite) Test_V2_ConcurrentUpdates() {
|
||||
// SQLite: Sequential updates only (write lock prevents concurrent writes)
|
||||
updateCount := 5
|
||||
for i := 0; i < updateCount; i++ {
|
||||
err := s.store.UpdateDownloadCount(s.ctx, "npm", "concurrent-test", "1.0.0")
|
||||
err = s.store.UpdateDownloadCount(s.ctx, "npm", "concurrent-test", "1.0.0")
|
||||
s.NoError(err)
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
package gormstore
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/go-gormigrate/gormigrate/v2"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
@@ -30,13 +32,51 @@ func GetMigrations() []*gormigrate.Migration {
|
||||
// return tx.Exec("ALTER TABLE packages DROP COLUMN new_field").Error
|
||||
// },
|
||||
// },
|
||||
{
|
||||
// Behavior change: aggregation worker now deletes raw download_events
|
||||
// in the same transaction as the hourly aggregation, using the same
|
||||
// cutoff. This prevents double-counting in multi-replica deployments
|
||||
// where the previous logic kept events for 24h and re-aggregated
|
||||
// them every run. No schema change required; we purge any stale
|
||||
// events older than the current aggregation cutoff so the new
|
||||
// invariant (events ⇔ unaggregated) holds from now on.
|
||||
ID: "202604280001",
|
||||
Migrate: func(tx *gorm.DB) error {
|
||||
// Best-effort cleanup. Safe to run repeatedly.
|
||||
cutoff := time.Now().Add(-5 * time.Minute)
|
||||
if err := tx.Exec(
|
||||
"DELETE FROM download_events WHERE downloaded_at < ?",
|
||||
cutoff,
|
||||
).Error; err != nil {
|
||||
// Table may not exist yet on a fresh install; tolerate.
|
||||
return nil
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Rollback: func(tx *gorm.DB) error {
|
||||
// Behavior change only — nothing to roll back schema-wise.
|
||||
return nil
|
||||
},
|
||||
},
|
||||
{
|
||||
// Add api_keys table for persistent authentication keys. Idempotent:
|
||||
// AutoMigrate is a no-op if the table already exists with the
|
||||
// expected columns.
|
||||
ID: "202604280002",
|
||||
Migrate: func(tx *gorm.DB) error {
|
||||
return tx.AutoMigrate(&APIKeyModel{})
|
||||
},
|
||||
Rollback: func(tx *gorm.DB) error {
|
||||
return tx.Migrator().DropTable("api_keys")
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// migrateToV2 creates the complete V2 schema
|
||||
func migrateToV2(tx *gorm.DB) error {
|
||||
// Get dialect name for database-specific features
|
||||
dialectName := tx.Dialector.Name()
|
||||
dialectName := tx.Name()
|
||||
|
||||
// Step 1: Create all tables using GORM AutoMigrate
|
||||
// This handles cross-database compatibility automatically
|
||||
@@ -59,11 +99,12 @@ func migrateToV2(tx *gorm.DB) error {
|
||||
}
|
||||
|
||||
// Step 3: Create database-specific optimizations
|
||||
if dialectName == "postgres" {
|
||||
switch dialectName {
|
||||
case "postgres":
|
||||
if err := createPostgreSQLOptimizations(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
} else if dialectName == "mysql" {
|
||||
case "mysql":
|
||||
if err := createMySQLOptimizations(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -192,6 +233,7 @@ func createMySQLOptimizations(tx *gorm.DB) error {
|
||||
func rollbackFromV2(tx *gorm.DB) error {
|
||||
// Drop in reverse order to respect foreign keys
|
||||
tables := []string{
|
||||
"api_keys",
|
||||
"audit_log",
|
||||
"download_stats_daily",
|
||||
"download_stats_hourly",
|
||||
@@ -206,13 +248,13 @@ func rollbackFromV2(tx *gorm.DB) error {
|
||||
}
|
||||
|
||||
// Drop PostgreSQL-specific objects
|
||||
if tx.Dialector.Name() == "postgres" {
|
||||
if tx.Name() == "postgres" {
|
||||
tx.Exec("DROP VIEW IF EXISTS v_vulnerable_packages")
|
||||
tx.Exec("DROP FUNCTION IF EXISTS create_next_month_partitions()")
|
||||
}
|
||||
|
||||
// Drop MySQL-specific objects
|
||||
if tx.Dialector.Name() == "mysql" {
|
||||
if tx.Name() == "mysql" {
|
||||
tx.Exec("DROP VIEW IF EXISTS v_vulnerable_packages")
|
||||
}
|
||||
|
||||
|
||||
+130
-101
@@ -18,13 +18,13 @@ type BaseModel struct {
|
||||
// RegistryModel represents package registries (normalized)
|
||||
// This eliminates repetition of "npm", "pypi", "go" across millions of rows
|
||||
type RegistryModel struct {
|
||||
ID int32 `gorm:"primaryKey;autoIncrement"`
|
||||
BaseModel
|
||||
Name string `gorm:"uniqueIndex:idx_registry_name;not null;size:50"` // npm, pypi, go
|
||||
DisplayName string `gorm:"not null;size:100"` // NPM Registry, PyPI, Go Modules
|
||||
UpstreamURL string `gorm:"not null;size:512"` // https://registry.npmjs.org
|
||||
ID int32 `gorm:"primaryKey;autoIncrement"`
|
||||
Enabled bool `gorm:"not null;default:true;index:idx_registry_enabled"`
|
||||
ScanByDefault bool `gorm:"not null;default:true"`
|
||||
BaseModel
|
||||
}
|
||||
|
||||
func (RegistryModel) TableName() string {
|
||||
@@ -33,45 +33,50 @@ func (RegistryModel) TableName() string {
|
||||
|
||||
// PackageModel represents the core package data (optimized)
|
||||
type PackageModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
RegistryID int32 `gorm:"not null;index:idx_package_registry_name_version,priority:1"` // Foreign key to registries
|
||||
Name string `gorm:"not null;size:255;index:idx_package_name;index:idx_package_registry_name_version,priority:2"`
|
||||
Version string `gorm:"not null;size:100;index:idx_package_registry_name_version,priority:3"`
|
||||
|
||||
// Cache management
|
||||
CachedAt time.Time `gorm:"not null;index:idx_package_cached_at"`
|
||||
LastAccessed time.Time `gorm:"not null;index:idx_package_last_accessed"` // For LRU eviction
|
||||
ExpiresAt *time.Time `gorm:"index:idx_package_expires_at"` // For cache invalidation
|
||||
LastScannedAt *time.Time `gorm:"index:idx_package_last_scanned"`
|
||||
Metadata *PackageMetadataModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
|
||||
// Relationships
|
||||
Registry RegistryModel `gorm:"foreignKey:RegistryID;constraint:OnUpdate:CASCADE,OnDelete:RESTRICT"`
|
||||
|
||||
BaseModel
|
||||
|
||||
Name string `gorm:"not null;size:255;index:idx_package_name;index:idx_package_registry_name_version,priority:2"`
|
||||
Version string `gorm:"not null;size:100;index:idx_package_registry_name_version,priority:3"`
|
||||
|
||||
// Storage information
|
||||
StorageKey string `gorm:"not null;uniqueIndex:idx_package_storage_key;size:512"`
|
||||
Size int64 `gorm:"not null;index:idx_package_size"` // For storage quota queries
|
||||
ChecksumMD5 string `gorm:"size:32;index:idx_package_md5"`
|
||||
ChecksumSHA256 string `gorm:"size:64;index:idx_package_sha256"`
|
||||
UpstreamURL string `gorm:"size:1024"`
|
||||
|
||||
// Cache management
|
||||
CachedAt time.Time `gorm:"not null;index:idx_package_cached_at"`
|
||||
LastAccessed time.Time `gorm:"not null;index:idx_package_last_accessed"` // For LRU eviction
|
||||
ExpiresAt *time.Time `gorm:"index:idx_package_expires_at"` // For cache invalidation
|
||||
AccessCount int64 `gorm:"not null;default:0;index:idx_package_access_count"` // Total access count (denormalized for performance)
|
||||
HighestSeverity string `gorm:"size:20;index:idx_package_severity"` // critical, high, medium, low, none
|
||||
AuthProvider string `gorm:"size:50;index:idx_package_auth_provider"` // github, gitlab, custom
|
||||
|
||||
// Security
|
||||
SecurityScanned bool `gorm:"not null;default:false;index:idx_package_security_scanned"`
|
||||
LastScannedAt *time.Time `gorm:"index:idx_package_last_scanned"`
|
||||
VulnerabilityCount int `gorm:"not null;default:0;index:idx_package_vuln_count"` // Denormalized for fast filtering
|
||||
HighestSeverity string `gorm:"size:20;index:idx_package_severity"` // critical, high, medium, low, none
|
||||
CriticalCount int `gorm:"not null;default:0"` // Count of critical vulnerabilities
|
||||
HighCount int `gorm:"not null;default:0"` // Count of high vulnerabilities
|
||||
ModerateCount int `gorm:"not null;default:0"` // Count of moderate vulnerabilities
|
||||
LowCount int `gorm:"not null;default:0"` // Count of low vulnerabilities
|
||||
|
||||
// Authentication
|
||||
RequiresAuth bool `gorm:"not null;default:false;index:idx_package_requires_auth"`
|
||||
AuthProvider string `gorm:"size:50;index:idx_package_auth_provider"` // github, gitlab, custom
|
||||
|
||||
BaseModel
|
||||
|
||||
// Relationships
|
||||
Registry RegistryModel `gorm:"foreignKey:RegistryID;constraint:OnUpdate:CASCADE,OnDelete:RESTRICT"`
|
||||
Metadata *PackageMetadataModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
ScanResults []ScanResultModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
Vulnerabilities []PackageVulnerabilityModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
Size int64 `gorm:"not null;index:idx_package_size"` // For storage quota queries
|
||||
AccessCount int64 `gorm:"not null;default:0;index:idx_package_access_count"` // Total access count (denormalized for performance)
|
||||
|
||||
VulnerabilityCount int `gorm:"not null;default:0;index:idx_package_vuln_count"` // Denormalized for fast filtering
|
||||
CriticalCount int `gorm:"not null;default:0"` // Count of critical vulnerabilities
|
||||
HighCount int `gorm:"not null;default:0"` // Count of high vulnerabilities
|
||||
ModerateCount int `gorm:"not null;default:0"` // Count of moderate vulnerabilities
|
||||
LowCount int `gorm:"not null;default:0"` // Count of low vulnerabilities
|
||||
|
||||
RegistryID int32 `gorm:"not null;index:idx_package_registry_name_version,priority:1"` // Foreign key to registries
|
||||
|
||||
// Security
|
||||
SecurityScanned bool `gorm:"not null;default:false;index:idx_package_security_scanned"`
|
||||
|
||||
// Authentication
|
||||
RequiresAuth bool `gorm:"not null;default:false;index:idx_package_requires_auth"`
|
||||
}
|
||||
|
||||
func (PackageModel) TableName() string {
|
||||
@@ -89,15 +94,15 @@ func (p *PackageModel) BeforeCreate(tx *gorm.DB) error {
|
||||
// PackageMetadataModel stores structured package metadata (1:1 with packages)
|
||||
// Separated from main table to reduce row size and improve query performance
|
||||
type PackageMetadataModel struct {
|
||||
PackageID int64 `gorm:"primaryKey;not null"` // 1:1 relationship
|
||||
RawMetadata JSONBField `gorm:"type:jsonb"` // Full metadata as JSONB (PostgreSQL) or JSON
|
||||
BaseModel
|
||||
Author string `gorm:"size:255;index:idx_metadata_author"`
|
||||
License string `gorm:"size:100;index:idx_metadata_license"`
|
||||
Homepage string `gorm:"size:512"`
|
||||
Repository string `gorm:"size:512"`
|
||||
Description string `gorm:"type:text"`
|
||||
Keywords PostgresArray `gorm:"type:text"` // JSONB array for PostgreSQL, JSON for MySQL/SQLite
|
||||
RawMetadata JSONBField `gorm:"type:jsonb"` // Full metadata as JSONB (PostgreSQL) or JSON
|
||||
BaseModel
|
||||
Keywords PostgresArray `gorm:"type:text"` // JSONB array for PostgreSQL, JSON for MySQL/SQLite
|
||||
PackageID int64 `gorm:"primaryKey;not null"` // 1:1 relationship
|
||||
}
|
||||
|
||||
func (PackageMetadataModel) TableName() string {
|
||||
@@ -106,21 +111,22 @@ func (PackageMetadataModel) TableName() string {
|
||||
|
||||
// ScanResultModel represents security scan results (optimized)
|
||||
type ScanResultModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
PackageID int64 `gorm:"not null;index:idx_scan_package_scanner,priority:1"` // Foreign key
|
||||
Scanner string `gorm:"not null;size:50;index:idx_scan_scanner;index:idx_scan_package_scanner,priority:2"`
|
||||
ScannedAt time.Time `gorm:"not null;index:idx_scan_scanned_at"`
|
||||
Status string `gorm:"not null;size:20;index:idx_scan_status"` // success, failed, pending
|
||||
VulnCount int `gorm:"not null;default:0;index:idx_scan_vuln_count"`
|
||||
CriticalCount int `gorm:"not null;default:0"`
|
||||
HighCount int `gorm:"not null;default:0"`
|
||||
MediumCount int `gorm:"not null;default:0"`
|
||||
LowCount int `gorm:"not null;default:0"`
|
||||
ScanDuration int `gorm:"not null;default:0"` // milliseconds
|
||||
Details JSONBField `gorm:"type:jsonb"` // Scanner-specific details
|
||||
ScannedAt time.Time `gorm:"not null;index:idx_scan_scanned_at"`
|
||||
Details JSONBField `gorm:"type:jsonb"` // Scanner-specific details
|
||||
BaseModel
|
||||
|
||||
Package PackageModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
Scanner string `gorm:"not null;size:50;index:idx_scan_scanner;index:idx_scan_package_scanner,priority:2"`
|
||||
Status string `gorm:"not null;size:20;index:idx_scan_status"` // success, failed, pending
|
||||
|
||||
Package PackageModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
PackageID int64 `gorm:"not null;index:idx_scan_package_scanner,priority:1"` // Foreign key
|
||||
VulnCount int `gorm:"not null;default:0;index:idx_scan_vuln_count"`
|
||||
CriticalCount int `gorm:"not null;default:0"`
|
||||
HighCount int `gorm:"not null;default:0"`
|
||||
MediumCount int `gorm:"not null;default:0"`
|
||||
LowCount int `gorm:"not null;default:0"`
|
||||
ScanDuration int `gorm:"not null;default:0"` // milliseconds
|
||||
}
|
||||
|
||||
func (ScanResultModel) TableName() string {
|
||||
@@ -129,16 +135,16 @@ func (ScanResultModel) TableName() string {
|
||||
|
||||
// VulnerabilityModel represents unique vulnerabilities (normalized)
|
||||
type VulnerabilityModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
PublishedAt time.Time `gorm:"not null;index:idx_vuln_published"`
|
||||
BaseModel
|
||||
CVEID string `gorm:"uniqueIndex:idx_vuln_cve_id;not null;size:50"` // CVE-2021-12345
|
||||
Title string `gorm:"not null;size:512"`
|
||||
Description string `gorm:"type:text"`
|
||||
Severity string `gorm:"not null;size:20;index:idx_vuln_severity"` // critical, high, medium, low
|
||||
CVSS float32 `gorm:"index:idx_vuln_cvss"` // CVSS score for sorting
|
||||
PublishedAt time.Time `gorm:"not null;index:idx_vuln_published"`
|
||||
FixedVersion string `gorm:"size:100"` // First version where it's fixed
|
||||
References PostgresArray `gorm:"type:text"` // URLs to advisories
|
||||
BaseModel
|
||||
FixedVersion string `gorm:"size:100"` // First version where it's fixed
|
||||
References PostgresArray `gorm:"type:text"` // URLs to advisories
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
CVSS float32 `gorm:"index:idx_vuln_cvss"` // CVSS score for sorting
|
||||
}
|
||||
|
||||
func (VulnerabilityModel) TableName() string {
|
||||
@@ -147,17 +153,18 @@ func (VulnerabilityModel) TableName() string {
|
||||
|
||||
// PackageVulnerabilityModel is a many-to-many relationship between packages and vulnerabilities
|
||||
type PackageVulnerabilityModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
PackageID int64 `gorm:"not null;index:idx_pkg_vuln_package,priority:1;index:idx_pkg_vuln_composite,priority:1"`
|
||||
VulnerabilityID int64 `gorm:"not null;index:idx_pkg_vuln_vuln,priority:1;index:idx_pkg_vuln_composite,priority:2"`
|
||||
Scanner string `gorm:"not null;size:50;index:idx_pkg_vuln_scanner"`
|
||||
DetectedAt time.Time `gorm:"not null;index:idx_pkg_vuln_detected"`
|
||||
Bypassed bool `gorm:"not null;default:false;index:idx_pkg_vuln_bypassed"`
|
||||
BypassID *int64 `gorm:"index:idx_pkg_vuln_bypass_id"` // Reference to bypass if applicable
|
||||
DetectedAt time.Time `gorm:"not null;index:idx_pkg_vuln_detected"`
|
||||
BypassID *int64 `gorm:"index:idx_pkg_vuln_bypass_id"` // Reference to bypass if applicable
|
||||
Vulnerability VulnerabilityModel `gorm:"foreignKey:VulnerabilityID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
BaseModel
|
||||
|
||||
Package PackageModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
Vulnerability VulnerabilityModel `gorm:"foreignKey:VulnerabilityID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
Scanner string `gorm:"not null;size:50;index:idx_pkg_vuln_scanner"`
|
||||
|
||||
Package PackageModel `gorm:"foreignKey:PackageID;constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
PackageID int64 `gorm:"not null;index:idx_pkg_vuln_package,priority:1;index:idx_pkg_vuln_composite,priority:1"`
|
||||
VulnerabilityID int64 `gorm:"not null;index:idx_pkg_vuln_vuln,priority:1;index:idx_pkg_vuln_composite,priority:2"`
|
||||
Bypassed bool `gorm:"not null;default:false;index:idx_pkg_vuln_bypassed"`
|
||||
}
|
||||
|
||||
func (PackageVulnerabilityModel) TableName() string {
|
||||
@@ -166,22 +173,22 @@ func (PackageVulnerabilityModel) TableName() string {
|
||||
|
||||
// CVEBypassModel represents CVE bypass rules (improved)
|
||||
type CVEBypassModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
Type string `gorm:"not null;size:20;index:idx_bypass_type"` // cve, package, registry
|
||||
Target string `gorm:"not null;size:512;index:idx_bypass_target"` // CVE-ID, package name, etc.
|
||||
Reason string `gorm:"not null;type:text"`
|
||||
CreatedBy string `gorm:"not null;size:255;index:idx_bypass_created_by"`
|
||||
ExpiresAt time.Time `gorm:"not null;index:idx_bypass_expires_at"`
|
||||
NotifyOnExpiry bool `gorm:"not null;default:false"`
|
||||
Active bool `gorm:"not null;default:true;index:idx_bypass_active"`
|
||||
UsageCount int64 `gorm:"not null;default:0"` // How many times this bypass has been used
|
||||
LastUsedAt *time.Time `gorm:"index:idx_bypass_last_used"`
|
||||
ExpiresAt time.Time `gorm:"not null;index:idx_bypass_expires_at"`
|
||||
LastUsedAt *time.Time `gorm:"index:idx_bypass_last_used"`
|
||||
|
||||
// Scope limiting (optional)
|
||||
RegistryID *int32 `gorm:"index:idx_bypass_registry"` // NULL = all registries
|
||||
PackageID *int64 `gorm:"index:idx_bypass_package"` // NULL = all packages
|
||||
|
||||
BaseModel
|
||||
Type string `gorm:"not null;size:20;index:idx_bypass_type"` // cve, package, registry
|
||||
Target string `gorm:"not null;size:512;index:idx_bypass_target"` // CVE-ID, package name, etc.
|
||||
Reason string `gorm:"not null;type:text"`
|
||||
CreatedBy string `gorm:"not null;size:255;index:idx_bypass_created_by"`
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
UsageCount int64 `gorm:"not null;default:0"` // How many times this bypass has been used
|
||||
NotifyOnExpiry bool `gorm:"not null;default:false"`
|
||||
Active bool `gorm:"not null;default:true;index:idx_bypass_active"`
|
||||
}
|
||||
|
||||
func (CVEBypassModel) TableName() string {
|
||||
@@ -191,17 +198,17 @@ func (CVEBypassModel) TableName() string {
|
||||
// DownloadEventModel represents raw download events (partitioned by month)
|
||||
// This table should use PostgreSQL partitioning or time-series DB features
|
||||
type DownloadEventModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
PackageID int64 `gorm:"not null;index:idx_download_package,priority:1"`
|
||||
RegistryID int32 `gorm:"not null;index:idx_download_registry"`
|
||||
DownloadedAt time.Time `gorm:"not null;index:idx_download_time;index:idx_download_package,priority:2"` // Partition key
|
||||
UserAgent string `gorm:"size:512"` // For analytics
|
||||
IPAddress string `gorm:"size:45;index:idx_download_ip"` // IPv6 support
|
||||
Authenticated bool `gorm:"not null;default:false"`
|
||||
Username string `gorm:"size:255;index:idx_download_user"`
|
||||
DownloadedAt time.Time `gorm:"not null;index:idx_download_time;index:idx_download_package,priority:2"` // Partition key
|
||||
UserAgent string `gorm:"size:512"` // For analytics
|
||||
IPAddress string `gorm:"size:45;index:idx_download_ip"` // IPv6 support
|
||||
Username string `gorm:"size:255;index:idx_download_user"`
|
||||
|
||||
// No BaseModel - this is append-only, no updates/deletes on individual rows
|
||||
// Partitioned tables handle cleanup via DROP PARTITION
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
PackageID int64 `gorm:"not null;index:idx_download_package,priority:1"`
|
||||
RegistryID int32 `gorm:"not null;index:idx_download_registry"`
|
||||
Authenticated bool `gorm:"not null;default:false"`
|
||||
}
|
||||
|
||||
func (DownloadEventModel) TableName() string {
|
||||
@@ -210,15 +217,16 @@ func (DownloadEventModel) TableName() string {
|
||||
|
||||
// DownloadStatsHourlyModel represents pre-aggregated hourly statistics (partitioned)
|
||||
type DownloadStatsHourlyModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
RegistryID int32 `gorm:"not null;index:idx_stats_hourly_composite,priority:1"`
|
||||
PackageID *int64 `gorm:"index:idx_stats_hourly_package"` // NULL = all packages in registry
|
||||
TimeBucket time.Time `gorm:"not null;index:idx_stats_hourly_composite,priority:2"` // Truncated to hour
|
||||
DownloadCount int64 `gorm:"not null;default:0"`
|
||||
UniqueIPs int64 `gorm:"not null;default:0"` // Unique downloaders
|
||||
AuthDownloads int64 `gorm:"not null;default:0"` // Authenticated downloads
|
||||
TimeBucket time.Time `gorm:"not null;index:idx_stats_hourly_composite,priority:2"` // Truncated to hour
|
||||
PackageID *int64 `gorm:"index:idx_stats_hourly_package"` // NULL = all packages in registry
|
||||
|
||||
BaseModel
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
DownloadCount int64 `gorm:"not null;default:0"`
|
||||
UniqueIPs int64 `gorm:"not null;default:0"` // Unique downloaders
|
||||
AuthDownloads int64 `gorm:"not null;default:0"` // Authenticated downloads
|
||||
|
||||
RegistryID int32 `gorm:"not null;index:idx_stats_hourly_composite,priority:1"`
|
||||
}
|
||||
|
||||
func (DownloadStatsHourlyModel) TableName() string {
|
||||
@@ -227,16 +235,16 @@ func (DownloadStatsHourlyModel) TableName() string {
|
||||
|
||||
// DownloadStatsDailyModel represents pre-aggregated daily statistics
|
||||
type DownloadStatsDailyModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
RegistryID int32 `gorm:"not null;index:idx_stats_daily_composite,priority:1"`
|
||||
PackageID *int64 `gorm:"index:idx_stats_daily_package"` // NULL = all packages in registry
|
||||
TimeBucket time.Time `gorm:"not null;index:idx_stats_daily_composite,priority:2"` // Truncated to day
|
||||
DownloadCount int64 `gorm:"not null;default:0"`
|
||||
UniqueIPs int64 `gorm:"not null;default:0"`
|
||||
AuthDownloads int64 `gorm:"not null;default:0"`
|
||||
TopUserAgents JSONBField `gorm:"type:jsonb"` // Top 10 user agents
|
||||
PackageID *int64 `gorm:"index:idx_stats_daily_package"` // NULL = all packages in registry
|
||||
TopUserAgents JSONBField `gorm:"type:jsonb"` // Top 10 user agents
|
||||
|
||||
BaseModel
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
DownloadCount int64 `gorm:"not null;default:0"`
|
||||
UniqueIPs int64 `gorm:"not null;default:0"`
|
||||
AuthDownloads int64 `gorm:"not null;default:0"`
|
||||
RegistryID int32 `gorm:"not null;index:idx_stats_daily_composite,priority:1"`
|
||||
}
|
||||
|
||||
func (DownloadStatsDailyModel) TableName() string {
|
||||
@@ -245,23 +253,43 @@ func (DownloadStatsDailyModel) TableName() string {
|
||||
|
||||
// AuditLogModel tracks all important changes (optional, for compliance)
|
||||
type AuditLogModel struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
EntityType string `gorm:"not null;size:50;index:idx_audit_entity_type"` // package, bypass, registry
|
||||
EntityID int64 `gorm:"not null;index:idx_audit_entity_id"`
|
||||
Action string `gorm:"not null;size:20;index:idx_audit_action"` // create, update, delete
|
||||
Username string `gorm:"not null;size:255;index:idx_audit_username"`
|
||||
Timestamp time.Time `gorm:"not null;index:idx_audit_timestamp"`
|
||||
Changes JSONBField `gorm:"type:jsonb"` // Before/after values
|
||||
Changes JSONBField `gorm:"type:jsonb"` // Before/after values
|
||||
EntityType string `gorm:"not null;size:50;index:idx_audit_entity_type"` // package, bypass, registry
|
||||
Action string `gorm:"not null;size:20;index:idx_audit_action"` // create, update, delete
|
||||
Username string `gorm:"not null;size:255;index:idx_audit_username"`
|
||||
IPAddress string `gorm:"size:45"`
|
||||
UserAgent string `gorm:"size:512"`
|
||||
|
||||
// No BaseModel - append-only audit log
|
||||
ID int64 `gorm:"primaryKey;autoIncrement"`
|
||||
EntityID int64 `gorm:"not null;index:idx_audit_entity_id"`
|
||||
}
|
||||
|
||||
func (AuditLogModel) TableName() string {
|
||||
return "audit_log"
|
||||
}
|
||||
|
||||
// APIKeyModel represents a persisted authentication API key.
|
||||
// Index on (id, project, revoked) is provided via composite index
|
||||
// idx_api_key_lookup so the auth manager can quickly enumerate
|
||||
// non-revoked keys per project at startup.
|
||||
type APIKeyModel struct {
|
||||
CreatedAt time.Time `gorm:"not null"`
|
||||
ExpiresAt *time.Time `gorm:"index:idx_api_key_expires"`
|
||||
LastUsedAt *time.Time `gorm:"index:idx_api_key_last_used"`
|
||||
ID string `gorm:"primaryKey;size:64;index:idx_api_key_lookup,priority:1"`
|
||||
KeyHash string `gorm:"not null;size:255"` // bcrypt hash
|
||||
Project string `gorm:"not null;size:255;index:idx_api_key_project;index:idx_api_key_lookup,priority:2"`
|
||||
Role string `gorm:"not null;size:32;index:idx_api_key_role"` // read_only, read_write, admin
|
||||
Permissions string `gorm:"type:text"` // JSON-encoded list (optional)
|
||||
Revoked bool `gorm:"not null;default:false;index:idx_api_key_revoked;index:idx_api_key_lookup,priority:3"`
|
||||
}
|
||||
|
||||
func (APIKeyModel) TableName() string {
|
||||
return "api_keys"
|
||||
}
|
||||
|
||||
// JSONBField is a custom type for JSONB (PostgreSQL) / JSON (MySQL/SQLite)
|
||||
type JSONBField map[string]interface{}
|
||||
|
||||
@@ -324,5 +352,6 @@ func GetAllModels() []interface{} {
|
||||
&DownloadStatsHourlyModel{},
|
||||
&DownloadStatsDailyModel{},
|
||||
&AuditLogModel{},
|
||||
&APIKeyModel{},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,12 +2,20 @@ package gormstore
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"time"
|
||||
|
||||
"github.com/rs/zerolog/log"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Whitelist regexes for partition table names — defense in depth before
|
||||
// we interpolate them into raw DDL.
|
||||
var (
|
||||
downloadEventPartitionRe = regexp.MustCompile(`^download_events_\d{4}_\d{2}$`)
|
||||
auditLogPartitionRe = regexp.MustCompile(`^audit_log_\d{4}_\d{2}$`)
|
||||
)
|
||||
|
||||
// PartitionManager handles automatic partition creation and cleanup for PostgreSQL
|
||||
type PartitionManager struct {
|
||||
db *gorm.DB
|
||||
@@ -21,7 +29,7 @@ func NewPartitionManager(db *gorm.DB) *PartitionManager {
|
||||
// EnsurePartitions ensures required partitions exist for current and future months
|
||||
func (pm *PartitionManager) EnsurePartitions() error {
|
||||
// Check if we're using PostgreSQL
|
||||
if pm.db.Dialector.Name() != "postgres" {
|
||||
if pm.db.Name() != "postgres" {
|
||||
log.Debug().Msg("Partitioning only supported on PostgreSQL, skipping")
|
||||
return nil
|
||||
}
|
||||
@@ -286,7 +294,7 @@ func (pm *PartitionManager) createPartitionFunction() error {
|
||||
|
||||
// CleanupOldPartitions drops partitions older than the retention period
|
||||
func (pm *PartitionManager) CleanupOldPartitions(retentionMonths int) error {
|
||||
if pm.db.Dialector.Name() != "postgres" {
|
||||
if pm.db.Name() != "postgres" {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -300,39 +308,45 @@ func (pm *PartitionManager) CleanupOldPartitions(retentionMonths int) error {
|
||||
|
||||
// Find and drop old download_events partitions
|
||||
var downloadPartitions []string
|
||||
err := pm.db.Raw(`
|
||||
if err := pm.db.Raw(`
|
||||
SELECT tablename FROM pg_tables
|
||||
WHERE tablename LIKE 'download_events_%'
|
||||
AND tablename < 'download_events_' || ?
|
||||
`, cutoffPartition).Scan(&downloadPartitions).Error
|
||||
|
||||
if err != nil {
|
||||
`, cutoffPartition).Scan(&downloadPartitions).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, partition := range downloadPartitions {
|
||||
// Defense in depth: even though tablename came from pg_tables, verify it
|
||||
// matches the expected partition naming scheme before interpolating into DDL.
|
||||
if !downloadEventPartitionRe.MatchString(partition) {
|
||||
log.Warn().Str("partition", partition).Msg("Skipping partition with unexpected name")
|
||||
continue
|
||||
}
|
||||
log.Info().Str("partition", partition).Msg("Dropping old partition")
|
||||
if err := pm.db.Exec(fmt.Sprintf("DROP TABLE IF EXISTS %s", partition)).Error; err != nil {
|
||||
log.Error().Err(err).Str("partition", partition).Msg("Failed to drop partition")
|
||||
if dropErr := pm.db.Exec(fmt.Sprintf("DROP TABLE IF EXISTS %s", partition)).Error; dropErr != nil {
|
||||
log.Error().Err(dropErr).Str("partition", partition).Msg("Failed to drop partition")
|
||||
}
|
||||
}
|
||||
|
||||
// Find and drop old audit_log partitions
|
||||
var auditPartitions []string
|
||||
err = pm.db.Raw(`
|
||||
if err := pm.db.Raw(`
|
||||
SELECT tablename FROM pg_tables
|
||||
WHERE tablename LIKE 'audit_log_%'
|
||||
AND tablename < 'audit_log_' || ?
|
||||
`, cutoffPartition).Scan(&auditPartitions).Error
|
||||
|
||||
if err != nil {
|
||||
`, cutoffPartition).Scan(&auditPartitions).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, partition := range auditPartitions {
|
||||
if !auditLogPartitionRe.MatchString(partition) {
|
||||
log.Warn().Str("partition", partition).Msg("Skipping audit partition with unexpected name")
|
||||
continue
|
||||
}
|
||||
log.Info().Str("partition", partition).Msg("Dropping old audit partition")
|
||||
if err := pm.db.Exec(fmt.Sprintf("DROP TABLE IF EXISTS %s", partition)).Error; err != nil {
|
||||
log.Error().Err(err).Str("partition", partition).Msg("Failed to drop audit partition")
|
||||
if dropErr := pm.db.Exec(fmt.Sprintf("DROP TABLE IF EXISTS %s", partition)).Error; dropErr != nil {
|
||||
log.Error().Err(dropErr).Str("partition", partition).Msg("Failed to drop audit partition")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -341,7 +355,7 @@ func (pm *PartitionManager) CleanupOldPartitions(retentionMonths int) error {
|
||||
|
||||
// GetPartitionInfo returns information about current partitions
|
||||
func (pm *PartitionManager) GetPartitionInfo() (map[string]interface{}, error) {
|
||||
if pm.db.Dialector.Name() != "postgres" {
|
||||
if pm.db.Name() != "postgres" {
|
||||
return map[string]interface{}{"status": "not_applicable"}, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -1,11 +1,19 @@
|
||||
// Package metadata defines the Store interface and shared model types for
|
||||
// package metadata persistence backends.
|
||||
package metadata
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ErrNotImplemented signals that an optional MetadataStore method is not
|
||||
// supported by the current backend (e.g. a backend may not persist API keys).
|
||||
// Callers should treat it as a non-fatal capability check.
|
||||
var ErrNotImplemented = errors.New("metadata: operation not implemented by this backend")
|
||||
|
||||
// Store is an alias for MetadataStore for convenience
|
||||
type Store = MetadataStore
|
||||
|
||||
@@ -62,10 +70,48 @@ type MetadataStore interface {
|
||||
// AggregateDownloadData aggregates raw download events and cleans up old data
|
||||
AggregateDownloadData(ctx context.Context) error
|
||||
|
||||
// SaveAPIKey persists (insert or update) an API key record. Backends that
|
||||
// do not implement persistent API keys MUST return ErrNotImplemented.
|
||||
SaveAPIKey(ctx context.Context, key *APIKey) error
|
||||
|
||||
// GetAPIKey retrieves a single API key by ID. Returns ErrNotImplemented if
|
||||
// the backend does not persist API keys. Returns a not-found error wrapped
|
||||
// from the backend if the key does not exist.
|
||||
GetAPIKey(ctx context.Context, id string) (*APIKey, error)
|
||||
|
||||
// ListAPIKeys returns all API keys (including revoked) so callers can
|
||||
// decide whether to filter. Returns ErrNotImplemented for backends that
|
||||
// do not persist API keys.
|
||||
ListAPIKeys(ctx context.Context) ([]*APIKey, error)
|
||||
|
||||
// DeleteAPIKey removes an API key by ID. Returns ErrNotImplemented for
|
||||
// backends that do not persist API keys.
|
||||
DeleteAPIKey(ctx context.Context, id string) error
|
||||
|
||||
// UpdateAPIKeyLastUsed updates the last-used timestamp without rewriting
|
||||
// the rest of the row. Implementations should make this cheap; auth.Manager
|
||||
// may invoke it asynchronously on every successful validation.
|
||||
UpdateAPIKeyLastUsed(ctx context.Context, id string, t time.Time) error
|
||||
|
||||
// Close closes the metadata store
|
||||
Close() error
|
||||
}
|
||||
|
||||
// APIKey is the storage-layer representation of an authentication API key.
|
||||
// It is intentionally separate from auth.APIKey to avoid an import cycle:
|
||||
// the auth package depends on metadata, never the other way around.
|
||||
type APIKey struct {
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
ExpiresAt *time.Time `json:"expires_at,omitempty"`
|
||||
LastUsedAt *time.Time `json:"last_used_at,omitempty"`
|
||||
ID string `json:"id"`
|
||||
KeyHash string `json:"key_hash"`
|
||||
Project string `json:"project"`
|
||||
Role string `json:"role"`
|
||||
Permissions string `json:"permissions,omitempty"` // JSON-encoded list (optional)
|
||||
Revoked bool `json:"revoked"`
|
||||
}
|
||||
|
||||
// Package represents package metadata
|
||||
type Package struct {
|
||||
CachedAt time.Time `json:"cached_at"`
|
||||
|
||||
Reference in New Issue
Block a user