mirror of
https://github.com/lukaszraczylo/gohoarder.git
synced 2026-07-13 04:57:11 +00:00
e39d6a0f0d
Multi-agent audit + fix pass covering bugs, security, dead config wiring,
and missing features. All quality gates green: build/vet/test -race/govulncheck/
golangci-lint. Frontend tests 47/47.
SECURITY & CORRECTNESS
- Scanner pipeline fail-closed: cache.go scan timeout now 503 (was goto servePkg);
scanner.go all-scanners-fail saves ScanStatusError; CheckVulnerabilities blocks
on missing/error result instead of allowing.
- Scanner argv corrections: govulncheck source-mode + go.mod discovery;
npm-audit generates lockfile via npm install --package-lock-only --ignore-scripts;
pip-audit per-extension dispatch (wheel direct / sdist extract+pyproject).
- GHSA version-range filtering implemented (was always-include); url.QueryEscape
on package name.
- pypi SSRF closed via host allowlist on original_url; url.QueryEscape on
rewriteURL output.
- Path traversal: cache temp file uses os.CreateTemp; smb keyToPath sanitized;
filesystem keyToPath returns error + filepath.Abs prefix-verify.
- Goroutine leaks plugged: cache.cleanupWorker stop channel; auth.ValidationCache
Stop() with sync.Once; ws.unregister buffered with non-blocking send.
- WebSocket double-close panic fixed via sync.Once Client.closeSend().
- Race conditions: gormstore.registryCache sync.RWMutex (was concurrent map
panic); auth.LastUsedAt no longer mutated under RLock; analytics strict
lock-ordering invariant (statsMu before downloadsMu).
- Auth validator fails CLOSED on transport/unknown-status errors (was returning
true,err 'allow cache fallback').
- Credential cache hash full SHA256 (was 8-byte truncate; eliminates 2^32
collision risk).
- filesystem.fs.used incremented after successful rename (no quota inflation
race).
- gormstore aggregation events deleted in same tx as insert (no double-counting).
- gormstore.SavePackage uses Updates(map) so zero-value security flags
(RequiresAuth=false) can be cleared.
- partition_manager validates partition name regex before DROP TABLE.
- vcs/git: version regex validation rejects --option-injection; checkout uses
--detach -- separator; removed TrimPrefix(repo,'v') that corrupted vault/
vitess/vim-go.
- WebSocket CheckOrigin allowlist (was return true; CSWSH closed); same-origin
default + ServerConfig.AllowedOrigins.
- fiber CVE GO-2026-4543 patched (v2.52.10 -> v2.52.12).
FUNCTIONAL FEATURES
- API key DB persistence: APIKeyModel + migration 202604280002, full CRUD,
async LastUsedAt updates with WaitGroup-tracked goroutines drained on Close.
- Admin bootstrap via GOHOARDER_BOOTSTRAP_ADMIN_KEY env var; idempotent
(skipped when non-revoked admin already exists).
- Auth middleware factory in pkg/app: RequireAuth, RequireRole, RequirePermission,
OptionalAuth. Mounted on proxy + read APIs when Auth.Enabled=true.
DELETE /api/packages/* requires admin role. /health public for k8s probes.
- NFS storage backend: pkg/storage/nfs wraps filesystem with /proc/mounts
detection (Linux), post-write fsync, read-after-write health probe.
- WebSocket real-time pipeline: pkg/events Broadcaster interface; cache + scanner
managers emit EventPackageCached / EventPackageDownloaded / EventScanComplete;
app.go runs 30s EventStatsUpdate ticker. Frontend has full WS client (reconnect
with exponential backoff 1s->30s, 25s heartbeat, subscriber pattern), Pinia
realtime store, Dashboard live indicator + activity feed + reactive stats
override.
- TLS termination: fiber.ListenTLS when Server.TLS.Enabled.
- Pre-warming: Prewarming.Enabled flag wired (was hardcoded false); Interval,
MaxConcurrent, TopPackages plumbed.
- NetworkConfig wired (timeouts, retry, rate-limit, circuit-breaker).
- AuthConfig.BcryptCost wired.
- Security.Scanners.Static.MaxPackageSize plumbed to cache.io.LimitReader.
- Handlers.{Go,NPM,PyPI}.Enabled gates route mounting.
- Graceful shutdown: authManager.Close drains async writes.
LINT/HYGIENE
- 80 golangci-lint issues resolved: errcheck (Close/Write/RemoveAll wrapped),
gofmt, gosec G104/G306, govet shadow renames + fieldalignment reorders,
staticcheck ST1000 pkg comments + QF1003 tagged switches + QF1008 embedded
field selectors + QF1001 De Morgan's law.
BEHAVIOR CHANGES
- Scanner now fail-closed: deployments without scanner binaries
(trivy/govulncheck/npm-audit/pip-audit/grype) BLOCK packages instead of
serving unscanned. Add binaries or plan a future allow-on-scan-error flag.
- WS origin defaults to same-origin; cross-origin dashboards must set
server.allowed_origins.
- Validator no longer fails open; private packages won't serve from cache when
validation can't be performed.
- download_events retention dropped from 24h to ~5min (deleted in agg tx).
Migration 202604280001 purges pre-upgrade events.
- DELETE /api/packages/* requires admin role when auth enabled.
NEW PACKAGES
- pkg/events - Broadcaster interface
- pkg/storage/nfs - NFS-aware filesystem wrapper
DEFERRED (out of scope this pass)
- Static scanner implementation (config defines AllowedLicenses/MaxPackageSize/
BlockSuspicious but pkg/scanner/static/ does not exist).
- AuthConfig.AuditLog wiring (no audit log infra yet).
- CacheConfig.TTLOverrides passthrough (would touch cache.Config beyond
integrator scope).
302 lines
7.2 KiB
TypeScript
302 lines
7.2 KiB
TypeScript
/**
|
|
* Lightweight WebSocket client with auto-reconnect, heartbeat, and pub/sub.
|
|
*
|
|
* Backend (pkg/websocket/server.go) emits envelopes shaped like:
|
|
* { "type": "package_cached", "timestamp": "2026-04-28T12:34:56Z", "data": {...} }
|
|
*
|
|
* We normalize this into RealtimeEvent { type, payload, timestamp(number ms) }.
|
|
*/
|
|
|
|
export type EventType =
|
|
| 'package_deleted'
|
|
| 'package_cached'
|
|
| 'package_downloaded'
|
|
| 'scan_complete'
|
|
| 'stats_update'
|
|
|
|
export interface RealtimeEvent {
|
|
type: EventType
|
|
payload: Record<string, unknown>
|
|
timestamp: number
|
|
}
|
|
|
|
type WildcardListener = '*'
|
|
type ListenerKey = EventType | WildcardListener
|
|
type Listener = (e: RealtimeEvent) => void
|
|
|
|
const KNOWN_EVENT_TYPES: ReadonlySet<string> = new Set<EventType>([
|
|
'package_deleted',
|
|
'package_cached',
|
|
'package_downloaded',
|
|
'scan_complete',
|
|
'stats_update',
|
|
])
|
|
|
|
const RECONNECT_BASE_MS = 1_000
|
|
const RECONNECT_MAX_MS = 30_000
|
|
const HEARTBEAT_INTERVAL_MS = 25_000
|
|
const RECONNECT_QUEUE_CAP = 50
|
|
|
|
interface OutboundMessage {
|
|
action: string
|
|
data?: unknown
|
|
}
|
|
|
|
function defaultUrl(): string {
|
|
if (typeof window === 'undefined' || !window.location) {
|
|
return 'ws://localhost/ws'
|
|
}
|
|
const proto = window.location.protocol === 'https:' ? 'wss:' : 'ws:'
|
|
return `${proto}//${window.location.host}/ws`
|
|
}
|
|
|
|
function isKnownEventType(value: unknown): value is EventType {
|
|
return typeof value === 'string' && KNOWN_EVENT_TYPES.has(value)
|
|
}
|
|
|
|
function coerceTimestamp(raw: unknown): number {
|
|
if (typeof raw === 'number' && Number.isFinite(raw)) {
|
|
return raw
|
|
}
|
|
if (typeof raw === 'string') {
|
|
const parsed = Date.parse(raw)
|
|
if (!Number.isNaN(parsed)) return parsed
|
|
}
|
|
return Date.now()
|
|
}
|
|
|
|
function parseEvent(text: string): RealtimeEvent | null {
|
|
let parsed: unknown
|
|
try {
|
|
parsed = JSON.parse(text)
|
|
} catch {
|
|
return null
|
|
}
|
|
if (!parsed || typeof parsed !== 'object') return null
|
|
const obj = parsed as Record<string, unknown>
|
|
|
|
// Server may also emit `{ type: "pong" }` style control frames; ignore those.
|
|
if (!isKnownEventType(obj.type)) return null
|
|
|
|
const payloadSource = obj.payload ?? obj.data
|
|
const payload =
|
|
payloadSource && typeof payloadSource === 'object'
|
|
? (payloadSource as Record<string, unknown>)
|
|
: {}
|
|
|
|
return {
|
|
type: obj.type,
|
|
payload,
|
|
timestamp: coerceTimestamp(obj.timestamp),
|
|
}
|
|
}
|
|
|
|
export class WSClient {
|
|
readonly url: string
|
|
private socket: WebSocket | null = null
|
|
private listeners: Map<ListenerKey, Set<Listener>> = new Map()
|
|
private heartbeatTimer: ReturnType<typeof setInterval> | null = null
|
|
private reconnectTimer: ReturnType<typeof setTimeout> | null = null
|
|
private reconnectAttempts = 0
|
|
private explicitlyClosed = false
|
|
private outboundQueue: OutboundMessage[] = []
|
|
private _connected = false
|
|
|
|
constructor(url?: string) {
|
|
this.url = url ?? defaultUrl()
|
|
}
|
|
|
|
get connected(): boolean {
|
|
return this._connected
|
|
}
|
|
|
|
connect(): void {
|
|
this.explicitlyClosed = false
|
|
this.openSocket()
|
|
}
|
|
|
|
close(): void {
|
|
this.explicitlyClosed = true
|
|
this.clearReconnect()
|
|
this.stopHeartbeat()
|
|
this._connected = false
|
|
if (this.socket) {
|
|
try {
|
|
this.socket.close()
|
|
} catch {
|
|
/* swallow */
|
|
}
|
|
this.socket = null
|
|
}
|
|
}
|
|
|
|
on(type: ListenerKey, cb: Listener): () => void {
|
|
let bucket = this.listeners.get(type)
|
|
if (!bucket) {
|
|
bucket = new Set()
|
|
this.listeners.set(type, bucket)
|
|
}
|
|
bucket.add(cb)
|
|
return () => {
|
|
const current = this.listeners.get(type)
|
|
if (!current) return
|
|
current.delete(cb)
|
|
if (current.size === 0) this.listeners.delete(type)
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Send a message. If not connected, queue (capped) and flush on open.
|
|
*/
|
|
send(msg: OutboundMessage): void {
|
|
if (this.socket && this._connected && this.socket.readyState === 1) {
|
|
try {
|
|
this.socket.send(JSON.stringify(msg))
|
|
return
|
|
} catch {
|
|
/* fall through to queue */
|
|
}
|
|
}
|
|
if (this.outboundQueue.length >= RECONNECT_QUEUE_CAP) {
|
|
this.outboundQueue.shift()
|
|
}
|
|
this.outboundQueue.push(msg)
|
|
}
|
|
|
|
private openSocket(): void {
|
|
const Ctor: typeof WebSocket | undefined =
|
|
typeof WebSocket !== 'undefined'
|
|
? WebSocket
|
|
: (globalThis as unknown as { WebSocket?: typeof WebSocket }).WebSocket
|
|
if (!Ctor) {
|
|
// No WebSocket available (e.g. SSR); silently bail.
|
|
return
|
|
}
|
|
|
|
let socket: WebSocket
|
|
try {
|
|
socket = new Ctor(this.url)
|
|
} catch {
|
|
this.scheduleReconnect()
|
|
return
|
|
}
|
|
this.socket = socket
|
|
|
|
socket.onopen = () => {
|
|
this._connected = true
|
|
this.reconnectAttempts = 0
|
|
this.startHeartbeat()
|
|
this.flushQueue()
|
|
}
|
|
|
|
socket.onmessage = (ev: MessageEvent) => {
|
|
const data = ev.data
|
|
if (typeof data !== 'string') return
|
|
const evt = parseEvent(data)
|
|
if (!evt) return
|
|
this.dispatch(evt)
|
|
}
|
|
|
|
socket.onerror = () => {
|
|
// onclose handles reconnect; nothing to do here.
|
|
}
|
|
|
|
socket.onclose = () => {
|
|
this._connected = false
|
|
this.stopHeartbeat()
|
|
this.socket = null
|
|
if (!this.explicitlyClosed) {
|
|
this.scheduleReconnect()
|
|
}
|
|
}
|
|
}
|
|
|
|
private dispatch(event: RealtimeEvent): void {
|
|
const exact = this.listeners.get(event.type)
|
|
if (exact) {
|
|
for (const cb of exact) {
|
|
try {
|
|
cb(event)
|
|
} catch (err) {
|
|
console.error('[ws] listener error', err)
|
|
}
|
|
}
|
|
}
|
|
const wild = this.listeners.get('*')
|
|
if (wild) {
|
|
for (const cb of wild) {
|
|
try {
|
|
cb(event)
|
|
} catch (err) {
|
|
console.error('[ws] listener error', err)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
private flushQueue(): void {
|
|
if (!this.socket || !this._connected) return
|
|
const queued = this.outboundQueue.splice(0)
|
|
for (const msg of queued) {
|
|
try {
|
|
this.socket.send(JSON.stringify(msg))
|
|
} catch {
|
|
// requeue and bail
|
|
this.outboundQueue.unshift(msg)
|
|
return
|
|
}
|
|
}
|
|
}
|
|
|
|
private startHeartbeat(): void {
|
|
this.stopHeartbeat()
|
|
this.heartbeatTimer = setInterval(() => {
|
|
if (this.socket && this._connected) {
|
|
try {
|
|
this.socket.send(JSON.stringify({ action: 'ping' }))
|
|
} catch {
|
|
/* swallow; reconnect path will handle */
|
|
}
|
|
}
|
|
}, HEARTBEAT_INTERVAL_MS)
|
|
}
|
|
|
|
private stopHeartbeat(): void {
|
|
if (this.heartbeatTimer !== null) {
|
|
clearInterval(this.heartbeatTimer)
|
|
this.heartbeatTimer = null
|
|
}
|
|
}
|
|
|
|
private scheduleReconnect(): void {
|
|
if (this.explicitlyClosed) return
|
|
this.clearReconnect()
|
|
const delay = Math.min(
|
|
RECONNECT_MAX_MS,
|
|
RECONNECT_BASE_MS * 2 ** this.reconnectAttempts,
|
|
)
|
|
this.reconnectAttempts += 1
|
|
this.reconnectTimer = setTimeout(() => {
|
|
this.reconnectTimer = null
|
|
this.openSocket()
|
|
}, delay)
|
|
}
|
|
|
|
private clearReconnect(): void {
|
|
if (this.reconnectTimer !== null) {
|
|
clearTimeout(this.reconnectTimer)
|
|
this.reconnectTimer = null
|
|
}
|
|
}
|
|
}
|
|
|
|
// Test-only helpers — exported under a namespace so production code does not
|
|
// reach into reconnect math accidentally.
|
|
export const __wsInternals = {
|
|
RECONNECT_BASE_MS,
|
|
RECONNECT_MAX_MS,
|
|
HEARTBEAT_INTERVAL_MS,
|
|
parseEvent,
|
|
defaultUrl,
|
|
}
|