Clear reset the counters with StoreInt64(0) after deleting entries, so
a Set racing the reset left a live entry while counters read zero, and
the next Delete drove them permanently negative.
Rewrite Clear to LoadAndDelete each key and subtract the linearized
returned entry, matching the rest of the file. Size the key slice from
the live entry count.
The SCAN MATCH pattern used the raw prefix, so a prefix containing
Redis glob metacharacters cleared the wrong keys or none. A Del error
also aborted the clear and discarded the remaining batches.
Escape the prefix for MATCH, continue past a per-batch Del error, and
return the joined error. Apply the same escaping to CountQueries.
truncateUTF8 validated the whole string each iteration. One
invalid byte emptied the log line at quadratic cost. A secret
whose closing quote fell past the truncation cut also leaked its
leading characters, because the redaction pattern needs a
terminator.
Back off over the last rune only, bounded by the maximum rune
width, and trim the partial rune before redaction runs. Strip a
trailing unterminated quoted value after the cut.
When the primary request panicked, waiters received a nil response
and nil error, which the proxy turned into an empty HTTP 200.
Recover in the deferred release, store an error response for the
waiters, then re-panic so the primary panic semantics are
unchanged. The test gates the panic on the live waiter count, so
every waiter is coalesced before the release.
The Set mutex serialised writes but left Delete, expiry, and
eviction racing the same counters, so double subtraction remained.
Eviction also pre-allocated from the configured maximum, which
could allocate hundreds of megabytes on a near-empty cache.
Use sync.Map.Swap in Set and adjust counters from the linearized
previous value. Size the eviction slice from the live entry count.
CleanExpiredEntries now subtracts the size of the entry it
actually deleted. A concurrency test recomputes expected counters
from the final map contents.
The handler returned while the ping goroutine could still write to
a connection that gofiber had recycled into its pool. This caused
a nil dereference panic or a ping on an unrelated socket.
Join the ping goroutine through a done channel before the handler
returns. Compute effective keepalive values once in the
constructor, clamp pongTimeout to at least pingInterval with one
warning, and pass the effective values to both proxy directions.
A negative interval or timeout disables keepalive. New test drives
the real HandleWebSocket wiring.
The read-error path in loadBannedUsers returned while it held the
shared file lock. One transient read failure blocked every later
ban write for the process lifetime. The reload also applied a stale
snapshot over a ban that landed between the read and the replace.
Acquire bannedUsersMu first, then the file lock, for the whole
read and replace. Release both locks with defer on every path.
A deterministic test proves the lock is released after a read
error.
The parser does not reject fragment cycles. A 60-byte query with a
self-referential fragment caused unbounded recursion and a stack
overflow, which recover cannot catch. This was a remote DoS when
BLOCK_SCHEMA_INTROSPECTION was enabled.
Thread a visited set through checkSelections so each fragment is
entered at most once per operation. Tests cover self cycles, mutual
cycles, and a deep acyclic chain that must still detect
introspection.
The shutdown component could observe the app before Listen ran.
fasthttp then treated Shutdown as a no-op and reset its stop flag.
The proxy started to serve after shutdown was requested.
Publish httpProxyApp from the OnListen hook and clear it after
Listen returns. Honor the shutdown context and retry until the
drain completes or the context expires. Gate the tests on the
publish signal with a bounded context.
cache.Shutdown only called Close() on an io.Closer, which only the Redis
backend (CacheWrapper) satisfied -- the in-memory Cache's Shutdown
(cancels ctx, stopping its cleanupRoutine background ticker) was never
reached, so that goroutine leaked until process exit (and with the
default set-up, and whenever Redis is unreachable and EnableCache falls
back to memory).
Add Close() error to Cache (calls Shutdown, idempotent, leaves the
cache usable) so cache.Shutdown handles both backends symmetrically.
Update Shutdown doc; add test TestMemoryCacheClose_IdempotentAndStillUsable.
PingInterval/PongTimeout were configured but unused, so a half-open peer
(TCP open but silent) left both proxy goroutines and the handler blocked
in ReadMessage forever. Wire both directions: read deadline refreshed by
real traffic or pong, plus a sidecar ping goroutine (WriteControl is
safe to call concurrently with WriteMessage). Add regression test
TestWebSocketProxy_ReapsSilentBackend_Keepalive proving a silent backend
is reaped via the read deadline instead of hanging.
- graphql.go: parseGraphQLQuery now indexes top-level FragmentDefinition nodes
and checkSelections resolves FragmentSpread by name, closing an
introspection-blocking bypass (`{ ...F }` where F spreads __schema was
allowed because the spread carried no selection set and definitions loop
skipped fragments).
- proxy.go/server.go: handleCaching stores the computed cache hash on the
request context; performProxyRequest reuses it instead of recomputing a
full MD5 of the request body per proxied request (fallback preserved for
direct callers).
- cache/memory: Set serializes the exists-check + counter update + store
with the embedded mutex so concurrent Sets of the same new key can't
double-increment entryCount/memoryUsage; regression test added.
Verification: go build, go vet, gofmt clean; full go test ./... and
go test -race ./... green (9 packages); golangci-lint 0 issues; trufflehog clean.
- tracing: validateEndpoint used fmt.Sscanf("%s:%d") which treats %s as
greedy, so every valid host:port endpoint failed and traced==nil disabled
tracing entirely. Validate with net.SplitHostPort (pure helper, port range
0-65535); test TestValidateEndpoint.
- proxy: createFasthttpClient used clientTimeout for every timeout slot,
ignoring the parsed CLIENT_READ_TIMEOUT/CLIENT_WRITE_TIMEOUT (silent
config loss, documented separately in README). Honour per-side read/write
timeouts, falling back to client timeout when unset.
- cache/memory: evictToFreeMemory capped candidates at maxCacheSize/5 and
stopped at the subset end, so a large write could leave the cache over
its memory limit. Collect all entries and sort.Slice (correct + O(n log n)
vs O(cap^2)), evict until the target is met or the cache is drained.
- cache/redis: the Redis cache client was never closed at shutdown (unlike
the metrics aggregator). Add RedisConfig.Close, CacheWrapper.Close, a
cache.Shutdown and register it with the ShutdownManager.
- ratelimit: custom UnmarshalJSON temp struct only carried interval/req,
silently dropping the configured 'burst' and 'endpoints' fields. Burst is
consumed at checkRateLimit (burst allowance) but could never be set via
config file. Carry all fields in the temp struct.
- sanitization: a large non-parsable body was truncated before redaction,
potentially leaking a secret in the first MaxLogBodySize bytes, and the
truncation could split a multi-byte UTF-8 rune. Redact the retained
prefix and truncate on a rune boundary (sanitizeTruncatedBody).
- tests: TestRateLimitConfigUnmarshalKeepsBurstAndEndpoints,
TestSanitizeTruncatedBodyRedactsAndKeepsUTF8, TestTruncateUTF8RuneBoundary.
The generic typed env reader only handled string/int/bool; any float64 key
fell through to default, so CIRCUIT_FAILURE_RATIO, CIRCUIT_BACKOFF_
MULTIPLIER and RETRY_BUDGET_TOKENS_PER_SEC were silently impossible to
override via environment (always the compile-time default).
Add a float64 case mirroring the int one: parse the GMP_ prefixed key,
then the plain key, else the default (envutil has no GetFloat).
Test: extended Test_getDetailsFromEnv with float-valid and float-malformed
cases.
bannedUsersIDs (sync.Map) was modified by admin Store/Delete (apiBanUser/
apiUnbanUser) and by loadBannedUsers->replaceBannedUsers, which does a
clear-then-rebuild (Range-Delete then Store). The flock only serialized
FILE I/O, not the in-memory map, so a reload could clear a just-applied
ban before storeBannedUsers snapshotted it (or persist the wiped set to
disk).
Add bannedUsersMu (mutex) held across both the map update and the file
write in the admin path, and across the clear-then-rebuild in
replaceBannedUsers. Lock ordering: bannedUsersMu before the file lock;
loadBannedUsers now releases its file read lock before applying the
snapshot, so no ordering inversion and no deadlock.
Also harden the connection-pool health type assertion against a missing
map key (apiConnectionPoolHealth).
Test: TestBannedUsersConcurrentModifyAndReload (concurrent modify +
reload must not lose a persisted ban; race-free under -race).
1) connection_pool.go: GetConnectionStats read lastRecoveryAttempt (a
multi-word time.Time) without any lock while checkAndRecover wrote it
under recoveryMutex -> data race on live admin API endpoints. Snapshot it
under recoveryMutex (the writer's lock). Regression test
TestGetConnectionStatsRace drives the read path concurrent with the locked
write under -race.
2) cache/memory memory.go: Cache.Get expired path used an unconditional
Delete + atomic decrement, so concurrent Gets on the same expired entry
each decremented entryCount/memoryUsage -> permanent negative drift. Use
LoadAndDelete-with-guard (as evictOldest/evictToFreeMemory already do) so
only the goroutine that actually removes the entry decrements. Regression
test TestMemoryCacheConcurrentGetOnExpiredEntry_CountersConsistent.
StartHealthChecking's goroutine captures startupProbe=true and blocks on
readinessChan until the readiness probe completes. WaitForBackendReady only
closed readinessChan on SUCCESS; on timeout it returned an error without
closing it.
main proceeds anyway after a timeout ('Staring proxy anyway'), so the
health-check goroutine stayed blocked forever on readinessChan: periodic
checks never began and bhm.IsHealthy() stayed false permanently, so the
backend could never be marked healthy again even after it recovered.
Release readinessChan on the timeout path too (via sync.Once for safety),
so the health-check goroutine proceeds to periodic checking.
Test: TestHealthChecksStartAfterStartupTimeout (after WaitForBackendReady
times out, readinessChan is closed).
GetGzipReader got a reader from the pool, Reset it, and on error returned a
fresh gzip.NewReader while never returning the pooled reader to the pool —
one leaked reader per reset error (non-gzip input), silently draining the
pool. Return the pooled reader without Close (Close panics because the
decompressor is nil until a header parses); the next caller's Reset
reinitialises it.
Test: TestGetGzipReader_ResetErrorPath (error path leaves the pool usable
for subsequent valid gzip reads).
In RequestCoalescer.Do, the primary path calls fn() before storing the
response, deleting the inflight key and calling wg.Done(). If fn() panics,
all three steps are skipped: the sync.Map entry leaks and every coalesced
waiter remains blocked on wg.Wait() forever (permanent deadlock for that
key). Route release through a defer so waiters are always released and the
entry cleared, even on panic.
Test: TestRequestCoalescer_PanicSafety (a panicking primary must not poison
the key for subsequent calls).
IsRetryable and GetStatusCode used a direct type assertion, so a ProxyError
wrapped with %w was never detected (status fell back to 500, retryable to
false). Use errors.As, which dereferences the existing Unwrap chain and
matches the file's own Unwrap method.
Test: Test_IsRetryable_GetStatusCode_Wrapped (wrapped error preserves both
flags).
Note: these helpers currently have no production callers (test-only
surface), so this is a correctness hardening of the error API.
Two defects in the bidirectional proxy:
1. Goroutine/socket leak on client disconnect. proxyClientToBackend and
proxyBackendToClient each block on their own Read; when the client drops,
the client-direction goroutine returns but the backend-direction goroutine
stays blocked on backend.ReadMessage, so wg.Wait() never returns and the
goroutine plus backend socket leak until the backend independently closes
or sends. Each direction now closes the opposite connection on exit,
unblocking the peer.
2. Backend->client messages were unbounded. client.SetReadLimit only bound
what the client could send; a large or misbehaving backend could push
arbitrarily large frames through. backend.SetReadLimit now bounds it.
Tests: TestWebSocketProxy_TeardownOnClientDisconnect (backend must be closed
after client disconnect), TestWebSocketProxy_BackendReadLimit (connection
torn down when backend exceeds the limit).
On signal, main waited on wg for StartHTTPProxy's server.Listen, which never
returns because nothing called server.Shutdown(); it timed out after 10s and
process-exited, abruptly killing in-flight GraphQL requests.
Expose the proxy Fiber app and register an http-proxy Shutdown component so
server.Listen returns after Shutdown, letting connections drain. Guard the
shared app handle with a mutex (written by the StartHTTPProxy goroutine, read
by the shutdown component). Verified with build, vet, full test and full
race suite.
Clear() used FlushDB, wiping every key in the selected Redis DB even if
that database is shared with other tenants. Switch to a prefix-scoped
SCAN+DEL so only this cache's keys (prefixed) are removed, matching the
memory cache's Clear semantics.
GetMemoryUsage made an Info() round-trip purely to return 0, stalling
the admin dashboard when Redis is slow; return 0 without the call.
UpdateConfig writes tokensPerSecond/maxTokens/enabled as plain fields while
the refill goroutine and AllowRetry readers touch them concurrently. Guard
all reads/writes of the mutable config with an RWMutex.
Adds TestRetryBudget_ConcurrentUpdateConfig, which races UpdateConfig
against AllowRetry/GetStats and is verified clean under -race.
Also drop the package-level min() that shadowed the Go 1.25 builtin.
doProxyRequestWithTimeout returns an error for every non-200 status and
executeProxyAttempt returned it as a plain, retryable error, so 4xx
client errors (400/404/405/...) were retried with exponential backoff
and the 4xx branch of isRetryableStatusCode was dead code.
Additionally, retry-go's finite-attempt loop only consults the custom
RetryIf, which overrode the default IsRecoverable gate, so retry.
Unrecoverable errors (non-retryable 4xx AND timeouts) were still
retried. Honor IsRecoverable in RetryIf so these never retry.
Now 5xx/429 are retried, other 4xx and timeouts fail fast.
Adds Test_proxyRetryPolicyByStatusCode locking in the policy.
The test hit live external endpoints (telegram-bot.app, google.com) and
failed once telegram-bot.app changed /v1/graphql to return 405. Replace
real hosts with local ok (200) and err (404) servers, preserving the
query->read-only / mutation->write routing semantics under test.
- Document WebSocket-streamed live metrics (2s) with 5s polling fallback, the overview KPIs and live charts, and the cluster-view toggle (aggregated vs this-node, disabled in single-node).
- Complete the API endpoint list (stats, cache, ws/stats, cluster endpoints, cache/clear) to match the registered routes. - Regenerate static/admin-dashboard.png for the redesigned dashboard.
Fixes the stale 'auto-refreshes every 5 seconds' claim.
BREAKING CHANGE: upgrades the HTTP framework from gofiber/fiber/v2 to gofiber/fiber/v3, and gofiber/websocket/v2 to gofiber/contrib/v3/websocket.
- Handlers now take fiber.Ctx (interface) instead of *fiber.Ctx. - DisableStartupMessage moved from fiber.Config to fiber.ListenConfig on the proxy/api/monitoring Listen calls. - cors AllowOrigins is now []string; c.BodyParser -> c.Bind().Body(). - app.Test takes fiber.TestConfig instead of an int timeout (tests updated). - WebSocket swapped to contrib/v3/websocket (same Conn/New/Config/IsCloseError/IsWebSocketUpgrade API; Conn.Query preserved). - Adopts fasthttp v1.71.0 (pulled by fiber v3), resolving the v1.71 Host-header enforcement that broke the prior pinned-v1.69 workaround.
fix(connpool): the background keep-alive goroutine no longer reads the mutable global cfg (captures HostGraphQL/HealthcheckGraphQL at pool creation). Under v3 timing it raced with parseConfig/test cfg reassignment; go test -race ./... is now clean (0 data races).
test: make the circuit-breaker cache-fallback and request-coalescing integration tests deterministic (cache TTL outlives the retry loop; coalescing asserts against a measured single-request baseline instead of a hard-coded count); drop the flaky timing-ratio assertion in TestTimingAttackResistance (constant-time guarantee comes from subtle.ConstantTimeCompare, not wall-clock timing).
Verified: go build, go vet, golangci-lint (0 issues), go test -race ./... (all packages pass, 0 races), govulncheck (no vulnerabilities). App + dashboard + WebSocket verified live under v3 in single-node and cluster modes.
Bump golang.org/x/net v0.52.0 -> v0.56.0, fixing GO-2026-5026 (idna Punycode label rejection) and GO-2026-4918 (HTTP/2 transport infinite loop). govulncheck now reports no vulnerabilities. Also bumps x/sync, x/sys, x/term, x/text, otel v1.43->v1.44, grpc v1.80->v1.81 and genproto.
Hold github.com/valyala/fasthttp at v1.69.0: v1.71.0 changes request handling such that the API auth/integration security suites fail (app.Test errors on benign requests), violating the behaviour-preserving upgrade rule. fasthttp had no flagged vulnerability, so pinning it back loses no security fix. Adopting v1.71.0 needs separate investigation.
Verified: go build, go vet, golangci-lint, go test -race ./... all pass; govulncheck clean.
Starting with ENABLE_CIRCUIT_BREAKER=true panicked at boot: parseConfig -> initCircuitBreaker registers a metrics gauge, but cfg.Monitoring is assigned later by StartMonitoringServer, so the registration dereferenced a nil *MetricsSetup.
Add nil-receiver guards to the MetricsSetup registration methods (RegisterMetricsGauge/GaugeFunc/Counter, RegisterFloatCounter, RegisterMetricsSummary/Histogram), matching their existing 'return a dummy to prevent panics' contract. The guard is a no-op for non-nil receivers, so existing callers are unaffected. The breaker and the admin dashboard (which reads gobreaker state directly) keep working.
Add regression tests for the nil-receiver methods and for initCircuitBreaker with nil monitoring.
RecordRequest() only ran inside proxyTheRequest, but cache hits return from handleCaching before reaching it. As a result the dashboard's current RPS read 0 whenever traffic was served from cache (e.g. a repeated query at 100% hit rate). Record the request on the cache-hit branch too; misses and proxied/GET requests still record in proxyTheRequest, so there is no double counting.
Verified: 100% cache-hit load now reports ~16-20k req/s instead of a flat 0.
Replace the AI-slop visual (indigo->purple gradient, glassmorphism, uniform rounded cards, hover-lift) with a deliberate warm-paper 'instrument panel': monospace tabular telemetry, GraphQL brand magenta accent, flat hairline cards, real type hierarchy, WCAG-AA contrast, focus-visible, prefers-reduced-motion and aria-live status. No new font/CDN deps.
Make the cluster-view toggle functional: the WebSocket stats stream now honours a ?view=local|cluster query param, so toggling streams this-node-only vs aggregated metrics instead of being cosmetic (the stream previously always sent cluster-aware data when an aggregator existed). The toggle is always visible -- disabled with a 'single node' hint when Redis cluster mode is off -- so it is discoverable.
Fix cluster-mode rendering bugs: derive backend health from healthy/total instances (health card no longer stuck on 'Loading'), show 'n/a' for Redis memory instead of a misleading 0 MB, render aggregated CB/retry/coalescing/connections on the polling path too, and surface the proxy version and backend-health detail.
Tests assert the served dashboard against stable markers instead of the cosmetic page title.
The repo's go.mod has required go 1.25.0 since the perf+coverage pass,
but the release workflow still pinned setup-go to 1.24 — the latest
1.24.X tool refuses to compile a 1.25 module with GOTOOLCHAIN=local,
breaking auto-release on every push.
The released v0.45.1 binary shipped with the source default
appVersion="dev" because .goreleaser.yaml had ldflags="-s -w" only,
so every startup ping was rejected by the receiver with HTTP 400
(invalid version: regex requires leading digit).
Two-layer fix:
1. .goreleaser.yaml now passes -X main.appVersion={{.Version}} so
goreleaser-built binaries report the actual release version.
2. Switch to telemetry.SendForModule which prefers
debug.ReadBuildInfo Main/Deps when available, falling back to
appVersion. This means `go install github.com/lukaszraczylo/
[email protected]` users also get correct versions
without relying on the ldflag.
Bumps oss-telemetry to v0.2.1 for SendForModule.
Discloses the single anonymous adoption ping sent on startup and points
users to the upstream README section for full opt-out instructions
instead of duplicating the table here.
Send a single fire-and-forget ping at startup to help track adoption
and version spread. No persistent identifiers are collected.
Adds main.appVersion var (defaulting to "dev"); wire ldflags
(-X main.appVersion=$VERSION) to populate it at release time.
Opt out via any of:
DO_NOT_TRACK=1
OSS_TELEMETRY_DISABLED=1
GRAPHQL_MONITORING_PROXY_DISABLE_TELEMETRY=1
lukaszraczylo
