Files
traefikoidc/universal_cache.go
lukaszraczylo 546ceb949c security: remediate audit findings (ranks 1–16 + 22 Lows) + yaegi load validation (#144)
* fix(security): encrypt session cookies + fail closed on invalid config

Batch 1 of security audit remediation (ranks 1, 2, 6).

- session.go: derive independent HMAC + AES-256 keys via stdlib HKDF-SHA256
  and build the gorilla cookie store with both, so session cookies are now
  encrypted, not merely signed. The single-key store previously left OIDC
  access/refresh/ID tokens recoverable from raw cookie bytes. Cookie format
  changes, so existing sessions are invalidated on deploy (one-time re-login).
- main.go: call config.Validate() at construction and error out on failure,
  instead of silently substituting a public hardcoded encryption key for
  empty/short keys (which allowed session forgery). The yaegi analyzer
  passes via .traefik.yml testData.
- settings.go: isValidSecureURL permits plaintext HTTP for loopback hosts
  only (RFC 8252); remote providers must still use HTTPS.
- tests: complete configs that did not satisfy Validate(); add regression
  tests in security_audit_fixes_test.go.

Configs below documented minimums (rateLimit < 10, key < 32 chars) are now
rejected at startup (fail closed).

* fix(security): validate discovered OIDC endpoints + pin introspection host

Batch 2 of security audit remediation (ranks 3, 4).

- url_helpers.go: add validateDiscoveredEndpoint, an SSRF screen for endpoints
  taken from the provider discovery document (jwks_uri, token, authorization,
  revocation, end_session, introspection, registration). Blocks link-local
  (cloud metadata 169.254.169.254), multicast, unspecified and private
  addresses (unless allowPrivateIPAddresses); blocks loopback unless the
  configured providerURL is itself loopback (dev/test). Cross-domain JWKS
  hosts (e.g. Google) stay allowed. Add sameHost helper.
- main.go: updateMetadataEndpoints screens every discovered endpoint and
  blanks any that fail (fail closed downstream). The introspection endpoint
  carries the client secret via HTTP Basic, so it is additionally pinned to
  the providerURL host to stop a poisoned discovery document exfiltrating the
  secret to an attacker-controlled host.
- tests: regression tests for the SSRF guard and the host pin.

* fix(security): close open redirects + anchor excluded-URL matching

Batch 3 of security audit remediation (ranks 5, 14, 15).

- auth_flow.go: run the stored incoming path through normalizeLogoutPath
  before using it as the post-login redirect, so //evil.com and /\evil.com
  payloads become host-relative (open-redirect, rank 5).
- url_helpers.go: excluded-URL matching is anchored at a natural boundary
  (exact, sub-path "/", or file extension "."), so excluding "/public" no
  longer also bypasses auth on "/publicsecret"; "/favicon" still matches
  "/favicon.ico" (rank 14).
- internal/utils: X-Forwarded-Host is sanitized (first value only; reject
  CRLF/whitespace/multi-value) before building redirect URLs (rank 15).
- helpers.go: the logout redirect used when there is no provider end-session
  endpoint is host-relative, never an absolute URL derived from the
  client-controllable request host (logout open-redirect, rank 15).
- tests: update two logout cases that asserted the old absolute redirect;
  add regression tests.

* fix(security): reject unverified Azure tokens; fix transport TLS reuse

Batch 4 of security audit remediation (ranks 7, 11).

- token_validation_rs.go: an Azure nonce-bearing access token that cannot be
  cryptographically verified no longer returns "authenticated" when there is
  no ID token to corroborate it; it refreshes (if possible) or forces
  re-authentication instead of failing open (rank 7).
- http_client_pool.go: the at-limit transport-reuse path now takes the write
  lock before mutating refCount (fixes a data race) and only reuses a
  transport whose TLS settings (CA pool + InsecureSkipVerify) match the
  caller's, never one with a different trust store; if none matches it returns
  nil so the caller falls back to a verifying default transport (rank 11).
- tests: add a transport-pool TLS-isolation regression test.

* fix(security): stop logging templated header values (token leak)

Batch 5 of security audit remediation (rank 16).

middleware.go: templated downstream headers commonly carry the access token
(e.g. "Authorization: Bearer {{.AccessToken}}"). The debug log line printed
the full header value, leaking credentials into logs. Log the header name and
byte length instead.

* fix(security): cache-key collision, cache-config divergence, fleet cleanup

Batch 6 of security audit remediation (ranks 9, 10, 12).

- token_manager.go: detectTokenType keys its cache on a SHA-256 hash of the
  full token instead of the first 32 chars (which are only the base64url JWT
  header). Distinct tokens sharing alg+kid no longer collide and get
  mis-classified (rank 10).
- cache_manager.go: the process-global cache manager is initialized once and
  shared across plugin instances; it now logs a loud warning when a later
  instance requests a different explicit Redis backend that is silently
  ignored, surfacing the cross-instance state-isolation hazard (rank 9).
- singleton_resources.go / main.go / utilities.go: track a process-global live
  instance count; the shared singleton-token-cleanup task is stopped only when
  the LAST instance shuts down, so one instance's Close() (e.g. a config reload)
  no longer kills cleanup for surviving instances (rank 12).
- tests: update TestDetectTokenTypeCaching for the new key; add regression tests.

* fix(security): bound introspection cache + cookie lifetime to config

Batch 7 of security audit remediation (ranks 8, 13).

- token_introspection.go: when requireTokenIntrospection is enabled, cap the
  positive introspection-result cache at 30s (instead of 5m) so a token
  revoked at the provider stops passing within ~30s, matching the operator's
  near-real-time revocation expectation (rank 8).
- session.go: bind the cookie store's MaxAge to the configured sessionMaxAge,
  so the cookie codec's cryptographic timestamp validity is no longer fixed at
  gorilla's 30-day default; a stolen cookie is valid only for the configured
  session lifetime (rank 13).
- tests: add a cookie-lifetime regression test.

* fix(security): low-severity hardening (cache, DoS caps, PKCE, throttle)

Batch 8 of security audit remediation — low severity
(ranks 24, 25, 27, 29, 31, 36, 37, 41, 45, 46, 49).

- universal_cache.go: updateLocalCache updates an existing key in place instead
  of orphaning its LRU element and double-counting currentSize/currentMemory
  (rank 36 — the only production-reachable bug in this batch).
- jwk.go / metadata_cache.go / token_introspection.go: bound response bodies
  with io.LimitReader (1 MiB) to prevent memory exhaustion from a hostile or
  buggy provider (ranks 24, 25).
- jwk.go: skip JWKs not usable for signature verification (use != sig, or
  key_ops without "verify") when building the key set (rank 49).
- auth_flow.go: fail closed at the callback when PKCE is enabled but the code
  verifier is missing, instead of silently dropping it (rank 27).
- utilities.go / main.go: match allowedUserDomains case-insensitively (rank 31).
- bearer_auth.go: a single success no longer wipes an active per-IP penalty;
  the counter resets only when no penalty is in effect (rank 29).
- main.go: handle (not discard) the NewSessionManager error (rank 37).
- error_recovery.go: take a write lock in isServiceDegraded (it deletes from a
  map); compare retryable-error substrings case-insensitively (ranks 45, 46).
- singleton_resources.go: bind the generic-cache cleanup goroutine to the
  resource-manager shutdown channel so it cannot outlive its owner (rank 41).
- tests: update the bearer throttle test to the corrected penalty semantics.

* fix(security): header sanitization, issuer pinning, fail-closed paths

Batch 9 of security audit remediation (ranks 18, 19, 20, 21, 22, 30, 33, 34).

- middleware.go / bearer_auth.go: sanitize claim-derived values on the cookie
  auth path before injecting them into downstream headers. Drop group/role and
  identifier values containing control chars, bidi-override runes, or the
  , ; = delimiters (a comma would inject phantom entries into X-User-Groups);
  reject control/bidi/over-length in rendered templated header output (but
  permit , ; = in free-form values such as a bearer token). The bearer path
  already sanitized; the cookie path did not (ranks 33, 34).
- main.go / metadata_cache.go: pin the discovered issuer to the configured
  provider host (sameHost) and refuse/never-cache a mismatch, so a poisoned
  discovery document cannot redefine the JWT trust anchor (ranks 21, 22).
- token_introspection.go: when a distinct API audience is configured, fail
  closed on a missing or mismatched introspection audience; aud parsed as
  string-or-array per RFC 7662 (rank 19).
- logout.go: front-channel logout requires a matching issuer; an empty iss is
  rejected (blocks unauthenticated forced-logout via a known sid) (rank 30).
- token_validation_rs.go: an opaque access token with no ID token and no
  successful introspection fails closed (re-auth) instead of authenticating
  (ranks 18, 20).
- tests: realistic same-host provider mocks; regression tests for the header
  sanitization distinction and the fail-closed paths.

* chore(security): remove unwired dead code with latent footguns

Batch 10 of security audit remediation — delete confirmed-dead, unwired
subsystems (ranks 26, 35, 50). None had a production caller (grep-verified);
removal eliminates the latent footguns and ~2.1k lines of dead code.

- token_validator.go (deleted): an unused *TokenValidator whose validateJWT set
  Valid=true with NO signature verification — a severe footgun if ever wired
  (rank 50). The wired RS-aware validators are unaffected.
- security_monitoring.go (deleted): an unused *SecurityMonitor / ExtractClientIP
  that trusted spoofable X-Forwarded-For / X-Real-IP. The live bearer throttle
  uses clientIPForBearer (RemoteAddr-only), unchanged (rank 35).
- dynamic_client_registration.go: removed the RFC 7592 management methods
  (Update/Read/DeleteClientRegistration) that dereferenced an attacker-
  influenced RegistrationClientURI with the registration token attached and no
  HTTPS/SSRF gate, and had no callers. The wired RFC 7591 RegisterClient and
  credential-store helpers are kept (rank 26).
- tests: removed the tests covering the deleted code.

* chore: add Makefile with yaegi load validation

No Makefile existed. The new `yaegi-validate` target interprets the plugin
under the yaegi interpreter the same way Traefik loads it, catching yaegi-only
incompatibilities (unsupported stdlib symbols, reflection edge cases) that the
native `go build` / `go test` toolchain does not. Importing the plugin forces
yaegi to interpret every file plus its vendored deps; CreateConfig + New
exercise the instantiation path.

- cmd/yaegicheck/main.go: the load driver, marked //go:build ignore so it is
  excluded from `go build ./...` (avoids VCS-stamping a main binary, which
  fails in git-worktree layouts) yet is run explicitly by yaegi.
- Makefile: build / fmt / vet / lint / test / vendor / yaegi-validate / check
  targets; `make check` runs vet + tests + yaegi-validate.

Verified: `make yaegi-validate` passes on this branch — the HKDF cookie
encryption, net-based endpoint validation, and claim sanitizers all interpret
and instantiate cleanly under yaegi.

* ci: bump workflow Go toolchain to 1.25; pin yaegi-validate to v0.16.1

Traefik v3.7.1 (the deployed version) is built with `go 1.25.0`, so the PR and
release workflows now use Go 1.25.x to match the toolchain Traefik uses.

Important distinction: the CI Go version is the build TOOLCHAIN. The plugin's
actual interpreter-compatibility ceiling is the yaegi version Traefik bundles
(v0.16.1, which declares go 1.21 and ships a ~Go 1.22 stdlib symbol surface),
NOT the CI Go version. That ceiling is enforced by `make yaegi-validate` plus
the go.mod language directive — e.g. it is why HKDF is hand-rolled with
hmac+sha256 rather than Go 1.24's crypto/hkdf, which yaegi v0.16.1 lacks.

Also pin Makefile YAEGI_VERSION to v0.16.1 (what Traefik v3.7.1 vendors) so
yaegi-validate exercises the real deployed interpreter instead of @latest,
which could pass on a newer yaegi that supports symbols the deployed one does
not.

* docs: align README/CONFIGURATION with branch behavior changes

- excludedURLs: documented as segment/extension-boundary matching (was
  "prefix-matched") — "/public" no longer also matches "/publicsecret" (rank 14).
- Front-channel logout now requires a matching `iss`; requests without one are
  rejected with 400 (rank 30).
- Add an "Upgrading from an earlier release" note: session cookies are now
  AES-256 encrypted with lifetime tracking sessionMaxAge (one-time re-login on
  upgrade), and invalid configuration (rateLimit < 10, key < 32 bytes, missing
  callbackURL, non-HTTPS remote providerURL) now fails closed at startup.

* fix: remove staticcheck-flagged unused functions; wire staticcheck into make check

CI Static Analysis (standalone staticcheck) failed with U1000 "unused":
- dynamic_client_registration.go: deleteCredentialsFromStore — its only caller
  was the RFC 7592 DeleteClientRegistration removed in the dead-code batch.
- token_test.go: createTestJWTSimple — its only callers were the TokenValidator
  tests removed in the same batch.
Both confirmed to have zero remaining callers and removed. build / vet /
go test ./... / staticcheck ./... all green.

The pre-commit hook runs golangci-lint, but CI runs standalone staticcheck
(which flags U1000). Add a `staticcheck` Makefile target and include it in
`make check` so this class of finding is caught locally before push.

* fix(test): stabilize flaky TestWorkerPool_TaskPanic

tasksFailed is incremented in the worker's deferred recover(), which runs after the panicking task's own defer wg.Done(). wg.Wait() could therefore return before the failure was recorded, so reading the counter immediately raced and flaked on slow CI runners. Poll until the failure lands (2s budget) instead. Verified 200x plain + 50x under -race/GOMAXPROCS=1.
2026-05-30 14:10:32 +01:00

1003 lines
28 KiB
Go

package traefikoidc
import (
"container/list"
"context"
"encoding/json"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/lukaszraczylo/traefikoidc/internal/cache/backends"
)
// CacheType defines the type of cache for optimized behavior
type CacheType string
const (
CacheTypeToken CacheType = "token"
CacheTypeMetadata CacheType = "metadata"
CacheTypeJWK CacheType = "jwk"
CacheTypeSession CacheType = "session"
CacheTypeGeneral CacheType = "general"
// maxCacheEntrySize defines the maximum size for a single cache entry (64 MiB)
// This prevents integer overflow when allocating memory for serialization
maxCacheEntrySize = 64 * 1024 * 1024
)
// UniversalCacheConfig provides configuration for the universal cache
type UniversalCacheConfig struct {
Strategy CacheStrategy
Logger *Logger
JWKConfig *JWKCacheConfig
MetadataConfig *MetadataCacheConfig
TokenConfig *TokenCacheConfig
Type CacheType
DefaultTTL time.Duration
CleanupInterval time.Duration
MaxMemoryBytes int64
MaxSize int
EnableAutoCleanup bool
EnableMemoryLimit bool
EnableMetrics bool
EnableCompression bool
SkipAutoCleanup bool
}
// TokenCacheConfig provides token-specific cache configuration
type TokenCacheConfig struct {
BlacklistTTL time.Duration
RefreshTokenTTL time.Duration
EnableTokenRotation bool
}
// MetadataCacheConfig provides metadata-specific cache configuration
type MetadataCacheConfig struct {
SecurityCriticalFields []string
GracePeriod time.Duration
ExtendedGracePeriod time.Duration
MaxGracePeriod time.Duration
SecurityCriticalMaxGracePeriod time.Duration
}
// JWKCacheConfig provides JWK-specific cache configuration
type JWKCacheConfig struct {
RefreshInterval time.Duration
MinRefreshTime time.Duration
MaxKeyAge time.Duration
}
// CacheItem represents a single cache entry
type CacheItem struct {
ExpiresAt time.Time
LastAccessed time.Time
Value interface{}
Metadata map[string]interface{}
element *list.Element
Key string
CacheType CacheType
Size int64
AccessCount int64
}
// UniversalCache provides a single, unified cache implementation
// that replaces all other cache types
type UniversalCache struct {
config UniversalCacheConfig
ctx context.Context
backend backends.CacheBackend
logger *Logger
lruList *list.List
items map[string]*CacheItem
cancel context.CancelFunc
cleanupTicker *time.Ticker
wg sync.WaitGroup
currentSize int64
currentMemory int64
hits int64
misses int64
evictions int64
mu sync.RWMutex
ownsBackend bool
}
// NewUniversalCache creates a new universal cache instance
func NewUniversalCache(config UniversalCacheConfig) *UniversalCache {
return createUniversalCache(config)
}
// NewUniversalCacheWithBackend creates a new universal cache with a specific backend
func NewUniversalCacheWithBackend(config UniversalCacheConfig, cacheBackend backends.CacheBackend) *UniversalCache {
cache := createUniversalCache(config)
cache.backend = cacheBackend
cache.ownsBackend = false // Shared backend, managed externally
return cache
}
// createUniversalCache is the internal constructor
func createUniversalCache(config UniversalCacheConfig) *UniversalCache {
// Apply type-specific defaults first (including MaxSize)
applyTypeDefaults(&config)
// Set general defaults only if not already set by type defaults
if config.MaxSize <= 0 {
config.MaxSize = 1000
}
if config.MaxMemoryBytes <= 0 {
config.MaxMemoryBytes = 50 * 1024 * 1024 // 50MB default
}
if config.DefaultTTL <= 0 {
config.DefaultTTL = 1 * time.Hour
}
if config.CleanupInterval <= 0 {
config.CleanupInterval = 5 * time.Minute
}
if config.Logger == nil {
config.Logger = GetSingletonNoOpLogger()
}
ctx, cancel := context.WithCancel(context.Background())
cache := &UniversalCache{
items: make(map[string]*CacheItem),
lruList: list.New(),
config: config,
logger: config.Logger,
ctx: ctx,
cancel: cancel,
}
// Start cleanup routine only if not skipped
// When cleanup is managed externally (e.g., by UniversalCacheManager),
// skip per-cache cleanup to reduce goroutine count
if !config.SkipAutoCleanup {
cache.startCleanup()
}
return cache
}
// applyTypeDefaults applies type-specific default configurations
func applyTypeDefaults(config *UniversalCacheConfig) {
switch config.Type {
case CacheTypeToken:
if config.TokenConfig == nil {
config.TokenConfig = &TokenCacheConfig{
BlacklistTTL: 24 * time.Hour,
RefreshTokenTTL: 7 * 24 * time.Hour,
EnableTokenRotation: true,
}
}
if config.MaxSize == 0 {
config.MaxSize = 5000 // Tokens need more entries
}
case CacheTypeMetadata:
if config.MetadataConfig == nil {
config.MetadataConfig = &MetadataCacheConfig{
GracePeriod: 5 * time.Minute,
ExtendedGracePeriod: 15 * time.Minute,
MaxGracePeriod: 30 * time.Minute,
SecurityCriticalMaxGracePeriod: 15 * time.Minute,
SecurityCriticalFields: []string{
"jwks_uri",
"token_endpoint",
"authorization_endpoint",
"issuer",
},
}
}
// Only set defaults if not already specified
if config.MaxSize == 0 {
config.MaxSize = 100 // Fewer providers
}
if config.DefaultTTL == 0 {
config.DefaultTTL = 1 * time.Hour
}
case CacheTypeJWK:
if config.JWKConfig == nil {
config.JWKConfig = &JWKCacheConfig{
RefreshInterval: 1 * time.Hour,
MinRefreshTime: 5 * time.Minute,
MaxKeyAge: 24 * time.Hour,
}
}
if config.MaxSize == 0 {
config.MaxSize = 200 // Limited number of keys
}
if config.DefaultTTL == 0 {
config.DefaultTTL = 1 * time.Hour
}
case CacheTypeSession:
if config.MaxSize == 0 {
config.MaxSize = 10000 // Many concurrent sessions
}
if config.DefaultTTL == 0 {
config.DefaultTTL = 30 * time.Minute
}
default:
// General cache defaults already set
}
}
// Set stores a value in the cache
func (c *UniversalCache) Set(key string, value interface{}, ttl time.Duration) error {
// Only use default TTL if ttl is exactly zero (not specified)
// Negative TTL means the item should expire in the past
if ttl == 0 {
ttl = c.config.DefaultTTL
}
// If we have a backend, use it for distributed caching
if c.backend != nil {
// Serialize the value
data, err := c.serialize(value)
if err != nil {
c.logger.Errorf("Failed to serialize value for key %s: %v", key, err)
return err
}
// Store in backend
ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond)
defer cancel()
if err := c.backend.Set(ctx, c.prefixKey(key), data, ttl); err != nil {
c.logger.Infof("Backend set error for key %s: %v", key, err)
// Continue with local cache even if backend fails
}
}
return c.setLocal(key, value, ttl)
}
// SetLocal stores a value only in the in-memory LRU, bypassing any
// distributed backend. Use for values that don't survive JSON round-tripping
// — interfaces holding concrete crypto keys, *big.Int, or types whose
// unexported fields yaegi exposes under an X prefix on Marshal. Each replica
// caches independently; correctness must not depend on cross-replica
// coherence for these keys.
func (c *UniversalCache) SetLocal(key string, value interface{}, ttl time.Duration) error {
if ttl == 0 {
ttl = c.config.DefaultTTL
}
return c.setLocal(key, value, ttl)
}
// setLocal performs the in-memory portion of a write. ttl must already be
// resolved against DefaultTTL by the caller.
func (c *UniversalCache) setLocal(key string, value interface{}, ttl time.Duration) error {
size := c.estimateSize(value)
c.mu.Lock()
defer c.mu.Unlock()
// Check memory limits
if c.config.MaxMemoryBytes > 0 {
// Evict items if necessary to make room
for c.currentMemory+size > c.config.MaxMemoryBytes && c.lruList.Len() > 0 {
c.evictOldest()
}
}
// Check size limits
if c.lruList.Len() >= c.config.MaxSize {
c.evictOldest()
}
// Update or create item
now := time.Now()
if existing, exists := c.items[key]; exists {
// Update existing item
c.currentMemory -= existing.Size
c.lruList.Remove(existing.element)
existing.Value = value
existing.Size = size
existing.ExpiresAt = now.Add(ttl)
existing.LastAccessed = now
existing.AccessCount++
// Move to front
existing.element = c.lruList.PushFront(key)
c.currentMemory += size
} else {
// Create new item
item := &CacheItem{
Key: key,
Value: value,
Size: size,
ExpiresAt: now.Add(ttl),
LastAccessed: now,
AccessCount: 1,
CacheType: c.config.Type,
Metadata: make(map[string]interface{}),
}
item.element = c.lruList.PushFront(key)
c.items[key] = item
c.currentSize++
c.currentMemory += size
}
if c.logger.IsDebug() {
c.logger.Debugf("UniversalCache[%s]: Set key=%s, ttl=%v, size=%d bytes",
c.config.Type, key, ttl, size)
}
return nil
}
// Get retrieves a value from the cache
func (c *UniversalCache) Get(key string) (interface{}, bool) {
// Try backend first if available (for distributed consistency)
if c.backend != nil {
ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond)
defer cancel()
data, _, exists, err := c.backend.Get(ctx, c.prefixKey(key))
if err != nil {
c.logger.Debugf("Backend get error for key %s: %v", key, err)
// Fall through to local cache
} else if exists {
// Deserialize the value
var value interface{}
if err := c.deserialize(data, &value); err != nil {
c.logger.Errorf("Failed to deserialize value for key %s: %v", key, err)
// Fall through to local cache
} else {
atomic.AddInt64(&c.hits, 1)
// Update local cache with backend value synchronously.
// Under yaegi, goroutine spawn is 5-10x costlier than compiled Go,
// and this path fires per-request on cold local cache.
// updateLocalCache is cheap (map write under mutex).
_ = c.updateLocalCache(key, value, c.config.DefaultTTL)
return value, true
}
}
}
return c.getLocal(key)
}
// GetLocal retrieves a value only from the in-memory LRU, never querying the
// distributed backend. Pair with SetLocal for values that aren't safe to
// serialize (see SetLocal docstring).
func (c *UniversalCache) GetLocal(key string) (interface{}, bool) {
return c.getLocal(key)
}
// getLocal returns the in-memory entry for key honoring expiry, grace
// periods, and the RLock fast path used by token/JWK/session caches.
func (c *UniversalCache) getLocal(key string) (interface{}, bool) {
// Fast read path for caches whose eviction is dominated by TTL rather than
// access-recency (token, JWK, session). Holding only an RLock here lets all
// concurrent readers verify cached tokens in parallel — under yaegi the
// previous unconditional Lock serialized every JWT verify on a single
// mutex and pinned a CPU under load.
switch c.config.Type {
case CacheTypeToken, CacheTypeJWK, CacheTypeSession:
c.mu.RLock()
item, exists := c.items[key]
if !exists {
c.mu.RUnlock()
atomic.AddInt64(&c.misses, 1)
return nil, false
}
if !time.Now().After(item.ExpiresAt) {
value := item.Value
c.mu.RUnlock()
atomic.AddInt64(&c.hits, 1)
return value, true
}
c.mu.RUnlock()
// Expired — return miss immediately. The periodic cleanup goroutine
// will evict the stale entry. NEVER fall through to the write-locked
// slow path for Token/JWK/Session caches: under Yaegi the write Lock
// at line 403 costs 10-100ms per acquisition, and Go's RWMutex
// writer-priority semantics block ALL new RLock callers while a Lock
// is pending. A single expired-token event turns every concurrent
// request from read-parallel into write-serialized — the exact
// convoy that produced the 737-goroutine pileup at 0x400275a608.
atomic.AddInt64(&c.misses, 1)
return nil, false
}
c.mu.Lock()
defer c.mu.Unlock()
item, exists := c.items[key]
if !exists {
atomic.AddInt64(&c.misses, 1)
return nil, false
}
// Check expiration
now := time.Now()
if now.After(item.ExpiresAt) {
// For metadata cache, check if we should apply grace period
// Grace periods are only extended if explicitly marked or if this is a retry after failure
if c.config.Type == CacheTypeMetadata && c.config.MetadataConfig != nil {
// Check if grace period has been explicitly activated (e.g., due to provider outage)
if gracePeriod, ok := item.Metadata["grace_period_active"].(bool); ok && gracePeriod {
if c.shouldExtendGracePeriod(item, now) {
newExpiry := c.calculateNewExpiry(item, now)
item.ExpiresAt = newExpiry
c.logger.Infof("UniversalCache[%s]: Extended grace period for key=%s until %v",
c.config.Type, key, newExpiry)
// Continue to return the cached value during grace period
} else {
// Grace period has expired completely
c.removeItem(key, item)
atomic.AddInt64(&c.misses, 1)
return nil, false
}
} else {
// No grace period active, remove expired item
c.removeItem(key, item)
atomic.AddInt64(&c.misses, 1)
return nil, false
}
} else {
// Non-metadata cache or no grace period config
c.removeItem(key, item)
atomic.AddInt64(&c.misses, 1)
return nil, false
}
}
// Update access time and count
item.LastAccessed = now
item.AccessCount++
// Move to front of LRU
c.lruList.MoveToFront(item.element)
atomic.AddInt64(&c.hits, 1)
return item.Value, true
}
// Delete removes a key from the cache
func (c *UniversalCache) Delete(key string) bool {
// Delete from backend if available
if c.backend != nil {
ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond)
defer cancel()
if _, err := c.backend.Delete(ctx, c.prefixKey(key)); err != nil {
c.logger.Debugf("Backend delete error for key %s: %v", key, err)
// Continue with local delete
}
}
c.mu.Lock()
defer c.mu.Unlock()
item, exists := c.items[key]
if !exists {
return false
}
c.removeItem(key, item)
return true
}
// Clear removes all items from the cache
func (c *UniversalCache) Clear() {
// Clear backend if available
if c.backend != nil {
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
if err := c.backend.Clear(ctx); err != nil {
c.logger.Infof("Backend clear error: %v", err)
// Continue with local clear
}
}
c.mu.Lock()
defer c.mu.Unlock()
c.items = make(map[string]*CacheItem)
c.lruList.Init()
c.currentSize = 0
c.currentMemory = 0
c.logger.Debugf("UniversalCache[%s]: Cleared all items", c.config.Type)
}
// Size returns the number of items in the cache
func (c *UniversalCache) Size() int {
c.mu.RLock()
defer c.mu.RUnlock()
return int(c.currentSize)
}
// MemoryUsage returns the current memory usage in bytes
func (c *UniversalCache) MemoryUsage() int64 {
c.mu.RLock()
defer c.mu.RUnlock()
return c.currentMemory
}
// GetMetrics returns cache metrics
func (c *UniversalCache) GetMetrics() map[string]interface{} {
c.mu.RLock()
defer c.mu.RUnlock()
hitRate := float64(0)
total := atomic.LoadInt64(&c.hits) + atomic.LoadInt64(&c.misses)
if total > 0 {
hitRate = float64(atomic.LoadInt64(&c.hits)) / float64(total)
}
return map[string]interface{}{
"type": c.config.Type,
"size": c.currentSize,
"entries": c.currentSize, // Alias for backward compatibility
"memory": c.currentMemory,
"hits": atomic.LoadInt64(&c.hits),
"misses": atomic.LoadInt64(&c.misses),
"evictions": atomic.LoadInt64(&c.evictions),
"hit_rate": hitRate,
"max_size": c.config.MaxSize,
"max_memory": c.config.MaxMemoryBytes,
}
}
// Cleanup manually triggers cleanup of expired items
func (c *UniversalCache) Cleanup() {
c.cleanup()
}
// Close shuts down the cache
func (c *UniversalCache) Close() error {
c.cancel()
// Stop cleanup ticker
if c.cleanupTicker != nil {
c.cleanupTicker.Stop()
}
// Wait for cleanup routine to finish with timeout
done := make(chan struct{})
go func() {
c.wg.Wait()
close(done)
}()
select {
case <-done:
// Cleanup routine finished normally
case <-time.After(2 * time.Second):
// Timeout waiting for cleanup routine
c.logger.Debug("UniversalCache[%s]: Timeout waiting for cleanup routine", c.config.Type)
}
// Clear all items
c.Clear()
// Close backend only if this cache owns it (not shared)
if c.backend != nil && c.ownsBackend {
if err := c.backend.Close(); err != nil {
c.logger.Infof("Failed to close cache backend: %v", err)
}
}
c.logger.Debugf("UniversalCache[%s]: Closed", c.config.Type)
return nil
}
// removeItem removes an item from the cache (must be called with lock held)
func (c *UniversalCache) removeItem(key string, item *CacheItem) {
delete(c.items, key)
c.lruList.Remove(item.element)
c.currentSize--
c.currentMemory -= item.Size
}
// evictOldest evicts the oldest item from the cache (must be called with lock held)
func (c *UniversalCache) evictOldest() {
elem := c.lruList.Back()
if elem == nil {
return
}
key, _ := elem.Value.(string) // Safe to ignore: cache internal type assertion
if item, exists := c.items[key]; exists && item.element == elem {
c.removeItem(key, item)
atomic.AddInt64(&c.evictions, 1)
if c.logger.IsDebug() {
c.logger.Debugf("UniversalCache[%s]: Evicted key=%s", c.config.Type, key)
}
return
}
// Defensive forward-progress guard: the back node is dangling — its key is
// absent from c.items, or c.items[key] points at a newer node (a stale
// duplicate). Drop the node directly so an eviction loop
// (`for ... && c.lruList.Len() > 0`) is guaranteed to terminate and can
// never spin holding c.mu.Lock(). With the updateLocalCache replace-in-place
// fix this branch should be unreachable, but it makes the spin impossible.
c.lruList.Remove(elem)
if c.currentSize > 0 {
c.currentSize--
}
}
// SetMaxSize sets the maximum size and evicts items if necessary
func (c *UniversalCache) SetMaxSize(newSize int) {
c.mu.Lock()
defer c.mu.Unlock()
oldSize := c.config.MaxSize
c.config.MaxSize = newSize
// If the new size is smaller, evict items until we meet the new limit
if newSize < oldSize {
for c.lruList.Len() > newSize {
c.evictOldest()
}
c.logger.Infof("UniversalCache[%s]: Resized from %d to %d, evicted %d items",
c.config.Type, oldSize, newSize, oldSize-c.lruList.Len())
}
}
// ActivateGracePeriod activates grace period for a specific key (e.g., due to provider outage)
func (c *UniversalCache) ActivateGracePeriod(key string) {
c.mu.Lock()
defer c.mu.Unlock()
if item, exists := c.items[key]; exists {
item.Metadata["grace_period_active"] = true
c.logger.Infof("UniversalCache[%s]: Activated grace period for key=%s", c.config.Type, key)
}
}
// startCleanup starts the background cleanup routine
func (c *UniversalCache) startCleanup() {
c.cleanupTicker = time.NewTicker(c.config.CleanupInterval)
c.wg.Add(1)
go func() {
defer c.wg.Done()
for {
select {
case <-c.ctx.Done():
return
case <-c.cleanupTicker.C:
c.cleanup()
}
}
}()
}
// cleanup removes expired items from the cache
func (c *UniversalCache) cleanup() {
c.mu.Lock()
defer c.mu.Unlock()
now := time.Now()
var toRemove []string
for key, item := range c.items {
if now.After(item.ExpiresAt) {
// Special handling for metadata cache grace periods
if c.config.Type == CacheTypeMetadata && c.config.MetadataConfig != nil {
// Only keep items that have active grace period and are still within limits
if gracePeriod, ok := item.Metadata["grace_period_active"].(bool); ok && gracePeriod {
if !c.shouldExtendGracePeriod(item, now) {
toRemove = append(toRemove, key)
}
} else {
// No grace period active, remove expired item
toRemove = append(toRemove, key)
}
} else {
toRemove = append(toRemove, key)
}
}
}
for _, key := range toRemove {
if item, exists := c.items[key]; exists {
c.removeItem(key, item)
}
}
if len(toRemove) > 0 {
c.logger.Debugf("UniversalCache[%s]: Cleaned up %d expired items",
c.config.Type, len(toRemove))
}
}
// estimateSize estimates the memory size of a value
func (c *UniversalCache) estimateSize(value interface{}) int64 {
// Basic size estimation - can be enhanced based on type
switch v := value.(type) {
case string:
return int64(len(v))
case []byte:
return int64(len(v))
case map[string]interface{}:
// Rough estimate for maps
return int64(len(v) * 100)
default:
// Default estimate
return 64
}
}
// shouldExtendGracePeriod determines if grace period should be extended
func (c *UniversalCache) shouldExtendGracePeriod(item *CacheItem, now time.Time) bool {
if c.config.MetadataConfig == nil {
return false
}
// Check if we're within the maximum grace period
maxGrace := c.config.MetadataConfig.MaxGracePeriod
// Check if this is a security-critical field
if fieldName, ok := item.Metadata["field"].(string); ok {
for _, critical := range c.config.MetadataConfig.SecurityCriticalFields {
if fieldName == critical {
maxGrace = c.config.MetadataConfig.SecurityCriticalMaxGracePeriod
break
}
}
}
// Calculate how long since the item originally expired
timeSinceExpiry := now.Sub(item.ExpiresAt)
return timeSinceExpiry <= maxGrace
}
// calculateNewExpiry calculates the new expiry time with progressive grace periods
func (c *UniversalCache) calculateNewExpiry(item *CacheItem, now time.Time) time.Time {
if c.config.MetadataConfig == nil {
return now.Add(c.config.DefaultTTL)
}
// Progressive grace period based on access count
var gracePeriod time.Duration
switch {
case item.AccessCount < 5:
gracePeriod = c.config.MetadataConfig.GracePeriod
case item.AccessCount < 10:
gracePeriod = c.config.MetadataConfig.ExtendedGracePeriod
default:
gracePeriod = c.config.MetadataConfig.MaxGracePeriod
}
// Apply security limits
if fieldName, ok := item.Metadata["field"].(string); ok {
for _, critical := range c.config.MetadataConfig.SecurityCriticalFields {
if fieldName == critical && gracePeriod > c.config.MetadataConfig.SecurityCriticalMaxGracePeriod {
gracePeriod = c.config.MetadataConfig.SecurityCriticalMaxGracePeriod
break
}
}
}
return now.Add(gracePeriod)
}
// Type-specific helper methods
// SetWithMetadata sets a value with additional metadata
func (c *UniversalCache) SetWithMetadata(key string, value interface{}, ttl time.Duration, metadata map[string]interface{}) error {
err := c.Set(key, value, ttl)
if err != nil {
return err
}
c.mu.Lock()
defer c.mu.Unlock()
if item, exists := c.items[key]; exists {
for k, v := range metadata {
item.Metadata[k] = v
}
}
return nil
}
// TokenCacheOperations provides token-specific operations
func (c *UniversalCache) BlacklistToken(token string, ttl time.Duration) error {
if c.config.Type != CacheTypeToken {
return fmt.Errorf("blacklist operation only available for token cache")
}
if ttl <= 0 && c.config.TokenConfig != nil {
ttl = c.config.TokenConfig.BlacklistTTL
}
return c.SetWithMetadata(token, true, ttl, map[string]interface{}{
"blacklisted": true,
"blacklisted_at": time.Now(),
})
}
// IsTokenBlacklisted checks if a token is blacklisted
func (c *UniversalCache) IsTokenBlacklisted(token string) bool {
if c.config.Type != CacheTypeToken {
return false
}
c.mu.RLock()
defer c.mu.RUnlock()
if item, exists := c.items[token]; exists {
if blacklisted, ok := item.Metadata["blacklisted"].(bool); ok {
return blacklisted
}
}
return false
}
// Getters for backward compatibility with tests
// Mutex returns the cache mutex for backward compatibility
func (c *UniversalCache) Mutex() *sync.RWMutex {
return &c.mu
}
// Strategy returns the cache strategy for backward compatibility
func (c *UniversalCache) Strategy() CacheStrategy {
return c.config.Strategy
}
// serialize converts a value to bytes for backend storage
func (c *UniversalCache) serialize(value interface{}) ([]byte, error) {
// If value is already a byte slice (e.g., pre-marshaled JSON from metadata_cache),
// store it directly with a marker to prevent double-encoding.
// This fixes the issue where []byte was being JSON-marshaled, causing Base64 encoding.
if bytes, ok := value.([]byte); ok {
// Validate size to prevent integer overflow
if len(bytes) > maxCacheEntrySize {
return nil, fmt.Errorf("cache entry size %d exceeds maximum allowed size %d", len(bytes), maxCacheEntrySize)
}
// Check for potential overflow when adding marker byte
if len(bytes) == maxCacheEntrySize {
return nil, fmt.Errorf("cache entry size would overflow when adding marker byte")
}
// Prepend marker byte 0x00 to indicate raw bytes (not JSON-encoded)
result := make([]byte, len(bytes)+1)
result[0] = 0x00
copy(result[1:], bytes)
return result, nil
}
// For all other types (maps, strings, etc.), use JSON encoding
// Prepend marker byte 0x01 to indicate JSON-encoded data
jsonData, err := json.Marshal(value)
if err != nil {
return nil, err
}
// Validate size to prevent integer overflow
if len(jsonData) > maxCacheEntrySize {
return nil, fmt.Errorf("serialized cache entry size %d exceeds maximum allowed size %d", len(jsonData), maxCacheEntrySize)
}
// Check for potential overflow when adding marker byte
if len(jsonData) == maxCacheEntrySize {
return nil, fmt.Errorf("serialized cache entry size would overflow when adding marker byte")
}
result := make([]byte, len(jsonData)+1)
result[0] = 0x01
copy(result[1:], jsonData)
return result, nil
}
// deserialize converts bytes from backend storage to a value
func (c *UniversalCache) deserialize(data []byte, value interface{}) error {
if len(data) == 0 {
return fmt.Errorf("cannot deserialize empty data")
}
// Check for type marker (added by serialize)
if data[0] == 0x00 {
// Raw bytes - strip marker and return as-is
rawBytes := data[1:]
if ptr, ok := value.(*interface{}); ok {
*ptr = rawBytes
return nil
}
return fmt.Errorf("cannot deserialize raw bytes into %T", value)
}
if data[0] == 0x01 {
// JSON-encoded - strip marker and unmarshal
return json.Unmarshal(data[1:], value)
}
// Legacy data without marker (for backward compatibility)
// Try to unmarshal as JSON
if err := json.Unmarshal(data, value); err != nil {
// If unmarshal fails, treat as raw bytes
if ptr, ok := value.(*interface{}); ok {
*ptr = data
return nil
}
return err
}
return nil
}
// prefixKey adds a cache type prefix to the key for backend storage
func (c *UniversalCache) prefixKey(key string) string {
return fmt.Sprintf("%s:%s", c.config.Type, key)
}
// updateLocalCache updates the local cache with a value from the backend
func (c *UniversalCache) updateLocalCache(key string, value interface{}, ttl time.Duration) error {
size := c.estimateSize(value)
c.mu.Lock()
defer c.mu.Unlock()
// Check memory limits
if c.config.MaxMemoryBytes > 0 {
for c.currentMemory+size > c.config.MaxMemoryBytes && c.lruList.Len() > 0 {
c.evictOldest()
}
}
// Check size limits
if c.lruList.Len() >= c.config.MaxSize {
c.evictOldest()
}
now := time.Now()
// Replace an existing entry in place: update the item and move its single
// list node to the front. Without this, a repeat populate of the same key
// (the per-request Get->backend-hit path) would PushFront a duplicate node
// and overwrite c.items[key], orphaning the previous node. Orphans inflate
// currentMemory/currentSize and, once eviction deletes the key, leave a
// Back() node whose key is absent from c.items — so evictOldest() spins
// while holding c.mu.Lock(): the 100%-CPU write-lock convoy seen in pprof.
// setLocal dedups the same way; evictOldest also guards any dangling node.
if existing, exists := c.items[key]; exists {
c.currentMemory -= existing.Size
c.lruList.Remove(existing.element)
existing.Value = value
existing.Size = size
existing.ExpiresAt = now.Add(ttl)
existing.LastAccessed = now
existing.AccessCount++
existing.element = c.lruList.PushFront(key)
c.currentMemory += size
return nil
}
item := &CacheItem{
Key: key,
Value: value,
Size: size,
ExpiresAt: now.Add(ttl),
LastAccessed: now,
AccessCount: 1,
CacheType: c.config.Type,
Metadata: make(map[string]interface{}),
}
item.element = c.lruList.PushFront(key)
c.items[key] = item
c.currentSize++
c.currentMemory += size
return nil
}