mirror of
https://github.com/lukaszraczylo/traefikoidc.git
synced 2026-06-06 22:49:43 +00:00
1b6c8616fd
* fix(refresh): wire RefreshCoordinator into the live refresh path
The RefreshCoordinator existed but was never instantiated. The actual
refresh path used only session.refreshMutex, which is per-SessionData
instance - and SessionData is pulled from a sync.Pool per request -
so concurrent requests sharing a refresh token had ZERO coordination.
Symptom: when access_token expired (e.g. 5min Zitadel default), every
in-flight request from a polling client (Grafana panels) entered the
refresh path simultaneously and POSTed the same refresh_token to the
IdP. With refresh-token rotation enabled (Zitadel/Authentik default),
only one grant succeeded; the rest got invalid_grant and each cleared
the entire session. Subsequent requests then thrashed in re-auth loops.
This commit:
- adds refreshCoordinator field on TraefikOidc
- instantiates it in NewWithContext with DefaultRefreshCoordinatorConfig
- shuts it down in Close() under shutdownOnce
- routes refreshToken() through the coordinator via coordinatedTokenRefresh,
which collapses concurrent grants to a single upstream call per
refresh_token hash
- exports refreshCoordinatorSessionID for both internal hashing and the
middleware-level wireup so dedup keys stay aligned
Behavioural notes:
- nil-coordinator fallback preserves existing tests that build TraefikOidc
literals without going through the constructor
- followers receive the same TokenResponse/error as the leader, so no
per-instance code paths change
- existing TestGetNewTokenWithRefreshToken_Concurrency still passes
because it hits GetNewTokenWithRefreshToken directly, below the
coordinator boundary
Tests:
- refresh_coordinator_wireup_test.go: 50 concurrent refreshes coalesce
to <=2 upstream calls; distinct tokens still run in parallel; nil
coordinator falls back cleanly
* perf(cache): bound L1 backfill goroutines in HybridBackend
Get() and GetMany() previously spawned a goroutine per L2 hit to write
the value through to L1. Under sustained polling traffic (e.g. a Grafana
dashboard refreshing every 30s with N panels) this minted thousands of
goroutines, each running in Yaegi - directly contributing to the
~1000% CPU spike that pairs with the refresh-token herd.
Replace the per-hit goroutines with a single l1BackfillWorker fed by
l1BackfillBuffer, mirroring the existing asyncWriteBuffer/asyncWriteWorker
pattern for L2 writes. Buffer overflow drops the backfill (counted via
l1BackfillDrops) - a dropped backfill just means the next L2 hit for
that key re-queues it, which is safe.
Tests:
- TestHybridBackend_L1BackfillBounded: 1000 distinct L2 hits keep
goroutine count within +20 of baseline (pre-fix it grew by ~1000)
- TestHybridBackend_L1BackfillFullDrops: drops are accounted for when
the buffer is saturated and the worker is stopped
* feat(refresh): implement isRefreshTokenExpired heuristic
Replace the placeholder `return false` with a real check based on the
issued_at timestamp that SetRefreshToken already stamps into the session.
Gated by a new MaxRefreshTokenAgeSeconds config field (default 21600 =
6h, matching the existing comment). 0 disables the check.
This wires the previously-dead refreshTokenExpired branch in middleware.go,
which short-circuits AJAX requests with a 401 instead of letting them
hammer the IdP for a refresh token that's almost certainly stale - the
classic Grafana-after-long-pause failure mode.
Behaviour:
- maxRefreshTokenAge=0 disables the check (preserves prior behaviour)
- legacy sessions without issued_at still attempt one refresh; the IdP
remains the source of truth on first try
- nil-receiver and nil-session guards keep test code that builds
TraefikOidc literals safe
Tests:
- TestIsRefreshTokenExpired_DisabledWhenAgeZero
- TestIsRefreshTokenExpired_LegacySessionWithoutTimestamp
- TestIsRefreshTokenExpired_WithinWindow
- TestIsRefreshTokenExpired_BeyondWindow
- TestIsRefreshTokenExpired_NilGuards
* perf(token): skip parseJWT on cache hit in VerifyToken
The token cache fast-return existed but ran AFTER parseJWT, so every
validation paid for base64 + JSON unmarshal even on a hit. Under bursty
traffic (e.g. 10+ concurrent panel requests on every Grafana dashboard
refresh, each calling validateStandardTokens which verifies BOTH the
access token and the ID token), this is two redundant parses per
request multiplied by the panel count.
Move the cache lookup ahead of parseJWT. On a hit the function returns
nil immediately. On a miss the original flow runs unchanged.
Also nil-guard t.tokenCache to keep partial-literal test instances safe
(matches the same pattern we already use for tokenBlacklist).
Tests:
- TestVerifyToken_CacheHitSkipsParse: cache pre-populated with claims
for a token whose body would fail parseJWT - returns nil iff the
fast-path bypasses the parse
- TestVerifyToken_CacheMissStillParses: a syntactically valid but
unsigned token still errors past parseJWT on cache miss
* feat(refresh): cross-replica refresh-grant dedup via shared cache
The in-process RefreshCoordinator added in 9f96d8c already collapses
concurrent refresh-token grants on a single Traefik replica. With the
plugin's existing Redis (Dragonfly) cache infrastructure available, we
can extend that dedup across replicas: if pod A refreshes a token at
T+0 and pod B receives a request for the same session at T+1, pod B
should reuse pod A's result rather than POSTing the now-rotated refresh
token to the IdP.
Implementation:
- Add a refreshResultCache to UniversalCacheManager (memory-only when
Redis is disabled, Redis-backed in production via the existing
hybrid/Redis-only mode selection)
- Expose it through CacheManager.GetSharedRefreshResultCache and on the
TraefikOidc struct as refreshResultCache (CacheInterface)
- Inside the closure passed to RefreshCoordinator.CoordinateRefresh,
consult the cache first; on hit return immediately, on miss exchange
with the IdP and populate the cache for peers
- 5s TTL: long enough for siblings to observe, short enough that a
rotated refresh token cannot be re-supplied after the IdP has moved on
- Errors are intentionally NOT cached - peers must always be able to
retry on their own
Pragmatic choice: optimistic cache rather than a hard distributed lock.
- A hard lock (SET NX + poll) doubles Redis RTT and risks dead-locks
if a Traefik pod dies mid-grant.
- The user's BGP+Local externalTrafficPolicy already pins ingress for
a session to one node in steady state, so cross-pod racing is rare.
- This optimistic path catches the rare failover case without adding
failure modes.
Tests:
- TestCoordinatedTokenRefresh_CrossReplicaCacheHit: pre-populated cache
short-circuits the upstream call entirely (0 IdP calls)
- TestCoordinatedTokenRefresh_PopulatesCrossReplicaCache: leader stores
a successful result for peers to find
- TestCoordinatedTokenRefresh_ErrorIsNotCached: invalid_grant must not
poison the dedup cache - peers must retry independently
344 lines
11 KiB
Go
344 lines
11 KiB
Go
// Package traefikoidc provides OIDC authentication middleware for Traefik.
|
|
// This file contains utility/helper methods extracted from main.go for better code organization.
|
|
package traefikoidc
|
|
|
|
import (
|
|
"crypto/sha256"
|
|
"encoding/hex"
|
|
"encoding/json"
|
|
"fmt"
|
|
"html"
|
|
"net/http"
|
|
"runtime"
|
|
"strings"
|
|
"time"
|
|
)
|
|
|
|
// safeLogDebug provides nil-safe logging for debug messages
|
|
func (t *TraefikOidc) safeLogDebug(msg string) {
|
|
if t.logger != nil {
|
|
t.logger.Debug("%s", msg)
|
|
}
|
|
}
|
|
|
|
// safeLogDebugf provides nil-safe logging for formatted debug messages
|
|
func (t *TraefikOidc) safeLogDebugf(format string, args ...interface{}) {
|
|
if t.logger != nil {
|
|
t.logger.Debugf(format, args...)
|
|
}
|
|
}
|
|
|
|
// safeLogError provides nil-safe logging for error messages
|
|
func (t *TraefikOidc) safeLogError(msg string) {
|
|
if t.logger != nil {
|
|
t.logger.Error("%s", msg)
|
|
}
|
|
}
|
|
|
|
// safeLogErrorf provides nil-safe logging for formatted error messages
|
|
func (t *TraefikOidc) safeLogErrorf(format string, args ...interface{}) {
|
|
if t.logger != nil {
|
|
t.logger.Errorf(format, args...)
|
|
}
|
|
}
|
|
|
|
// safeLogInfo provides nil-safe logging for info messages
|
|
func (t *TraefikOidc) safeLogInfo(msg string) {
|
|
if t.logger != nil {
|
|
t.logger.Info("%s", msg)
|
|
}
|
|
}
|
|
|
|
// isAllowedUser checks if a user identifier is authorized based on the configured user identifier claim.
|
|
// When using email as the identifier (default), it validates against allowedUsers and allowedUserDomains.
|
|
// When using non-email identifiers (sub, oid, upn, etc.), it only validates against allowedUsers
|
|
// since domain-based validation doesn't apply to non-email identifiers.
|
|
//
|
|
// Parameters:
|
|
// - userIdentifier: The user identifier to validate (email, sub, oid, upn, etc.).
|
|
//
|
|
// Returns:
|
|
// - true if the user is authorized, false otherwise.
|
|
func (t *TraefikOidc) isAllowedUser(userIdentifier string) bool {
|
|
// If no restrictions are configured, allow all authenticated users
|
|
if len(t.allowedUserDomains) == 0 && len(t.allowedUsers) == 0 {
|
|
return true
|
|
}
|
|
|
|
// Check if user is explicitly allowed
|
|
if len(t.allowedUsers) > 0 {
|
|
_, userAllowed := t.allowedUsers[strings.ToLower(userIdentifier)]
|
|
if userAllowed {
|
|
t.logger.Debugf("User identifier %s is explicitly allowed in allowedUsers", userIdentifier)
|
|
return true
|
|
}
|
|
}
|
|
|
|
// For email-based identifiers, also check domain restrictions
|
|
// Only apply domain validation if using email as identifier AND identifier looks like an email
|
|
if t.userIdentifierClaim == "email" && strings.Contains(userIdentifier, "@") {
|
|
return t.isAllowedDomain(userIdentifier)
|
|
}
|
|
|
|
// For non-email identifiers with allowedUserDomains configured, log a warning
|
|
if len(t.allowedUserDomains) > 0 && t.userIdentifierClaim != "email" {
|
|
t.logger.Debugf("AllowedUserDomains is configured but userIdentifierClaim is '%s', not 'email'. Domain validation skipped for: %s",
|
|
t.userIdentifierClaim, userIdentifier)
|
|
}
|
|
|
|
// User not found in allowedUsers list
|
|
if len(t.allowedUsers) > 0 {
|
|
t.logger.Debugf("User identifier %s is not in the allowed users list", userIdentifier)
|
|
}
|
|
|
|
return false
|
|
}
|
|
|
|
// isAllowedDomain checks if an email address is authorized based on domain or user whitelist.
|
|
// It validates against both allowed user domains and specific allowed users.
|
|
// Parameters:
|
|
// - email: The email address to validate.
|
|
//
|
|
// Returns:
|
|
// - true if the email is authorized (domain or user allowed), false if not authorized
|
|
// or if the email format is invalid.
|
|
func (t *TraefikOidc) isAllowedDomain(email string) bool {
|
|
if len(t.allowedUserDomains) == 0 && len(t.allowedUsers) == 0 {
|
|
return true
|
|
}
|
|
|
|
if len(t.allowedUsers) > 0 {
|
|
_, userAllowed := t.allowedUsers[strings.ToLower(email)]
|
|
if userAllowed {
|
|
t.logger.Debugf("Email %s is explicitly allowed in allowedUsers", email)
|
|
return true
|
|
}
|
|
}
|
|
|
|
if len(t.allowedUserDomains) > 0 {
|
|
parts := strings.Split(email, "@")
|
|
if len(parts) != 2 {
|
|
t.logger.Errorf("Invalid email format encountered: %s", email)
|
|
return false
|
|
}
|
|
|
|
domain := parts[1]
|
|
_, domainAllowed := t.allowedUserDomains[domain]
|
|
|
|
if domainAllowed {
|
|
t.logger.Debugf("Email domain %s is allowed", domain)
|
|
return true
|
|
} else {
|
|
t.logger.Debugf("Email domain %s is NOT allowed. Allowed domains: %v",
|
|
domain, keysFromMap(t.allowedUserDomains))
|
|
}
|
|
} else if len(t.allowedUsers) > 0 {
|
|
t.logger.Debugf("Email %s is not in the allowed users list: %v",
|
|
email, keysFromMap(t.allowedUsers))
|
|
}
|
|
|
|
return false
|
|
}
|
|
|
|
// keysFromMap extracts string keys from a map for logging purposes.
|
|
// Helper function to get keys from a map for logging.
|
|
// Parameters:
|
|
// - m: The map to extract keys from.
|
|
//
|
|
// Returns:
|
|
// - A slice of string keys.
|
|
func keysFromMap(m map[string]struct{}) []string {
|
|
keys := make([]string, 0, len(m))
|
|
for k := range m {
|
|
keys = append(keys, k)
|
|
}
|
|
return keys
|
|
}
|
|
|
|
// sendErrorResponse sends an appropriate error response based on the request's Accept header.
|
|
// It sends JSON responses for clients that accept JSON, otherwise sends HTML error pages.
|
|
// Parameters:
|
|
// - rw: The HTTP response writer.
|
|
// - req: The HTTP request (used to check Accept header).
|
|
// - message: The error message to display.
|
|
// - code: The HTTP status code to set for the response.
|
|
func (t *TraefikOidc) sendErrorResponse(rw http.ResponseWriter, req *http.Request, message string, code int) {
|
|
acceptHeader := req.Header.Get("Accept")
|
|
|
|
if strings.Contains(acceptHeader, "application/json") {
|
|
t.logger.Debugf("Sending JSON error response (code %d): %s", code, message)
|
|
rw.Header().Set("Content-Type", "application/json")
|
|
rw.WriteHeader(code)
|
|
_ = json.NewEncoder(rw).Encode(map[string]interface{}{
|
|
"error": http.StatusText(code),
|
|
"error_description": message,
|
|
"status_code": code,
|
|
}) // Safe to ignore: error response write
|
|
return
|
|
}
|
|
|
|
t.logger.Debugf("Sending HTML error response (code %d): %s", code, message)
|
|
|
|
returnURL := "/"
|
|
// Escape message to prevent XSS attacks
|
|
escapedMessage := html.EscapeString(message)
|
|
|
|
htmlBody := fmt.Sprintf(`
|
|
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<title>Authentication Error</title>
|
|
<style>
|
|
body { font-family: sans-serif; padding: 20px; background-color: #f8f9fa; color: #343a40; }
|
|
h1 { color: #dc3545; }
|
|
a { color: #007bff; text-decoration: none; }
|
|
a:hover { text-decoration: underline; }
|
|
.container { max-width: 600px; margin: auto; background: #fff; padding: 20px; border-radius: 5px; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<div class="container">
|
|
<h1>Authentication Error</h1>
|
|
<p>%s</p>
|
|
<p><a href="%s">Return to application</a></p>
|
|
</div>
|
|
</body>
|
|
</html>`, escapedMessage, returnURL)
|
|
|
|
rw.Header().Set("Content-Type", "text/html; charset=utf-8")
|
|
rw.WriteHeader(code)
|
|
_, _ = rw.Write([]byte(htmlBody)) // Safe to ignore: error response write
|
|
}
|
|
|
|
// Close gracefully shuts down the TraefikOidc middleware instance.
|
|
// It cancels contexts, stops background goroutines, closes HTTP connections,
|
|
// cleans up caches, and releases all resources. Safe to call multiple times.
|
|
// Returns:
|
|
// - An error if shutdown times out or resource cleanup fails.
|
|
func (t *TraefikOidc) Close() error {
|
|
var closeErr error
|
|
t.shutdownOnce.Do(func() {
|
|
t.safeLogDebug("Closing TraefikOidc plugin instance")
|
|
|
|
// Get resource manager for cleanup
|
|
rm := GetResourceManager()
|
|
|
|
// Stop singleton tasks related to this instance
|
|
_ = rm.StopBackgroundTask("singleton-token-cleanup") // Safe to ignore: best effort cleanup
|
|
// Stop metadata refresh task using same hash-based name as startMetadataRefresh
|
|
if t.providerURL != "" {
|
|
hash := sha256.Sum256([]byte(t.providerURL))
|
|
taskName := "singleton-metadata-refresh-" + hex.EncodeToString(hash[:])[0:6]
|
|
_ = rm.StopBackgroundTask(taskName) // Safe to ignore: best effort cleanup
|
|
}
|
|
|
|
// Remove reference for this instance
|
|
rm.RemoveReference(t.name)
|
|
|
|
if t.cancelFunc != nil {
|
|
t.cancelFunc()
|
|
t.safeLogDebug("Context cancellation signaled to all goroutines")
|
|
}
|
|
|
|
// Clean up legacy stop channels if they exist
|
|
if t.tokenCleanupStopChan != nil {
|
|
close(t.tokenCleanupStopChan)
|
|
t.safeLogDebug("tokenCleanupStopChan closed")
|
|
}
|
|
if t.metadataRefreshStopChan != nil {
|
|
close(t.metadataRefreshStopChan)
|
|
t.safeLogDebug("metadataRefreshStopChan closed")
|
|
}
|
|
|
|
if t.refreshCoordinator != nil {
|
|
t.refreshCoordinator.Shutdown()
|
|
t.safeLogDebug("refreshCoordinator shut down")
|
|
}
|
|
|
|
if t.goroutineWG != nil {
|
|
done := make(chan struct{})
|
|
go func() {
|
|
t.goroutineWG.Wait()
|
|
close(done)
|
|
}()
|
|
|
|
select {
|
|
case <-done:
|
|
t.safeLogDebug("All background goroutines stopped gracefully")
|
|
case <-time.After(10 * time.Second):
|
|
t.safeLogError("Timeout waiting for background goroutines to stop")
|
|
}
|
|
} else {
|
|
t.safeLogDebug("No goroutineWG to wait for (likely in test)")
|
|
}
|
|
|
|
if t.httpClient != nil {
|
|
if transport, ok := t.httpClient.Transport.(*http.Transport); ok {
|
|
transport.CloseIdleConnections()
|
|
t.safeLogDebug("HTTP client idle connections closed")
|
|
}
|
|
}
|
|
|
|
if t.tokenHTTPClient != nil {
|
|
if transport, ok := t.tokenHTTPClient.Transport.(*http.Transport); ok {
|
|
transport.CloseIdleConnections()
|
|
t.safeLogDebug("Token HTTP client idle connections closed")
|
|
}
|
|
if t.tokenHTTPClient.Transport != t.httpClient.Transport {
|
|
if transport, ok := t.tokenHTTPClient.Transport.(*http.Transport); ok {
|
|
transport.CloseIdleConnections()
|
|
t.safeLogDebug("Token HTTP client transport closed (separate from main)")
|
|
}
|
|
}
|
|
}
|
|
|
|
if t.tokenBlacklist != nil {
|
|
t.tokenBlacklist.Close()
|
|
t.safeLogDebug("tokenBlacklist closed")
|
|
}
|
|
if t.metadataCache != nil {
|
|
t.metadataCache.Close()
|
|
t.safeLogDebug("metadataCache closed")
|
|
}
|
|
if t.tokenCache != nil {
|
|
t.tokenCache.Close()
|
|
t.safeLogDebug("tokenCache closed")
|
|
}
|
|
|
|
if t.jwkCache != nil {
|
|
t.jwkCache.Close()
|
|
t.safeLogDebug("t.jwkCache.Close() called as per original instruction.")
|
|
}
|
|
|
|
// Shutdown session manager and its background cleanup routines
|
|
if t.sessionManager != nil {
|
|
if err := t.sessionManager.Shutdown(); err != nil {
|
|
t.safeLogErrorf("Error shutting down session manager: %v", err)
|
|
} else {
|
|
t.safeLogDebug("sessionManager shutdown completed")
|
|
}
|
|
}
|
|
|
|
// Clean up error recovery manager
|
|
if t.errorRecoveryManager != nil && t.errorRecoveryManager.gracefulDegradation != nil {
|
|
t.errorRecoveryManager.gracefulDegradation.Close()
|
|
t.safeLogDebug("Error recovery manager graceful degradation closed")
|
|
}
|
|
|
|
// Stop all global background tasks
|
|
taskRegistry := GetGlobalTaskRegistry()
|
|
taskRegistry.StopAllTasks()
|
|
t.safeLogDebug("All global background tasks stopped")
|
|
|
|
// Note: Centralized pool in internal/pool is singleton-managed and doesn't require explicit cleanup
|
|
t.safeLogDebug("Memory pools managed by singleton pattern")
|
|
|
|
// Force garbage collection to help with memory cleanup after shutdown
|
|
runtime.GC()
|
|
t.safeLogDebug("Forced garbage collection after shutdown")
|
|
|
|
t.safeLogDebug("TraefikOidc plugin instance closed successfully.")
|
|
})
|
|
return closeErr
|
|
}
|