lukaszraczylo/traefik-llmgateway
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/traefik-llmgateway.git synced 2026-10-21 05:57:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: per-group cache TTL

Browse Source 
GroupConfig gains CacheTTL (json cacheTTL, empty inherits the global
cache.ttl). Parsed and validated at construction (newAuthStore,
groupCacheTTL in cache.go): a malformed duration, a duration <= 0, or
CacheTTL set while the global cache block is not configured is a
constructor error naming the group, mirroring Cache:true's existing
"nothing to inherit from" check.

group carries the resolved time.Duration (0 means inherit).
responseCache.store gains an explicit ttl parameter; routes_unified.go
computes effectiveTTL(g.cache, grp) immediately before each store call,
so setEx's Redis-native EX seconds reflect the group's own override.

Tests: constructor-error table, empty-inherits and valid-override
resolution, effectiveTTL inheritance/override, a scripted-RESP-server
proof that store's SET...EX carries a different value per group, and an
end-to-end Gateway.ServeHTTP test with two groups asserting the actual
wire EX value (30 vs 60) differs per group.

README: GroupConfig.cacheTTL config row and a Caching-section paragraph.
This commit is contained in:
lukaszraczylo
2026-08-20 23:25:06 +01:00
parent 8eb06a4b8b
commit 1ae8555255
8 changed files with 329 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
llmgateway.go
+12 -1
View File
@@ -71,7 +71,18 @@ type GroupConfig struct {
// group in — but only when the global cache block is actually
// configured (newAuthStore rejects true otherwise, a constructor
// error, since there is nothing to inherit ttl/maxBodyBytes from).
Cache *bool `json:"cache,omitempty"`
Cache *bool `json:"cache,omitempty"`
// CacheTTL overrides the global cache.ttl (a Go duration string, e.g.
// "5m") for this group's cache entries only. Empty (the default)
// inherits the global TTL. Set means: parsed via time.ParseDuration at
// construction (newAuthStore, auth.go); a malformed duration, a
// duration <= 0, or CacheTTL set while the global cache block is not
// configured (cfg.Cache.Enabled false — there is no global TTL to
// override) is a constructor error naming the group. CacheTTL is
// independent of Cache above: it names the TTL to use WHEN this
// group's requests are cached, whether that caching comes from the
// global default or from this group's own Cache:true.
CacheTTL string `json:"cacheTTL,omitempty"`
MCPServers []string `json:"mcpServers,omitempty"`
Agents []string `json:"agents,omitempty"`
}