From 945f41b9b0f7603c18d12de502c54d92c2060b68 Mon Sep 17 00:00:00 2001 From: Lukasz Raczylo Date: Thu, 20 Aug 2026 23:38:17 +0100 Subject: [PATCH] docs: kubernetes hot-reload guidance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README Kubernetes section: what a Middleware CR edit actually does (Traefik builds a new instance, warm-fills discovery, swaps it in; requests in flight finish on the old one), what survives a rebuild (Redis-backed counters and cache — windowKey's key never carries instance identity) vs what resets (in-memory fallback counters, down-latches, the discovery cache), why new models via discovery need no config edit, why the users Secret must be a whole-volume mount (subPath never receives a kubelet update) and how the plugin's own 5s mtime poll picks up the resulting symlink swap, and why a file-provider dynamic config needs directory: rather than filename: in Kubernetes (ConfigMap symlink swap) and under Docker Desktop (VirtioFS fsnotify) alike. examples/kubernetes.yaml: annotated the existing (already-correct, no-subPath) users Secret volume mount explaining why subPath would silently break Secret updates, so a future edit doesn't 'simplify' it into one. --- README.md | 54 ++++++++++++++++++++++++++++++++++++++++ examples/kubernetes.yaml | 8 ++++++ 2 files changed, 62 insertions(+) diff --git a/README.md b/README.md index b07cbe5..2069093 100644 --- a/README.md +++ b/README.md @@ -880,6 +880,60 @@ The short version: rather than standing up a dedicated instance for this plugin — the distributed limiter is a client, not a datastore. +### Config hot-reload + +Editing the `Middleware` custom resource applies with no pod restart and no +manual reload step. The Kubernetes CRD provider watches `Middleware` +resources natively: on a change, Traefik builds a whole new plugin +instance from the updated config, lets it warm-fill discovery for every +provider with `discovery: true` (see [Model routing](#model-routing)), then +swaps it in for new requests. Requests already in flight keep running +against the old instance until they finish; only new requests reach the +new one. + +A rebuilt instance keeps some state and resets the rest: + +- **Survives a rebuild**: request/token/cost counters and cached responses, + both stored in Redis. A counter's key comes from `windowKey`'s + `(kind, id, metric, window, time)` alone (see + [Limits and accounting](#limits-and-accounting)) — never from anything + specific to one Gateway instance — so a new instance reads the exact + counters the old one wrote. The response cache's keys work the same way. +- **Resets on a rebuild**: the in-process fallback counters used only when + `redis` is not configured, the limiter's and cache's Redis-outage + down-latches, and the model registry's discovery cache — a new instance + always runs its own full warm-fill before serving, rather than + inheriting the old instance's listing. + +Two practical implications follow: + +- **New models need no config edit at all.** A provider's own + `discoveryInterval` polls the upstream directly on its own schedule; + adding a model upstream needs neither a `Middleware` change nor a + rebuild. +- **The users `Secret` must be a whole-volume mount, never `subPath`.** A + `subPath` volume mount binds once at pod start — the kubelet never + updates it again, so a `Secret` change behind a `subPath` mount stays + invisible to the running pod for its whole lifetime (a documented + kubelet limitation, not a plugin one). Mount the whole `Secret` as its + own volume instead, exactly as `examples/kubernetes.yaml`'s + `additionalVolumeMounts` snippet does: a whole-volume mount updates by + the kubelet swapping a symlink, and the plugin's own 5-second mtime poll + (`reloadEvery`, `users_file.go`) picks that swap up with no pod restart. + +If you run Traefik's file provider in Kubernetes instead of the CRD +provider — a `ConfigMap`-mounted dynamic config file rather than +`Middleware` resources — point it at the mount with `directory:`, never +`filename:`. A `ConfigMap` update reaches the pod the same way a `Secret` +does: the kubelet swaps a symlink for the whole mounted directory, which +Traefik's single-file `filename:` watch never observes; `directory:` +watches the directory itself and picks the swap up. The same +`directory:` requirement also holds for local/Docker Desktop development +against this plugin: a single file's fsnotify watch does not fire +reliably over Docker Desktop's VirtioFS bind mount, which is why this +repository's own integration stack (`integration/traefik/traefik.yml`) +uses `directory:` rather than `filename:`. + ## Development ```sh diff --git a/examples/kubernetes.yaml b/examples/kubernetes.yaml index d333498..65be24b 100644 --- a/examples/kubernetes.yaml +++ b/examples/kubernetes.yaml @@ -234,6 +234,14 @@ spec: # secret: {secretName: llmgw-users} # deployment: # additionalVolumeMounts: +# # Whole-volume mount — no subPath — is required, not a style +# # choice: a subPath mount binds once at pod start and the +# # kubelet never updates it again, so editing the llmgw-users +# # Secret above would never reach this pod. A whole-volume mount +# # updates via the kubelet swapping a symlink for the whole +# # directory, which users.file's own 5-second mtime poll +# # (reloadEvery, users_file.go) picks up with no pod restart — +# # see ../README.md#config-hot-reload. # - name: llmgw-users # mountPath: /secrets/llmgw-users # readOnly: true