mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-14 05:18:54 +00:00
docs: v0.2 features
README, examples, and a comment update covering v0.2's four features:
- README: new Caching section (key semantics incl. requestedModel +
endpoint, hit/miss header, count-requests-free-tokens, group override,
redis-required) and Admin section (enable flag, admin:true users,
browser key-entry flow, sessionStorage, what's exposed/redacted,
poll-counts-against-quota, unauthenticated-shell). Retry and Image/audio
sections already existed from T3/T4; left as-is. Config reference gains
cache/admin rows at the top level, GroupConfig.cache, UserConfig.admin,
and full CacheConfig/AdminConfig tables. Known limitations gains
images/audio-not-cost-accounted and cache-shares-entries-across-users.
- examples/kubernetes.yaml: adds cache{enabled:true,ttl:5m} (this example
already configures redis) and admin{enabled:false} with a comment on
enabling it, plus an inline admin:true user gated behind that flag.
Round-trips through New() cleanly (verified with a throwaway test).
- examples/docker-compose.yml: adds retry, cache (left disabled — this
minimal demo runs no Redis), and admin (left disabled, with a comment
on enabling it). Round-trips through New() cleanly (verified the same
way).
- admin.go: pluginVersion's doc comment now says to keep it in sync with
the release tag.
This commit is contained in:
@@ -60,6 +60,23 @@ configs:
|
||||
- name: demo
|
||||
group: default
|
||||
apiKey: "${LLMGW_DEMO_KEY:-sk-demo-local}"
|
||||
# v0.2 features — see ../README.md#retry, #caching, #admin.
|
||||
retry:
|
||||
enabled: true
|
||||
attempts: 1
|
||||
backoff: 250ms
|
||||
cache:
|
||||
# Left disabled: this minimal demo runs no Redis, and
|
||||
# cache.enabled: true with no `redis` block configured
|
||||
# silently stays off anyway (one warning logged) — see
|
||||
# ../README.md#caching. Add a `redis` service and block
|
||||
# to this compose file to actually exercise it.
|
||||
enabled: false
|
||||
admin:
|
||||
# Flip to true, then add `admin: true` to the "demo"
|
||||
# user above, to try the dashboard at
|
||||
# http://localhost:${LLMGW_DEMO_PORT:-8080}/admin.
|
||||
enabled: false
|
||||
|
||||
services:
|
||||
whoami:
|
||||
|
||||
@@ -86,6 +86,16 @@ spec:
|
||||
requestsPerDay: 500
|
||||
users:
|
||||
file: /secrets/llmgw-users/users.json
|
||||
inline:
|
||||
# Grants the admin dashboard (see the "admin" block above),
|
||||
# which stays unreachable until admin.enabled is true. In a
|
||||
# real deployment this key still belongs in a Secret via
|
||||
# env:/file: the same as every other apiKey here — inline only
|
||||
# to keep this one example self-contained.
|
||||
- name: dashboard-admin
|
||||
group: engineering
|
||||
apiKey: "env:LLMGW_ADMIN_KEY"
|
||||
admin: true
|
||||
redis:
|
||||
address: redis.shared-resources.svc:6379
|
||||
password: "env:REDIS_PASSWORD"
|
||||
@@ -94,6 +104,19 @@ spec:
|
||||
enabled: true
|
||||
attempts: 2 # retries after the first try: 3 tries total
|
||||
backoff: 250ms
|
||||
# Opt-in response cache for non-streaming chat/embeddings — see
|
||||
# ../README.md#caching. Requires the redis block above; without it,
|
||||
# this would silently stay disabled (one warning logged) rather than
|
||||
# fail construction.
|
||||
cache:
|
||||
enabled: true
|
||||
ttl: 5m
|
||||
# Disabled here deliberately: flip to `enabled: true`, then flag at
|
||||
# least one user `admin: true` (see the inline "dashboard-admin"
|
||||
# user under users.inline below) to try the read-only dashboard at
|
||||
# https://llm.example.com/admin — see ../README.md#admin.
|
||||
admin:
|
||||
enabled: false
|
||||
pricing:
|
||||
gpt-5-mini: {inputPerM: 0.25, outputPerM: 2.0}
|
||||
mcpServers:
|
||||
|
||||
Reference in New Issue
Block a user