lukaszraczylo/traefik-llmgateway
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/traefik-llmgateway.git synced 2026-10-17 05:37:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(config): add maxRequestBytes for the unified body cap

Browse Source 
The unified chat and embeddings routes reject bodies above 4 MiB. Long
agent sessions exceed this limit and receive a 413.

Add Config.maxRequestBytes. Zero or a negative value keeps the 4 MiB
default. A positive value raises the cap, clamped to 64 MiB.

An adversarial body decodes to about 12x live heap. An operator who raises
the cap must also set maxInFlightBodyRequests and the pod memory limit.
This commit is contained in:
lukaszraczylo
2026-10-16 00:34:14 +01:00
parent 4ecfa10532
commit a328a8687e
4 changed files with 96 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1
View File
@@ -384,6 +384,7 @@ config accepts them as YAML, which decodes to the same JSON shape.
| `stripClientAnthropicBeta` | `bool` | `false` | When `true`, `POST /v1/messages` does not forward the client's `anthropic-beta` header to the provider, as native passthrough already does. `false` forwards it, so a client can enable beta features on the operator's own Anthropic account. |
| `modelMeta` | `map[string]*ModelMetaConfig` | `{}` | Per-model context-window and cost overrides, keyed by exact `provider/model` or a bare model id — see [Model metadata](#model-metadata). Omitted or empty means no overrides; every model's metadata still resolves through discovery and the built-in table. |
| `passthroughUnknown` | `bool` | `false` | `false`: a request matching none of the plugin's routes gets a 404 JSON envelope. `true`: it falls through to the router's own backing service. **Does not cover the three media routes** — see the upgrade note below. |
| `maxRequestBytes` | `int` | `0` (4 MiB) | Raises the unified chat/embeddings request-body cap. `0` or negative keeps 4 MiB; a positive value is clamped to 64 MiB. Pair with `maxInFlightBodyRequests` and the pod memory limit: an adversarial body decodes to ~12x live heap. |
| `maxInFlightBodyRequests` | `int` | `0` (self-tuned) | Caps how many unified/media requests may concurrently be inside their own request-body read+decode step — see [Body-admission limit](#body-admission-limit). `0` self-tunes from the host's own `GOMAXPROCS`; a positive value pins an exact cap instead, clamped to a maximum of 10,000. |
> **Upgrading from v0.1**: `POST /v1/images/generations`, `POST