perf(openai): presize upstream JSON response reads from Content-Length

forwardJSON read upstream response bodies with io.ReadAll, which grows
its buffer many times. readAllLimited returns the same data and errors,
but it uses the declared Content-Length as a capacity hint. A body of
known length then usually needs one allocation.

A wrong Content-Length still reads to EOF or to the limit. The hint is
clamped to 1 MiB, so a peer that declares a large length and then stalls
cannot make the gateway allocate more than that before data arrives. The
client request path keeps io.ReadAll, because the client controls its
own Content-Length.

Tests compare readAllLimited with io.ReadAll for exact, short, long,
unknown and oversized lengths, for a mid-body error and for the clamp.

With both commits, the yaegi harness measures 729565 bytes per embeddings
request instead of 925587 (-21%), 12.6% fewer allocations, and 1.15 ms
instead of 1.31 ms. With 16 embeddings per request, allocation falls from
5.0 MB to 3.0 MB.

Claude-Session: https://claude.ai/code/session_01KwtV5ecajCrftcf8TkfPad
This commit is contained in:
2026-09-17 15:27:00 +01:00
parent 5f791d3700
commit 56e112a3ed
3 changed files with 145 additions and 1 deletions
+1 -1
View File
@@ -214,7 +214,7 @@ type chatUsagePayload struct {
// since a malformed-but-readable body is the upstream's problem to answer
// for, not a reason to swallow the response.
func (a *openaiAdapter) forwardJSON(w http.ResponseWriter, resp *http.Response) (usage, error) {
body, err := io.ReadAll(io.LimitReader(resp.Body, maxResponseBytes))
body, err := readAllLimited(resp.Body, resp.ContentLength, maxResponseBytes)
if err != nil {
return usage{}, fmt.Errorf("%w: read response body: %w", errUpstream, err)
}