mirror of
https://github.com/lukaszraczylo/traefik-llmgateway.git
synced 2026-10-14 05:18:54 +00:00
lukaszraczylo
fdb81640ec
Adds modelRegistry: per-provider explicit+discovered model sets, bare-id and provider-prefixed resolution with sorted-name collision precedence, group authorization pairing (allowsModel AND allowsProvider), and a group-filtered OpenAI-compatible /v1/models catalog. Discovery is lazy and throttled per provider: newGateway runs a synchronous 5s-bounded warm fill for discovery-enabled providers (non-fatal on error), and Gateway.ServeHTTP triggers a throttled, inFlight-guarded background refresh per provider per interval (default 1h, per-provider DiscoveryInterval parsed at construction). A failed refresh keeps the previous discovered set (stale-while-error). Wires GET /v1/models into ServeHTTP (auth required, 401 envelope otherwise) and gives Gateway its adapters/registry fields, both built in newGateway. newModelRegistry's signature gained an error return beyond the task brief's literal listing: an invalid DiscoveryInterval must be a constructor error per the controller ruling, which the brief's bare *modelRegistry signature could not express.
24 lines
664 B
Go
24 lines
664 B
Go
package traefikllmgateway
|
|
|
|
import (
|
|
"encoding/json"
|
|
"net/http"
|
|
)
|
|
|
|
// handleModels implements GET /v1/models: an authenticated caller gets back
|
|
// an OpenAI-compatible {"object":"list","data":[...]} envelope of the
|
|
// models their group can see, via modelRegistry.listFor.
|
|
func (g *Gateway) handleModels(w http.ResponseWriter, r *http.Request) {
|
|
_, grp, ok := g.auth.identify(r)
|
|
if !ok {
|
|
writeOAIError(w, http.StatusUnauthorized, "authentication_error", "invalid or missing API key")
|
|
return
|
|
}
|
|
|
|
w.Header().Set("Content-Type", "application/json")
|
|
_ = json.NewEncoder(w).Encode(map[string]any{
|
|
"object": "list",
|
|
"data": g.registry.listFor(grp),
|
|
})
|
|
}
|