Adds real OpenAI function-calling support end to end and two test tranches.
Client (internal/llm): Request.Tools advertises tools with tool_choice
auto; Response.ToolCalls carries the model's calls, decoding each
function's JSON-encoded argument string into an object (malformed
arguments preserved raw with Decoded=false). Verified live: the gateway
round-trips tools and the qwen/deepseek tool-call parsers emit the
expected format.
Runner: when a test declares Tools, it serializes the model's tool calls
into a canonical JSON envelope and scores that, not the free text.
Eval (toolcall.go): ToolCalled, NoToolCalled, ToolCallWithArgs (subset
match, number/string coercion), ToolSequence, FirstToolIs, ToolNotCalled,
ToolArgValuesCover - all parse the envelope, no llm dependency.
Tests: 12 function-calling tests (tools/function-calling) covering tool
selection, argument extraction, enum constraints, chained and parallel
calls, no-tool-needed and missing-param abstention, destructive-tool
safety, and name-vs-description traps; 8 ultra-hard single-shot tasks in
existing subcategories. Ground-truth self-consistency tests assert every
documented answer and ideal tool envelope scores 1.0.
Six qwen3.8-27b-uncensored requests in the 2026-08-30 run failed
instantly with zero tokens while the replica was warming; the gateway
reported them in-band and the client gave up on the first attempt. Both
shapes now retry within the existing backoff budget.
Reasoning models burn thousands of completion tokens before the
answer. Small per-test caps caused silent truncation and zero scores.
The runner now floors max_tokens at the config default (12000). The
client parses finish_reason. Reports mark truncated cells and the
JSON format carries response text for debugging. A stderr progress
reporter prints one line per completed request; --quiet disables it.
Framework runs a category/subcategory test catalog against multiple
LLMs through an OpenAI-compatible endpoint. Deterministic evaluators
score each response. Reports render as table, markdown, or JSON.
lukaszraczylo