## Per-test scores

| Test | Category | Subcategory | uni/deepseek-v4-flash-0731 | uni/qwen3.6-27b | uni/btl-4 |
| --- | --- | --- | --- | --- | --- |
| agent-deleg-batch-vs-separate | agents | delegation | 1.00 (161tok) | 1.00 (367tok) | 1.00 (894tok) |
| agent-deleg-build-then-verify | agents | delegation | 1.00 (382tok) | 1.00 (494tok) | 1.00 (455tok) |
| agent-deleg-escalation-to-human | agents | delegation | 1.00 (157tok) | 1.00 (202tok) | 1.00 (683tok) |
| agent-deleg-handoff-context | agents | delegation | 1.00 (198tok) | 1.00 (633tok) | 1.00 (1037tok) |
| agent-deleg-main-thread-vs-delegate | agents | delegation | 1.00 (154tok) | 1.00 (200tok) | 1.00 (456tok) |
| agent-deleg-minimal-privilege | agents | delegation | 1.00 (471tok) | 1.00 (579tok) | 1.00 (632tok) |
| agent-deleg-parallel-dispatch-safety | agents | delegation | 1.00 (231tok) | 1.00 (284tok) | 1.00 (901tok) |
| agent-deleg-reviewer-independence | agents | delegation | 1.00 (679tok) | 1.00 (836tok) | 1.00 (1355tok) |
| agent-deleg-task-to-specialist | agents | delegation | 1.00 (628tok) | 1.00 (654tok) | 1.00 (1050tok) |
| agent-deleg-verify-vs-trust | agents | delegation | 1.00 (155tok) | 1.00 (269tok) | 1.00 (726tok) |
| agent-plan-critical-path | agents | planning | 1.00 (567tok) | 1.00 (470tok) | 1.00 (1079tok) |
| agent-plan-cycle-detect | agents | planning | 1.00 (306tok) | 1.00 (315tok) | 1.00 (1195tok) |
| agent-plan-idempotency | agents | planning | 1.00 (499tok) | 1.00 (553tok) | 1.00 (1335tok) |
| agent-plan-milestone-date | agents | planning | 1.00 (794tok) | 1.00 (236tok) | 1.00 (892tok) |
| agent-plan-minimal-replan | agents | planning | 1.00 (406tok) | 1.00 (451tok) | 1.00 (865tok) |
| agent-plan-ordering | agents | planning | 1.00 (415tok) | 1.00 (379tok) | 1.00 (905tok) |
| agent-plan-parallel-steps | agents | planning | 1.00 (385tok) | 1.00 (322tok) | 1.00 (1120tok) |
| agent-plan-repair | agents | planning | 1.00 (346tok) | 1.00 (312tok) | 1.00 (907tok) |
| agent-plan-resource-constrained | agents | planning | 1.00 (1216tok) | 1.00 (577tok) | 1.00 (1633tok) |
| agent-plan-rollback-trigger | agents | planning | 1.00 (370tok) | 1.00 (442tok) | 1.00 (1142tok) |
| agent-tool-routing | agents | tool-routing | 1.00 (323tok) | 1.00 (434tok) | 1.00 (461tok) |
| agent-tool-routing-ambiguous | agents | tool-routing | 1.00 (277tok) | 1.00 (336tok) | 1.00 (526tok) |
| agent-tool-routing-chaining | agents | tool-routing | 1.00 (218tok) | 1.00 (302tok) | 1.00 (301tok) |
| agent-tool-routing-cheapest | agents | tool-routing | 1.00 (260tok) | 1.00 (331tok) | 1.00 (421tok) |
| agent-tool-routing-distractors | agents | tool-routing | 1.00 (555tok) | 1.00 (543tok) | 1.00 (602tok) |
| agent-tool-routing-missing-param | agents | tool-routing | 1.00 (201tok) | 1.00 (478tok) | 0.50 (502tok) |
| agent-tool-routing-multistep | agents | tool-routing | 1.00 (247tok) | 1.00 (326tok) | 1.00 (358tok) |
| agent-tool-routing-no-tool-needed | agents | tool-routing | 1.00 (291tok) | 1.00 (362tok) | 1.00 (669tok) |
| agent-tool-routing-parallel | agents | tool-routing | 1.00 (319tok) | 1.00 (380tok) | 1.00 (391tok) |
| agent-tool-routing-safety | agents | tool-routing | 1.00 (194tok) | 1.00 (253tok) | 1.00 (232tok) |
| llm-429-retry-backoff | ai | llm-integration | 1.00 (188tok) | 1.00 (278tok) | 1.00 (886tok) |
| llm-context-overflow-strategy | ai | llm-integration | 1.00 (340tok) | 1.00 (739tok) | 1.00 (1373tok) |
| llm-embedding-batch-math | ai | llm-integration | 1.00 (164tok) | 1.00 (252tok) | 1.00 (619tok) |
| llm-field-semantics | ai | llm-integration | 1.00 (373tok) | 1.00 (401tok) | 1.00 (983tok) |
| llm-role-placement | ai | llm-integration | 1.00 (181tok) | 1.00 (206tok) | 1.00 (418tok) |
| llm-sse-stream-done | ai | llm-integration | 1.00 (297tok) | 1.00 (352tok) | 1.00 (792tok) |
| llm-stop-sequence-trace | ai | llm-integration | 1.00 (450tok) | 1.00 (587tok) | 1.00 (1297tok) |
| llm-temperature-zero-caveat | ai | llm-integration | 1.00 (974tok) | 1.00 (1957tok) | 1.00 (2400tok) |
| llm-token-budget-reasoning | ai | llm-integration | 1.00 (185tok) | 1.00 (237tok) | 1.00 (297tok) |
| llm-tool-call-handling | ai | llm-integration | 1.00 (195tok) | 1.00 (318tok) | 1.00 (806tok) |
| rag-chunk-size-tradeoff | ai | rag | 1.00 (222tok) | 1.00 (312tok) | 1.00 (788tok) |
| rag-citation-grounding | ai | rag | 1.00 (712tok) | 1.00 (236tok) | 1.00 (842tok) |
| rag-context-selection-vs-stuffing | ai | rag | 1.00 (204tok) | 1.00 (316tok) | 1.00 (602tok) |
| rag-eval-metric-choice | ai | rag | 1.00 (330tok) | 1.00 (395tok) | 1.00 (924tok) |
| rag-hallucination-mitigation-ordering | ai | rag | 1.00 (331tok) | 1.00 (401tok) | 1.00 (1374tok) |
| rag-index-staleness | ai | rag | 1.00 (291tok) | 1.00 (317tok) | 1.00 (895tok) |
| rag-multihop-decomposition | ai | rag | 1.00 (358tok) | 1.00 (442tok) | 1.00 (1185tok) |
| rag-preassembly-dedup | ai | rag | 1.00 (1395tok) | 1.00 (1194tok) | 1.00 (1304tok) |
| rag-reranker-placement | ai | rag | 1.00 (182tok) | 1.00 (256tok) | 1.00 (306tok) |
| rag-retrieval-failure-mode | ai | rag | 1.00 (150tok) | 1.00 (335tok) | 1.00 (480tok) |
| vec-cosine-vs-dot | ai | vector-search | 1.00 (217tok) | 1.00 (298tok) | 1.00 (653tok) |
| vec-distance-to-similarity | ai | vector-search | 1.00 (238tok) | 1.00 (306tok) | 1.00 (882tok) |
| vec-embedding-dimension-tradeoff | ai | vector-search | 1.00 (244tok) | 1.00 (425tok) | 1.00 (524tok) |
| vec-hnsw-efsearch-tradeoff | ai | vector-search | 1.00 (298tok) | 1.00 (261tok) | 1.00 (722tok) |
| vec-index-build-query-tradeoff | ai | vector-search | 1.00 (319tok) | 1.00 (361tok) | 1.00 (521tok) |
| vec-near-duplicate-threshold | ai | vector-search | 1.00 (349tok) | 1.00 (408tok) | 1.00 (1202tok) |
| vec-pq-memory-math | ai | vector-search | 1.00 (238tok) | 1.00 (338tok) | 1.00 (865tok) |
| vec-pre-vs-post-filtering | ai | vector-search | 1.00 (195tok) | 1.00 (455tok) | 1.00 (1043tok) |
| vec-recall-at-k | ai | vector-search | 1.00 (258tok) | 1.00 (313tok) | 1.00 (811tok) |
| vec-rrf-fusion | ai | vector-search | 1.00 (690tok) | 1.00 (618tok) | 1.00 (1466tok) |
| pg-bloat-vacuum | databases | postgres | 1.00 (211tok) | 1.00 (288tok) | 1.00 (453tok) |
| pg-deadlock-lock-order | databases | postgres | 1.00 (327tok) | 1.00 (391tok) | 1.00 (1138tok) |
| pg-explain-seq-vs-index | databases | postgres | 1.00 (347tok) | 1.00 (622tok) | 1.00 (1166tok) |
| pg-index-choice | databases | postgres | 1.00 (287tok) | 1.00 (433tok) | 1.00 (612tok) |
| pg-isolation-anomaly | databases | postgres | 1.00 (240tok) | 1.00 (377tok) | 1.00 (813tok) |
| pg-listen-notify | databases | postgres | 1.00 (603tok) | 1.00 (254tok) | 1.00 (1204tok) |
| pg-n-plus-one | databases | postgres | 1.00 (213tok) | 1.00 (253tok) | 1.00 (566tok) |
| pg-partial-index | databases | postgres | 1.00 (356tok) | 1.00 (433tok) | 1.00 (988tok) |
| pg-pool-sizing | databases | postgres | 1.00 (160tok) | 1.00 (155tok) | 1.00 (265tok) |
| pg-replica-failover | databases | postgres | 1.00 (254tok) | 1.00 (331tok) | 1.00 (1404tok) |
| redis-cache-stampede | databases | redis | 1.00 (534tok) | 1.00 (477tok) | 1.00 (1341tok) |
| redis-incr-atomicity | databases | redis | 1.00 (345tok) | 1.00 (593tok) | 1.00 (1025tok) |
| redis-keyspace-anti-pattern | databases | redis | 1.00 (275tok) | 1.00 (378tok) | 1.00 (673tok) |
| redis-lua-atomicity | databases | redis | 1.00 (135tok) | 1.00 (324tok) | 1.00 (646tok) |
| redis-memory-estimate | databases | redis | 1.00 (184tok) | 1.00 (228tok) | 1.00 (992tok) |
| redis-multi-vs-pipeline | databases | redis | 1.00 (275tok) | 1.00 (1734tok) | 0.00 (594tok) |
| redis-noeviction-write | databases | redis | 1.00 (163tok) | 1.00 (182tok) | 1.00 (490tok) |
| redis-pubsub-delivery | databases | redis | 1.00 (240tok) | 1.00 (588tok) | 1.00 (1009tok) |
| redis-scan-vs-keys | databases | redis | 0.50 (774tok) | 0.50 (1063tok) | 0.50 (1968tok) |
| redis-structure-choice | databases | redis | 1.00 (417tok) | 1.00 (417tok) | 1.00 (1616tok) |
| sql-aggregate-sum | databases | sql-tuning | 1.00 (280tok) | 1.00 (380tok) | 1.00 (1101tok) |
| sql-composite-index-skip-column | databases | sql-tuning | 1.00 (746tok) | 1.00 (1933tok) | 1.00 (1082tok) |
| sql-covering-index | databases | sql-tuning | 1.00 (160tok) | 1.00 (529tok) | 1.00 (676tok) |
| sql-equivalent-rewrite | databases | sql-tuning | 1.00 (330tok) | 1.00 (509tok) | 1.00 (1043tok) |
| sql-groupby-having | databases | sql-tuning | 1.00 (610tok) | 1.00 (660tok) | 1.00 (1376tok) |
| sql-join-counts | databases | sql-tuning | 1.00 (462tok) | 1.00 (662tok) | 1.00 (1644tok) |
| sql-keyset-pagination | databases | sql-tuning | 1.00 (205tok) | 1.00 (256tok) | 1.00 (827tok) |
| sql-null-in-where | databases | sql-tuning | 1.00 (331tok) | 1.00 (421tok) | 1.00 (917tok) |
| sql-orderby-index-serving | databases | sql-tuning | 1.00 (249tok) | 1.00 (383tok) | 1.00 (680tok) |
| sql-window-rank | databases | sql-tuning | 1.00 (286tok) | 1.00 (382tok) | 1.00 (414tok) |
| docker-build-arg-vs-env | delivery | containers | 1.00 (243tok) | 1.00 (408tok) | 1.00 (978tok) |
| docker-copy-vs-add | delivery | containers | 1.00 (195tok) | 1.00 (276tok) | 1.00 (856tok) |
| docker-dockerignore-effect | delivery | containers | 1.00 (648tok) | 1.00 (389tok) | 1.00 (400tok) |
| docker-entrypoint-cmd-trace | delivery | containers | 1.00 (197tok) | 1.00 (313tok) | 1.00 (1055tok) |
| docker-healthcheck-semantics | delivery | containers | 1.00 (143tok) | 1.00 (472tok) | 1.00 (810tok) |
| docker-image-size-math | delivery | containers | 1.00 (230tok) | 1.00 (284tok) | 1.00 (526tok) |
| docker-layer-cache-bust | delivery | containers | 1.00 (337tok) | 1.00 (568tok) | 1.00 (550tok) |
| docker-layer-count | delivery | containers | 1.00 (665tok) | 1.00 (423tok) | 1.00 (1668tok) |
| docker-multistage-size-benefit | delivery | containers | 1.00 (1683tok) | 1.00 (1174tok) | 1.00 (1560tok) |
| docker-nonroot-user-caps | delivery | containers | 1.00 (554tok) | 1.00 (5843tok) | 1.00 (1866tok) |
| git-bisect-steps | delivery | git | 1.00 (4242tok) | 1.00 (4859tok) | 1.00 (2517tok) |
| git-cherry-pick-vs-revert | delivery | git | 1.00 (204tok) | 1.00 (240tok) | 1.00 (432tok) |
| git-conventional-commit-classify | delivery | git | 1.00 (429tok) | 1.00 (453tok) | 1.00 (1451tok) |
| git-detached-head-recovery | delivery | git | 1.00 (1389tok) | 0.00 (12166tok)! | 0.00 (2045tok) |
| git-force-with-lease-vs-force | delivery | git | 0.50 (1465tok) | 0.50 (2207tok) | 1.00 (1206tok) |
| git-gitignore-negation-trace | delivery | git | 0.75 (834tok) | 1.00 (955tok) | 0.75 (1210tok) |
| git-hook-choice | delivery | git | 1.00 (279tok) | 1.00 (255tok) | 1.00 (756tok) |
| git-rebase-vs-merge | delivery | git | 1.00 (236tok) | 1.00 (239tok) | 1.00 (948tok) |
| git-stash-pop-conflict | delivery | git | 1.00 (328tok) | 1.00 (436tok) | 1.00 (1168tok) |
| git-worktree-use-case | delivery | git | 1.00 (502tok) | 1.00 (2334tok) | 1.00 (1292tok) |
| rel-canary-vs-bluegreen | delivery | release-engineering | 1.00 (261tok) | 1.00 (318tok) | 1.00 (678tok) |
| rel-checksums-signing-purpose | delivery | release-engineering | 1.00 (697tok) | 1.00 (286tok) | 0.50 (563tok) |
| rel-clean-tree-requirement | delivery | release-engineering | 1.00 (902tok) | 0.50 (360tok) | 0.50 (732tok) |
| rel-commit-changelog-mapping | delivery | release-engineering | 1.00 (378tok) | 1.00 (378tok) | 1.00 (989tok) |
| rel-hotfix-flow-ordering | delivery | release-engineering | 0.00 (838tok) | 0.00 (1591tok) | 1.00 (2009tok) |
| rel-pin-vs-range-policy | delivery | release-engineering | 1.00 (228tok) | 1.00 (307tok) | 1.00 (1018tok) |
| rel-pipeline-stage-ordering | delivery | release-engineering | 0.00 (989tok) | 0.00 (340tok) | 0.00 (1054tok) |
| rel-rollback-ordering | delivery | release-engineering | 1.00 (265tok) | 1.00 (976tok) | 1.00 (1610tok) |
| rel-semver-bump-changelog | delivery | release-engineering | 1.00 (275tok) | 1.00 (345tok) | 1.00 (939tok) |
| rel-tag-to-release-sequence | delivery | release-engineering | 1.00 (333tok) | 1.00 (464tok) | 1.00 (1374tok) |
| k8s-argocd-outofsync | operations | kubernetes | 1.00 (415tok) | 1.00 (529tok) | 1.00 (1304tok) |
| k8s-cnpg-pdb | operations | kubernetes | 1.00 (625tok) | 1.00 (4965tok) | 1.00 (1733tok) |
| k8s-crashloop-gitops | operations | kubernetes | 0.75 (782tok) | 0.75 (2285tok) | 1.00 (2392tok) |
| k8s-imagepullbackoff | operations | kubernetes | 1.00 (305tok) | 1.00 (378tok) | 1.00 (724tok) |
| k8s-networkpolicy-dns-block | operations | kubernetes | 1.00 (964tok) | 1.00 (2399tok) | 1.00 (2102tok) |
| k8s-pending-taints | operations | kubernetes | 1.00 (268tok) | 1.00 (398tok) | 1.00 (1052tok) |
| k8s-pvc-storageclass-mismatch | operations | kubernetes | 1.00 (308tok) | 1.00 (440tok) | 1.00 (1030tok) |
| k8s-qos-class | operations | kubernetes | 1.00 (299tok) | 1.00 (319tok) | 1.00 (871tok) |
| k8s-service-selector-mismatch | operations | kubernetes | 1.00 (296tok) | 1.00 (360tok) | 1.00 (639tok) |
| k8s-traefik-ingressroute-host | operations | kubernetes | 1.00 (324tok) | 1.00 (344tok) | 1.00 (517tok) |
| linux-cgroup-oom-diagnosis | operations | linux | 1.00 (200tok) | 1.00 (450tok) | 1.00 (1058tok) |
| linux-cron-expression | operations | linux | 1.00 (222tok) | 1.00 (231tok) | 1.00 (1033tok) |
| linux-journalctl-filter | operations | linux | 1.00 (240tok) | 1.00 (271tok) | 1.00 (758tok) |
| linux-lvm-extend-order | operations | linux | 1.00 (426tok) | 1.00 (517tok) | 1.00 (1248tok) |
| linux-nftables-dnat | operations | linux | 1.00 (401tok) | 1.00 (740tok) | 1.00 (1146tok) |
| linux-pct-exec | operations | linux | 1.00 (593tok) | 1.00 (6904tok) | 1.00 (968tok) |
| linux-proc-meminfo-available | operations | linux | 1.00 (272tok) | 1.00 (511tok) | 1.00 (1087tok) |
| linux-ssh-port-forward | operations | linux | 1.00 (431tok) | 0.50 (1566tok) | 1.00 (837tok) |
| linux-systemd-oneshot | operations | linux | 1.00 (715tok) | 0.75 (3848tok) | 1.00 (580tok) |
| linux-systemd-timer | operations | linux | 1.00 (781tok) | 0.00 (12134tok)! | 0.67 (741tok) |
| macos-apfs-snapshot | operations | macos | 1.00 (827tok) | 0.00 (12121tok)! | 1.00 (1083tok) |
| macos-caffeinate | operations | macos | 1.00 (777tok) | 1.00 (1285tok) | 1.00 (1223tok) |
| macos-launchctl-bootstrap | operations | macos | 1.00 (1416tok) | 1.00 (2506tok) | 1.00 (1744tok) |
| macos-launchd-cron | operations | macos | 1.00 (1535tok) | 1.00 (4637tok) | 1.00 (2191tok) |
| macos-mdfind-spotlight | operations | macos | 1.00 (604tok) | 1.00 (4675tok) | 1.00 (1221tok) |
| macos-plutil-plist-read | operations | macos | 1.00 (448tok) | 1.00 (741tok) | 1.00 (1164tok) |
| macos-sed-inplace | operations | macos | 1.00 (483tok) | 1.00 (492tok) | 1.00 (819tok) |
| macos-ssh-identity-agent | operations | macos | 1.00 (1021tok) | 1.00 (3259tok) | 1.00 (1655tok) |
| macos-stat-bsd-gnu | operations | macos | 1.00 (444tok) | 1.00 (468tok) | 1.00 (1257tok) |
| macos-timeout-portability | operations | macos | 1.00 (1417tok) | 0.00 (12140tok)! | 0.67 (3692tok) |
| c-array-decay-sizeof | programming | c | 1.00 (181tok) | 1.00 (354tok) | 1.00 (842tok) |
| c-bitmask-ops | programming | c | 1.00 (173tok) | 1.00 (260tok) | 1.00 (942tok) |
| c-integer-promotion-overflow | programming | c | 1.00 (159tok) | 1.00 (316tok) | 1.00 (967tok) |
| c-macro-expansion-pitfall | programming | c | 1.00 (253tok) | 1.00 (668tok) | 1.00 (760tok) |
| c-pointer-arithmetic | programming | c | 1.00 (289tok) | 1.00 (327tok) | 1.00 (1179tok) |
| c-string-function-output | programming | c | 1.00 (698tok) | 1.00 (880tok) | 1.00 (780tok) |
| c-struct-bitfield-packing | programming | c | 1.00 (486tok) | 1.00 (657tok) | 1.00 (1188tok) |
| c-struct-size | programming | c | 1.00 (312tok) | 1.00 (365tok) | 1.00 (1135tok) |
| c-undefined-behavior-spot | programming | c | 1.00 (260tok) | 1.00 (331tok) | 1.00 (1651tok) |
| c-union-size-lp64 | programming | c | 1.00 (307tok) | 1.00 (408tok) | 1.00 (1342tok) |
| go-bounded-fanout | programming | golang | 1.00 (1628tok) | 0.00 (2714tok) | 1.00 (2483tok) |
| go-channel-deadlock | programming | golang | 1.00 (277tok) | 1.00 (410tok) | 1.00 (561tok) |
| go-context-cancellation-trace | programming | golang | 1.00 (283tok) | 1.00 (1093tok) | 1.00 (927tok) |
| go-defer-recover-trace | programming | golang | 1.00 (392tok) | 1.00 (415tok) | 1.00 (1090tok) |
| go-generics-constraint-fix | programming | golang | 1.00 (709tok) | 1.00 (6720tok) | 1.00 (1776tok) |
| go-method-set-interface | programming | golang | 1.00 (256tok) | 1.00 (1828tok) | 1.00 (1278tok) |
| go-semver-classify | programming | golang | 1.00 (616tok) | 1.00 (450tok) | 1.00 (1211tok) |
| go-slice-append-aliasing | programming | golang | 1.00 (433tok) | 1.00 (365tok) | 1.00 (1087tok) |
| go-struct-align | programming | golang | 1.00 (517tok) | 1.00 (1682tok) | 1.00 (2084tok) |
| go-worker-pool | programming | golang | 1.00 (972tok) | 0.00 (12193tok)! | 0.00 (894tok) |
| py-asyncio-gather-trace | programming | python | 1.00 (249tok) | 1.00 (342tok) | 1.00 (1352tok) |
| py-cosine | programming | python | 1.00 (4455tok) | 1.00 (3928tok) | 1.00 (3882tok) |
| py-dict-comprehension-trace | programming | python | 1.00 (346tok) | 1.00 (337tok) | 0.00 (12090tok)! |
| py-generator-exhaustion-trace | programming | python | 1.00 (207tok) | 1.00 (266tok) | 1.00 (1208tok) |
| py-json-transform | programming | python | 1.00 (434tok) | 1.00 (648tok) | 1.00 (1079tok) |
| py-log-triage | programming | python | 1.00 (1727tok) | 1.00 (2375tok) | 1.00 (1645tok) |
| py-mutable-default-arg | programming | python | 1.00 (299tok) | 1.00 (269tok) | 1.00 (950tok) |
| py-pathlib-rewrite | programming | python | 1.00 (657tok) | 1.00 (1705tok) | 1.00 (659tok) |
| py-regex-log-extraction | programming | python | 0.00 (956tok) | 1.00 (1616tok) | 1.00 (1249tok) |
| py-softmax | programming | python | 1.00 (2134tok) | 1.00 (2861tok) | 1.00 (1834tok) |
| ts-array-method-chain | programming | typescript | 1.00 (197tok) | 1.00 (263tok) | 1.00 (1146tok) |
| ts-computed-vs-watch | programming | typescript | 1.00 (226tok) | 1.00 (300tok) | 1.00 (823tok) |
| ts-debounce-composable | programming | typescript | 1.00 (687tok) | 1.00 (8871tok) | 1.00 (1521tok) |
| ts-discriminated-union-narrowing | programming | typescript | 1.00 (328tok) | 1.00 (436tok) | 1.00 (1624tok) |
| ts-eslint-flat-config | programming | typescript | 1.00 (903tok) | 1.00 (3197tok) | 1.00 (4007tok) |
| ts-generic-utility-type | programming | typescript | 1.00 (2547tok) | 0.25 (12152tok)! | 1.00 (1759tok) |
| ts-optional-chaining-nullish | programming | typescript | 1.00 (323tok) | 1.00 (392tok) | 1.00 (1513tok) |
| ts-promise-allsettled-trace | programming | typescript | 1.00 (186tok) | 1.00 (282tok) | 1.00 (287tok) |
| ts-tsconfig-strict-flag | programming | typescript | 1.00 (210tok) | 1.00 (665tok) | 1.00 (529tok) |
| ts-vue-ref-reactive-unwrap | programming | typescript | 1.00 (247tok) | 0.00 (12087tok) | 0.50 (629tok) |
| code-bigo-dedup | research | codebase | 1.00 (278tok) | 1.00 (313tok) | 1.00 (920tok) |
| code-bug-line-diff | research | codebase | 1.00 (344tok) | 1.00 (607tok) | 1.00 (892tok) |
| code-commit-bisect | research | codebase | 1.00 (474tok) | 1.00 (585tok) | 1.00 (996tok) |
| code-dead-functions | research | codebase | 1.00 (225tok) | 1.00 (312tok) | 1.00 (404tok) |
| code-generic-return-type | research | codebase | 1.00 (171tok) | 1.00 (248tok) | 1.00 (599tok) |
| code-import-cycle | research | codebase | 1.00 (150tok) | 1.00 (230tok) | 1.00 (973tok) |
| code-race-variable | research | codebase | 1.00 (188tok) | 1.00 (409tok) | 1.00 (643tok) |
| code-trace-go | research | codebase | 1.00 (766tok) | 1.00 (690tok) | 1.00 (1514tok) |
| code-trace-python | research | codebase | 1.00 (378tok) | 1.00 (345tok) | 1.00 (981tok) |
| code-trace-ts | research | codebase | 1.00 (447tok) | 1.00 (356tok) | 1.00 (958tok) |
| web-canonical-vs-redirect | research | web | 1.00 (249tok) | 1.00 (400tok) | 1.00 (767tok) |
| web-cors-missing-header | research | web | 1.00 (221tok) | 1.00 (369tok) | 1.00 (999tok) |
| web-dns-mx-record | research | web | 1.00 (186tok) | 1.00 (204tok) | 1.00 (529tok) |
| web-hreflang-reciprocal | research | web | 1.00 (496tok) | 1.00 (540tok) | 1.00 (1319tok) |
| web-html-product-extract | research | web | 1.00 (203tok) | 1.00 (315tok) | 1.00 (956tok) |
| web-http-status-scenarios | research | web | 1.00 (425tok) | 1.00 (516tok) | 1.00 (1719tok) |
| web-robots-ai-crawlers | research | web | 1.00 (517tok) | 1.00 (591tok) | 1.00 (1469tok) |
| web-security-headers-audit | research | web | 1.00 (327tok) | 1.00 (409tok) | 1.00 (1250tok) |
| web-sitemap-max-urls | research | web | 1.00 (127tok) | 1.00 (272tok) | 1.00 (958tok) |
| web-url-parse-components | research | web | 1.00 (367tok) | 1.00 (316tok) | 1.00 (987tok) |
| paper-attention-scale-factor | research | whitepapers | 1.00 (300tok) | 1.00 (378tok) | 1.00 (608tok) |
| paper-bloom-fp-rate | research | whitepapers | 1.00 (5487tok) | 1.00 (5125tok) | 1.00 (2928tok) |
| paper-btree-height-levels | research | whitepapers | 1.00 (501tok) | 1.00 (541tok) | 1.00 (1499tok) |
| paper-cap-availability-choice | research | whitepapers | 1.00 (269tok) | 1.00 (366tok) | 1.00 (879tok) |
| paper-hnsw-params | research | whitepapers | 1.00 (346tok) | 1.00 (541tok) | 1.00 (1337tok) |
| paper-lsm-write-amplification | research | whitepapers | 1.00 (416tok) | 1.00 (529tok) | 1.00 (1351tok) |
| paper-pq-compression-ratio | research | whitepapers | 1.00 (387tok) | 1.00 (468tok) | 1.00 (1112tok) |
| paper-raft-quorum-failures | research | whitepapers | 1.00 (392tok) | 1.00 (418tok) | 1.00 (1350tok) |
| paper-recall-at-k | research | whitepapers | 1.00 (430tok) | 1.00 (492tok) | 1.00 (1158tok) |
| paper-tfidf-term-score | research | whitepapers | 1.00 (527tok) | 1.00 (604tok) | 1.00 (1595tok) |
| sec-csrf-requirement | security | appsec | 1.00 (187tok) | 1.00 (297tok) | 1.00 (726tok) |
| sec-idor-spot | security | appsec | 1.00 (282tok) | 1.00 (424tok) | 1.00 (658tok) |
| sec-input-validation-boundary | security | appsec | 0.00 (12089tok)! | 1.00 (2320tok) | 1.00 (1382tok) |
| sec-open-redirect-spot | security | appsec | 1.00 (179tok) | 1.00 (241tok) | 1.00 (502tok) |
| sec-path-traversal-spot | security | appsec | 1.00 (277tok) | 1.00 (331tok) | 1.00 (628tok) |
| sec-rate-limit-placement | security | appsec | 1.00 (184tok) | 1.00 (397tok) | 1.00 (1185tok) |
| sec-secret-log-spot | security | appsec | 1.00 (305tok) | 1.00 (292tok) | 1.00 (471tok) |
| sec-sqli-spot | security | appsec | 1.00 (229tok) | 1.00 (311tok) | 1.00 (504tok) |
| sec-ssrf-spot | security | appsec | 1.00 (285tok) | 1.00 (485tok) | 1.00 (628tok) |
| sec-xss-sink-spot | security | appsec | 1.00 (225tok) | 1.00 (312tok) | 1.00 (606tok) |
| sec-aes-gcm-nonce-reuse | security | crypto | 1.00 (2811tok) | 1.00 (10585tok) | 1.00 (3603tok) |
| sec-cert-chain-validation | security | crypto | 1.00 (244tok) | 1.00 (326tok) | 1.00 (1189tok) |
| sec-constant-time-compare | security | crypto | 1.00 (468tok) | 1.00 (462tok) | 1.00 (874tok) |
| sec-hash-vs-encrypt-pii | security | crypto | 1.00 (201tok) | 1.00 (250tok) | 1.00 (543tok) |
| sec-hmac-vs-signature | security | crypto | 1.00 (488tok) | 1.00 (375tok) | 1.00 (577tok) |
| sec-jwt-alg-none | security | crypto | 1.00 (210tok) | 1.00 (337tok) | 1.00 (1050tok) |
| sec-password-hash-choice | security | crypto | 0.50 (1597tok) | 0.50 (638tok) | 0.50 (1936tok) |
| sec-rand-source-choice | security | crypto | 0.50 (435tok) | 1.00 (839tok) | 0.50 (1270tok) |
| sec-rotate-order | security | crypto | 1.00 (354tok) | 1.00 (665tok) | 1.00 (6504tok) |
| sec-tls-floor-version | security | crypto | 1.00 (155tok) | 1.00 (311tok) | 1.00 (959tok) |
| sec-diff-secret-spot | security | secrets | 1.00 (250tok) | 1.00 (394tok) | 1.00 (1159tok) |
| sec-fork-pr-ci-exposure | security | secrets | 1.00 (344tok) | 1.00 (289tok) | 1.00 (376tok) |
| sec-gitignore-history-nuance | security | secrets | 1.00 (376tok) | 1.00 (311tok) | 1.00 (819tok) |
| sec-hardcoded-secret-spot | security | secrets | 1.00 (170tok) | 1.00 (202tok) | 1.00 (243tok) |
| sec-k8s-secret-base64 | security | secrets | 1.00 (148tok) | 1.00 (199tok) | 1.00 (328tok) |
| sec-least-privilege-scope | security | secrets | 1.00 (177tok) | 1.00 (230tok) | 1.00 (770tok) |
| sec-remediation-order | security | secrets | 1.00 (1843tok) | 1.00 (506tok) | 0.00 (12098tok)! |
| sec-rotation-no-downtime | security | secrets | 1.00 (451tok) | 1.00 (501tok) | 1.00 (465tok) |
| sec-ssh-agent-identity | security | secrets | 1.00 (1689tok) | 1.00 (2416tok) | 1.00 (2843tok) |
| sec-vault-tradeoff | security | secrets | 1.00 (305tok) | 1.00 (407tok) | 1.00 (740tok) |

! = response truncated by the token budget (finish_reason=length); increase max_tokens_default to avoid

## Category rollup (mean score)

| Category | uni/deepseek-v4-flash-0731 | uni/qwen3.6-27b | uni/btl-4 |
| --- | --- | --- | --- |
| agents | 1.00 | 1.00 | 0.98 |
| ai | 1.00 | 1.00 | 1.00 |
| databases | 0.98 | 0.98 | 0.95 |
| delivery | 0.91 | 0.87 | 0.89 |
| operations | 0.99 | 0.87 | 0.98 |
| programming | 0.97 | 0.91 | 0.94 |
| research | 1.00 | 1.00 | 1.00 |
| security | 0.93 | 0.98 | 0.93 |

## Model summary

| Model | Mean | Passed | Partial | Failed | Errors | Mean latency | Total tokens | Tok/s |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| uni/deepseek-v4-flash-0731 | 0.97 | 240 | 6 | 4 | 0 | 5.685104556s | 138851 | 71.6 |
| uni/qwen3.6-27b | 0.95 | 233 | 8 | 9 | 0 | 23.689755872s | 291412 | 40.7 |
| uni/btl-4 | 0.96 | 234 | 10 | 6 | 0 | 13.135007924s | 295086 | 77.7 |