test(bench): cross-library benchmarks vs top 5 go telegram libraries

Adds test/benchmarks/ as a separate Go module so competitor deps
(go-telegram-bot-api/v5, telebot.v3, go-telegram/bot, telego,
echotron/v3) stay out of the root go.mod.

Hot paths covered:
  - Webhook decode  (small Update -> typed Update struct)
  - Large unmarshal (Update with entities + reply markup + photo array)
  - API round-trip  (sendMessage against httptest.Server)
  - Dispatch route  (20 handlers, last-registered matches)

Results on Apple M4 Max / go1.26.2: ours wins 3 of 4 paths and is
2nd of 5 in the round-trip path. Full report at
docs/benchmarks/2026-05-10-comparison.md, raw output committed under
test/benchmarks/results/.

Caveats called out in the report:
  - codec asymmetry (we ship goccy/go-json; competitors mostly stdlib)
  - echotron call bench skipped — built-in rate limiter not externally
    configurable; would measure throttling, not the library
  - dispatch bench limited to libs with a public sync entry point
    (ours, telebot, gobot); gotba has no dispatcher, telego/echotron
    use channel/per-chat paradigms not directly comparable

Also gitignores docs/superpowers/ (local brainstorm/spec scratch)
and regenerates docs/reference/dispatch.md after the new
Router.Process method.
This commit is contained in:
2026-05-10 21:52:00 +01:00
parent c9a062ea04
commit 607c3e8ddd
14 changed files with 1935 additions and 4 deletions
+113
View File
@@ -0,0 +1,113 @@
# Benchmarks vs top 5 Go Telegram libraries
**Date:** 2026-05-10
**Environment:** Apple M4 Max · darwin/arm64 · `go1.26.2`
**Methodology:** `go test -count=10 -bench=. -benchmem`, summarised with `benchstat` (golang.org/x/perf)
**Source:** [`test/benchmarks/`](../../test/benchmarks/) · raw output: [`results/raw.txt`](../../test/benchmarks/results/raw.txt) · benchstat: [`results/benchstat.txt`](../../test/benchmarks/results/benchstat.txt)
## Libraries
| Lib | Module |
|-----|--------|
| **ours** | `github.com/lukaszraczylo/go-telegram` (this repo) |
| gotba | `github.com/go-telegram-bot-api/telegram-bot-api/v5` |
| telebot | `gopkg.in/telebot.v3` (tucnak) |
| gobot | `github.com/go-telegram/bot` |
| telego | `github.com/mymmrac/telego` |
| echotron | `github.com/NicoNex/echotron/v3` |
## TL;DR
- **Webhook decode** (small Update): ours is **1519% faster** than every competitor and ties telego for the lowest alloc count (11).
- **Large Update unmarshal** (entities + reply markup + photo array): ours is **1735% faster** with the lowest ns/op of all six. telego edges us on alloc count (31 vs 34) at the cost of ~17% more time.
- **API call round-trip** (mock HTTP server): telego wins (36.3 µs / 48 allocs) thanks to its custom binder; ours is second (38.95 µs / 104 allocs) and beats gotba, telebot, gobot.
- **Dispatcher routing** (20 handlers, last matches): ours is **2.5× faster than telebot and gobot** (101 ns vs 269 / 252 ns).
## How to read these numbers
- One machine, single workload, fixtures defined in [`shared/fixtures.go`](../../test/benchmarks/shared/fixtures.go). Re-run on your hardware before drawing conclusions.
- Codecs differ across libs (we use `goccy/go-json`; most competitors use stdlib `encoding/json`). Codec choice is part of the library's value prop, so we benchmark each library as it ships, not in some artificial common-codec mode.
- "Equivalent code path" was chosen via each library's idiomatic public API for the same logical operation. The exact code is in the bench files alongside each `BenchmarkXxx_<lib>` function — read them.
---
## 1. Webhook decode — small Update (text message)
Decode `shared.SmallUpdateJSON` into the library's typed `Update` struct.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| **ours** | **1.743 µs ±3%** | 2.180 KiB | **11** |
| gotba | 2.016 µs ±3% | 1.461 KiB | 17 |
| telebot | 2.073 µs ±3% | 1.773 KiB | 17 |
| gobot | 1.999 µs ±1% | 1.789 KiB | 16 |
| telego | 2.026 µs ±2% | 3.060 KiB | **11** |
| echotron | 1.973 µs ±0% | 1.680 KiB | 16 |
**Notes.** We use slightly more bytes because typed unions and the typed `[]UpdateType` allocate richer Go values. We win on time and tie telego on alloc count.
## 2. Large Update unmarshal — entities + reply markup + photo array
Decode `shared.LargeUpdateJSON` (text + 3 entities + 2x3 inline keyboard + 3-size photo array). Stresses each library's union/discriminator decoding.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| **ours** | **6.667 µs ±4%** | 5.881 KiB | 34 |
| gotba | 8.321 µs ±2% | 3.438 KiB | 56 |
| telebot | 10.240 µs ±4% | 5.594 KiB | 60 |
| gobot | 8.150 µs ±2% | 4.703 KiB | 50 |
| telego | 7.797 µs ±1% | 6.621 KiB | **31** |
| echotron | 8.072 µs ±0% | 4.219 KiB | 56 |
**Notes.** Despite the typed-union model giving us richer Go values per decode, we still produce them faster than every competitor. telego edges us by 3 allocs but pays 17% more time.
## 3. API call round-trip — `sendMessage` against a mock HTTP server
Build params → POST to local `httptest.Server` returning `{"ok":true,"result":Message}` → decode response.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| ours | 38.95 µs ±3% | 11.17 KiB | 104 |
| gotba | 41.95 µs ±2% | 10.95 KiB | 125 |
| telebot | 43.63 µs ±0% | 13.16 KiB | 139 |
| gobot | 61.11 µs ±1% | 13.51 KiB | 176 |
| **telego** | **36.31 µs ±1%** | **6.556 KiB** | **48** |
| echotron | *skipped — see below* | — | — |
**Notes.**
- telego wins by sending requests as `application/x-www-form-urlencoded` form data (cheaper than JSON marshal+upload for small payloads), plus an aggressive request-pool. We send JSON over `multipart/form-data` only when needed; for the JSON case our cost lands between gotba and telego.
- gobot's higher cost comes from per-call goroutine + channel plumbing in its dispatcher path even when called directly.
- **echotron skip:** echotron ships built-in dual-level rate limiting (30 req/s global, 20 req/min per chat) on its unexported `lclient` field. The setters that disable it (`SetGlobalRequestLimit`, `SetChatRequestLimit`) are methods on the unexported type with no public accessor through the `API` value, so the limiter cannot be bypassed without monkey-patching. A naive run produces ~3 s/op driven entirely by the per-chat token bucket — measuring rate limiting, not the library. We skip rather than publish a misleading number. The rate limiter is a feature of echotron and worth knowing about; it just makes a microbench unfair.
## 4. Dispatcher routing — 20 handlers, last one matches
Register 20 command handlers (`/cmd0``/cmd19`); feed an update matching `/cmd19` so the bench measures worst-case filter chain traversal.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| **ours** | **100.7 ns ±3%** | 128 B | 3 |
| telebot | 269.2 ns ±5% | 678 B | 5 |
| gobot | 251.5 ns ±4% | **48 B** | **1** |
**Notes.** We dispatch ~2.5× faster than telebot and gobot. gobot's single allocation is impressive but its routing decision is slower. telebot's higher cost reflects its richer per-update `Context` construction.
**Coverage caveats.**
- **gotba** ships no built-in dispatcher; users route via a manual `switch` on `Update` fields. Benchmarking that against framework-based dispatchers would be apples-to-oranges, so it's omitted.
- **telego** routes via a buffered channel + goroutine pool inside `telegohandler.BotHandler`. There is no public sync entry point, so the bench would conflate channel + goroutine overhead with routing cost.
- **echotron** uses a chat-ID-keyed `Dispatcher` that fans out to per-chat `Bot` instances — a different paradigm (stateful per-chat bot loop), not directly comparable to "match this update against N handlers".
---
## How to reproduce
```bash
cd test/benchmarks
go test -count=10 -bench=. -benchmem | tee results/raw.txt
benchstat results/raw.txt > results/benchstat.txt
```
Install `benchstat` if missing: `go install golang.org/x/perf/cmd/benchstat@latest`.
## Bench code
All bench source lives under [`test/benchmarks/`](../../test/benchmarks/) as a separate Go module so competitor dependencies stay out of the root `go.mod`. The fixtures (the JSON each library decodes, the mock HTTP server) are in [`shared/fixtures.go`](../../test/benchmarks/shared/fixtures.go) — every library decodes the same bytes.
+16 -4
View File
@@ -62,6 +62,7 @@ Package dispatch provides a typed router for Telegram updates. It consumes any t
- [func \(r \*Router\) OnRemovedChatBoost\(h Handler\[\*api.ChatBoostRemoved\]\)](<#Router.OnRemovedChatBoost>)
- [func \(r \*Router\) OnShippingQuery\(h Handler\[\*api.ShippingQuery\]\)](<#Router.OnShippingQuery>)
- [func \(r \*Router\) OnText\(pattern string, h Handler\[\*api.Message\]\)](<#Router.OnText>)
- [func \(r \*Router\) Process\(ctx context.Context, u \*api.Update\) error](<#Router.Process>)
- [func \(r \*Router\) Run\(ctx context.Context, u transport.Updater\) error](<#Router.Run>)
- [func \(r \*Router\) Use\(mw Middleware\[\*api.Update\]\)](<#Router.Use>)
- [type RouterOption](<#RouterOption>)
@@ -600,18 +601,29 @@ OnText registers a handler for messages whose Text matches the regex.
Panics at registration time if pattern is not a valid regular expression.
<a name="Router.Run"></a>
### func \(\*Router\) [Run](<https://github.com/lukaszraczylo/go-telegram/blob/main/dispatch/router.go#L303>)
<a name="Router.Process"></a>
### func \(\*Router\) [Process](<https://github.com/lukaszraczylo/go-telegram/blob/main/dispatch/router.go#L310>)
```go
func (r *Router) Run(ctx context.Context, u transport.Updater) error
func (r *Router) Process(ctx context.Context, u *api.Update) error
```
Run consumes the Updater and dispatches each update. It blocks until the Updater's channel is closed or ctx is cancelled.
By default updates are processed concurrently \(up to WithMaxConcurrency\(50\) goroutines\). Handlers for different updates may therefore run simultaneously; shared state must be protected. Pass WithMaxConcurrency\(0\) to New to restore serial \(legacy\) behaviour.
Run waits for all in\-flight handlers to finish before returning.
Run waits for all in\-flight handlers to finish before returning. Process runs a single update through the router's middleware and handler chain synchronously. Entry point for callers sourcing updates outside the standard transport.Updater flow — custom webhook frameworks, message\-bus consumers, or tests driving the router without spinning up Run.
Honours the router's global middleware \(Use\) but bypasses the concurrency semaphore wired up by Run; the caller controls parallelism.
<a name="Router.Run"></a>
### func \(\*Router\) [Run](<https://github.com/lukaszraczylo/go-telegram/blob/main/dispatch/router.go#L322>)
```go
func (r *Router) Run(ctx context.Context, u transport.Updater) error
```
<a name="Router.Use"></a>
### func \(\*Router\) [Use](<https://github.com/lukaszraczylo/go-telegram/blob/main/dispatch/router.go#L141>)