lukaszraczylo/go-telegram
1
0
Fork 0
mirror of https://github.com/lukaszraczylo/go-telegram.git synced 2026-06-05 22:43:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

test(bench): cross-library benchmarks vs top 5 go telegram libraries

Browse Source 
Adds test/benchmarks/ as a separate Go module so competitor deps
(go-telegram-bot-api/v5, telebot.v3, go-telegram/bot, telego,
echotron/v3) stay out of the root go.mod.

Hot paths covered:
  - Webhook decode  (small Update -> typed Update struct)
  - Large unmarshal (Update with entities + reply markup + photo array)
  - API round-trip  (sendMessage against httptest.Server)
  - Dispatch route  (20 handlers, last-registered matches)

Results on Apple M4 Max / go1.26.2: ours wins 3 of 4 paths and is
2nd of 5 in the round-trip path. Full report at
docs/benchmarks/2026-05-10-comparison.md, raw output committed under
test/benchmarks/results/.

Caveats called out in the report:
  - codec asymmetry (we ship goccy/go-json; competitors mostly stdlib)
  - echotron call bench skipped — built-in rate limiter not externally
    configurable; would measure throttling, not the library
  - dispatch bench limited to libs with a public sync entry point
    (ours, telebot, gobot); gotba has no dispatcher, telego/echotron
    use channel/per-chat paradigms not directly comparable

Also gitignores docs/superpowers/ (local brainstorm/spec scratch)
and regenerates docs/reference/dispatch.md after the new
Router.Process method.
This commit is contained in:
Lukasz Raczylo
2026-05-10 21:52:00 +01:00
parent c9a062ea04
commit 607c3e8ddd
14 changed files with 1935 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/benchmarks/2026-05-10-comparison.md
+113
View File
@@ -0,0 +1,113 @@
# Benchmarks vs top 5 Go Telegram libraries
**Date:** 2026-05-10
**Environment:** Apple M4 Max · darwin/arm64 · `go1.26.2`
**Methodology:** `go test -count=10 -bench=. -benchmem`, summarised with `benchstat` (golang.org/x/perf)
**Source:** [`test/benchmarks/`](../../test/benchmarks/) · raw output: [`results/raw.txt`](../../test/benchmarks/results/raw.txt) · benchstat: [`results/benchstat.txt`](../../test/benchmarks/results/benchstat.txt)
## Libraries
| Lib | Module |
|-----|--------|
| **ours** | `github.com/lukaszraczylo/go-telegram` (this repo) |
| gotba | `github.com/go-telegram-bot-api/telegram-bot-api/v5` |
| telebot | `gopkg.in/telebot.v3` (tucnak) |
| gobot | `github.com/go-telegram/bot` |
| telego | `github.com/mymmrac/telego` |
| echotron | `github.com/NicoNex/echotron/v3` |
## TL;DR
- **Webhook decode** (small Update): ours is **15–19% faster** than every competitor and ties telego for the lowest alloc count (11).
- **Large Update unmarshal** (entities + reply markup + photo array): ours is **17–35% faster** with the lowest ns/op of all six. telego edges us on alloc count (31 vs 34) at the cost of ~17% more time.
- **API call round-trip** (mock HTTP server): telego wins (36.3 µs / 48 allocs) thanks to its custom binder; ours is second (38.95 µs / 104 allocs) and beats gotba, telebot, gobot.
- **Dispatcher routing** (20 handlers, last matches): ours is **2.5× faster than telebot and gobot** (101 ns vs 269 / 252 ns).
## How to read these numbers
- One machine, single workload, fixtures defined in [`shared/fixtures.go`](../../test/benchmarks/shared/fixtures.go). Re-run on your hardware before drawing conclusions.
- Codecs differ across libs (we use `goccy/go-json`; most competitors use stdlib `encoding/json`). Codec choice is part of the library's value prop, so we benchmark each library as it ships, not in some artificial common-codec mode.
- "Equivalent code path" was chosen via each library's idiomatic public API for the same logical operation. The exact code is in the bench files alongside each `BenchmarkXxx_<lib>` function — read them.
---
## 1. Webhook decode — small Update (text message)
Decode `shared.SmallUpdateJSON` into the library's typed `Update` struct.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| **ours** | **1.743 µs ±3%** | 2.180 KiB | **11** |
| gotba | 2.016 µs ±3% | 1.461 KiB | 17 |
| telebot | 2.073 µs ±3% | 1.773 KiB | 17 |
| gobot | 1.999 µs ±1% | 1.789 KiB | 16 |
| telego | 2.026 µs ±2% | 3.060 KiB | **11** |
| echotron | 1.973 µs ±0% | 1.680 KiB | 16 |
**Notes.** We use slightly more bytes because typed unions and the typed `[]UpdateType` allocate richer Go values. We win on time and tie telego on alloc count.
## 2. Large Update unmarshal — entities + reply markup + photo array
Decode `shared.LargeUpdateJSON` (text + 3 entities + 2x3 inline keyboard + 3-size photo array). Stresses each library's union/discriminator decoding.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| **ours** | **6.667 µs ±4%** | 5.881 KiB | 34 |
| gotba | 8.321 µs ±2% | 3.438 KiB | 56 |
| telebot | 10.240 µs ±4% | 5.594 KiB | 60 |
| gobot | 8.150 µs ±2% | 4.703 KiB | 50 |
| telego | 7.797 µs ±1% | 6.621 KiB | **31** |
| echotron | 8.072 µs ±0% | 4.219 KiB | 56 |
**Notes.** Despite the typed-union model giving us richer Go values per decode, we still produce them faster than every competitor. telego edges us by 3 allocs but pays 17% more time.
## 3. API call round-trip — `sendMessage` against a mock HTTP server
Build params → POST to local `httptest.Server` returning `{"ok":true,"result":Message}` → decode response.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| ours | 38.95 µs ±3% | 11.17 KiB | 104 |
| gotba | 41.95 µs ±2% | 10.95 KiB | 125 |
| telebot | 43.63 µs ±0% | 13.16 KiB | 139 |
| gobot | 61.11 µs ±1% | 13.51 KiB | 176 |
| **telego** | **36.31 µs ±1%** | **6.556 KiB** | **48** |
| echotron | *skipped — see below* | — | — |
**Notes.**
- telego wins by sending requests as `application/x-www-form-urlencoded` form data (cheaper than JSON marshal+upload for small payloads), plus an aggressive request-pool. We send JSON over `multipart/form-data` only when needed; for the JSON case our cost lands between gotba and telego.
- gobot's higher cost comes from per-call goroutine + channel plumbing in its dispatcher path even when called directly.
- **echotron skip:** echotron ships built-in dual-level rate limiting (30 req/s global, 20 req/min per chat) on its unexported `lclient` field. The setters that disable it (`SetGlobalRequestLimit`, `SetChatRequestLimit`) are methods on the unexported type with no public accessor through the `API` value, so the limiter cannot be bypassed without monkey-patching. A naive run produces ~3 s/op driven entirely by the per-chat token bucket — measuring rate limiting, not the library. We skip rather than publish a misleading number. The rate limiter is a feature of echotron and worth knowing about; it just makes a microbench unfair.
## 4. Dispatcher routing — 20 handlers, last one matches
Register 20 command handlers (`/cmd0` … `/cmd19`); feed an update matching `/cmd19` so the bench measures worst-case filter chain traversal.
| Lib | sec/op | B/op | allocs/op |
|-----|--------|------|-----------|
| **ours** | **100.7 ns ±3%** | 128 B | 3 |
| telebot | 269.2 ns ±5% | 678 B | 5 |
| gobot | 251.5 ns ±4% | **48 B** | **1** |
**Notes.** We dispatch ~2.5× faster than telebot and gobot. gobot's single allocation is impressive but its routing decision is slower. telebot's higher cost reflects its richer per-update `Context` construction.
**Coverage caveats.**
- **gotba** ships no built-in dispatcher; users route via a manual `switch` on `Update` fields. Benchmarking that against framework-based dispatchers would be apples-to-oranges, so it's omitted.
- **telego** routes via a buffered channel + goroutine pool inside `telegohandler.BotHandler`. There is no public sync entry point, so the bench would conflate channel + goroutine overhead with routing cost.
- **echotron** uses a chat-ID-keyed `Dispatcher` that fans out to per-chat `Bot` instances — a different paradigm (stateful per-chat bot loop), not directly comparable to "match this update against N handlers".
---
## How to reproduce
```bash
cd test/benchmarks
go test -count=10 -bench=. -benchmem | tee results/raw.txt
benchstat results/raw.txt > results/benchstat.txt
```
Install `benchstat` if missing: `go install golang.org/x/perf/cmd/benchstat@latest`.
## Bench code
All bench source lives under [`test/benchmarks/`](../../test/benchmarks/) as a separate Go module so competitor dependencies stay out of the root `go.mod`. The fixtures (the JSON each library decodes, the mock HTTP server) are in [`shared/fixtures.go`](../../test/benchmarks/shared/fixtures.go) — every library decodes the same bytes.