docs(examples): reCAPTCHA v2 audio-challenge recipe

Add examples/playwright/solve-recaptcha-audio.mjs — annotated Playwright recipe for solving reCAPTCHA v2 via the audio challenge. - Spawns MCP server, applies stealth fingerprint (same seed as vision example) - Checks solver_info.capabilities.audio before launching browser; exits cleanly on unsupported provider with a clear message - Clicks the anchor iframe checkbox, switches to audio challenge if image grid shown first, reads MP3 URL via .rc-audiochallenge-tdownload-link / #audio-source - Passes URL directly to solve_audio_captcha (server fetches + decodes) - Types transcription into #audio-response, clicks #recaptcha-verify-button - Reads g-recaptcha-response from host page; logs outcome, exits 0 on no token (headless environments commonly produce no token — documented in comments) - node --check passes; no Go changes
2026-10-12 00:45:28 +00:00 · 2026-06-07 17:42:21 +01:00
parent 87088e5065
commit f42282d70e
2 changed files with 303 additions and 3 deletions
@@ -80,9 +80,9 @@ All nine tools are registered in `internal/mcp/server.go`. Every image/audio inp
 | Tool | Inputs | Output |
 |------|--------|--------|
 | `detect_captcha` | `html?`, `url?`, `screenshot?` | `{detected:[{type,sitekey?,iframe_url?,notes?,llm_solvable}],summary}` — deterministic; v3/Turnstile → `llm_solvable:false` |
-| `solve_text_captcha` | `image`, `hint?`, `charset?`, `length?`, `case_sensitive?` | `{text,confidence}` (OCR) |
+| `solve_text_captcha` | `image`, `hint?`, `charset?`, `length?`, `case_sensitive?`, `samples?`, `upscale?` | `{text,confidence}` (OCR) |
 | `solve_audio_captcha` | `audio`, `language?` | `{text,confidence}`; errors clearly if provider has no transcription |
-| `solve_grid_captcha` | `screenshot`, `instruction`, `rows`, `cols`, `image_width?`, `image_height?`, `captcha_type?` | `{tiles,centroids?,confidence,recheck_recommended}` (reCAPTCHA v2 / hCaptcha) |
+| `solve_grid_captcha` | `screenshot`, `instruction`, `rows`, `cols`, `image_width?`, `image_height?`, `captcha_type?`, `samples?`, `upscale?`, `annotate?` | `{tiles,centroids?,confidence,no_targets,recheck_recommended}` (reCAPTCHA v2 / hCaptcha) |
 | `solve_rotation_captcha` | `screenshot`, `instruction?` | `{choice,rotations,confidence}` (FunCaptcha, best-effort) |
 | `get_stealth_script` | `engine?` = `playwright`\|`cdp`\|`generic`, `os?`, `browser?`, `locale?`, `seed?` | `{script,apply,fingerprint,included_evasions}` — JS init script (templated to match `fingerprint`) + how to inject it |
 | `generate_fingerprint` | `os?`, `browser?`, `locale?`, `seed?` | `{fingerprint,header_set}` — coherent identity + matching headers (seeded = stable) |
@@ -92,6 +92,13 @@ All nine tools are registered in `internal/mcp/server.go`. Every image/audio inp
 - `solve_grid_captcha` returns `centroids` (pixel `{x,y}` click points) only when
  `image_width` + `image_height` are supplied; otherwise the caller maps tile indices to
  pixels itself.
+- `solve_grid_captcha` new accuracy params: `samples` (run N times, majority-vote tiles;
+  `samples:3` raises Pass@1 ~70% → Success@3 ~97%); `upscale` (1–6, upscale image before
+  solving — helps small tiles); `annotate` (overlay numbered grid cells to reduce
+  off-by-one errors). Output now includes `no_targets:true` when no tile matches — caller
+  should click Verify instead of looping.
+- `solve_text_captcha` new params: `samples` (majority-vote the transcription) and `upscale`
+  (upscale image before OCR).
 - `solve_audio_captcha` requires an audio-capable provider. `anthropic` is **vision-only**
  here and will return a clear "unsupported" error — use `openai` or `gemini`
  (set `CAPTCHA_LLM_AUDIO_MODEL`).
@@ -135,7 +142,10 @@ The server is the brain; your browser is the hands. The loop for an image-grid c
   click), re-screenshot and repeat steps 3–5 until the grid clears.
 7. **Harvest** — read the `g-recaptcha-response` token from the page and submit your form.

-Runnable reference implementations are in [`examples/`](./examples).
+Runnable reference implementations are in [`examples/`](./examples):
+
+- [`examples/playwright/solve-recaptcha.mjs`](./examples/playwright/solve-recaptcha.mjs) — image-grid path (vision model)
+- [`examples/playwright/solve-recaptcha-audio.mjs`](./examples/playwright/solve-recaptcha-audio.mjs) — **audio path** (highest reliability; needs an audio-capable provider)

 ## Usage

@@ -169,6 +179,69 @@ export CAPTCHA_LLM_MODEL=openai/gpt-5.4        # or zai/GLM-4.6V-FlashX, claude-
 ./bin/captcha-solver-mcp
 ```

+## Maximizing accuracy
+
+### Image grids (reCAPTCHA v2 / hCaptcha)
+
+- Use `samples: 3` — the server runs the vision model three times and majority-votes the
+  tile selection. Published research (COGNITION, arXiv:2512.02318) shows this raises a ~70%
+  Pass@1 model to ~97% Success@3. Cost: 3× LLM calls per round.
+- Use `annotate: true` — overlays numbered cell labels on the grid image, which reduces
+  off-by-one selection errors on dense or ambiguous grids.
+- Use `upscale: 2` (or higher) for small tile grids — upscaling before the vision pass
+  helps models resolve fine detail in compressed tiles.
+- Use a strong vision model (GPT-5, Gemini-2.5, Claude Opus 4).
+- Watch `no_targets` in the response — when `true`, no tile matches the instruction and you
+  should click **Verify** rather than looping on an empty selection.
+- Watch `recheck_recommended` — when `true` the grid is dynamic (tiles re-render after each
+  click); re-screenshot and loop.
+
+### reCAPTCHA v2 — audio route (highest reliability)
+
+The **audio challenge is the most reliable programmatic route** for reCAPTCHA v2. It converts
+to a speech-to-text problem that modern ASR models handle at 70–97% accuracy (vs ~70–80%
+Pass@1 for image grids). See
+[`examples/playwright/solve-recaptcha-audio.mjs`](./examples/playwright/solve-recaptcha-audio.mjs)
+for a runnable annotated recipe.
+
+Requires an audio-capable provider:
+
+```sh
+CAPTCHA_LLM_PROVIDER=openai
+CAPTCHA_LLM_AUDIO_MODEL=whisper-1
+# or
+CAPTCHA_LLM_PROVIDER=gemini
+CAPTCHA_LLM_AUDIO_MODEL=gemini-2.0-flash
+```
+
+`anthropic` is vision-only and will return a clear "unsupported" error.
+
+### Distorted text captchas
+
+- `samples: 3` + `upscale: 2` for heavily distorted or low-resolution images.
+- Modern vision models (GPT-4o, Gemini-2.5) handle mild distortion zero-shot at ~99%
+  without upscaling; reserve upscaling for genuinely blurry or tiny inputs.
+
+### Cost note
+
+`samples: N` = N× LLM calls per tool invocation. Use `samples: 1` (the default) when
+latency or cost matters more than marginal accuracy gain.
+
+## Accuracy expectations (from published research)
+
+High solver accuracy does not guarantee a token. reCAPTCHA v2 and hCaptcha gate final token
+minting on the caller's browser + IP behavioral score — a high recognition rate in isolation
+does not compensate for headless-browser signals, suspicious IP reputation, or missing
+cookies. Use a real browser profile (`CAPTCHA_E2E_HEADED=1`) for best odds.
+
+| Captcha | Approach | Reported accuracy | Source |
+|---------|----------|-------------------|--------|
+| reCAPTCHA v2 image grid | fine-tuned YOLOv8 (not LLM) | ~100% image-solving | ETH Zurich "Breaking reCAPTCHAv2", arXiv:2409.08831 |
+| Recognition grids (MLLM) | GPT-5/Gemini-2.5, optimized prompt | >80% Pass@1; ~97% Success@3 | COGNITION, arXiv:2512.02318 |
+| Visual captchas (agentic VLM) | general solver | 60.7% controlled / 70.6% wild | Halligan, USENIX Security 2025 |
+| reCAPTCHA v2 audio | speech-to-text (Whisper / Gemini) | ~90% (unCaptcha2); 70–97% modern ASR | unCaptcha2, USENIX WOOT |
+| Distorted text | multimodal LLM + upscale | ~99% on mild distortion | modern VLM OCR benchmarks |
+
 ## Testing

 ```sh
@@ -244,6 +317,12 @@ One piece remains **designed but deferred** (see
 [`docs/superpowers/specs/`](./docs/superpowers/specs) and
 [`docs/superpowers/plans/`](./docs/superpowers/plans)):

+- **YOLOv8 grid classifier** — a fine-tuned YOLOv8 classifier/segmenter (per ETH
+  arXiv:2409.08831) embedded via an ONNX runtime would beat zero-shot LLM accuracy on
+  standard reCAPTCHA grids (~100% image-solving vs ~80% Pass@1 for zero-shot MLLMs), at the
+  cost of training data and a CV runtime dependency. Deferred as a separate effort from the
+  LLM path.
+
 - **Deterministic (offline) browser e2e harness** — driving Playwright Chromium *and*
  Lightpanda against a *local* fake-captcha page, to exercise the full brain/hands loop in
  CI without hitting real sites. A **live** browser harness already exists (`make
@@ -0,0 +1,221 @@
+// Reference: solve a reCAPTCHA v2 challenge via the AUDIO path with Playwright as the
+// "hands" and captcha-solver-mcp as the "brain".
+//
+// WHY AUDIO? The audio challenge converts to a speech-to-text problem, which modern ASR
+// models handle at 70–97% accuracy (unCaptcha2 reports ~90% on the USENIX WOOT 2019 dataset;
+// modern Whisper-class models typically land 80–97% depending on audio quality). This is
+// generally higher than the zero-shot vision path on image grids (~70–80% Pass@1), making
+// the audio route the most reliable programmatic path for reCAPTCHA v2.
+//
+// PROVIDER NOTE: `solve_audio_captcha` requires an audio-capable provider.
+// Anthropic is vision-only and will return a clear "unsupported" error. Use one of:
+//   CAPTCHA_LLM_PROVIDER=openai   +  CAPTCHA_LLM_AUDIO_MODEL=whisper-1
+//   CAPTCHA_LLM_PROVIDER=gemini   +  CAPTCHA_LLM_AUDIO_MODEL=gemini-2.0-flash  (or similar)
+//
+// Run it yourself (this repo does not vendor node_modules):
+//   npm i playwright
+//   node examples/playwright/solve-recaptcha-audio.mjs
+//
+// Required env:
+//   CAPTCHA_BIN=/abs/path/to/bin/captcha-solver-mcp
+//   CAPTCHA_LLM_PROVIDER=openai          # or gemini — NOT anthropic
+//   CAPTCHA_LLM_API_KEY=sk-...
+//   CAPTCHA_LLM_MODEL=gpt-4o             # vision model for general use
+//   CAPTCHA_LLM_AUDIO_MODEL=whisper-1    # transcription model
+//
+// HONEST CAVEAT: Google's reCAPTCHA infrastructure tracks behavioral signals beyond what
+// this script can control (TLS fingerprint, IP reputation, headless-browser detection). High
+// ASR accuracy does NOT guarantee a token — especially from headless automation.
+// Set CAPTCHA_E2E_HEADED=1 to launch a visible browser for a real-profile attempt that
+// significantly improves the odds. Headless reCAPTCHA v2 may never mint a token regardless
+// of solver accuracy; that is Google's anti-automation defense working as intended.
+//
+// AUTHORIZED USE ONLY. Run this against captcha integrations you own or are permitted to
+// test. Do not use it to evade protections on sites you do not control.
+
+import { chromium } from 'playwright';
+import { McpStdioClient } from '../mcp-client.mjs';
+
+const TARGET_URL = 'https://2captcha.com/demo/recaptcha-v2';
+const CAPTCHA_BIN = process.env.CAPTCHA_BIN || '../../bin/captcha-solver-mcp';
+const HEADED = process.env.CAPTCHA_E2E_HEADED === '1';
+
+// ── Step 1: Spawn the MCP server ───────────────────────────────────────────────
+// Pass LLM config from env straight through. The server validates on startup.
+const serverEnv = {
+  CAPTCHA_LLM_PROVIDER:     process.env.CAPTCHA_LLM_PROVIDER,
+  CAPTCHA_LLM_BASE_URL:     process.env.CAPTCHA_LLM_BASE_URL,
+  CAPTCHA_LLM_API_KEY:      process.env.CAPTCHA_LLM_API_KEY,
+  CAPTCHA_LLM_MODEL:        process.env.CAPTCHA_LLM_MODEL,
+  // whisper-1 (openai) or a Gemini model — the server routes audio to this separately.
+  CAPTCHA_LLM_AUDIO_MODEL:  process.env.CAPTCHA_LLM_AUDIO_MODEL,
+};
+
+const mcp = new McpStdioClient(CAPTCHA_BIN, serverEnv);
+await mcp.initialize();
+
+// Verify the configured provider supports audio before we start a browser.
+const info = await mcp.callTool('solver_info', {});
+if (!info.capabilities?.audio) {
+  console.error(
+    `[error] provider "${info.provider}" has no audio capability. ` +
+    'Set CAPTCHA_LLM_PROVIDER=openai or gemini and CAPTCHA_LLM_AUDIO_MODEL.',
+  );
+  mcp.close();
+  process.exit(1);
+}
+console.log(`[mcp] provider=${info.provider} audio_model=${serverEnv.CAPTCHA_LLM_AUDIO_MODEL ?? info.model}`);
+
+// ── Step 1b: Build a coherent stealth identity ─────────────────────────────────
+// generate_fingerprint + get_stealth_script with the SAME seed give a coherent pair:
+// the evasion script's WebGL/platform strings match the header set exactly.
+const fp = await mcp.callTool('generate_fingerprint', {
+  os: 'macos', browser: 'chrome', locale: 'en-US', seed: 42,
+});
+const stealth = await mcp.callTool('get_stealth_script', {
+  engine: 'playwright', os: 'macos', browser: 'chrome', locale: 'en-US', seed: 42,
+});
+console.log('[mcp] stealth evasions:', stealth.included_evasions);
+
+// ── Launch Chromium ────────────────────────────────────────────────────────────
+const browser = await chromium.launch({ headless: !HEADED, slowMo: HEADED ? 200 : 0 });
+const context = await browser.newContext({
+  userAgent: fp.header_set['User-Agent'] || fp.header_set['user-agent'],
+  locale:    'en-US',
+  extraHTTPHeaders: fp.header_set,
+});
+// Inject the evasion script BEFORE any page script runs — this is the correct order.
+await context.addInitScript({ content: stealth.script });
+
+const page = await context.newPage();
+
+// ── Step 2: Navigate and click the "I'm not a robot" checkbox ─────────────────
+// The checkbox lives inside the reCAPTCHA anchor iframe.
+console.log('[browser] navigating to', TARGET_URL);
+await page.goto(TARGET_URL, { waitUntil: 'domcontentloaded' });
+
+// NOTE: Selector `iframe[title="reCAPTCHA"]` is the anchor (checkbox) iframe.
+// The exact title varies slightly by locale; adjust if needed.
+const anchorFrame = page.frameLocator('iframe[title="reCAPTCHA"]');
+await anchorFrame.locator('#recaptcha-anchor').click();
+console.log('[browser] clicked checkbox, waiting for challenge iframe...');
+
+// The challenge iframe appears after Google evaluates the initial click signal.
+// Its title contains "challenge" (e.g. "recaptcha challenge expires in two minutes").
+const challengeFrame = page.frameLocator('iframe[title*="challenge"]');
+
+// Wait for the challenge to load (image grid OR audio button present).
+await challengeFrame.locator('#recaptcha-audio-button, .rc-imageselect-challenge').waitFor({ timeout: 15_000 });
+
+// ── Step 3: Switch to the audio challenge ─────────────────────────────────────
+// If an image grid is shown first, click the audio button to switch.
+// The audio button selector is stable across reCAPTCHA versions.
+const audioBtn = challengeFrame.locator('#recaptcha-audio-button');
+const audioBtnVisible = await audioBtn.isVisible().catch(() => false);
+if (audioBtnVisible) {
+  await audioBtn.click();
+  console.log('[browser] switched to audio challenge');
+  // Wait for the audio player to appear.
+  await challengeFrame.locator('.rc-audiochallenge-tdownload-link, #audio-source').waitFor({ timeout: 10_000 });
+} else {
+  // We might already be on the audio challenge (unlikely on first load, but handle it).
+  console.log('[browser] audio challenge already visible');
+}
+
+// ── Step 4: Read the audio MP3 URL ────────────────────────────────────────────
+// Two selectors cover different reCAPTCHA versions:
+//   .rc-audiochallenge-tdownload-link  — the "Download" link (href = MP3 URL)
+//   #audio-source                      — the <source> element inside <audio>
+// We try the download link first (more reliable); fall back to audio-source src.
+//
+// NOTE: Google periodically changes these selectors. Treat them as illustrative.
+let audioUrl = null;
+
+const downloadLink = challengeFrame.locator('.rc-audiochallenge-tdownload-link');
+if (await downloadLink.isVisible().catch(() => false)) {
+  audioUrl = await downloadLink.getAttribute('href');
+}
+
+if (!audioUrl) {
+  const audioSource = challengeFrame.locator('#audio-source');
+  if (await audioSource.isVisible().catch(() => false)) {
+    audioUrl = await audioSource.getAttribute('src');
+  }
+}
+
+if (!audioUrl) {
+  console.warn('[warn] could not locate audio URL — Google may have blocked the audio endpoint.');
+  console.warn('       Try CAPTCHA_E2E_HEADED=1 for a real-profile attempt.');
+  mcp.close();
+  await browser.close();
+  process.exit(0); // non-fatal: document the outcome, don't throw
+}
+console.log('[browser] audio URL:', audioUrl);
+
+// ── Step 5: Pass the audio URL to solve_audio_captcha ─────────────────────────
+// imageutil.Decode on the server accepts http(s) URLs directly — no need to fetch
+// and base64-encode on our side. The server downloads and decodes it.
+//
+// If the MP3 URL requires a session cookie to download, you would instead:
+//   const resp = await fetch(audioUrl, { headers: { Cookie: ... } });
+//   const buf = Buffer.from(await resp.arrayBuffer());
+//   const audioInput = `data:audio/mpeg;base64,${buf.toString('base64')}`;
+// For the public reCAPTCHA demo the URL is publicly fetchable without auth.
+console.log('[mcp] calling solve_audio_captcha...');
+let solveResult;
+try {
+  solveResult = await mcp.callTool('solve_audio_captcha', {
+    audio: audioUrl,
+    language: 'en',
+  });
+} catch (err) {
+  console.error('[error] solve_audio_captcha failed:', err.message);
+  mcp.close();
+  await browser.close();
+  process.exit(0); // non-fatal exit so CI doesn't fail on provider misconfiguration
+}
+
+const transcription = solveResult.text ?? '';
+const confidence    = solveResult.confidence ?? 0;
+console.log(`[mcp] transcription="${transcription}" confidence=${confidence.toFixed(2)}`);
+
+if (!transcription) {
+  console.warn('[warn] empty transcription — cannot proceed');
+  mcp.close();
+  await browser.close();
+  process.exit(0);
+}
+
+// ── Step 6: Type the answer and click Verify ──────────────────────────────────
+// #audio-response is the text input inside the challenge iframe.
+// #recaptcha-verify-button submits the answer.
+await challengeFrame.locator('#audio-response').fill(transcription);
+await challengeFrame.locator('#recaptcha-verify-button').click();
+console.log('[browser] submitted answer, waiting for response...');
+
+// Give Google a moment to evaluate the submission.
+await page.waitForTimeout(2000);
+
+// ── Step 7: Read the token ────────────────────────────────────────────────────
+// The caller (us) reads the token — the MCP server never sees it.
+// g-recaptcha-response is a hidden textarea injected into the HOST page (not the iframe).
+const token = await page.evaluate(() => {
+  const el = document.querySelector('textarea[name="g-recaptcha-response"]');
+  return el ? el.value : '';
+});
+
+if (token) {
+  console.log(`[result] token obtained (length=${token.length}) — challenge passed.`);
+} else {
+  console.warn(
+    '[result] no token. Possible reasons:\n' +
+    '  • Google rejected the audio answer (try again or use a better audio model)\n' +
+    '  • Headless behavioral signals flagged the session (try CAPTCHA_E2E_HEADED=1)\n' +
+    '  • The audio endpoint was blocked before we fetched the MP3\n' +
+    '  • Google served an image challenge even after clicking the audio button\n' +
+    'This is expected in many headless environments — not a bug in the solver.',
+  );
+}
+
+mcp.close();
+await browser.close();