diff --git a/README.md b/README.md index a6f75f6..d223081 100644 --- a/README.md +++ b/README.md @@ -80,9 +80,9 @@ All nine tools are registered in `internal/mcp/server.go`. Every image/audio inp | Tool | Inputs | Output | |------|--------|--------| | `detect_captcha` | `html?`, `url?`, `screenshot?` | `{detected:[{type,sitekey?,iframe_url?,notes?,llm_solvable}],summary}` — deterministic; v3/Turnstile → `llm_solvable:false` | -| `solve_text_captcha` | `image`, `hint?`, `charset?`, `length?`, `case_sensitive?` | `{text,confidence}` (OCR) | +| `solve_text_captcha` | `image`, `hint?`, `charset?`, `length?`, `case_sensitive?`, `samples?`, `upscale?` | `{text,confidence}` (OCR) | | `solve_audio_captcha` | `audio`, `language?` | `{text,confidence}`; errors clearly if provider has no transcription | -| `solve_grid_captcha` | `screenshot`, `instruction`, `rows`, `cols`, `image_width?`, `image_height?`, `captcha_type?` | `{tiles,centroids?,confidence,recheck_recommended}` (reCAPTCHA v2 / hCaptcha) | +| `solve_grid_captcha` | `screenshot`, `instruction`, `rows`, `cols`, `image_width?`, `image_height?`, `captcha_type?`, `samples?`, `upscale?`, `annotate?` | `{tiles,centroids?,confidence,no_targets,recheck_recommended}` (reCAPTCHA v2 / hCaptcha) | | `solve_rotation_captcha` | `screenshot`, `instruction?` | `{choice,rotations,confidence}` (FunCaptcha, best-effort) | | `get_stealth_script` | `engine?` = `playwright`\|`cdp`\|`generic`, `os?`, `browser?`, `locale?`, `seed?` | `{script,apply,fingerprint,included_evasions}` — JS init script (templated to match `fingerprint`) + how to inject it | | `generate_fingerprint` | `os?`, `browser?`, `locale?`, `seed?` | `{fingerprint,header_set}` — coherent identity + matching headers (seeded = stable) | @@ -92,6 +92,13 @@ All nine tools are registered in `internal/mcp/server.go`. Every image/audio inp - `solve_grid_captcha` returns `centroids` (pixel `{x,y}` click points) only when `image_width` + `image_height` are supplied; otherwise the caller maps tile indices to pixels itself. +- `solve_grid_captcha` new accuracy params: `samples` (run N times, majority-vote tiles; + `samples:3` raises Pass@1 ~70% → Success@3 ~97%); `upscale` (1–6, upscale image before + solving — helps small tiles); `annotate` (overlay numbered grid cells to reduce + off-by-one errors). Output now includes `no_targets:true` when no tile matches — caller + should click Verify instead of looping. +- `solve_text_captcha` new params: `samples` (majority-vote the transcription) and `upscale` + (upscale image before OCR). - `solve_audio_captcha` requires an audio-capable provider. `anthropic` is **vision-only** here and will return a clear "unsupported" error — use `openai` or `gemini` (set `CAPTCHA_LLM_AUDIO_MODEL`). @@ -135,7 +142,10 @@ The server is the brain; your browser is the hands. The loop for an image-grid c click), re-screenshot and repeat steps 3–5 until the grid clears. 7. **Harvest** — read the `g-recaptcha-response` token from the page and submit your form. -Runnable reference implementations are in [`examples/`](./examples). +Runnable reference implementations are in [`examples/`](./examples): + +- [`examples/playwright/solve-recaptcha.mjs`](./examples/playwright/solve-recaptcha.mjs) — image-grid path (vision model) +- [`examples/playwright/solve-recaptcha-audio.mjs`](./examples/playwright/solve-recaptcha-audio.mjs) — **audio path** (highest reliability; needs an audio-capable provider) ## Usage @@ -169,6 +179,69 @@ export CAPTCHA_LLM_MODEL=openai/gpt-5.4 # or zai/GLM-4.6V-FlashX, claude- ./bin/captcha-solver-mcp ``` +## Maximizing accuracy + +### Image grids (reCAPTCHA v2 / hCaptcha) + +- Use `samples: 3` — the server runs the vision model three times and majority-votes the + tile selection. Published research (COGNITION, arXiv:2512.02318) shows this raises a ~70% + Pass@1 model to ~97% Success@3. Cost: 3× LLM calls per round. +- Use `annotate: true` — overlays numbered cell labels on the grid image, which reduces + off-by-one selection errors on dense or ambiguous grids. +- Use `upscale: 2` (or higher) for small tile grids — upscaling before the vision pass + helps models resolve fine detail in compressed tiles. +- Use a strong vision model (GPT-5, Gemini-2.5, Claude Opus 4). +- Watch `no_targets` in the response — when `true`, no tile matches the instruction and you + should click **Verify** rather than looping on an empty selection. +- Watch `recheck_recommended` — when `true` the grid is dynamic (tiles re-render after each + click); re-screenshot and loop. + +### reCAPTCHA v2 — audio route (highest reliability) + +The **audio challenge is the most reliable programmatic route** for reCAPTCHA v2. It converts +to a speech-to-text problem that modern ASR models handle at 70–97% accuracy (vs ~70–80% +Pass@1 for image grids). See +[`examples/playwright/solve-recaptcha-audio.mjs`](./examples/playwright/solve-recaptcha-audio.mjs) +for a runnable annotated recipe. + +Requires an audio-capable provider: + +```sh +CAPTCHA_LLM_PROVIDER=openai +CAPTCHA_LLM_AUDIO_MODEL=whisper-1 +# or +CAPTCHA_LLM_PROVIDER=gemini +CAPTCHA_LLM_AUDIO_MODEL=gemini-2.0-flash +``` + +`anthropic` is vision-only and will return a clear "unsupported" error. + +### Distorted text captchas + +- `samples: 3` + `upscale: 2` for heavily distorted or low-resolution images. +- Modern vision models (GPT-4o, Gemini-2.5) handle mild distortion zero-shot at ~99% + without upscaling; reserve upscaling for genuinely blurry or tiny inputs. + +### Cost note + +`samples: N` = N× LLM calls per tool invocation. Use `samples: 1` (the default) when +latency or cost matters more than marginal accuracy gain. + +## Accuracy expectations (from published research) + +High solver accuracy does not guarantee a token. reCAPTCHA v2 and hCaptcha gate final token +minting on the caller's browser + IP behavioral score — a high recognition rate in isolation +does not compensate for headless-browser signals, suspicious IP reputation, or missing +cookies. Use a real browser profile (`CAPTCHA_E2E_HEADED=1`) for best odds. + +| Captcha | Approach | Reported accuracy | Source | +|---------|----------|-------------------|--------| +| reCAPTCHA v2 image grid | fine-tuned YOLOv8 (not LLM) | ~100% image-solving | ETH Zurich "Breaking reCAPTCHAv2", arXiv:2409.08831 | +| Recognition grids (MLLM) | GPT-5/Gemini-2.5, optimized prompt | >80% Pass@1; ~97% Success@3 | COGNITION, arXiv:2512.02318 | +| Visual captchas (agentic VLM) | general solver | 60.7% controlled / 70.6% wild | Halligan, USENIX Security 2025 | +| reCAPTCHA v2 audio | speech-to-text (Whisper / Gemini) | ~90% (unCaptcha2); 70–97% modern ASR | unCaptcha2, USENIX WOOT | +| Distorted text | multimodal LLM + upscale | ~99% on mild distortion | modern VLM OCR benchmarks | + ## Testing ```sh @@ -244,6 +317,12 @@ One piece remains **designed but deferred** (see [`docs/superpowers/specs/`](./docs/superpowers/specs) and [`docs/superpowers/plans/`](./docs/superpowers/plans)): +- **YOLOv8 grid classifier** — a fine-tuned YOLOv8 classifier/segmenter (per ETH + arXiv:2409.08831) embedded via an ONNX runtime would beat zero-shot LLM accuracy on + standard reCAPTCHA grids (~100% image-solving vs ~80% Pass@1 for zero-shot MLLMs), at the + cost of training data and a CV runtime dependency. Deferred as a separate effort from the + LLM path. + - **Deterministic (offline) browser e2e harness** — driving Playwright Chromium *and* Lightpanda against a *local* fake-captcha page, to exercise the full brain/hands loop in CI without hitting real sites. A **live** browser harness already exists (`make diff --git a/examples/playwright/solve-recaptcha-audio.mjs b/examples/playwright/solve-recaptcha-audio.mjs new file mode 100644 index 0000000..960ca9f --- /dev/null +++ b/examples/playwright/solve-recaptcha-audio.mjs @@ -0,0 +1,221 @@ +// Reference: solve a reCAPTCHA v2 challenge via the AUDIO path with Playwright as the +// "hands" and captcha-solver-mcp as the "brain". +// +// WHY AUDIO? The audio challenge converts to a speech-to-text problem, which modern ASR +// models handle at 70–97% accuracy (unCaptcha2 reports ~90% on the USENIX WOOT 2019 dataset; +// modern Whisper-class models typically land 80–97% depending on audio quality). This is +// generally higher than the zero-shot vision path on image grids (~70–80% Pass@1), making +// the audio route the most reliable programmatic path for reCAPTCHA v2. +// +// PROVIDER NOTE: `solve_audio_captcha` requires an audio-capable provider. +// Anthropic is vision-only and will return a clear "unsupported" error. Use one of: +// CAPTCHA_LLM_PROVIDER=openai + CAPTCHA_LLM_AUDIO_MODEL=whisper-1 +// CAPTCHA_LLM_PROVIDER=gemini + CAPTCHA_LLM_AUDIO_MODEL=gemini-2.0-flash (or similar) +// +// Run it yourself (this repo does not vendor node_modules): +// npm i playwright +// node examples/playwright/solve-recaptcha-audio.mjs +// +// Required env: +// CAPTCHA_BIN=/abs/path/to/bin/captcha-solver-mcp +// CAPTCHA_LLM_PROVIDER=openai # or gemini — NOT anthropic +// CAPTCHA_LLM_API_KEY=sk-... +// CAPTCHA_LLM_MODEL=gpt-4o # vision model for general use +// CAPTCHA_LLM_AUDIO_MODEL=whisper-1 # transcription model +// +// HONEST CAVEAT: Google's reCAPTCHA infrastructure tracks behavioral signals beyond what +// this script can control (TLS fingerprint, IP reputation, headless-browser detection). High +// ASR accuracy does NOT guarantee a token — especially from headless automation. +// Set CAPTCHA_E2E_HEADED=1 to launch a visible browser for a real-profile attempt that +// significantly improves the odds. Headless reCAPTCHA v2 may never mint a token regardless +// of solver accuracy; that is Google's anti-automation defense working as intended. +// +// AUTHORIZED USE ONLY. Run this against captcha integrations you own or are permitted to +// test. Do not use it to evade protections on sites you do not control. + +import { chromium } from 'playwright'; +import { McpStdioClient } from '../mcp-client.mjs'; + +const TARGET_URL = 'https://2captcha.com/demo/recaptcha-v2'; +const CAPTCHA_BIN = process.env.CAPTCHA_BIN || '../../bin/captcha-solver-mcp'; +const HEADED = process.env.CAPTCHA_E2E_HEADED === '1'; + +// ── Step 1: Spawn the MCP server ─────────────────────────────────────────────── +// Pass LLM config from env straight through. The server validates on startup. +const serverEnv = { + CAPTCHA_LLM_PROVIDER: process.env.CAPTCHA_LLM_PROVIDER, + CAPTCHA_LLM_BASE_URL: process.env.CAPTCHA_LLM_BASE_URL, + CAPTCHA_LLM_API_KEY: process.env.CAPTCHA_LLM_API_KEY, + CAPTCHA_LLM_MODEL: process.env.CAPTCHA_LLM_MODEL, + // whisper-1 (openai) or a Gemini model — the server routes audio to this separately. + CAPTCHA_LLM_AUDIO_MODEL: process.env.CAPTCHA_LLM_AUDIO_MODEL, +}; + +const mcp = new McpStdioClient(CAPTCHA_BIN, serverEnv); +await mcp.initialize(); + +// Verify the configured provider supports audio before we start a browser. +const info = await mcp.callTool('solver_info', {}); +if (!info.capabilities?.audio) { + console.error( + `[error] provider "${info.provider}" has no audio capability. ` + + 'Set CAPTCHA_LLM_PROVIDER=openai or gemini and CAPTCHA_LLM_AUDIO_MODEL.', + ); + mcp.close(); + process.exit(1); +} +console.log(`[mcp] provider=${info.provider} audio_model=${serverEnv.CAPTCHA_LLM_AUDIO_MODEL ?? info.model}`); + +// ── Step 1b: Build a coherent stealth identity ───────────────────────────────── +// generate_fingerprint + get_stealth_script with the SAME seed give a coherent pair: +// the evasion script's WebGL/platform strings match the header set exactly. +const fp = await mcp.callTool('generate_fingerprint', { + os: 'macos', browser: 'chrome', locale: 'en-US', seed: 42, +}); +const stealth = await mcp.callTool('get_stealth_script', { + engine: 'playwright', os: 'macos', browser: 'chrome', locale: 'en-US', seed: 42, +}); +console.log('[mcp] stealth evasions:', stealth.included_evasions); + +// ── Launch Chromium ──────────────────────────────────────────────────────────── +const browser = await chromium.launch({ headless: !HEADED, slowMo: HEADED ? 200 : 0 }); +const context = await browser.newContext({ + userAgent: fp.header_set['User-Agent'] || fp.header_set['user-agent'], + locale: 'en-US', + extraHTTPHeaders: fp.header_set, +}); +// Inject the evasion script BEFORE any page script runs — this is the correct order. +await context.addInitScript({ content: stealth.script }); + +const page = await context.newPage(); + +// ── Step 2: Navigate and click the "I'm not a robot" checkbox ───────────────── +// The checkbox lives inside the reCAPTCHA anchor iframe. +console.log('[browser] navigating to', TARGET_URL); +await page.goto(TARGET_URL, { waitUntil: 'domcontentloaded' }); + +// NOTE: Selector `iframe[title="reCAPTCHA"]` is the anchor (checkbox) iframe. +// The exact title varies slightly by locale; adjust if needed. +const anchorFrame = page.frameLocator('iframe[title="reCAPTCHA"]'); +await anchorFrame.locator('#recaptcha-anchor').click(); +console.log('[browser] clicked checkbox, waiting for challenge iframe...'); + +// The challenge iframe appears after Google evaluates the initial click signal. +// Its title contains "challenge" (e.g. "recaptcha challenge expires in two minutes"). +const challengeFrame = page.frameLocator('iframe[title*="challenge"]'); + +// Wait for the challenge to load (image grid OR audio button present). +await challengeFrame.locator('#recaptcha-audio-button, .rc-imageselect-challenge').waitFor({ timeout: 15_000 }); + +// ── Step 3: Switch to the audio challenge ───────────────────────────────────── +// If an image grid is shown first, click the audio button to switch. +// The audio button selector is stable across reCAPTCHA versions. +const audioBtn = challengeFrame.locator('#recaptcha-audio-button'); +const audioBtnVisible = await audioBtn.isVisible().catch(() => false); +if (audioBtnVisible) { + await audioBtn.click(); + console.log('[browser] switched to audio challenge'); + // Wait for the audio player to appear. + await challengeFrame.locator('.rc-audiochallenge-tdownload-link, #audio-source').waitFor({ timeout: 10_000 }); +} else { + // We might already be on the audio challenge (unlikely on first load, but handle it). + console.log('[browser] audio challenge already visible'); +} + +// ── Step 4: Read the audio MP3 URL ──────────────────────────────────────────── +// Two selectors cover different reCAPTCHA versions: +// .rc-audiochallenge-tdownload-link — the "Download" link (href = MP3 URL) +// #audio-source — the element inside