The circuit breaker ignored the trip-on-timeout, trip-on-4xx, and
trip-on-5xx flags and never cleared its counts, so the failure ratio
became lifetime-cumulative. The backoff-multiplier and max-backoff
settings did nothing. Circuit-open surfaced as HTTP 500.
Failure classification now honours the trip flags, the breaker uses a
rolling count window, and repeated trips extend the open state up to the
configured maximum. A helper maps circuit-open to HTTP 503. The request
body is copied once per request instead of once per retry. Defaults
reproduce the prior behaviour: the breaker is off by default and a
backoff multiplier of one keeps a constant open window.
doProxyRequestWithTimeout returns an error for every non-200 status and
executeProxyAttempt returned it as a plain, retryable error, so 4xx
client errors (400/404/405/...) were retried with exponential backoff
and the 4xx branch of isRetryableStatusCode was dead code.
Additionally, retry-go's finite-attempt loop only consults the custom
RetryIf, which overrode the default IsRecoverable gate, so retry.
Unrecoverable errors (non-retryable 4xx AND timeouts) were still
retried. Honor IsRecoverable in RetryIf so these never retry.
Now 5xx/429 are retried, other 4xx and timeouts fail fast.
Adds Test_proxyRetryPolicyByStatusCode locking in the policy.
The test hit live external endpoints (telegram-bot.app, google.com) and
failed once telegram-bot.app changed /v1/graphql to return 405. Replace
real hosts with local ok (200) and err (404) servers, preserving the
query->read-only / mutation->write routing semantics under test.
BUG FIX: The endpoint routing logic was inside the loop that
processes all GraphQL definitions. This caused mutations to be incorrectly
routed to read-only endpoints when followed by other definitions (queries,
fragments, etc).
The bug manifested as: mutations → read-only Hasura → read-only pooler →
PostgreSQL replica → "cannot set transaction read-write mode during recovery"
Changes:
- Move endpoint routing logic AFTER the definition processing loop
- Ensures mutations are ALWAYS routed to write endpoint regardless of
subsequent definitions in the document
- Add 3 comprehensive regression tests covering:
1. Mutation with multiple operations
2. Mutation followed by fragment
3. Complex main-bot style mutation document
Tests: All pass including new regression tests
Impact: Fixes database write failures in main-bot and other services
* This change introduces ability to set additional endpoint leading to the
instance of the graphql server connected to the read only database.
If regular query is detected and endpoint for `HOST_GRAPHQL_READONLY` value is set,
the query will be proxied to it. Mutations and non-graphql will be sent
to the `HOST_GRAPHQL` endpoint.
lukaszraczylo
