Detection
How to detect breaking changes in third-party APIs before they hit production
Vendor changes are invisible to your tests because your tests mock the vendor. The channels that make a vendor change visible, ranked, and a weekly pipeline that uses the best two.
Updated 9 October 2026 · 7 min read
Your test suite cannot see a vendor breaking change, because the vendor is mocked: the mock has the old shape, and it will keep the old shape until a human rewrites it. This is the structural reason integrations break with tests green. A third-party API breaking change reaches production through one of a small number of channels, and the channels are not equally good. This article ranks them with evidence, then assembles the machine-readable ones into a weekly pipeline.
The five channels, ranked
1. The vendor's published OpenAPI specification, diffed over time. This is the only channel that is complete, per-operation, dated, and diffable by machine. Three of the biggest vendors publish their full REST surface in a public repository with per-commit history: github/rest-api-description, stripe/openapi, openai/openai-openapi. GitHub versions its REST API by date, and the X-GitHub-Api-Version header selects a version. Its specification carries a deprecated flag on each operation, so a deprecation appears there as a change you can diff.
Diffing between a pinned baseline and current answers one precise question: did any operation my code calls change shape, gain a required parameter, or get marked deprecated. The six-month GitHub measurement behind apibreak's /changes pages shows why the completeness matters: 1,107 operations became 1,224, with 25 removed and 7 newly deprecated, a volume nobody tracks by skimming a changelog.
What it misses: vendors without published specs, and behavior changes invisible to the contract.
2. The changelog or deprecations page. The vendor's own prose, the channel they intend you to read. OpenAI's deprecations page is the entire lifecycle contract for their models: email to affected customers plus a documentation entry, with minimum notice periods (six months for generally available models, three for specialized, potentially two weeks for preview). Stripe's changelog lists the changes in each API version; the 2026-09-30.endive version has one entry removing the payment method types parameter from Checkout Sessions and another removing it from Payment Intents and Setup Intents. Strengths: intent, narrative, links. Weaknesses: prose, opt-in reading, and vendors under-label severity.
Twilio is the case where the changelog is the only sensor: their versioned API spec did not change one operation in the six months apibreak measured (197 operations before, 197 after), so the mechanical channel is quiet and the changelog carries everything, including the regional api.REGION.twilio.com domains that Twilio said would stop working on April 28, 2026.
3. Lifecycle headers on responses. The Deprecation header was published as RFC 9745 in March 2025 (the same RFC defines rel=deprecation links), and Sunset is RFC 8594 (2019). A well-behaved vendor tells you per-response when a resource is deprecated and when it will stop. Measured on 2026-10-08 with one unauthenticated request each, api.github.com, api.stripe.com, api.openai.com and api.twilio.com sent neither. One request to a current endpoint shows little, though: GitHub documents that it sends both headers for an API version that is closing down, and a deprecated GitHub endpoint requested the same day carried both. Adoption is partial and per resource, so log the headers and alert on their appearance, but do not make them a primary channel yet.
4. Monitoring for production errors. The last line of defense, and it sees the change only after it ships: structured logging on integration call sites (vendor error code families, enum-parse failures, null-field reads), and anomaly alerts on the metrics. This channel sees what no contract can: behavior changes without a spec change, rate-limit shifts, undocumented error text. It is also the most expensive channel, because the unit is an incident.
5. Semver-checked SDKs and pinned versions. Strictly, this is not detection but insulation: pin Stripe-Version, send X-GitHub-Api-Version, pin a client library version. It converts constant drip into scheduled decisions, which makes every other channel more useful, but the pin itself is not a sensor and the pin ages (GitHub supports the previous API version for at least 24 months after a new one is released).
The weekly pipeline over the top two channels
Channels 1 and 2 are the two that can run without a human deciding to read something, so they become the pipeline. The spec channel is the apibreak check shape: declare the endpoints you call per vendor, pin a baseline, run it as a scheduled GitHub Actions job (fail-on: breaking), and let the report name the two revisions compared. On the example manifest, ten endpoints checked, that is 7 breaking, 1 deprecation, 9 advisory, 19 not compared, 210 additive counted but never listed, with the filter reducing the vendors' hundreds of changes to the ones that can hurt you. The changelog channel is a scheduled fetch of the changelog page or feed per vendor, diffed against last week's copy, with a human triaging the entries that the spec diff did not already explain. Both jobs run in one repository, one per vendor shape, and the steady-state signal is small by design: after a fix-and-repin cycle, the next unexpected row is genuinely unexpected.
name: apibreak
on:
schedule: [{ cron: '0 13 * * 1' }]
workflow_dispatch:
jobs:
apibreak:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: Skyblueballykid/apibreak@v1
with:
manifest: apibreak.json
fail-on: breakingWhat a week looks like when it works (illustrative)
- Monday 13:00 UTC: the vendor check runs. A row like the one in apibreak's example report appears: POST /v1/checkout/sessions, ui_mode no longer accepts "custom", "embedded" or "hosted". A work item is created; production is unaffected while your pinned Stripe-Version predates the change.
- A newly deprecated operation surfaces on a GitHub endpoint you call, months ahead of removal, because the spec is the notice.
- A changelog fetch catches a behavior change the spec cannot represent, and a human attaches it to the endpoint list in the manifest.
- Production monitoring stays the last line of defense, but it stops being the first.
The ranking is the whole strategy. Spec diffs first (complete, machine-readable, dated), changelogs second (intent, coverage where specs are quiet), headers third (logged, rarely present), production errors last (real, expensive, residual), pins throughout (insulation, not detection). A team with the pipeline green and the triage small has converted vendor change from an incident into a queue.
Check the endpoints you actually call
APIBreak reads the endpoints you declare, compares the vendor's own OpenAPI specification against the revision you pinned, and fails the build only when a change touches one of yours. It runs in your CI with no credentials and nothing uploaded, and the check is free.
Install the check · read a real report · what changed at each vendor