Contract testing vs schema diffing: Pact or an OpenAPI diff?

Contract testing and schema diffing both promise to catch breaking API changes before production, and both can fail a CI job. They look at different evidence. A contract test, in the Pact sense, runs requests against the real provider and checks the answers. A schema diff compares two versions of an OpenAPI document and never calls the API. This article explains what each one actually checks, where each is blind, and how to choose based on who owns the consumer and the provider. Pact details come from docs.pact.io, read on 31 October 2026.

How Pact contract testing works

Per Pact's own description, the consumer's tests send real requests to a mock provider created by the Pact framework. Every interaction the tests exercise is written to a pact file. Later, provider verification sends each recorded request to the real provider and compares the actual response with the minimal expected response. If the provider has changed in a way that breaks a recorded interaction, verification fails.

Matching is deliberately loose on responses. Pact's matching guide describes type matching (the value must be present and of the expected type), eachLike for arrays (each element has the expected shape, and there is at least one), and regex matching for strings. Unexpected fields in a JSON response body are ignored during verification, which the guide calls perfectly OK. Requests are matched strictly. The result: a provider can add fields freely, and only the parts of the response some consumer actually recorded are protected.

How schema diffing works

A schema diff takes two versions of an OpenAPI document, usually the base branch and the pull request, and classifies each difference: an operation removed, a response field removed, a parameter made required, a type changed. Tools such as oasdiff, OpenAPITools openapi-diff, pb33f openapi-changes and apibreak diff do this; the comparison of OpenAPI diff tools covers their differences. The diff sees every documented change, including ones no consumer test exercises. It sees nothing the document does not say. If the server stops matching its spec without a spec change, the diff stays green.

That second gap is real in practice. Running apibreak docs over listmonk's API docs against listmonk's own published spec found 14 disagreements, and checked against listmonk's router, most of them were the spec lagging the server rather than the docs being wrong. A diff gate on a spec like that faithfully protects a document that has already drifted from the code.

What each one catches

• A field removed from a response that a consumer reads: Pact catches it if a recorded interaction includes the field; the diff catches it if the spec is updated.
• A field removed that no consumer test covers: Pact does not see it; the diff does.
• The server changes behaviour but the spec is not updated: Pact catches it for recorded interactions; the diff does not.
• A provider adds a response field: Pact ignores it by design; most diff tools report it as non-breaking.
• A provider adds a value to a response enum: Pact passes if the consumer used a type matcher; diff tools disagree, some flag it as breaking by default and some as advisory.

Where Pact does not fit: public and third-party APIs

Pact's documentation is direct about this. Its page on what Pact is good for lists as its first good fit the case where you control the development of both the consumer and the provider, and lists among the poor fits: public APIs whose consumers cannot be individually identified, APIs where the team maintaining the other side will not also use Pact, and situations where you cannot control the data used to generate the provider's responses. Provider verification needs the provider team to run your pact against their service, and a public API vendor does not run its customers' pacts.

There is a middle path for providers that publish an OpenAPI spec. PactFlow, SmartBear's commercial Pact Broker, offers bi-directional contract testing: a static comparison of the consumer's pact against the provider's OpenAPI document. Its guide says it is not intended to replace Pact testing, rates it as suited to unknown consumers and third-party APIs, and lists weaker guarantees as the trade-off. It is a schema check driven by what your consumer uses, which makes it a close relative of a filtered schema diff.

Choosing by ownership

1. You own both sides (internal services): Pact. Consumer tests record what each service needs, and provider verification runs in the provider's pipeline before it deploys.
2. You own the provider and the consumers are the public: a schema diff on every pull request, because you cannot enumerate your consumers. Keep the spec honest by generating it from code or checking it against your docs.
3. You own the consumer and a vendor owns the provider: neither of the above runs where you need it. Diff the vendor's published spec between the revision you built against and the current one, filtered to the endpoints you call, on a schedule. That is the job apibreak check does for GitHub and Stripe.
4. In all three cases, any mock or fixture of the other side is a copy of a contract. Record which spec revision it came from and regenerate it when the contract moves.

Running both in one pipeline

For an internal service with a published spec, the two checks can run on the same pull request without overlapping much. The spec diff covers the contract as documented:

name: apibreak diff
on:
  pull_request:
    paths: [openapi.yaml]
jobs:
  diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
        with:
          fetch-depth: 0
      - run: npx apibreak diff --base-ref origin/${{ github.base_ref }} openapi.yaml

Provider verification covers the contract as used, against the running service. When they disagree, the disagreement is information. A breaking spec change that Pact verification passes means no recorded consumer uses that part of the API, or your pacts have a coverage gap; check which before you merge. A Pact failure with a clean spec diff means the server changed behaviour without a spec change, and the spec needs fixing as well as the code.

Neither tool replaces the other. The diff sees every documented change and nothing undocumented. Pact sees real behaviour, but only for interactions some consumer recorded, and only where the provider team runs it. Pick by who owns each side, and add the scheduled vendor diff for the APIs nobody on your team owns.