How to prepare for failure of an external API

Your order process depends on a shipping carrier's API to calculate rates. Your invoicing depends on a bank or payment provider's API to confirm transactions. Your CRM depends on an email service's API to send confirmations. And one Tuesday morning, one of those APIs is down — or worse, it's up but returning malformed data — and nobody notices until a customer calls asking why their order never shipped.

This is not a hypothetical. Third-party APIs fail: they go down for maintenance without enough warning, they get rate-limited during your busiest hour, they change a response format without telling integration partners, or they simply have an outage. If your business software calls out to any external service — a webshop platform, an accounting package like Exact Online, a shipping API, a payment gateway, an AI service — you need a plan for what happens when that call doesn't come back the way you expect. This article walks through how to actually build that plan, not just acknowledge the risk.

Why does API failure matter more than it used to?

Ten years ago, most business software was a closed system: one database, one application, maybe a nightly export. Today, a typical FileMaker or ERP setup for a mid-sized company might talk to five, ten, or more external services in a single business process — a customer places an order, and behind the scenes that single click triggers calls to a payment provider, a shipping carrier, an accounting system, and a marketing platform.

Each one of those is a dependency you don't control. You don't control their uptime, their release schedule, or their support response time. A single failed call in the middle of that chain can leave you with an order that's paid but not booked in accounting, or a shipment that's booked but never invoiced — and untangling that manually, order by order, is exactly the kind of hidden cost that never shows up on a project plan but eats a support team's week.

What actually goes wrong when an API fails?

It helps to be specific, because "the API failed" covers several very different failure modes, and each needs a different fix:

• Hard downtime — the endpoint returns a connection timeout or a 5xx error. Easy to detect, hard to predict when it will resolve.
• Rate limiting — the API works, but starts rejecting requests once you exceed a quota (common with shipping and marketing APIs during peak season).
• Authentication expiry — an API key, OAuth token, or certificate expires silently, and every call starts failing with a 401 until someone notices.
• Silent schema changes — the vendor changes a field name, a data type, or removes a field from their response. The call succeeds, but your system either crashes on unexpected data or, worse, processes it wrong without erroring at all.
• Partial failure — the request succeeds on the vendor's side (a payment is actually charged) but the confirmation response never reaches you, so your system thinks it failed and retries — now you've charged the customer twice.

That last one is the scenario that causes the most damage in practice, because it looks like nothing went wrong until a customer or your finance team spots it days later.

How do you find out an API has failed before your customer does?

You cannot prepare for a failure you don't know is happening. The single highest-leverage investment here is monitoring and alerting, not fancier retry code.

1. Log every outbound API call — request, response, status code, and timestamp — somewhere queryable, not just in a scrolling log file nobody reads.
2. Set an explicit timeout on every call. A call with no timeout doesn't fail, it hangs — and a hanging FileMaker script or scheduled server process can block everything behind it.
3. Alert on error rate, not just outright downtime. A carrier API returning 10% errors is a warning sign long before it returns 100% errors.
4. Watch for silent schema drift by validating the response shape, not just the status code — a 200 response with a missing field should still trigger an alert.
5. Track token and certificate expiry dates in a calendar or automated check, not in someone's memory.

What should your system do the moment a call fails?

Design the failure path with the same care you design the happy path. In practice this means:

• Retry with backoff, not immediately and not forever. Retrying an already-overloaded API instantly just makes things worse; retrying every 30 seconds for three attempts, then giving up and queuing, is usually the right shape.
• Queue what you can't process immediately. If the shipping API is down, don't lose the shipment request — write it to a local queue table so it can be replayed automatically once the service is back.
• Fail loudly to a human, not silently to a log. A queued record that nobody looks at for two weeks is not resilience, it's a slower failure.
• Make retries idempotent. Before you retry a payment or an order-creation call, make sure retrying it twice can't create it twice — use idempotency keys or check-before-create logic.
• Have a manual fallback for critical paths. If the carrier API is down for shipping labels, can staff generate a label manually for a few hours without the whole order process grinding to a halt?

Should you build your own retry logic or use a middleware / connector layer?

For a single integration, hand-rolled retry logic inside a script is fine. Once you're maintaining several external connections — accounting, shipping, payments, email, AI services — that logic tends to get duplicated, inconsistently applied, and hard to audit.

A dedicated integration or API connector layer sitting between your core system (e.g. FileMaker) and the outside world gives you one place to standardize logging, retries, timeouts, and alerting, instead of five slightly different implementations scattered across scripts. It also isolates the blast radius: if one connector needs to be rebuilt because a vendor changed their API, you're not touching the core application logic that runs the rest of the business.

This is part of the broader discipline covered in how to secure and maintain business-critical software — treating your integrations as maintained infrastructure, not one-time setup work.

How do you test your failure handling before it's tested on you?

Most teams only discover their failure handling doesn't work during an actual incident. Test it deliberately instead:

1. Point a test call at an invalid URL and confirm your system times out and alerts, instead of hanging.
2. Temporarily revoke or expire a test API key and confirm your team gets notified before customers do.
3. Simulate a malformed response (missing field, wrong data type) and confirm the system flags it instead of silently processing bad data.
4. Run a load test that intentionally hits a vendor's documented rate limit and confirm your queueing kicks in.
5. Review, once a quarter, every external dependency your system has — is it still actively used, still supported, still on a maintained version?

What should a written contingency plan cover?

Even a one-page document per critical integration is far better than nothing. For each external API your business depends on, write down:

• What business process breaks if this API is down, and for how long is that tolerable?
• Who gets alerted, and how (email, Slack, SMS)?
• Is there a manual workaround, and does staff know it exists?
• Where are failed/queued transactions stored, and who reviews them?
• Who is the point of contact at the vendor, and what's their support SLA?

FAQ

How long should we tolerate an API being down before escalating? It depends on the process, not a fixed number — a marketing email API can usually wait hours, a payment API usually cannot wait minutes. Define this per integration in your contingency plan rather than using one blanket rule.

Can AI tools help detect these failures earlier? Yes — anomaly detection on API response logs (unusual error rates, unusual response shapes, unusual latency) can flag a problem before it becomes visible to customers, and can be added on top of existing log data without redesigning the integration itself.

Is it worth paying for an API's premium/enterprise tier just for better uptime guarantees? For genuinely business-critical calls (payments, core accounting sync), often yes — the cost of a guaranteed SLA is usually far lower than the cost of one bad outage during a peak sales period.

What's the single most common mistake companies make here? Assuming a working integration will keep working forever. Vendors change APIs, deprecate versions, and rotate credentials — an integration that isn't monitored is an integration that's quietly aging toward failure.

Checklist: is your business ready for an API failure?

• Every outbound API call has an explicit timeout
• Every API call is logged with status and timestamp
• Alerting exists for error rate, not just total downtime
• Failed requests are queued and automatically retried, not lost
• Retries are idempotent
• Token/certificate expiry dates are tracked proactively
• A manual fallback exists for each business-critical integration
• A one-page contingency plan exists per critical API
• Failure handling has been tested deliberately, not just assumed

If your business runs on FileMaker with connections into accounting, shipping, payment, or AI services, it's worth mapping out exactly which of those dependencies would hurt most if they failed tomorrow. Loggix can help build that resilience directly into a custom FileMaker solution or a dedicated API connector layer, add monitoring and AI-based anomaly detection to existing integrations, or simply sit down with your team for a focused consultancy session to map every critical dependency and close the gaps before they become an incident.