What should happen when an external service is unavailable?
A practical guide to designing integrations that survive outages — with retry logic, queues, and fallback rules for FileMaker, ERP, and API connections.
It's 9:14 AM and your webshop just stopped syncing orders to your ERP because the shipping carrier's API returned a timeout. Nobody notices for two hours, until a customer calls asking why their order confirmation never arrived. This scenario plays out in some form at almost every company running integrated systems — a payment gateway, an accounting package, a CRM, or a carrier API goes down for ten minutes, and the ripple effect lasts all day.
Most businesses only think about this after it has already happened once. This article walks through what should actually happen — technically and organizationally — the moment an external service becomes unreachable, so the failure stays small instead of becoming a crisis.
Why does this keep happening, even with "reliable" APIs?
Every external service you depend on — a payment provider, a webshop platform, a logistics API, an accounting system like Exact Online or Twinfield — will go down eventually. Not because the vendor is careless, but because 100% uptime doesn't exist. Add your own network hiccups, expired API tokens, rate limits, and scheduled maintenance windows, and you're looking at dozens of small unavailability events per year across a typical integration landscape.
The mistake isn't that the outage happens. The mistake is building an integration that assumes it never will.
What actually breaks when nobody planned for downtime?
Here's a concrete example we've seen repeatedly in FileMaker-based systems: a script calls an external API synchronously — the user clicks "Send Order," FileMaker waits for a response, and if the API doesn't answer within the timeout, the script either hangs, throws an unhandled error, or silently fails.
The consequences stack up fast:
- The order was created in FileMaker but never reached the ERP — no error shown, so nobody follows up.
- The user, frustrated by the hang, clicks "Send" again — creating a duplicate order once the service comes back.
- A record gets marked as "synced" even though the call failed, because the flag was set before the response was checked.
- The failure only surfaces days later, when someone reconciles order counts between two systems by hand.
None of this is caused by the outage itself. It's caused by an integration with no defined behavior for the moment things go wrong.
What should happen the moment a call fails?
There's a predictable sequence a well-designed integration should follow. Think of it as a decision tree, not a single fix:
- Detect the failure correctly. Distinguish between "no response" (timeout), "service says no" (4xx/5xx error), and "service accepted it but slowly" (202 Accepted, still processing). Each needs a different response.
- Never let the failure block the user. The person entering the order should not be staring at a frozen screen. Queue the request and let them continue working.
- Retry — but with limits and backoff. A single retry immediately after a timeout often hits the same overloaded server. Wait, then retry with increasing delays (e.g. 30 seconds, 2 minutes, 10 minutes), and cap the number of attempts.
- Store the failed request somewhere durable. Not in a variable that disappears when FileMaker closes — in a table, a log, or a queue that survives a restart.
- Flag it for a human, eventually. If retries are exhausted, someone needs to know — not by discovering it themselves three days later.
- Never silently mark something as done that isn't done. This is the single most common and most damaging shortcut we see in the field.
How do you actually build a retry queue that works?
In practice, this means adding a small piece of infrastructure that most integrations skip: a queue table.
A typical design looks like this:
- A queue table with fields for: payload, target endpoint, status (pending / retrying / failed / sent), attempt count, next retry timestamp, and last error message.
- A scheduled script (running every 1–5 minutes via FileMaker Server's schedule, or a scripting engine like FMBetterForms if the sync runs from a web layer) that picks up pending and due-for-retry records and attempts them again.
- An exponential backoff rule, so retry #1 waits 30 seconds, retry #2 waits 2 minutes, retry #3 waits 10 minutes, and after that it's marked failed instead of retried forever.
- A visible dashboard or alert — even something as simple as a layout listing all records with status "failed," checked once a day, or better, an automated email/Slack alert the moment a record crosses a failure threshold.
This pattern applies whether the external service is a payment API, a shipping carrier, an accounting package, or an AI service called from inside FileMaker (for example, a Klai-powered lookup or enrichment step). The AI call is no different from any other API call in this respect — if it's unreachable, the workflow needs a defined fallback, not a crash.
What should the end user actually see?
This is where a lot of integrations quietly fail their own users. The technical retry logic can be flawless, and the experience still feels broken if the interface doesn't communicate honestly.
Good practice:
- Show a clear, non-alarming status: "Order saved. Sending to warehouse system — this may take a few minutes." Not a spinning wheel with no explanation, and not a scary red error for something that's actually just queued.
- Never say "Success" until it's actually true. If the record is queued, say "Queued," not "Sent."
- Give the user (or an admin) a way to see the current state of any pending sync without digging into logs.
- If a failure is permanent (e.g. invalid data was rejected, not a timeout), say so specifically — "Rejected: postal code missing" is actionable; "Sync error" is not.
Should every failure trigger a retry?
No — and this is a distinction that's often missed. Not all errors are equal:
| Type of failure | What it means | What should happen |
|---|---|---|
| Timeout / connection refused | Service is temporarily down | Retry with backoff |
| 429 Too Many Requests | You're rate-limited | Retry, but wait longer |
| 401 Unauthorized | Token expired or revoked | Don't retry blindly — refresh the token first, then retry once |
| 400 Bad Request | Your data is invalid | Don't retry — it will fail again. Flag for correction |
| 500 Internal Server Error | Something broke on their end | Retry a few times, then escalate |
Retrying a bad request endlessly just fills your queue with noise and delays the alert that something needs a human's attention.
How do you avoid duplicate records once the service comes back?
This is the classic follow-on problem: the API was actually down for 90 seconds, the retry succeeded, but a user had already manually re-entered the order in the meantime out of frustration. Now there are two.
The fix is designing for idempotency from the start:
- Generate a unique request ID (a UUID) on the FileMaker side before the first attempt, and send it with every retry of that same request.
- Configure or check whether the receiving system can deduplicate based on that ID — many modern APIs support an "idempotency key" header for exactly this reason.
- If the external system doesn't support that, build a local check: "has this order number already been confirmed as sent?" before allowing a manual resend.
What does this cost to build — and is it worth it?
A basic retry queue with backoff and a status dashboard is typically a few days of development work for a mid-sized FileMaker/ERP integration — small compared to the cost of one bad afternoon of duplicate orders, missed shipments, or a customer escalation. The real investment isn't the code; it's deciding, in advance, what "failure" should look like for each integration point, instead of discovering it live in production.
This kind of resilience planning is exactly what we cover in more depth in our guide on how to design reliable integrations between business systems — this article zooms in on one specific piece of that puzzle: the moment things go wrong.
A quick checklist before you go live
- Every external call has a defined timeout — not the default, a deliberate one.
- Failed calls are stored in a durable queue, not a temporary variable.
- Retries use backoff and a maximum attempt count.
- Permanent errors (bad data, auth) are distinguished from temporary ones (timeout, 5xx).
- Every request carries a unique ID to prevent duplicates on retry.
- There's a visible status for pending/failed items — for users and for admins.
- Someone gets alerted automatically when retries run out, not just when a customer complains.
FAQ
Does this apply to AI integrations too? Yes. An AI service called from inside a FileMaker workflow — for enrichment, classification, or lookups via a tool like Klai — is still just an API call over the network. It needs the same timeout, retry, and fallback logic as any accounting or shipping integration; the only difference is deciding what the workflow should do if the AI step is skipped (e.g. proceed without the suggestion, rather than blocking the whole process).
Can FileMaker handle retry queues natively? Yes, using a standard table for the queue and FileMaker Server's schedule feature to run a periodic script. For web-facing workflows built with something like FMBetterForms, the same queue table can be checked from both the web layer and the native client.
How long should we retry before giving up? There's no universal number, but a common pattern is 3–5 attempts over roughly 15–30 minutes for time-sensitive processes (orders, payments), and longer windows (hours) for background sync tasks like updating stock levels.
What if the external service has no status page or webhook to tell us it's back? Poll it. A lightweight "health check" call every few minutes, separate from the actual data transfer, can tell your queue when it's safe to resume full retries instead of hammering a still-recovering endpoint.
If your systems currently go quiet — or worse, go wrong — the moment one external service hiccups, that's usually a sign the integration was built for the happy path only. Loggix helps map out where those failure points sit across your FileMaker, ERP, and API connections, and builds the retry logic, queues, and fallback rules — including AI-driven steps — so a ten-minute outage stays a ten-minute outage instead of a week of manual cleanup.