How Do You Ask Your Business Database a Question and Get a Real-Time Answer?

A colleague pings you: "How many students are currently in the trial-lesson stage?" or "What's the latest contract we have with that customer in Rotterdam?" You know the answer is in the system somewhere. But getting it means opening a layout, setting a find, maybe cross-referencing two or three related tables, and then typing the result back into an email or a Slack message.

Multiply that by every question your team asks in a week, and you're looking at hours of manual lookup work — work a computer should be able to finish in seconds. This article walks through why the obvious fix (bolt on an AI chatbot) often fails, and what actually works instead: connecting an LLM directly to your live data instead of a static copy of it.

Why does adding an AI chatbot to your data usually disappoint?

The default approach to adding AI to a business system is Retrieval-Augmented Generation, or RAG: export your data, chop it into chunks, embed those chunks into a vector database, and let an LLM search across them.

That works well for static, unstructured content — PDFs, manuals, scanned contracts, policy documents. It works badly for live operational data. A vector index is a snapshot, frozen at the moment it was built. The instant a contract status changes, an invoice gets paid, or a student moves from "trial" to "enrolled," that snapshot is wrong — and it stays wrong until someone re-runs the ingestion job, which in most setups happens overnight at best.

For a question like "who currently has an unpaid invoice," a snapshot from last night isn't good enough. You need an answer based on what the database says right now, at the second the question is asked.

What's the alternative to indexing your data for AI search?

Instead of copying data into a separate search index, you can give an LLM a direct, real-time line into your database's own API. When a question comes in, the system works through five steps:

1. Reads the question to figure out what's actually being asked — a person, a status, a count, a date range.
2. Queries the live database at that exact moment, using its native API rather than a cached copy.
3. Gathers related records — for a person lookup, that might mean pulling together contracts, payment terms, invoices, notes, and open tickets in one pass.
4. Assembles the results into context for the LLM, translating internal codes and field names into plain language a person would actually use.
5. Returns a natural-language answer, ready to read or paste straight into an email.

Because step 2 hits the live system every single time, the answer is always current. There's no re-ingestion, no stale index, no "as of last night's sync" caveat attached to the answer.

What does this look like with an actual question?

Someone types: "How many contracts currently have status 'In Progress'?"

The system:

• Recognizes this as a count question tied to a specific status field
• Queries the contracts table live, filtered on that exact status
• Counts the matching records
• Returns a plain-language answer: "There are 12 contracts currently in progress."

No report needed to exist in advance. No dashboard needed to already cover that exact slice of data. The question itself defines the query — which is exactly what makes this useful for the long tail of one-off questions that never justified building a dedicated report.

Can it handle a real back-and-forth conversation, or just one-off lookups?

A well-built version of this pattern also remembers context between questions, so a user can follow a natural thread instead of repeating themselves:

"Show me the Jansen account." → "What's their contract status?" → "Do they have any unpaid invoices?"

The system recognizes short follow-up questions and pronouns like "their" and "they," and reuses the person or record from the previous question automatically. Without this, every follow-up would need to restate the full customer name — which is exactly the kind of friction that makes people give up on a chatbot after the second question.

Is your database actually a good fit for this approach?

Run through this checklist before investing in either RAG or a live-query approach:

• Your data changes frequently — statuses, payments, or records get updated daily or more
• Your database exposes an API, either natively or through a connector
• People regularly ask the same handful of question types — counts, lookups, filters, follow-ups
• You want answers grounded in current data, not a periodic export
• You'd rather not maintain a separate vector database and re-ingestion pipeline

If most of these are true, a direct-to-live-data approach will serve you better than a RAG-based one. If instead your "data" is mostly a folder of manuals and PDFs that rarely change, RAG is still the right tool — the two approaches solve different problems.

Frequently asked questions

Does this modify my data? No. This pattern is read-only — it queries your existing system through its API and never writes back, so your system of record stays untouched.

Is my data sent to an external AI provider? Only the specific records retrieved for a given question, plus the question itself, are sent to the LLM. If that's a concern, a locally hosted model can keep everything on-premises.

Does my data need to be restructured first? No. The system reads directly from your existing tables and fields — it works with the structure you already have, not a redesigned copy of it.

What if a name or term is misspelled in the question? Good implementations include partial-matching and fallback logic, so a near-miss on a name or customer ID still finds the right record.

Is this the same as a chatbot bolted onto a search index? No — that's the RAG approach. This pattern skips the index entirely and queries the source system directly, which is what keeps answers current.

Does this replace my existing reports and dashboards? No. Reports are still the right tool for questions you ask repeatedly and want visualized. This pattern covers the one-off questions that never justified building a dedicated report in the first place.

Where does this fit in your broader data strategy?

Real-time question answering isn't a replacement for reporting or analytics — it's a complement to them. Reports and dashboards are for the questions you already know you'll ask again and again. This pattern is for the long tail of daily one-off questions that pop up in Slack, in a meeting, or in a phone call with a customer — the ones nobody ever built a report for because there wasn't time or reason to.

Before building anything, map out the handful of question types your team actually asks day to day — counts, lookups, status checks, follow-ups on a specific record. That list becomes the real starting point: it defines what the system needs to handle well before it needs to handle everything.

If this sounds like a good fit for how your team works but you're not sure where your own database — FileMaker, an ERP, or a custom system — stands on API access or data structure, that's exactly the kind of question worth mapping out with a technical partner first. Loggix builds this kind of real-time AI connection directly into existing FileMaker systems and other business databases, and can also help wire it up across multiple systems through API integrations where the answer to one question lives in more than one place.