Can an AI agent actually find security holes in your business software?

Your ERP system, your customer portal, your custom FileMaker solution, that internal API you connected to your webshop last year — someone in your company has probably said "we should really get that tested for security" at some point. And then it didn't happen, because a proper penetration test from a specialized firm costs thousands of euros, takes weeks to schedule, and produces a PDF that's out of date the moment you ship your next update.

Now there's a new category of tool trying to close that gap: autonomous AI security agents. One of the more talked-about open-source examples is Strix, which bills itself as an "AI agents that hack your apps like a pro hacker" — it doesn't just scan for known patterns, it actively explores your running application, tries to exploit what it finds, and reports back with a proof of concept.

This article answers the question that actually matters for your business: is this something you can rely on for your custom-built software, or is it a developer toy that gives a false sense of safety?

What does a tool like Strix actually do differently from a normal scanner?

Most security scanning tools you've encountered — whether it's a plugin in your CI pipeline or an annual "vulnerability scan" report from your hosting provider — work by pattern matching. They check your code or your running app against a list of known signatures: outdated library versions, missing headers, common misconfigurations. Useful, but shallow. They tell you "this dependency has a known CVE," not "I logged in as a normal user, then walked through your API and ended up reading another customer's invoices."

Agentic tools like Strix work differently. They:

• Spin up your application in a sandboxed environment (or point at a running instance).
• Use browser and terminal tools to actually use the app the way a real attacker would — clicking through flows, sending crafted requests, inspecting responses.
• Chain findings together. A minor information leak in one endpoint might, combined with a weak session check somewhere else, become a full account takeover — a human pentester would spot that chain, and this is what agentic tools are specifically designed to attempt.
• Validate exploitability instead of just flagging a theoretical issue. Strix's own documentation emphasizes producing evidence-based findings with reproducible proof-of-concept steps, not a generic severity score.
• Generate a structured report mapped to common frameworks (OWASP Top 10 categories, business logic flaws, authentication and authorization issues, injection vulnerabilities, and so on).

That's a meaningfully different capability than "we ran a scanner and got a list of CVEs." It's closer — not identical, but closer — to what a junior-to-mid-level human pentester does in the first few hours of an engagement: reconnaissance, exploration, exploitation attempts, and reporting.

Where does this genuinely help a business running custom software?

If you run a custom-built system — a bespoke FileMaker solution handling orders and customer data, an internal ERP module, or an API layer connecting your webshop to your warehouse system — you're in a specific bind: this software is unique to you, so there's no off-the-shelf security scanner tuned to it, and it's usually too small a project to justify a full external pentest every quarter.

That's exactly the gap where an autonomous agent earns its keep:

1. Pre-release checks on new features. Before you push a new module live — say, a customer self-service portal you just bolted onto your FileMaker back office — running an agent against a staging copy can catch the obvious stuff: an endpoint that doesn't check permissions properly, a form field that accepts more than it should, a session that doesn't expire.
2. Regression testing after integrations. Every time you connect a new system via an API — your CRM to your accounting package, your webshop to your inventory database — you introduce a new attack surface. An agent can be pointed at that integration specifically, rather than waiting for the annual audit to (maybe) notice it.
3. A second opinion between real pentests. If you do commission a proper external penetration test once a year (which you still should for anything handling sensitive data), an AI agent running monthly or after each release can catch drift in between — the new bug introduced three sprints after the last human pentest, sitting unnoticed until now.
4. Documenting what "good" looks like. Reproducible, evidence-based reports are useful not just for fixing bugs, but for showing a client, an auditor, or your own management that security testing is a routine part of your development process — not a one-time checkbox.

Where should you still be cautious?

Being honest about the limits matters more than the marketing pitch here.

• It tests what it can reach. An agent exploring your app from the outside will find what a real remote attacker could find. It's much weaker at catching issues that require deep domain knowledge of your business logic — for example, whether a specific FileMaker script step correctly re-checks a user's role before allowing a price override, something only someone who understands your actual workflow would think to test.
• False confidence is the real risk, not false positives. A clean report from an autonomous scan can lull a business owner into thinking "we're covered," when really the tool simply didn't try the one attack path that matters for your specific data model.
• It needs a safe environment. Running an exploitation-capable agent against your live production system is a bad idea — you want a staging environment that mirrors production closely enough to be meaningful, without the risk of an aggressive test knocking over a system your team is using right now.
• Findings still need a human to prioritize. A report listing fifteen findings is not the same as knowing which three actually put customer data at risk this week. That triage step — deciding what's a real business risk versus a low-severity theoretical issue — still benefits enormously from someone who knows the system and the business.
• It's a moving target. These tools are new, evolving fast, and (like any LLM-driven system) can behave inconsistently between runs. Treat results as a strong starting point for investigation, not a certified compliance artifact.

How do you actually put this to safe use, step by step?

1. Never point it at production. Stand up a staging copy of your application with realistic (but anonymized or synthetic) data.
2. Scope it clearly. Tell the agent which application, which URLs or endpoints, and which credentials/test accounts it's allowed to use — the same way you'd brief a human pentester.
3. Run it after every significant change, not just once a year — new feature, new integration, new API connector, new user role.
4. Read every finding with your own business logic in mind. Ask: could this specific flaw expose a specific customer's data, or let a specific role do something it shouldn't?
5. Fix, retest, document. Treat the report the same way you'd treat a bug list — assign owners, fix, verify the fix closes the reported exploit path, and keep the report as evidence of due diligence.
6. Still budget for a periodic human-led pentest, especially if you process payment data, health data, or anything under GDPR scrutiny. Autonomous agents complement human expertise; for now, they don't replace it for anything regulatory-grade.

Frequently asked questions

Is an AI pentesting agent enough on its own for GDPR or compliance purposes? Generally no. Most compliance frameworks and cyber-insurance policies expect evidence of a qualified, often certified, human-led assessment. Use AI agents for continuous, frequent testing between those formal assessments.

Can this replace our developers' own security awareness? No — if anything it raises the bar on what developers need to understand, because they now need to interpret and validate AI-generated findings, not just receive them.

Does this work for a custom FileMaker system, or only for standard web apps? Tools like Strix are built around exploring HTTP-based applications (web apps, APIs), so they work well against the web-facing parts of a FileMaker deployment — a WebDirect interface, a Data API layer, or a connected web app. The native FileMaker client layer itself needs a different, more manual review approach alongside this.

What's the first thing we should test with a tool like this? Any externally reachable entry point you've added recently: a new customer portal, a new webhook receiver, a new API integration. Those are the pieces least likely to have already been reviewed by a human.

The takeaway

Autonomous AI security agents like Strix are a genuinely useful addition to a modern security routine — especially for businesses running custom software that will never get the attention of a big enterprise's dedicated security team. They're not a replacement for understanding your own system, and they're not a substitute for a real pentest when the stakes are high. Used as a frequent, low-friction check between bigger reviews, they close a gap that used to simply go unaddressed for months at a time.

If you're running a custom FileMaker system, a connected ERP setup, or a web application built specifically for your business, this kind of testing is only as good as your understanding of what the findings actually mean for your data and your customers. Loggix builds and maintains exactly these kinds of tailored systems — custom FileMaker solutions, API integrations between the tools you already use, and the web applications that sit around them — and can help you think through where your real exposure is, which parts of your setup deserve the closest scrutiny, and how to fold a security check like this into your regular development process rather than treating it as a once-a-year fire drill.