Better Defensive Testing Builds Better Software

Ask your team what gets checked before a release goes out and you will get a list of things that are supposed to work. 

The login works, the form submits, the integration passes data along, the dashboard shows the right number.

That list covers the normal path. It tells you the software does what it was asked to do, and almost nothing about what happens when a permission gets tested, an unexpected input arrives, or a dependency underneath you turns out to carry a flaw. 

Those are the questions your clients' security reviewers ask, and the cost of answering them climbs the longer you wait.

‍

What does defensive testing look for in software?

Defensive testing asks what your software does when it gets used in ways it was never designed for, which is a different question from whether the feature works.

The questions a defensive review is trying to answer:

• Can a user reach data their role should not see
• Can a form accept input the developer did not plan for
• Can an integration pass information to a system that should not have it
• Can an admin tool be used by the wrong person
• Can an error message reveal how the system is built
• Can a workflow skip a review step the business depends on
• Can sensitive data move without leaving an audit trail
• Can a dependency underneath the application carry a known flaw

In July 2024 CrowdStrike pushed a content update that crashed 8.5 million Windows machines. Delta canceled 7,000 flights and sued in October, putting its losses above $500 million. 

The update passed whatever check stood in front of it, and this was a security company with a mature release process, so the gap was in what that process thought to look at.

‍

How does Big Pixel test software for security weaknesses?

We are cleared through Anthropic's Cyber Verification Program for cybersecurity capability it keeps closed to the public, and we point it at our own builds before a client's software has users.

The capability is worth describing plainly, because the generation of models behind it finds zero-day vulnerabilities and writes working exploits for them without a person driving. 

Anthropic restricts access for that reason and vets the practitioners who get it. We went and got vetted so the first serious attempt on a client's software would be ours.

On an engagement that means pointing it at the authentication path, the permission boundaries, the integrations carrying data in and out, and anything that moves patient or claims data. 

Alongside that we review the decisions that determine how much damage a weakness can do in the first place:

• How data moves between systems and where it comes to rest
• Which roles can see which fields
• What happens when authentication fails
• Who holds administrative access and what that access can reach
• What error messages tell a user about the system behind them
• What gets logged, and whether the log would answer an auditor

The timing is the whole argument. A weak permission boundary found while the code is still ours costs a developer an afternoon. 

The same weakness found by a researcher a year in costs an emergency release, a disclosure window, and a call to your compliance officer, with your name on the notification. 

We also have the hours to do this properly, because we get 30 to 40 percent faster delivery out of AI against an industry average gain of about 20 percent, and the time that gives back goes into the work that gets cut first when a deadline tightens.

We believe that business is built on transparency and trust, and that good software is built the same way, which on security work means you see what we tried to break and what we changed before the release went out.

‍

Who reviews what an AI security tool finds?

A finding is a claim about your software until a developer who knows the system confirms it, sizes it, and works out what a fix would break.

In July 2025 an AI agent at Replit deleted a production database during an explicit code freeze, then generated fake test results and fabricated records covering what it had done. 

Replit's chief executive called it "a catastrophic error of judgement." The agent was fast and it was wrong, and the only thing that catches that combination is a person who has to approve what reaches production.

So every finding ours produces goes through a developer who answers:

• Is the finding real in this environment
• How severe is it against this business, rather than in general
• Which system and which workflow does it actually affect
• What is the safest fix available
• What else could that fix break
• How will the fix get tested before it goes out
• Does the architecture need to change, or only the code

For Witherspoon Rose Culture we built Quickrose, a custom operations platform running their orders, inventory, spray mix tracking, and month-end finances, with a Shopify integration, automated statement generation and tax reporting, and an Android app for field staff. 

The month-end workflow is the one we would test hardest, because it runs without a person watching it and it produces the numbers their tax reporting depends on.

‍

Why does this matter most in healthcare and insurance?

A member portal in a health plan is usually reading claims history from one system and holding uploaded documents in another, which means a permission mistake inside the portal reaches well past the portal.

On November 18, 2025, Cloudflare made a database permissions change that doubled a bot-management configuration file past the size its proxy layer expected, and core traffic returned errors for about five hours and forty-six minutes. The permissions change was correct on its own terms. 

What it broke sat two systems away from where the work happened, and connected healthcare environments are built the same way.

Health-ISAC flagged the new generation of offensive AI capability as a sector-level concern for that reason. The software holding member and patient data is connected to more things than its own screens suggest, and the organizations running it answer to regulators who ask for dates rather than intentions. 

A weakness in that environment arrives as a notification obligation before it ever becomes an engineering ticket. On a healthcare or insurance build we start the defensive review at those connection points for that reason, rather than at the screens a user sees.

‍

What should you expect from a software partner on security?

A software partner should be able to tell you what they tried to break in your system and what happened when they tried.

Ask for these on your next engagement:

1. Security inside the design phase. Roles, permissions, and data boundaries decided with the architecture rather than added once the screens exist.
2. Testing past the happy path. Edge cases, bad input, and the workflows that would hurt most if they failed.
3. A defensive review of the code. A developer looking for weak points on purpose, with a written record of what they looked at.
4. A system you can patch. Tests, a documented deployment, and current dependencies, so a finding does not turn into a project.
5. Documentation that outlasts one person. What depends on what, written down outside one developer's memory.
6. Care after launch. Monitoring and maintenance, because the risk picture changes after you go live.

The fair objection is that this reads like a longer timeline and a bigger number. Some of it is, and the comparison that matters is against the alternative, where the same weakness surfaces in a client security review or a breach notification and gets fixed under a disclosure clock by a team that now has an audience. 

The work also scopes down well, since a review aimed at authentication, permissions, and whatever moves your most sensitive data covers most of the real exposure without touching every screen.

Ask your current partner what they tried to break on your last release and what they found. 

If the answer is that everything passed, the useful follow-up is what exactly was tested.