The Coal Face · 2026-07-08

Is AI-assisted software secretly making business systems more fragile?

by Paul Bappoo

Something Feels Off — But You Can't Quite Name It

You didn't build the software yourself. You described what you needed to an AI tool, clicked through a few prompts, maybe hired someone who works the same way, and ended up with something that mostly works. Orders go through. Invoices get sent. Customers can log in.

But lately there's a low-level unease. A billing run that doubled some charges and missed others. A customer record that just... disappeared. An insurance policy that got flagged as cancelled when it wasn't. Nothing catastrophic, but enough that you've started checking things you used to trust.

And you've started wondering: is this just my situation, or is something broader happening? Are all these AI-built systems quietly more fragile than what came before?

That instinct is right. Something is happening. And it has a pretty clear explanation — one that doesn't require you to understand code at all.

Read the rest — free

Free membership unlocks the full article and the PDF download — plus the rest of the series as it lands.

Building something with AI?

Take the free AI Mandate Self-Score at bappoo.com/?selfscore. Or email paul@bappoo.com for a personal review of what you've built — or what you're thinking of building — before it gets expensive.

These articles are general information and opinion, not professional advice. Verify anything important for your own situation — full disclaimer.