The Bridge Looked Fine Too

This is the fourth post in Craft & Code, a short Friday series about what carpentry can teach us about AI, skill and the future of software. Last week I worried about where the next generation’s judgement will come from. This week, why we may not notice it is missing until it is too late.

My father built me shelves in an alcove when I was small, and I mentioned in the first post that they may still be there for eternity. The other side of that story is the one every household knows: the shelf that is not quite right. The one that sags under a row of books, or sits a degree off true so that anything round rolls gently to one end. You do not need to be a carpenter to see it. A bad joint, a door that will not close, a shelf that dips — the material tells on the maker, immediately and to everyone.

That is the comforting version of the analogy, and the one I expected to write: carpentry is honest about its failures because they are visible, while software can look polished and be rotten underneath. A wonky shelf looks wonky; bad software looks finished. It is a tidy line, and there is real truth in it.

But it is only half the truth, and the more interesting half should worry us — because the moment you go up from a shelf to a serious piece of engineering, the comfort falls away completely.

Consider two of the most admired structures of the last century.

The Tacoma Narrows Bridge was designed by one of the leading suspension-bridge engineers of his day: elegant, slender, celebrated. It opened in the summer of 1940 and tore itself apart in the wind that November, twisting like a ribbon because the design had not reckoned with how the deck would behave aerodynamically. Nobody had seen a wonky bridge; it looked magnificent. The flaw was real, fundamental, and invisible until the wind found it.

The Citicorp Center in New York, finished in 1977, was a triumph of structural engineering, raised dramatically on great columns at the midpoints of its sides. Only after it was complete and occupied did its own engineer rework the numbers and realise the building was vulnerable to diagonal winds the original calculations had missed — made worse because some welded joints had been changed to bolted ones during construction. On paper, a storm of the kind that arrives every couple of decades could have toppled it. The fix was carried out in secret, at night, while the building was full of people who had no idea anything was wrong.

These were not amateurs cutting corners. They were experts at the very top of their craft, with every incentive and resource to get it right. And they still shipped beautiful things with fatal flaws nobody could see by looking. That is the part the wonky-shelf story leaves out. Visible failure is the privilege of simple work. The more complex and ambitious the thing, the more its flaws can hide beneath a surface that looks not just acceptable but superb.

Software is the most complex, most ambitious, least physically constrained engineering most of us will ever touch. It belongs firmly in the company of the bridge and the tower, not the shelf.

This is where the analogy bites, and where I want to be careful, because the democratisation I keep returning to in this series is genuine and worth defending. More people can build useful things now than at any point in my career — internal tools, automations, dashboards, prototypes that solve a real problem for a real team. Much of it is good, and much of it would never have existed otherwise. What follows is not an argument that amateurs should not build software. The experts got caught too; that is rather the point.

Software hides its failures even better than a building does, and the stakes are not always visible at the moment of creation. The screen renders. The login works. The form submits and shows a friendly tick. The demo lands and the room nods. None of that tells you whether the permissions model is sound, whether the data model survives a year of growth, whether the security boundaries hold, whether anyone could recover the system at three in the morning when it falls over. A bridge that is unsound will, at least, eventually announce itself by falling down in public. Software can carry a latent, fatal flaw indefinitely, silently, and then express it everywhere at once — because unlike a bridge it is copied, deployed, and handed to thousands of people at no cost. The same properties that make software miraculous apply in full to its mistakes.

So the ceiling is not defined by who is building. It is defined by what the software touches: customer data, money, legal obligations, regulated workflows, security boundaries, large numbers of strangers depending on it, and the long grey grind of having to keep it alive for years. Below that ceiling, build away; if it breaks, the blast radius is small. Above it, a thing that merely looks finished is not an asset but a liability in the costume of one.

Here is the part I find genuinely unsettling, not merely cautionary.

Everything I have described so far happened with expert humans firmly in charge. The bridge, the tower, the skyscraper — and, more recently, the cloud. In October 2025 a single fault in Amazon’s US-EAST-1 region took a large part of the internet down for most of a day. The root cause was not a hardware failure or an attack; it was a latent flaw in an automated system that managed DNS records, which quietly produced an empty record and cascaded across dozens of dependent services. Nine days later Microsoft’s Azure suffered its own global outage, in which a bad configuration change sailed straight through the very safeguards built to catch it, because those safeguards themselves had a defect. These are among the most sophisticated engineering organisations that have ever existed — and in both cases the automation built to prevent failure was the thing that failed, in a way nobody saw coming until it had already happened at global scale.

Now ask what happens as we hand more of the building, and eventually more of the operating, to AI systems whose own failure modes we do not yet understand — with fewer and fewer experienced humans in the loop, for all the reasons I worried about last week. I am not claiming to know the answer; honestly, it is an unknown. But the two outages are a foretaste of the shape of it: systems too complex for any one person to hold in their head, defended by automation that can itself harbour the fatal flaw, producing a surface that looks entirely healthy right up until the moment it does not. AI does not invent this problem. It widens it, accelerates it, and removes some of the people who used to be standing there to catch it.

And there is one more thing software has that no bridge or tower ever did: an enemy.

Every failure I have described so far came by accident — a bad assumption, an unlucky configuration, a latent flaw nobody saw. A suspension bridge is only ever attacked by the wind, and the wind does not learn, bear a grudge, or go looking for weaknesses on purpose. Software is attacked by people: intelligent, motivated, well-resourced people whose entire occupation is to hunt for exactly the hidden, plausible-looking flaws this whole post has been about. That turns the question from “might this break?” into “might this be broken, deliberately, at the worst possible moment?”

And the blast radius is now national. Ransomware has shut down hospitals, fuel pipelines, and the back offices of whole companies and public authorities — rarely through exotic genius, usually by finding the one unsound assumption nobody went back to check. We have already seen how fragile the concentration is even without an attacker: when Spain’s card payments collapsed in 2025 — once in a nationwide power blackout, and again in the very cloud fault I described above — an entire country was thrown back to cash within minutes. Both were accidents. That is the unsettling part. If an accident can flatten a nation’s payments for a day, the version where someone is actively trying does not take much imagination.

AI sits on both sides of this. It will help the defenders, certainly. But it also arms the attackers, and it pours out code whose hidden flaws no experienced human may ever have reviewed — more surface that looks finished, produced faster than anyone can probe it, in a world where someone is always probing.

The judgement that matters here was never the magic of typing the right syntax. Anyone, and now any tool, can produce syntax that runs and a screen that looks finished. The judgement is knowing where things break: which assumptions are load-bearing, which corners cannot be cut, what a system does on its worst day rather than in its demo.

And even that is not a guarantee, which is the most sobering lesson of all. The engineer of the Citicorp Center was brilliant, and he still very nearly presided over a catastrophe. What saved the building was not that he got it right first time. It was that he kept looking after everyone else had stopped — that someone with the experience to know what to check went back and checked, long after the ribbon had been cut and the building declared a success.

That habit, of continuing to scrutinise the thing that already looks finished, is precisely what a fast, confident, plausible AI surface discourages. The work looks done. Why would you go back? A wonky shelf at least nags at you from across the room. The bridge looked fine too — right up until it didn’t.

Next week, the last in the series: the lecturer. What my father’s final career taught me about where craft goes when the tools take over the doing.