There is no doubt AI is advancing fast. Governments are backing it, companies are racing to adopt it, and investment is booming. But despite the excitement, we are seeing growing numbers of failures – often serious ones.<\/span> <\/p>\n I approach this from a risk and governance perspective. Boards and management need to know what is being built – and ensure proper oversight. Not later. Not after it fails.<\/span> <\/p>\n The business world has changed. Systems used to take years to built and deployed. Now, it is quarters – or weeks. AI enables that speed, but it also increases risk.<\/span> <\/p>\n Governments are starting to react. Regulation is coming – sometimes strong, sometimes vague. In the UK, company law applies, but that may be too slow. The EU\u2019s upcoming AI Act (February 2025) includes major fines – up to \u20ac35 million or 7% of global turnover.<\/span> <\/p>\n This is not just about new AI systems. It applies to upgrades, scripts, patches – everything. Assumptions that “it\u2019ll work” can lead to serious consequences.<\/span> <\/p>\n Tech-savvy companies like Booking.com and Amazon run A\/B or split tests before full deployment. They compare two versions, watch how users react and refine.<\/span> <\/p>\n Many AI systems are now being developed by non-technical teams – often without the board even knowing. A Salesforce study in Q4 2023 showed that 50% of employees using GenAI did so without approval.<\/span> <\/p>\n – A well-known delivery company\u2019s chatbot spiraled out of control. It was taken offline after going viral.<\/span> <\/p>\n
\n<\/span>
\n<\/span>These failures are not due to bad luck or poor tech. They are often the result of weak governance, inadequate testing, and teams cutting corners to keep up with the pace.<\/span>
\n<\/span>
\n<\/span>Even the big player\u2019s struggle. In February 2025, an Apple AI feature wrongly claimed a BBC article said someone had taken their own life – completely false. Apple responded quickly, but if it can happen to Apple, it can happen to anyone.<\/span>
\n<\/span>
\n<\/span>I am not anti-AI. I believe AI brings innovation, creativity, and speed. But it must be checked, tested, and governed before it is trusted. A quick “eyeball test” is not enough in complex systems.<\/span> \u00a0<\/span><\/p>\nAI needs control<\/h3>\n
\n<\/span>
\n<\/span>Today, I would label all AI projects with a red flag – not because AI is bad, but because many teams are skipping QA and testing steps or doing them poorly. In a rush to ship new tools, too many organizations have gotten sloppy.<\/span>
\n<\/span>
\n<\/span>If professional teams get it wrong, what happens when non-technical departments build and deploy AI without formal processes? It is happening more than you think.<\/span>
\n<\/span>
\n<\/span>AI is powerful, but we cannot let it run unchecked like it is infallible. That is a fast track to failure.<\/span> \u00a0<\/span>Remember AI is usually overconfident and devoid of accountability. That\u2019s a key differentiator between a human doing the work or proper controls in place vs. uncontrolled AI \u2013 the AI has no concern or repercussions if it\u2019s wrong or not, despite having a huge negative impact on the business.<\/span><\/p>\nSpeed is the pressure<\/h3>\n
\n<\/span>
\n<\/span>With AI high on every CEO\u2019s agenda, solutions are often rushed. The faster we move, the more we need governance, QA, and testing to catch up. Sadly, those areas have often been left behind.<\/span>
\n<\/span>
\n<\/span>Teams now assume the tech will \u201cjust work.\u201d But it does not – not without control.<\/span>
\n<\/span>
\n<\/span>We also need to look more closely at the tools we use. Some tools (like DeepSeek) may have regional or political biases. We must apply due diligence – not just to the AI itself, but also to the systems behind it.<\/span><\/p>\nThe legal view<\/h3>\n
\n<\/span>
\n<\/span>One proposed law in California used the word \u201ccarefulness.\u201d I like that. It is what we need: governance and tests – before things break.<\/span><\/p>\nThe Importance of testing<\/h3>\n
\n<\/span>
\n<\/span>Think CrowdStrike in 2024. The global impact was massive. Some companies suffered for months.<\/span>
\n<\/span>
\n<\/span>We need to test properly. Always.<\/span>
\n<\/span>
\n<\/span>I recently spoke with Sandeep Shah at Webtrends Optimize (disclosure: I am a shareholder). His view? \u201cTesting is common sense, not rocket science.\u201d Good testing teams follow best practices, use smart tools, and experiment before going live.<\/span><\/p>\nExperimentation (A\/B Testing)<\/h3>\n
\n<\/span>
\n<\/span>Start small. Test. Improve. Then go live. That is what professionals do.<\/span><\/p>\nShadow AI projects: A real risk<\/h3>\n
\n<\/span>
\n<\/span>That is a problem. These teams often lack the skills to test or govern what they build. Without oversight, these systems go live – untested, unreviewed, and risky.<\/span>
\n<\/span>
\n<\/span>Companies today are complex. Systems talk to each other. If one breaks, many break. You need ecosystem-level visibility and governance.<\/span><\/p>\nReal-world examples<\/h3>\n
\n<\/span>– A GM car dealership in the US sold a $76,000 car for $1 (thankfully, not honored).<\/span>
\n<\/span>– A UK train company\u2019s ticketing app issued incorrect tickets and had to be shut down.<\/span>
\n<\/span>– A BBC report in Feb 2025 found that AI chatbots failed to accurately summarize even basic news content.<\/span>
\n<\/span>
\n<\/span>We cannot have another Post Office Horizon-style disaster in ten years, with executives claiming \u201cWe thought the AI was right.\u201d<\/span>
\n<\/span>
\n<\/span>These incidents all point to the same thing: failure to test, govern, and communicate clearly – especially at the board level.<\/span><\/p>\nWhat should companies do?<\/h3>\n