UK AI Weekly: AI found the PHP bug. Now comes the bill

In late October, AI-assisted researchers uncovered a way to turn an apparently ordinary PHP crash into a serious security hole. The so-called PHP2Shell vulnerability could let attackers run malicious code on servers and read sensitive data, putting major websites—including NASA, Wikipedia and the White House site—at risk.

The exploit was reported before criminals got hold of it, and a patch was released. So far, so good. But this is not really a story about one bug. It is a story about AI confronting the enormous technical debt hiding inside software built over the past four decades.

This is not a British story: the key organisations named by IEEE Spectrum are California-based Calif, which conducted an independent audit of PHP sponsored by OpenAI, and the PHP Foundation. The UK part is what happens next, because this kind of vulnerability does not politely respect national borders.

PHP was created in 1994 and remains common on legacy websites. The underlying bug had reportedly reached the PHP Foundation as early as 2024. It caused an out-of-memory error: allocated memory was too small for the data. Normally, that means the program crashes. It is annoying, but it does not trigger the same urgency as a vulnerability allowing arbitrary code execution.

Then came the change in context. According to IEEE Spectrum, a Calif researcher using AI tools found a way to abuse that error to execute malicious code, install malware or read critical data. The crash became a security incident.

The wider significance is blunt. Much of today’s software was written before modern security practices, much of it is no longer actively maintained, and finding the dangerous edge cases can require painfully detailed reasoning through decades of accumulated behaviour. AI appears to make that search faster.

That is encouraging. It is also not a free lunch.

What this means

For the UK, the immediate lesson is that AI is becoming useful as a vulnerability-finding tool, but the responsibility for remediation still lands with software owners. Patching PHP helps only if organisations update it promptly, inventory what they run and check whether exposure occurred.

The regulatory picture remains recognisably British: principles-based and sector-led. There is no AI-specific statute and no single AI regulator. If the PHP2Shell flaw led to unauthorised access to personal information, the ICO would consider the relevant data-protection obligations under the UK GDPR and the Data (Use and Access) Act 2025.

The AI Security Institute is not the obvious home for this issue. Its independent pre-deployment evaluations concern frontier models, whereas PHP2Shell is fundamentally a software-maintenance and vulnerability-management problem. The government’s pro-innovation stance may help AI-assisted security research scale; it does not remove the unglamorous work of actually updating old systems.

There is also a useful policy question here. If AI can help surface decades of latent vulnerabilities at remarkable speed, what would it take for UK public services and critical infrastructure operators to make systematic use of that capability—without outsourcing judgement to a tool whose findings still need experts to validate?

AI has found the warning in the coal mine. The harder task is turning every exposed legacy system into something less likely to become tomorrow’s headline.

The takeaway: AI can accelerate the discovery of old software flaws, but only sustained investment and competent remediation will stop technical debt from becoming the next major cyber incident.

Source: AI is forcing us to pay off 40 years of technical debt — 1150 points on Hacker News