Open by Default: Why open source matters for public sector AI

MLOps.WTF Edition #33

Ahoy there 🚢,

Original article by Tom published on digileaders.com

Last month, NHS England told its teams to make all code repositories private by default. The reason given: the threat from advanced AI models such as Anthropic’s Mythos and their supposed ability to find previously unknown, critical security vulnerabilities. Two weeks later and following a petition to ‘keep things open’, GDS and DSIT published guidance reaffirming the open by-default government policy that was first put in place almost 10 years ago.

Two government departments with opposite instructions. A panic response followed by a measured reminder to stay cool.

I sympathise with the reaction from the NHS. Many of us were shocked when we first read about the Mythos capabilities. However, the main flaw in the NHS response is that anyone planning to use Mythos (if it’s ever released) to attack NHS code almost certainly stored a copy long before it went private. Closing the repositories after the fact changes very little for attackers. What it does do is remove access for the engineers who might have caught and fixed those vulnerabilities first.

In principle, open source code is more secure than closed code. Linus’s law (named after Linus Torvalds, the inventor of Linux) says that “given enough eyeballs, all bugs are shallow”. In the NHS case, it’s likely that the open code repositories didn’t have that many eyeballs on them and I expect this is part of what drove the NHS response. It’s a fair concern. It just doesn’t justify going private by default.

The main reason I make this point isn’t to bash the NHS, but to provide context for a much bigger opportunity: our government’s chance to test the case for open source (and sovereign AI) in a procurement decision affecting another of our key public services The Police.

The College of Policing’s £115m Police AI programme is about to become one of the most significant technology procurement decisions in UK public services. The frameworks already in place point clearly in one direction. The recent covenant for AI in policing, which forms the basis of the upcoming Police AI tender, mandates efficiency and transparency as core requirements. The NPCC’s own procurement guidance advocates for avoiding vendor lock-in by “requesting open source” and “requiring tools that are supplier agnostic.” This is the opposite of the heavily proprietary models companies like Palantir are offering.

We can either be digital landlords, building on AI systems we own and control. Or we can be digital tenants, paying rent to overseas vendors indefinitely, on their terms.

The case for open source is about more than security, especially when applied to AI systems. Open source allows for full transparency of how a system was built and how it reaches its decisions. And in policing, where AI can influence decisions that affect people’s liberty, that transparency is non-negotiable. Courts will ask questions that need openness to answer. An AI system running on a closed proprietary platform from an overseas vendor simply can’t be scrutinised and trusted in the same way.

Security and transparency are strong technical reasons to adopt an open source approach. But there’s another, softer reason that is potentially even more compelling. And that’s how we can inspire the next generation of talent.

Andy Burnham said, at last year’s Manchester Tech Festival, that young people in his region “can see the skyscrapers from their bedroom windows but don’t know the pathways for them to work in them.” Open source public sector AI is one of those pathways. If the code that runs our public services is locked inside a vendor, those pathways are closed.

If it’s open and properly maintained then the pathways are very tangible. A young developer in one of Manchester’s boroughs could, theoretically, find a bug in the AI being used to support policing in their own city. Raise the bug, fix it, and have that improvement rolled out across forces nationwide. That’s how every successful open source project works. It just so happens that this open source project is part of running the infrastructure in their own country. Imagine that.

None of this is untested theory. GDS and the Government Design System proved that an open-source approach can work at scale in government. It’s the framework that underpins most government department websites. That same methodology, applied to AI, is what the Police AI programme has the chance to become.

The case for sovereign AI has been building recently. And I am very much in favour of it. Yes, you could achieve sovereign AI by building closed systems owned by British companies. But a closed British system still can’t be inspected by a court when a decision is challenged, and it can’t be reused or built on across departments. Open source is what makes sovereignty work in practice.

Our minister for AI, Kanishka Narayan MP, put it plainly at Founders Forum earlier this year: “We need greater British technology ownership before we can demand deeper British technology influence.” He’s also said he wants Britain to be “the home of global open source AI talent.”

Police AI is the perfect opportunity to turn both of those ambitions from political speeches into practical reality.

It’s a harder route than buying a ready-made system from the US. But from the conversations I’ve been having, the appetite to build this sovereign open future is there. We just need the courage to choose it.

Article first published on https://digileaders.com/will-public-sector-ai-be-a-digital-landlord-or-tenant/

Similar articles - The Case for Sovereign Open-Source AI: Digital Landlord, Not Digital Tenant

---

And finally…

Come to our next MLOps.WTF event! Wednesday 15th July.

“Agents can be cheap or useful. Not both.”

A panel evening looking into AI vs ROI - how does AI show up, or not show up, on your bottom line. Should be a really interesting one exploring token usage and how do we make outputs as efficient as possible at scale.

Doors at 5:40. Domino’s and drinks included.

---

About Fuzzy Labs

We’re Fuzzy Labs. A Manchester-rooted open-source MLOps consultancy, founded in 2019.

Currently hiring:

🚀 MLOps Fellow

🚀 MLOps Engineer

🚀 Senior MLOps Engineer

🚀 Lead MLOps Engineer

🚀 Marketing Manager (12-month FTC)

🍅 Want to share the sauce? Share and subscribe to receive MLOps.WTF episodes straight to your inbox.

You can also follow us on LinkedIn to be part of the wider Fuzzy community.