The Lethal Trifecta, the Crown Jewels, and a dog named Seaweed
MLOps.WTF Edition #32
Your rundown of MLOps.WTF Meetup #9, where three brilliant panellists worked through whether AI agents can be useful and secure at the same time and what we really mean by trust.
Last Wednesday saw an excellent turnout for MLOps.WTF #9 at DiSH in Manchester. This ninth edition tackled agentic security, and between Matt Squire’s mandatory fire safety, an opening joke about logicians and a quick explanation of why he’s dressed as a police officer on the slide deck (not his fault, he has nothing to do with it) we settled in for a proper debate.
The panel: Leanne Fitzpatrick - The Financial Times, Geraint North - Arm and Danny Wood - Fuzzy Labs.
The statement:
“Agents can be useful or secure, but not both.”
Settle in. Here’s how it went down 👇
Watch the Full Panel
So, what does it mean when we say “agents can be useful or secure, but not both”?
Most of us still think about AI assistants the way we think about search engines. You put something in, you get something back, maybe it’s a bit more detailed and nuanced but the premise is the same. But now the playing field isn’t just chatbots and LLMS, agents are different. They act beyond a result recall, using LLMs as their “brains”, and as a result, the risk is naturally greater.
Danny Wood is the Lead AI Research Scientist at Fuzzy Labs, and his argument “Agents can be useful or secure, but not both” rests on a concept from programmer and AI blogger Simon Willison, who identified what he calls the lethal trifecta.
What is the lethal trifecta?
The lethal trifecta is a concept/framework that an agent becomes structurally insecure when it does all three of the following simultaneously:
Reads from untrusted sources
Has access to your private data
Can communicate externally
But to be inherently useful, or to do useful things on your behalf, such as checking your calendar and emails, how much are you willing to put your data at risk?
And, as such, the dilemma begins.
So why don’t we just add filters? Check what goes in and out, scan for sensitive data, add prompt injection guards?
Danny: “That’s not really solving the problem. That’s just making things slightly better.”
The real answer is the principle of least privilege. Not guarding against misuse of a permission. But not granting it from the outset.
It’s kind of like having a work experience student who’s incredibly smart but kind of naive about the world. You can tell them all the things they can do to be trusted with your credit card. But actually you just want a system where you don’t give it to them in the first place.
So, how do you put agents into production ?
Leanne Fitzpatrick is Director of Data Science and AI at the Financial Times.
We are extremely conservative when it comes to agents.
But conservative definitely doesn’t mean timid.
The Financial Times (FT) has stayed relevant for over a century precisely because it adopts new technology early and with intention. When ChatGPT launched, they took out a commercial licence and gave it to the whole company — with a lightweight governance process designed to bring people in rather than let them go rogue. “We’re going to welcome this with arms open wide,” was the internal message.
Broad LLM access is great for productivity but, as we’ve discussed already, agentic systems are different.
Suppose you want an agent to pay your window cleaner or help sort your dog’s hydrotherapy. To do that reliably, it needs to check your emails (did the window cleaner actually come?), your WhatsApp (did the dog get in the pool?), and your digital banking (to make the payments). Three completely reasonable tasks for an agent to help make your life easier - but you’ve just handed something you don’t fully understand a view into your inbox, your messages, and your bank account.
Now. Scale it to a newsroom.
The FT’s bread and butter isn’t code or a customer database. It’s journalism. “It’s our crown jewels. It’s the keys to our castle.” Plug an agent into the archive and you’re doing the leaking for everyone else, in an environment where copyright is already under serious strain.
The vibe-coding dilemma
When it comes to agentic systems, that governance process only increases. The FT, in their forward-thinking, has rolled Codex across engineering and enabled everyone to unify their workflow. But when everyone can “code”, who counts as an engineer?
We are using agents from an AI coding system perspective, but we are getting into challenges now about how we define who is an engineer at the FT, because actually we’ve enabled everybody to vibe code.
When anyone can spin up a tool, shadow tech doesn’t just become possible — it becomes inevitable. People are no longer limited by their lack of coding knowledge, but the risk lies in the limited public understanding of security and data safety. How can we help people think security first, and make sure what they’re building isn’t putting themselves or the company at risk?
The FT’s logic is to run ahead of it, provide structure, own the process, they have incredible legal teams who are clued up, giving education and applying governance. Creating a really good culture around cyber.
Industry wide, new job titles are also emerging: such as BISOS or business information security officers - responding to rapidly changing workplaces and this shift in agentic usage.
How do we then think about building agents which are both useful and secure?
Geraint North is a Fellow in AI and Developer Platforms at Arm.
In the IoT world you can trust almost nothing
His experience working in IoT gave such an interesting perspective. Coming in to counter Danny’s opening statement. His view: agents have to be secure to be useful.
“A lot of what gets called a security problem is actually a trust problem.” Security sets the guardrails. Trust is the question of how far you’re willing to let it operate within those limits. The danger lies in trusting something more than you should.
When we look at agents in the real world, there are probably three major things that we have to worry about that maybe don’t apply if you’re deploying agents inside your enterprise:
The diversity of products. In enterprise, you’re working with controlled, standardised systems. In IoT, the hardware landscape is massive and fragmented, and there’s no single security model that fits across all of it.
The confused deputy problem. In enterprise you can often funnel everything through a single trusted API. You can’t do that with physical devices. You can’t virtualise a camera. And when you lose that unified gateway, the confused deputy problem arrives: a legitimate, trusted component gets tricked into acting on behalf of something it shouldn’t. The architecture of IoT makes this harder to solve.
Proving trust in an adversarial environment. Deploy inside a company and you can assume a reasonable degree of physical security. Deploy in the real world and your device is out there, exposed, potentially interfered with. How do you prove that what you’re talking to is actually what it says it is? The answer Arm works toward is a hardware chain of trust, so integrity can be proven before anything built on top.
But the underlying problem — that security controls can be imperfect or bypassed — isn’t unique to IoT. It’s why trust, and consequence design, need to be there from the beginning and can’t be an afterthought.
What expectations do we have more broadly for security?
The reason why [agentic] security is so hard is because it requires a kind of different mindset than traditional cyber security. - Danny Wood
In traditional cyber security, you have a list of things that you want your system to do and a list of things you definitely don’t want it to do. You can test explicitly. It’s black and white. An agent is much smarter, and it’s technicolour in terms of its responses. They may find ways to trick you, especially if there’s an adversary giving it messages on the other side.
Which means you have these two different security systems. Number 1. Component-level security. “Can I make my LLM harder to trick?” Making the LLMs less likely to misbehave by adding guardrails for prompt injections etc.
But as Danny added “That’s kind of not really solving the problem; that’s just making things slightly better.”
The real way to look at this is creating a system where it just can’t do the things which would be harmful: ie. not let the LLM do more than it needs in the first place. Which is Danny’s point two. System-level security: Ensuring the system behaves as expected or fails safely even if the LLM goes off piste
By limiting it in the first place. We can avoid problems from the start rather than just retrospectively managing them.
So, if you constrain agents enough to make them safe, do you lose what makes them useful?
Bad security is more permission than you need
Think of it as a line graph. On one axis, the utility you want the agent to provide. On the other, the permissions you’ve granted.
Bad security is granting far more utility than the task requires, leaving the agent free to roam past where the job ends. Good security is landing on the line: exactly the utility required, no more. The principle of least privilege, applied to your agents.
“Bad security is more permission than you need. Good security is just the right amount.” - Geraint
If the system is too secure to get what you need done, then you need more permissions. But most of the time we don’t scale up, adding on capabilities, such as read my email, to help get the right result. We actually start by giving the agent more capabilities than needed to get the job done. This often happens because controls are not fine-grained enough, how can we navigate fine-tuning if the option to fine-tune simply isn’t there?
So why is it that we can’t trust LLMs but we can trust browsers?
Here is where Danny pushed back on himself. A web browser also ticks all three boxes: it reads untrusted data, accesses private data, communicates externally. We trust browsers with extraordinary amounts of sensitive information every day. So what makes an LLM different?
Danny’s argument. The browser was built to do one thing: browse the web. Its design, its security model, its decades of hardening - all aimed at that one task. An LLM was trained to do essentially everything that can be done with language, and then some. When you build an agent with one, your job is to constrain it back down to the specific task at hand. Not the thousand other tasks it could complete if someone prompted it right.
“I don’t trust an LLM as much as I trust a web browser.”
For most applications, Danny thinks we’re safe enough. If you’re using Claude Code to write a feature, the realistic risk of it deciding to post your environment variables somewhere is small but nonzero.
Ultimately can we have an agent that is constrained, that the risk surface is minimised but it’s still useful?
The agreement here is that yes, in theory. But in reality we haven’t seen enough maturity in this space, it’s all still very young, and larger more risk-averse and regulated sectors, such as aviation, banking ledgers, stock market - are not deploying agents at scale, they are simply not going to take the risk.
I just don’t think we’ve had enough of a maturity curve in non-deterministic capabilities out there yet. I just think the maturity curve just needs to keep going - Leanne
The other thing Geraint raised is trust. Yes, you can make your LLM more constrained. You can make it more deterministic, but one of the things about trust is understanding and taking into account the consequences of it going wrong. What if the consequences were more culturally understood, and therefore reduced?
What if “it wasn’t me, it was the LLM” becomes a socially acceptable thing to do? What if credit card companies reimburse you because your LLM made a payment that you didn’t actually ask it to do?
When we look at it through this potential lens - “that there is more protection for people when LLMs make mistakes” - if the consequences are lower, is the risk lower? Would we be more trusting, or willing to use LLMs/agents if there was greater consumer protection and cultural understanding? Perhaps.
Wrapping up
What is clear is that we are still at the beginning of the conversation when it comes to agentic security. Alongside the maturity curve being needed to see how agents will fare in terms of safety, we are still young in our maturity curve for agentic adoption. The real world impact is still being defined.
In a year, even three months time, or even tomorrow, the conversation may be vastly different again - but at least Simon Willison has put a statement together which better allows us to talk about security and usefulness as a starting block.
The panel itself gave much to think about, more so than what we have captured here, so make sure you watch the full panel to be able to muse on what we missed. In the meantime, here is the TLDR:
Takeaways from MLOps.WTF #9
Guardrails help, but they’re not the answer. The real fix is not granting permissions you don’t need in the first place.
The lethal trifecta is a useful frame. If your agent does all three: reads untrusted content, has access to private data, can communicate externally. Is it really safe? And can you still do what you’re trying to do without as many permissions granted?
Vibe-coding has changed who’s building things. When everyone can spin up a tool, shadow tech is arguably inevitable. The people building now don’t always have the security instincts that used to come with the ability to build.
What looks like a security problem is often a trust problem. Security sets the guardrails. Trust is how far you’re willing to let it operate within them.
And if you see Matt in a police uniform between now and then, don’t worry. It’s just the slides. He had nothing to do with it. 🍅
Final Bits
What’s coming up:
Wednesday 15 July: MLOps.WTF #10. Save the date. Theme TBD, location also TBD, but the timings are spot on. Suggestions and speaker applications are very welcome. Please reach out to take part!
Women’s Hackathon, Friday July 12th: Fuzzy Labs are hosting a female hackathon at DiSH in Manchester, open to undergraduates, postgraduates and those in the early stages of their career. Everyone who takes part will build their own agent, with support from women across Manchester’s AI community. Places limited.
We’re hiring. Fuzzy Labs is growing. We’re looking for engineers at all levels: mid, senior, and team lead. Check out our latest roles! 🫶
We’ve also just launched the Fuzzy Labs Fellowship, a nine-month training programme for fresh graduates entering MLOps.
Read more about the programme here: MLOps Fellow with Fuzzy Labs
And please do not hesitate to reach out if you’d like to know more.
About Fuzzy Labs
We’re Fuzzy Labs. A Manchester-rooted open-source MLOps consultancy, founded in 2019.
Liked this? Please share the sauce! 🍅 Forward it to someone wrestling with getting their ML models into production. Or give us a follow on LinkedIn to be part of the wider Fuzzy Labs community.
Not subscribed yet? Come on. You’re already here.