What's Missing in AI Security Today

Noah Labhart: [00:00:00] Hello listeners. Today we are dropping another episode in our series, The AI Control Loop: How Enterprises Govern the AI They've Already Deployed, sponsored by our friends at Wallarm. Wallarm is the AI control platform for enterprise AI, protecting every AI workload, API, and application in production, giving CISOs the governance they need and CIOs the speed they demand.

Organizations choose Wallarm for a complete inventory of APIs, AI agents, and AI apps using patented AI/ML-based threat detection and blocking that operates at production speeds. In today's episode, Craig Thomas, senior solutions engineer at Wallarm, returns to the show to dive into why runtime behavior is the critical blind spot and what CISOs should demand if they want to move from policy to control.

Craig, welcome back to the show. Thank you for being in a [00:01:00] return episode on Code Story.

Craig Thomas: Yeah, great to be back. I know last time we talked about rogue AI, so excited to dive in deeper about the threats and how to solve some of the AI problems today.

Noah Labhart: Absolutely. Really digging in, really enjoying this, the series we're putting together here.

Title for today is What's Missing in AI Security Today? We're gonna dive into runtime behavior being a critical blind spot, and what CISOs should demand, and let's dive right into it. So security teams are used to detecting incidents and responding after the fact, right? We've covered that quite a bit in this series.

Why is that model becoming insufficient for AI-driven systems?

Craig Thomas: Yeah, great question, Noah. First of all, just kinda like to frame this whole discussion, right? That traditional security really was built to answer one question: Who gets access? Firewall rules, identity access policies, network segmentation, all of it's fundamentally controlling the gate.

But AI has to move that problem past the gate. And [00:02:00] so you asked about detect and respond systems. Traditional IR assumes a human attacker moving at human speed. You have time to detect, triage, contain that attack. But as we talked about before in some of the other episodes, AI compresses that timeline dramatically.

A model can exfiltrate, manipulate, corrupt data in milliseconds versus hours, days, or even weeks. And so this incident with AI may not look like an incident. Looks like normal API calls, normal outputs and traffic. I like to think about it about how incident response was designed. Traditionally, someone breaches the perimeter, they move laterally out over hours, days, weeks.

Your SIEM fires an alert, SOC analyst opens a ticket. That men-mental model all assumes time. AI just doesn't give you that time anymore. And so if that prompt injection or that attack causes your AI agent to start dumping customer records to an external endpoint, that loop closes in seconds. So by the time your SIEM would normally [00:03:00] fire, the data's already gone.

And detect and respond built for a world where attackers are slower than defenders. But as, and as we've talked about, AI flips that assumption today.

Noah Labhart: It sure does. It moves so fast, and this, these models that have been built in the past just really don't, they really don't keep up. And building on that, when we talk about response today, and enforcement often means actions like restarting pods or rotating credentials or shutting down services, things like that.

Why can those measures come too late, even in an AI environment?

Craig Thomas: Those are stateless remediations applied to stateful harm. The damage is already done. It's already materialized. AI agents often hold open sessions with those external services, so rotating a credential after the agent has already acted doesn't undo the action.

And so that's really what we have to fundamentally look at. I like to equate it to restarting a pod is like locking a door after someone's already walked out with your files. It's good, [00:04:00] they're clean, you're good now, but it doesn't matter. And really in that traditional sense, that traditional web app, the worst case of a single compromise is usually bounded.

A one-record touch, a single session hijacked, but with an AI agent that has that tool used, that single compromised session might have already browsed your internal wiki, drafted an email, scheduled a meeting, queried your CRM. All that happened before your orchestration layer or those pod restarts happened, right?

You can't unsend that email by rotating their credential or simply restarting that pod.

Noah Labhart: Right on. Okay, so if traditional response isn't enough, and I think we have established that for sure, why does AI behavior require controls that operate so much closer to runtime? And the engineer in me is being like, we're touching on this in different areas, but I wanna be clear in why that's important.

Craig Thomas: It's really static controls, these firewall rules, IM policies, they can't evaluate intent. And so a lot of looking at AI is really about that intent, [00:05:00] that semantic drift. It's moved... This threat's moved from who gets in to what happens after they're in. So you're assuming these credentials are legit, but then once they get in, that spreads.

So the shift really moves from traditional security being essentially that gate I talked a little bit about earlier. Uh, you check their credentials. Check, they're good. You check the traffic. Yep, that looks good. You let it through or you don't. But once that AI agent with legitimate credentials, legitimate network access happens, your gate controls are done.

They're in. They've already done their job. So now that question, but it moves from what the agent does next. Does it behave consistently with business intent that you desire? Does it stay in scope? Does it just keep within that small frame that you wanted it to do? Or does it start doing things that look fine individually, but harmful in aggregate?

None of that happens at the gate. Really, that's the runtime question. So if you don't have controls that operate at [00:06:00] runtime, as we've been talking about for AI security and API security for a while, right? If you don't have those controls that operate at runtime, you really have no controls for that class of the problem.

Don't like to just be a scaremonger, right? There, there are solutions, and we're starting to see those solutions for how a CISO can gain this access and enable the business, but it has to shift from gate controls to runtime controls.

Noah Labhart: Right on. Okay. So we're, we're going a level down there, and when people hear runtime enforcement, they may think of existing security controls, right?

What changes when enforcement happens at the kernel level in the runtime rather than only at the network identity or application layers?

Craig Thomas: Yeah. So those network identity layers we talked a little bit about already, but they see who and where, but they aren't really seeing what The application layer does see inputs and outputs, but no kinda intermediate behavior, what tools were called, what memory was accessed, what reasoning steps were taking.

And so shifting that to kernel [00:07:00] level visibility, it actually sees those system calls, the file IO, the process behavior, what model it-- what the model is actually doing versus just what model is called on the host and not just what it's saying at that API boundary. And if you're only watching certain layers, you're only watching that front door aspect.

And so as you dive into the kernel layer, you get the aggregate of all these things, and you need all those things at runtime to truly see and shift and protect in real time the whole picture. And then, like we talked about, that enables you to leverage AI within your environments to get the most from the AI that you're investing in for both internal processes as well as customers.

Noah Labhart: Can you make that a bit more tangible for us, and I think for the audience? What does it actually mean to revoke or contain a compromised AI session without disrupting the broader deployment? 'Cause if you think linear and it's all part of the same-- a similar workflow and there's a [00:08:00] process, if you cut a chink in that workflow, kills the workflow.

So tell me a little bit more about that. Make it more tangible for us.

Craig Thomas: Yeah. Yeah, great question. You've got to shift from these broad controls to, to being more scalpel-like, right? So session-level containment talks about terminating or sandboxing just a single agent's execution without taking down the model, the infrastructure, or adjacent sessions, right?

So you have to continue to enable the business while atta-- you know, while blocking that individual attacker or rogue AI agent from there. So look at it like killing a single process without rebo-rebooting the whole server. But in order to do that, you ha-- you need the instrumentation to identify the right process.

In practice, Noah, you, you revoke the session token or you freeze pending tool calls in that execution context. You also need to preserve the session artifacts for forensics, so you can see what has happened, but you allow those other sessions to continue. So I always like to, from [00:09:00] an analogy perspective, yo, if you have a compromised account in an enterprise SaaS environment like Salesforce, if a single user session gets hijacked, you don't shut down for everyone.

You kill that session, you invalidate that token, you audit what that session touched. AI needs those same perimeters-- primitives. The problem is most organizations don't have them yet. They kill the pod, which we talked about earlier, which kills everything, or they do nothing. There's really no middle option So real, what real containment looks like, that I can identify the session that's misbehaving, terminate exactly that execution context, and preserve what it did for investigation.

Providing some feedback, we're gonna talk about how do you provide feedback to developers, but provide that feedback, but then leave everything else running. So that's a technical capability gap for most teams right now that folks are looking at. How do we do that? But that's really the end goal of being able to revoke or contain that session without [00:10:00] breaking everything.

Noah Labhart: That makes sense. Okay, so when we move into this type of real-time containment and thinking about the risk for AI agents and them having access to things, how does this kind of real-time containment change the risk equation? When AI agents have access to sensitive data, external services, production workflows, they're essentially doing the work of a, that you would, that a human would do.

How does the real-time containment change the risk?

Craig Thomas: Let's look at what it looks like without. Without that containment, that blast radius of a compromised agent is unbounded, right? It has whatever access it was granted. So if that user or that session has full access to go do whatever it wants, you're unbounded.

With containment, that blast radius shrinks dramatically. It's bounded by detection latency. The faster you see it, the less damage can accumulate. And s- and so this changes how you reason about those agent permissions. You can grant broader access if you have confidence in your ability to contain quickly.

Without [00:11:00] that confidence, you're forced into over-restriction, which then ultimately limits utility. And I think it's really important reframe for CISOs thinking about AI agent architecture, right? As a pr- as a previous CISO and dealing with those day in and day out, right now the dominant strategy is least privilege taken to an extreme.

You give the agent almost nothing and hope it can still be useful. Then the business comes back to you. They're mad that this agent isn't very useful a- and you're the bad guy again, right? But that, that... So that works until the business decides it really needs that agent to actually be useful, and now you're in a fight.

But the good news is if you have real containment capability, that really that calculus can change, can grant broader permissions 'cause you're confident that if something goes wrong, you can catch it fast and the damage is bounded like we talked about. Security stops being the ceiling on that utility and starts being the foundation that makes higher utility possible.

I really like to... [00:12:00] The CISO can now enable the business, provide the security, provide the governance versus just having to stop the progress a- as you start to get the this additional visibility and confidence.

Noah Labhart: Right on. What are some examples of AI behaviors that organizations should be able to stop immediately when they move to your runtime enforcement?

Craig Thomas: Yeah. A couple jump to mind right away. Know a data movement outside the approved endpoint. If a model is exfiltrating to unexpected destinations, you need to know when your data is leaving, right? Because that becomes a real, a real pain point a- and has some real legal and monetary consequences. So that's one.

Uh, another is prompt injection executing, especially when it alters the agent's stated goal or override systems instructions. A tool that exceeds its scope. It's supposed to do A and B, but now it's doing C, right? Or it's supposed to have read access only, but now it's making write and delete calls. You should be able to see and stop that immediately.

[00:13:00] Another one similar to we- we've looked at for APIs for a while, we've looked at even WAFs and DDoS rate- rate anomalies. If you have sudden spikes in query volume, especially to sensitive systems, that, that suggests the agent's being used as a proxy, and so you need to be able to detect and block that.

And then traditional security perspective, lateral movement. If you have one agent accessing credentials or context intended for a different agent, now you certainly have a problem, and you need to be able to see that and block it pretty quickly.

Noah Labhart: So you mentioned this a bit already, where security teams don't want to create or limit the functionalities, or you have business leaders coming back to you and being like, "Hey, this isn't very useful because it's completely locked down."

Security teams don't wanna be the bottleneck. How do organizations balance strong enforcement with the need to keep AI development and deployment moving fast, as fast as innovation can move?

Craig Thomas: For too often that a lot of organizations, the [00:14:00] security organization is operated kinda outside the bounds of or outside the parameters that we ask developers to do.

So I'd start with policy as code. Those enforcement rules can live in version control right alongside the agent code. They're reviewed in your CI/CD pipeline before reaching production. So take a similar approach to your taking the code with policy as well. Observe before enforce, right? That crawl, walk, run approach.

Start in audit mode. Understand the baseline behavior. Before writing those blocking rules This helps avoid false positives that slow development. But then you have to come back to it. You have to move to enforce. You can't just leave it in observe for forever. Tiered enforcement could be another one similar to we've done with other blocking mechanisms in the past.

No, like dev and staging might have more permissive monitoring, whereas production has enforced policy. Same tooling in place, just different modes. And then developer self-service. I believe being able to allow teams to see their own agent's [00:15:00] behavioral profile, understand what it will block before they deploy.

Because one of the things that I think kills developer velocity, and we've seen this over and over, right, it's an enforcement, but it's enforcement that nobody understands. As a CISO and security professional, I saw this too often. Security blocks or flags things, but don't give that feedback back to developers, so the insight as to when and how and why.

And if that rule blocks your agent, you have no idea why. You file a ticket, you wait, you're frustrated. But if you can look at that dashboard that shows, "Hey, here's what your agent did in staging. Here's which of those calls would have been blocked in production. Here's why," that's actually a developer accelerant, right?

They catch issues before deployment. The security team isn't the bottleneck or reactive after it's in prod. They actually enable the business and the developers. And so the shift really then is from security as the team that says no at the end to security as the team that [00:16:00] gives you the signal you need to be able to ship that code faster with confidence.

Noah Labhart: It really turns traditional stereotypes of security on its head and really enables things to move fast and be secure, which is incredibly important and really exciting, honestly, when you think about it. Okay, I got one more question. When organizations have the ability to discover, observe, and enforce AI behavior in real time, which we're- we're- which you are illustrating is so important, how does that change accountability at the enterprise level?

What does good governance look like from there?

Craig Thomas: You see this in modern EGRC tools. You see this in modern security programs, right? Governance becomes evidence-based rather than attestation-based. It's not just a check mark once a year, once every three years, when you just roll it out. You're not just asking do you have a policy, you're producing logs and evidence that show the policy was actually enforced.

So that accountability then, though it gets tied to specific AI [00:17:00] sessions, specific agent identities, specific tool calls, not just, "Hey, we use this model." And those arti-artifacts now are continuous, not a point in time when a board or regulator asks you pull the timeline, not just a self-assessment from six months ago.

This also can now limit the impact if there is an actual security event, right? A, a story I've heard in the past is once you have that actual attestation base, even if you do have an incident, you can say how it was limited, how it was contained, right? Those notifications don't have to go to all five million, 10 million customers.

You can say, "Hey, these five customers were impacted." So fewer notifications, lesser fines, even if there is an incident, because you have that attestation and that limit. And so then that risk quantification becomes possible. You can say, "This agent accessed this data four thousand times in Q2. Every access was in-within policy," rather than, "Hey, we think our [00:18:00] controls are good."

And so fundamentally, that changes... The thing that changes most for the CISO is the conversation with the board. Right now, the board asks, "How do we know our AI systems are behaving appropriately?" The honest answer for most organization is, "Hey, we reviewed the design. We approved the deployment. We have a policy document."

That's attestation, right? That's not evidence. What governance looks like when you have a closed control loop is, I can pull a timeframe of every action that a- that every agent took, every tool it called, every piece of data it touched, and I can show you exactly where policy was enforced and where it wasn't.

So that's fundamentally a different conversation, and that's what moves AI security from a framework exercise to an operational capability. So these, uh, I like to say those good runtime controls enable broader AI a-adoption, not restricting it, and this reframe lands well with CISOs tired of being seen as [00:19:00] blockers, and security becomes the foundation, not the ceiling.

Noah Labhart: Absolutely. Craig, that just makes too much sense. I really appreciate you returning to the show today. It's clear that businesses have to move to runtime containment. I think we have nailed that on the head today. The days of older, slower models are over. CISOs have to acknowledge this blind spot and change the way they think about their operation.

You have to catch these issues fast and lower the blast radius, and in doing so, you actually maintain speed of innovation rather than hinder it. And enterprises can move this to an operational capability through these runtime controls. I think you've made that very clear, Craig. Really appreciate you coming back to the show and telling us what's missing in AI security today.

Thanks, Noah. It was great to have Craig back on the show to point out what's missing in AI security today, and that is changing the models of security away from reactive to proactive at runtime.

Craig Thomas: If you'd like to learn more about Wallarm,

Noah Labhart: you can visit [00:20:00] wallarm.com. That's W-A-L-L-A-R-M.com. And thanks again for listening.