AI Discovery isn't just AI

Noah Labhart: [00:00:00] Hello listeners. Today we are dropping another episode in our series, The AI Control Loop, how enterprises govern the AI they've already deployed. Sponsored by our friends at Wallarm. Wallarm is the AI control platform for enterprise AI, protecting every AI workload, API, and application in production, giving CISOs the governance they need and CIOs the speed they demand.

Organizations choose Wallarm for a complete inventory of APIs, AI agents, and AI apps using patented AI ML-based threat detection and blocking that operates at production traffic speeds. We all know that you can't secure what you can't see, which is why AI discovery is a first principle for AI security.

But what's really required for AI discovery? It's more than just LLMs and agents. Today's episode is entitled AI Discovery Isn't Just AI, and joining us is Tim Ebbers, [00:01:00] Field CTO at Wallarm. Tim and I discuss the real requirements for AI discovery and why the connections between assets and infrastructure are part of the puzzle

Tim, thanks for being on the show today. Thank you for being on Code Story.

Tim Ebbers: Thanks for having me. It's a pleasure to be here.

Noah Labhart: Excited to dive into our topic today, all around AI discovery. Before we do, tell me and my audience a little bit about you.

Tim Ebbers: Sure. I've been in and around the industry for many years. I've spent half my time o- as a customer of vendors, and then the other half as a vendor.

I not only understand what customers go through, but have been a customer and have gone through it. I, I love being an operator, but I love data as much as being an operator, and AI seems to put both of them together very quickly, which is why I jumped into the middle of, of the AI things that we're doing at Walmart.

Noah Labhart: Yeah, for sure, and it's an exciting time, no doubt. What do you do for fun?

Tim Ebbers: I actually live on a ranch. My son and I are ropers, so we go help ranchers with their cows. My daughter trains horses, as well as my son does. So we, we [00:02:00] spend a lot of time on the ranch doing the usual stuff that you do on a ranch.

The analog-to-digital switch is very interesting and exciting, though I did put the wireless down into the barn this weekend 'cause my kids begged me to.

Noah Labhart: Of course, of course. Okay, let's dive into, let's dive into our topic for today. Again, we're discussing real requirements around AI discovery, and let's dive into what security teams often say, right?

They often say, "You can't secure what you can't see," which is totally true and feels just right. In the context of AI, what exactly do they need to see?

Tim Ebbers: That's really interesting because when you go from a deterministic world to a non-deterministic world, you have to understand more about what's going on in the data flows rather than just knowing that a request happened.

And so gone are the days that you can just look at a log and go, "Hey, that person did this," but that person did this and then did something else. And so it's not just the logs and the flows that you need to care about anymore, and so security needs to understand the data flows and what that [00:03:00] LLM and those models are actually triggering through the requests and the response of the, of the non-deterministic LLM.

And so not only can you... You need to understand the models, you need to understand the endpoints, what those models are calling, and the data that's flowing between them, and then really everything around it so that you know that this person did this thing- With this AI technology, and these are the downstream effects of what that happened and the data that they came back from.

Noah Labhart: That just makes so much sense, and I'm glad you brought up the deterministic versus non-deterministic or probabilistic nature of AI. It's one of the biggest concerns I have with AI and with AI ruling the world, quote, unquote. But moving into infrastructure, what supporting infrastructure matters most when mapping AI risks, such as APIs, but also cloud services, Kubernetes workloads, data storage identities, all the things, and external integrations?

What's the most important supporting infrastructure there?

Tim Ebbers: I think what you'll find is actually the identity is the [00:04:00] most important thing because the infrastructure is there to support a transaction. That transaction is there to support a person or an agent doing something. So in the old days, it was relatively easy.

You have a person doing a thing, or you have a service with a known API key doing a thing. Now we need to understand what these agents are doing and where they're taking those things. I don't know that any one thing matters more than the other. I actually think they all matter the same, but where I would start is probably APIs and gateways because that's, that's what's gonna drive where the data is because the data is still the most critical thing here, which is what we like to talk about a lot is where's the data, who's accessing the data, and when.

And the problem with MCP servers and AI gateways is your identity may change as you move from the, from the initial flow to and through the gateway to an MCP server. And so that can cause issues as well when one identity is making a request on behalf of another. How do you know that first person should actually have the data [00:05:00] that you're getting from the, the initial database?

Noah Labhart: The authentication and authorization problems expand now to beyond people to agents and knowing who's doing what. I definitely am tracking with what you're saying there Well, I have a question around shadow AI, and as far as I understand it, shadow AI is the use of AI tools and things, right, without formal approval by IT.

So a company has a formal process to approve tools, and people are using AI kinda outside of the normal process of their company. Where does shadow AI typically appear first inside of an enterprise environment? And I think even, even more importantly, how can it be prevented?

Tim Ebbers: I think we, we see it a lot at the edges of productivity before it shows up in production, right?

So in or- in order to get AI into production, you s- you have to develop the thing that's gonna do AI in development, and most of the people are developing with AI, which is where you see it first. So the developers, the business users is where you typically see [00:06:00] AI first, and everybody trying to go faster and everybody using AI to their advantage is really where you're seeing shadow AI.

And that shadow AI can be really anything from an MCP server that you hook up to Cloud on your desktop to, to an API that lives somewhere in the data center that you're hitting to get data that you may or may not be able to, to have access. So it's really everywhere. As you start your AI transformation journey, there's really no place that it can't be, but it usually starts at the end users trying to drive speed in their business, in their organization.

Noah Labhart: Yeah, that makes total sense. How can it be prevented? How can businesses get in front of that, so to speak?

Tim Ebbers: I would actually tell you that it shouldn't be prevented because then you're going back to the shadow IT ways of the business is just trying to get stuff done, and now you're putting a block in their place so that they can't get stuff done again.

What I would tell you is- We should watch what's going on, so discover, and then add policies so that we know the behavior of those agents [00:07:00] so that we can then be notified if they're going left or right of that particular policy. So the policy doesn't necessarily have to prevent somebody from doing something, but it can also just notify you that something happened, and it's either good or bad that it happened.

Noah Labhart: Ah, that makes sense. So it's shifting the focus from limiting the amount of tools or solutions that people are using and more of understanding who's doing what.

Tim Ebbers: Yes, and it's not that you shouldn't have standards and these are the tools that you should use, but when you get into the, in, into that prevention right away of, "I wanna prevent this from happening," what you're gonna do is make enemies out of those people that are trying to m- make everything go faster, trying to make the organization be better.

And if you come take a step back and now you're just watching, and then going and talking with a group of, "Hey, what do you need to do? And maybe we can find a better, more secure way," rather than, "Nope, you can't do this."

Noah Labhart: Fantastic. No, that makes total sense to me. How do relationships between assets change the risk picture?

And I think to, for [00:08:00] example, or maybe to put that in different terms here, why does it matter which API an agent can call or which data source a workflow can reach, right? Why does the authorization part of it matter?

Tim Ebbers: Yeah, I think that's, it's an important shift that we're watching, and the same model becomes high risk as soon as you connect a tool to it that can act on a data source.

So a read-only search API versus one that can call payments or an email send API, those are wildly different but still the same agent. And so the relationships between an agent and the tools that they can use are really critical, and understanding and identifying those tools is really key in creating a policy around those tools as well, so that this agent can have access to these tools but not those tools.

And when they try to access some other tool, then you get notified when that happens. And that risk relationship isn't really a property of the asset, it's the property of the edges, right? So an [00:09:00] agent isn't necessarily good or bad, it's the edge between that agent and the tool that you need to put the risk on.

So it changes the way you think about it, where an API service always did the same thing, your agent may not do the same thing over and over again.

Noah Labhart: Oh, that makes it clear, then. Okay, let's dig into AI discovery a bit. What makes AI discovery harder than traditional application or even cloud asset discovery?

What are the similarities and what are the differences?

Tim Ebbers: So let's start with what's the same, 'cause that's probably the easiest. What's the same is it's still discovery, right? You're still watching and looking for what applications are doing between each other or what two entities are doing between each other, and I really think that's the easiest part.

It's still the same inventory and ownership and continuous coverage that you need. It's really those fundamentals that we've had that are still in place. What's different is it's behavioral and not static, and the boundaries are fuzzy, right? Before it used to be very specific, [00:10:00] this person can do this thing.

It's not that way anymore. So those boundaries have really gotten fuzzy, especially when they can start looking at your third party APIs that you can't scan or you can't see, and you can only see them by their traffic. And that's a change from where we were, and data is part of that asset now. So understanding the data inside of a RAG is needed in order to look at the asset's risk or that edge's risk when the agent calls it.

And the bigger problem, and I think the one that we've been thinking about a lot, is It changes in minutes, not release cycles. Like, it's changes so quick because everything has just gotten so much faster. So it's the same muscle, but it's new terrain. The teams that do well will treat AI discovery as an extension of their API and cloud discovery.

It's not a replacement, right? It's additive to that, which is really what we've been focused on as well.

Noah Labhart: Yeah, having it alongside the API discovery and [00:11:00] application discovery, that makes, that just makes so much sense as you're expanding the infrastructure you're using as a company. And then when you go about AI discovery, obviously you come back with some results, right?

How should organizations prioritize what they find? Is every AI asset equally risky? And I think I could cherry-pick for some of your answers maybe where you're gonna go, but I'm curious what you have to say.

Tim Ebbers: Understanding the behavior first of the agent, what you want it to be able to do and what you don't want it to be able to is a good start.

And then looking at the edges, which is what does each one call in the behaviors that are really important to look at risk at those edges and the data that's flowing through them. And that's really where I've been focused for the last couple months is how do I take data that's flowing through transactions at real time and expose them for customers that can look at them to see if they're doing what they want them to do.

Noah Labhart: Excellent. What does continuous discovery mean? In a world where AI services can be deployed, connected, and changed [00:12:00] in minutes, what does it mean to be in continuous discovery?

Tim Ebbers: I think no longer can discovery be a quarterly scan or a spreadsheet because by the time you've cataloged it, it's probably changed 10 or 15 times over.

So it's really all about always on, watching the traffic, watching for change in the applications, not just the existence of them there. Great that you have an application, but if you've done a release, what changed in that release? And you need to know that real time, not at a quarterly scan. And when you change an agent, what did they change?

What are the new libraries that they're using? Do those have CVEs? Understanding everything about the whole flow. It's not just about we have somebody that does applications, and we have somebody that does infrastructure, and we have somebody that does data. You have to do it all when it comes to AI. It all needs to be in, in scope and viewed holistically rather than pieces and parts.

Noah Labhart: So things are moving so fast, you need to be doing this on a continuous basis, otherwise you're gonna be left behind. The days of quarterly scans for this particular thing are [00:13:00] over.

Tim Ebbers: Yeah, and inventory is, it's like a live stream, right? Not a snapshot. If your discovery cadence is slower than your deployment cadence, you're permanently behind So your discovery has to be faster than the deployments, which are getting faster and faster every day.

Noah Labhart: For sure. Okay, Tim, I got one more question for you. Once an organization has visibility into its AI footprint, what's next? What are the biggest gaps in today's AI security programs?

Tim Ebbers: Observability, visibility, and then you need to be able to govern what's going on and understand the behaviors of the things in your environment.

So from inventory to enforcement, right? Programs now list their AI and the... and, but very few can act on it, right? So great, I have AI, but can you... do you know what's going on with it and can you act on it? Can you block an unsanctioned provider or an unsanctioned model? Or can you cut off a misbehaving session in real time?

Can you make it so that you don't have PII leaving, leaving your egress? [00:14:00] So those are the kinds of things that you need to look for. Runtime protection, prompt injections, data leakage, model abuse are all things that you need to be looking at after you have visibility. And then audit. How can you prove that no PII has left?

And how are you going to make sure that the auditors understand that? And then ownership. Who owns what? It seems simple, but with the speed at which we're going, sometimes in a large corporation, that's hard to figure out is who owns what. So we have been spending a lot of time looking at not only observability, but also what to do after observability, and it's really exciting to see the what's after because that's really where you're making a difference in an organization.

Noah Labhart: Certainly. Oh, that's very clear. Tim, thank you for being on the show today. I think that it's been really interesting digging into the discovery process around AI. I hear you in saying identity is the most important thing to, to realize who does what and when and whether it's a person or an agent. And you define [00:15:00] shadow AI as living at the edges of productivity, and that AI discovery should be running alongside other regular discoveries in a continuous nature because the industry is moving fast, and you don't wanna find yourself as a business permanently behind, which I really appreciate you strongly wording that, 'cause I think that's really important.

The biggest gaps in business programs are clear: visibility, observability, and from inventory to enforcement. Businesses need to take this seriously, make AI discovery a regular part of their everyday operations. I really appreciate you being on the show today.

Tim Ebbers: Thanks for having me, Noah.

Noah Labhart: Tim makes it clear that with the speed of the industry, AI discovery needs to be a part of businesses' everyday operation, continuously monitoring new tools, and more importantly, who and what is taking action on their systems internally If you'd like to learn more about Wallarm, you can visit wallarm.com.

That's W-A-L-L-A-R-M.com. [00:16:00] And thanks again for listening.