Runtime AI Governance on AWS: Deployment to Control

Aliaksei Ivanou: We have a web application firewall. We are covered. I hear that a lot. But a WAF doesn't see prompt ejection. It looks like normal HTTPS traffic. Enforcement for AI operates at multiple layers simultaneously. As a perimeter, WAF, like, AWS love with the AI Activity Dashboard. feature. It sees both and agent traffic patterns. At the model layer, better guardrails filtering inputs and outputs on every invocation, catching prop injection, redacting PII, as you mentioned as one of the concerns team. And, at the identity layer, identity and access management scoping permissions per agent.

And that's runtime inside the cluster, and that's where Walarm, I guess, enforces based on behavior. So, four layers, all enforcing independently. That's a defense in depth for AI.

Tim Erlin: Yeah, I think that this is a great example of where defense in depth… a great example of defense in depth, because you have this layered approach. And I think talking about some of the other controls is important. You know, a WAF, as an example. Just because the WAF doesn't see the AI interaction doesn't mean that it's not a valuable and valid control. It's doing exactly what its job is, which is protecting web applications. And that's true for all of the controls in the layer, but that ability to connect the dots, if you will, and enforce behavior at runtime is a real advantage, and it's something that's material to AI.

I think it's also important to mention the ability to connect those interactions back to the originating identity, which is part of that observability chain that I didn't talk about before. Because it… you know, as you look at sort of a layered approach, the user who started with the prompt can get lost through the multiple interactions that occur, and that context is valuable to the security practitioner. So being able to trace the entire interaction and map it back to that identity. gives you the ability to understand where, you know, an enforcement action further in the chain originated. And I think that's a valuable component of the AI control loop.

Aliaksei Ivanou: Yes, we definitely need to connect the dots.

Tim Erlin: Awesome. Okay, so last but not least in the AI control loop is, governance, or govern. And this is an interesting one to me, because You know, it's at the… I talked a little bit about the pattern that we see with new technology, you know, that starts with understanding what you have, discovery, that's the first place, right? I've got a new technology, I don't know what's out there. Moves to observability, what's it doing, and then to enforcement, how do I block bad things? Governance sort of comes last, and you could argue that it shouldn't come last. But it's really driven by the need to manage the entire system. And before you can manage that system, you kind of have to understand what's out there and how it's working.

On the other hand, governance certainly has a tight relationship or, you know, is tightly coupled to compliance and, you know, regulatory standards as well. And we've seen regulatory efforts in technology and cybersecurity evolve over the years. So, whereas in the past, you might have seen more reactive, regulatory responses, the EU AI Act actually you know, was drafted a couple years back, before AI adoption really took off to where it is today, and aims to address some of those challenges with governance from a compliance angle, from a regulatory compliance angle. Now, not all of the EU AI Act is enforcement now, and there are potentially some changes around The dates of enforcement.

As well, but some parts of it are definitely in place. And so, it becomes, a challenge for organizations to understand whether or not their deployed AI is under control, is it managed, is it governed, and how do the… the controls that you put in place map to the standards that are out there. And the EU AI Act isn't the only standard. I've put up here a mapping of other standards and how they apply to the steps in the AI control loop. GDPR is a good example of a standard that's existed for years, but applies to AI, because AI deals with personal data and privacy, so GDPR still applies, even though you've put a new technology on top of it.

There are still some newer, newer regulations, like, the Colorado one that are coming, and then, of course. There are voluntary standards, the NIST AI Risk Management Framework, the ISO standard, that apply to AI, but are designed not specifically for a regulatory compliance need, but for a other standards-based approaches. So I expect that the regulatory and standards environment for AI will continue to increase in complexity, and that organizations will need tools to help them manage that complexity. And that is part of the overall AI control loop. That needs to be in place for organizations to govern AI. Alexi, what can AWS add to this equation or this conversation around governance?

Aliaksei Ivanou: On AWS, Security Hub CSPN gives you continuous compliance scoring against CIS, NIST, PCI DSS, right? CloudTrail gives you immutable audit trails, and Security Instant Response, one of my favorites, gives you case history and monthly reporting. That's your evidence foundation. WallArm complements with AI-specific evidence which agent, which session, which data flow, which model provider was involved. That's the traceability layer regulators are looking for. And the time to build that evidence trail is now, while you still have runway, because

EU AI Act enforcement starts August 2026, right? So that's 2 months from now. The question for most organizations isn't whether they need governance evidence for their AI systems, it's whether they can produce it today if a regulator asks.

Tim Erlin: Yeah. Yeah, and I think it… it's important for… for… If you're responsible for compliance in some way, I think it's really valuable to think about the controls and the audit in terms of, you know, if you're talking about tools, does this tool provide a control? Does it validate that a control is in place, or does it support a control in some way, make it easier to use or easier to deploy? Because understanding that having a control is not the same thing as validating that control, as being able to prove that it's in place, is important from an audit perspective. And the validation… the requirements for validation are driven by those auditors. So, if you are…

If you're worried about compliance, but you're not in a position where you have to comply with something, nobody's showing up to audit today, now is the time to put in place the controls, and then the auditors can drive the validation of those controls as the regulations become enforceable. And you can do… it doesn't have to be one step. It can be put-in-place controls today. CloudTrail's a great example, right? Anything that's collecting immutable logs, that's a control that applies across many different compliance standards, IT compliance standards.

Proving that you have the right logs can be a second step when an auditor shows up. That's the validation step that follows implementing the control. And so with AI, we're going to see that pattern where putting in place controls today is valuable. And then, as the regulations and standards develop, augmenting those controls with the appropriate validation will become apparent as the auditors define what they're looking for and what format they need, and that kind of thing as well. Agreed. Awesome. Alright, thank you. So we've gone through the whole control loop. Actually, I'm just gonna do a little backwards step here.

So we've gone through the whole control loop on each of these steps, Discover, Observe, and force govern. We've talked about what's required at each step, we've talked a little bit about how AWS provides capabilities and how WallArm, provides, runtime visibility and runtime capabilities to, you know, augment what AWS provides natively. And that gives you a picture of the full loop that we're talking about. And really, you know, the reason we do this webinar is because we believe that there's a Better Together story for AWS and WallArm. And I think, you know, Alexi, do you just want to give a summary of the AWS role in, you know, governing AI. We talked about the details, but what's the summary for folks?

Aliaksei Ivanou: Well, AWS is designed and built to be the most secure cloud to run your AI workloads. So we have a portfolio of services that we covered already, partially, during this presentation that help you to secure and monitor and observe and enforce, right, your AI workloads. But when you combine these layers, native AWS controls and runtime governance. What you get is the ability to move fast on AI without choosing between speed and security. That's the outcome. It's not about adding more tools, it's about each layer doing one job well, so nothing falls through. available on AWS Marketplace. I believe today we have this, wallarm solution as well that, complements What our native services provide, and… help you to build even better AI workloads.

Tim Erlin: And I think, you know, on the wallarm side, you mentioned it, the runtime AI visibility, the inline enforcement. You know, those are things that complement what AWS provides natively, that, you know, extend to some of the… if you're using services that aren't AWS native, as you mentioned, Alexi. Gaining some visibility there. You know, there's a real story of combining the two to produce that outcome, and I love that you focused on the outcome. Our goal, in partnership with AWS, that Better Together story, is to close that gap between deployment and governance, is to give organizations the ability to move quickly, with AI and AI deployments, and also move securely and, you know, with governance as they, as they develop.

So I'm excited about the ability for customers to benefit from the combination. So with that, we've reached some final takeaways. So I always like to remind people, you know, maybe you weren't paying attention to the webinar up until now, this is the point at which you should pay attention, these are the final takeaways. Start with discovery. I think, Alexi, you said it at the beginning, I've said it many times in other contexts, you can't secure what you don't know about, you can't govern what you can't see.

Aliaksei Ivanou: Yes.

Tim Erlin: Understanding what's your environment is the starting point.

Aliaksei Ivanou: Yeah, I think overall, one takeaway is to go answer those four questions we talked through. Discovery. Observation, enforcement, governance. Wherever you said, not yet, that's your starting point this week. To think about.

Tim Erlin: And number two, for me, on observe, of a takeaway, if you want to remember one thing about the observability piece, is that AI governance is about more than AI. I think focusing on just the LLM, just the model, just the agent, is an easy trap to fall into, if you will. But when you deploy AI capabilities in your environment. you're deploying them on top of AWS infrastructure that is more than just the model. You're deploying tools, you know, agents that connect to other tools, that use MCP servers to get to resources that connect to data, that, you know, connect to APIs to take actions. And so that governance is not just about those

AI-specific pieces, it's about the whole picture. And it's important to think of that whole picture and understand what's there in order to achieve the AI governance you're after. On enforcement, I think the real takeaway here for me is that detection is not enough. Detecting events, detecting policy violations, detecting attacks is only part of the puzzle. If you can't actually stop those attacks from happening, you're not doing the whole job. detection is a requirement, it's a necessary but not sufficient requirement in order to accomplish AI governance, in order to complete that AI control loop.

Aliaksei Ivanou: And what's important is to have enforcement capabilities on multiple layers within your infrastructure. Getting back to this defensive desk, right? Absolutely.

Tim Erlin: That's right, that's right. Enforcement isn't a single point in the infrastructure of the process. Enforcement is an action that can be taken at multiple points across the layers. That's the defense-in-depth approach.

Aliaksei Ivanou: Starting with evaluation on the infrastructure and compute layer, then moving to identity and data protection, so least privilege, right? depends in depth, in a way, like, you should think about KMS keys that you use and the key policies, so even if something is compromised, you know, the keys cannot be used within the workloads that are not supposed to be using these encryption keys. And finally, we say, yeah. protection layer of AI application. Guardrails is a good starting point, where you can filter input and output, and use even some prune algorithms, like, for example. Better guardrails of automated reasoning, where, you know, you can Verify every answer, whether it's a hallucination or it's… Grant. to some.

Knowledge base or specific data that you have.

Tim Erlin: we didn't… we didn't talk about it in this webinar, and I don't… I don't want to, you know, it's a… it could be a whole webinar by itself, but I think when people talk about defense in depth, and they talk about all of the places that enforcement might exist at all those layers, it can be pretty overwhelming, and it can be hard to figure out where to start, and I think threat modeling is a great tool to use to figure out where to…

Aliaksei Ivanou: Always.

Tim Erlin: your limited resources, right? You don't have to address every threat all at the same time. Figure out which of them are the most important, or the most likely for your organization, and then go look at the controls that are there to help you address that threat, those threats.

Aliaksei Ivanou: I agree. Start modeling definitely brings this systematic approach, thinking that we mentioned multiple times throughout the webinar, answering those four key questions, like what we are working on, right? What can go wrong. What can we do about it, and did we do a good job? Definitely helps you to understand your risk. Portfolio right to perform risk assessments and think of remediation and mitigation techniques to address those risks.

Tim Erlin: The last takeaway, which, you know, sounds a little bit like a threat, compliance is coming. So you may not find that you're subject to compliance standards or regulations around AI today, but there can be no doubt that it's coming. And that the EU AI Act is sort of the tip of the spear. The reality is that you probably are subject to compliance. You know, if you have AI that's interacting with PII in any way. you're probably subject to GDPR with some limitations. So, you know, take a careful look at what might apply today, but be aware that, you know, if you don't have an auditor knocking on your door.

today to talk about AI and compliance in your AI deployments, that it is coming, and now is the time to put in place the controls that you'll need in order to be compliant. Alright. We do have a few questions, so that's the end of the presentation that we've prepared. There are some questions for us to answer, so I'm gonna, go dig up those questions and ask them and see who… which of us can answer them. And so if you have questions, feel free to put them into the Q&A, you still have time. And let's take a look at the list of questions. Alright, so… Given that most companies are bolting on AI components versus being AI-native built. Leading to AI security being built around tech versus intertwined.

What's your most common first step with clients? There are two questions here. So, first of all, I think… I think this… this person is, going back to, sort of, your point, Alexi, that it's better to build, AI on a secure foundation, or a foundation of security, than bolt-on security after the fact, but the reality is. That that isn't always possible. So, if you've got, AI, you know, companies adding AI components to existing infrastructure, what do you think is the first step they should take, in terms of, you know, governance and security for those, those bolt-on components?

Aliaksei Ivanou: Yeah, I can take this one. Speaking of the first step. as far as I'm concerned, it should be visibility, because most teams don't know what AI is running in their environment. So, start by discovering what's there, sanctioned and unsanctioned. Because you can secure what you can see, obviously.

Tim Erlin: And I think there's an important component there, that if you have if that AI is running in AWS native services, you've got AWS tools to find and discover what's there. And then if it's not, with WallArm, you've got that runtime visibility layer to identify the things that aren't in AWS. Or native in AWS. And so you're creating sort of that complete picture, which I think is pretty valuable. So discovery is a first step, I agree 100%. The second question, I think, is a little harder to answer, a little more challenging, which is, how do you evolve to get security intertwined versus bolted on? And to me, that's…

that's more of an organizational process question than a technology question. I don't know, Alexi, if you have any thoughts about that.

Aliaksei Ivanou: Well, that's a very reasonable question to follow the previous one. I think that building security into the AI development lifecycle, not around it, is the reasonable next steps. That means guardrails at the model endpoint from day one. Identity per agent from the first deploy, and not added after production. the framework score that we mentioned, like, the frame… the principles that I talked about, right? You aren't adding security to AI. You're building AI on top of security, as you already summarized. So, that shift happens, when security is in the pipeline, not reviewing after the fact. So.

keep adding those important AI security components into your pipeline, into your development lifecycle, that would be a reasonable evolvement on top of just bringing the visibility as a step one.

Tim Erlin: I think one of the challenges I've seen with organizations is that and I… it's amazing to me how this has happened, but that more and more organizations than with previous technology evolutions are willing, with AI, to sort of throw out their best practices in favor of velocity, in favor of a fast deployment and adoption. And that creates this challenge of, you know, I've now deployed an AI capability, an application, what have you, and I need to bolt security on.

because I gave up, or I skipped… I accepted the risk, would be the right way to put it, of, you know, deploying it without the usual best practices. And so, to me, that, as I said, I feel like that's an organizational challenge of getting the senior leadership to understand the risks That exist. And then they can either choose to accept those risks or to you know, do something to mitigate them. And the mitigation is the… is, what, Alexa, you were talking about. Mitigating those risks can be building the security into the pipeline, into the development process, is one of those steps, for sure.

Aliaksei Ivanou: Yeah.

Tim Erlin: So, another question, a little bit related, but different, slightly different question, is what's the biggest challenge in controlling AI after deployment? I think there are probably a few challenges we could talk about, but from your perspective, Alexi, what do you think the biggest challenge in controlling AI after deployment is?

Aliaksei Ivanou: From my perspective, one of the biggest challenges is behavior changes without code changes. Traditional applications are deterministic. Same input, same output. AI agents adapt, they change decisions, and produce different outcomes from the same prompt. You cannot write a static rule for that. So, int continues monitoring of what the AI is actually doing at the runtime. Instead of just what it was designed to do in deploy time. So this… Monitoring, behavioral analysis. It's really important to stay secure and make informed decisions whether something is going off.

Tim Erlin: Yeah, I agree with you, actually. I think that is probably the biggest challenge. I'll give you an example, like, a personal example of where that occurs. I built an agent to do a particular task. The task isn't important. And I asked that agent to sort of self-evaluate the ways that it could go wrong. And the first thing it came up with which just sort of surprised me, was the number one first thing that it said could go wrong is I could just completely ignore your instructions. that was its assessment of risk. So, in this case, the AI tool I was using assessed its own risk as just ignoring the instructions and doing something different. And so, to your point about the outputs being non-deterministic.

It, it is a… it is a big challenge, because we… We often, expect deterministic behavior, but the tool, the technology, is specifically designed not to be deterministic, and that's going to result in unexpected behaviors at runtime. You're absolutely right.

Aliaksei Ivanou: Yeah, and user input, it contains instructions as well as user requests, right? So basically, that's where all these new threats that we didn't think about before AI workloads come into play, like prompt injection, for example.

Tim Erlin: Yeah, yeah, absolutely. Let's see if we've got any other questions to answer. Let's see, I've got one here. How do organizations typically discover they have a shadow AI problem? Is it usually an incident that triggers it, or proactive assessment? That's a valid question. By definition, if you have shadow AI, you didn't know it was there. And so, how do you find out?

Aliaksei Ivanou: I would approach it from two angles. For AI built on AWS, services like Agent Registry that catalogs what's sanctions, CloudTrail, that we mentioned a million times already, shows every Bedrock and SageMaker and Vacation. So the delta is your unsanctioned AI within AWS. For AI usage outside of AWS, that simply is calling some third-party providers, right, and APIs directly. CloudTrail doesn't see that, but you need a network-level visibility, like network firewall with CLS inspection on ingress, VPC flow logs, or DNS logs, that's your visibility into what's going out of your network. To figure out that something probably might be unsanctioned.

Tim Erlin: I want to point out the language you're using, because although I fully expect that Wall Arms Marketing will continue to use the term shadow AI, I actually don't like it that much. It's not… it's not, It's not accurate enough. I like the term sanctioned, sanctioned and unsanctioned, and I think there's a third term, and, you know, we could debate what the right word is, but there's sanctioned there's, you know, sort of, known about, and then there's unsanctioned. So you've got, you know, things that you specifically approve, things that you know are there and aren't approved, and things that you specifically prohibit. And I think that's a more…

accurate, detailed way of describing the quote-unquote shadow AI problem than just saying shadow AI. But shadow AI is very good for marketing. All right, I think we've got a last question here. Let me just take a look and make sure I'm not missing any. Okay. So I think we've got a last question here. I always like this question, actually. It comes up in lots of webinars. Is there a quick win? Like, you know, you're walking away from this webinar today, you want to do something. What could you do? as a quick win, you know, for AI governance. Alexi, what do you suggest is a quick win?

Aliaksei Ivanou: What I would personally recommend is to enable bedrock guardrails on your model endpoints. It takes minutes to configure and immediately gives you input-output filtering from injection detection and PII reduction, so… It works with patchwork-hosted models and external models for Agent Core Gateway, and at the highest failure single action for ESP security, that doesn't change any design, you know, changes or implementation changes, it's just a configuration click.

Tim Erlin: So, I'm gonna… I'm gonna do my job as a corporate spokesperson and say that my quick win recommendation is exactly what's on the slide here, which is to… to run Wall Arm Infrastructure Discovery. It's available on the Marketplace as a free trial, or a free tier as well. It doesn't require that you install anything, and it gives you a picture of your AWS environment with some pretty interesting results. And so, that's a great quick win, and a good place to start as well. So, with that, we're at the end of our time. Alexia, I want to thank you for joining us. I think it was a great webinar, really interesting content. I think, you know, the partnership between AWS and WallArm is a strong, better-together story for customers.

Aliaksei Ivanou: Yes.

Tim Erlin: And I'm excited to see where we can go with it. And I want to thank everyone else for joining us for the webinar as well. There will be a recording, so if you want to see it again, or share it, you can. And feel free to reach out to us if you have any questions. Thank you so much.

Aliaksei Ivanou: Thank you, everyone.