Privacy settings
We use cookies and similar technologies that are necessary to run the website. Additional cookies are only used with your consent. You can consent to our use of cookies by clicking on Agree. For more information on which data is collected and how it is shared with our partners please read our privacy and cookie policy: Cookie policy, Privacy policy
We use cookies to access, analyse and store information such as the characteristics of your device as well as certain personal data (IP addresses, navigation usage, geolocation data or unique identifiers). The processing of your data serves various purposes: Analytics cookies allow us to analyse our performance to offer you a better online experience and evaluate the efficiency of our campaigns. Personalisation cookies give you access to a customised experience of our website with usage-based offers and support. Finally, Advertising cookies are placed by third-party companies processing your data to create audiences lists to deliver targeted ads on social media and the internet. You may freely give, refuse or withdraw your consent at any time using the link provided at the bottom of each page.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
See Wallarm's AI Control Platform In Action
Talk to an Expert
12 Questions to Ask Any WAF, WAAP, API, or AI Security Vendor
Every vendor on your shortlist will tell you they cover your APIs, stop bad actors, and are "AI-ready." Those aren't answers — they're the sentence a slide deck is built around. Below are twelve questions for any WAF, WAAP, API, or AI security vendor, designed to get past it, along with what a real answer sounds like versus a hedge.
Instructions
How to Use This
Ask these live, on the call, not by email. Write down what the vendor actually said, not how confident they sounded. Then grade it:
Solid
A specific, falsifiable claim you could verify in a POV.
Hedge
Technically true, engineered to sound better than it is.
Red Flag
Deflection, a non-answer, or a claim that contradicts how the technology works.
This pairs with our Application Security Stack Audit Kit, which has space to score every vendor call, and our WAF vs. WAAP explainer if you're still mapping which category you're buying in.
WAAP vendor evaluation questions: testing a coverage claim
Start here if you're evaluating a WAF or WAAP platform's core traffic coverage, before the conversation moves to APIs or AI.
Question 01
Show me the APIs in my environment that I never told you about.
Any vendor can enforce rules against the endpoints you list in a spreadsheet. The harder and more useful claim is discovering the ones nobody documented: an old partner integration, a staging endpoint that made it to prod, a mobile backend the app team stood up without a ticket.
Solid
A live discovery pass against your traffic that surfaces undocumented or "shadow" APIs by observing actual requests — not by scanning a repo or ingesting an OpenAPI spec you already had.
Hedge
"We support API discovery," with no offer to run it against your environment before you sign.
Red Flag
Discovery that only works if you first upload a complete API inventory — the exact problem you're trying to solve.
Question 02
What is your inspection limit, how is it measured, and what happens to the part of a request past that point?
Every inspection engine has a limit somewhere. The question is what the limit is measured in, and what happens when a request hits it.
Many cloud firewalls measure it in kilobytes. They stop reading the request body after a fixed size to protect their own performance. Everything past that point is either blocked, which breaks legitimate large uploads, or passed through uninspected. A fixed size limit also has a second weakness. It treats a large, harmless file upload the same as a small, deeply nested payload built to be expensive to decode.
Published limits worth knowing before the call: AWS WAF inspects only the first 8 KB of a request body when protecting an ALB (16 KB default on other services, 64 KB maximum), and its common default lets oversized requests through uninspected. Cloudflare stops reading at 128 KB, including on its top Enterprise tier. Going higher is a negotiated exception, not a setting you flip on.
A time budget works differently. Instead of cutting off at a size, the engine gets a fixed amount of processing time per request. The limit follows the work the request actually costs, and an operator can set a different budget for a different endpoint. A file-upload endpoint can get more time than a login endpoint.
Attackers also nest payloads inside layers of wrapping: a malicious string encoded once, tucked inside a token, inside a JSON field, encoded again on top. A firewall that reads only the outer layer sees gibberish and waves it through.
Solid
A stated limit, whether in size or in processing time, and a named default for what happens past it: block, flag, or pass with logging. Look for a limit you can tune per endpoint, not one global number. Every request that hits the limit should be recorded as a visible event, so you can see how often it happens in your traffic. Ask the same of malformed payloads: unexpected content types, or nesting deeper than the schema expects.
Hedge
"Our engine handles edge cases gracefully." The sentence has no limit and no default behavior attached to it. Or a limit that exists but is a single global number nobody can adjust for the endpoints that need it.
Red Flag
The vendor doesn't know their own limit. Or oversized and unparseable requests pass through unread and unlogged. A silent pass-through is a coverage gap you'll never see in a dashboard.
Question 03
Which OWASP API Top 10 items do your signatures actually catch, and which need something else?
No single control catches all ten. BOLA (broken object-level authorization, also called IDOR) and business logic abuse require behavioral and identity-aware analysis, not pattern matching. A vendor claiming full coverage from one engine is describing the marketing page, not the product.
Solid
A category-by-category breakdown — which items are caught by request inspection, which need behavioral baselining, which are partially covered, and which are out of scope. Partial coverage stated honestly is a better signal than blanket coverage.
Hedge
"We cover the OWASP API Top 10," with no item-level detail when you ask for it.
Red Flag
A claim of 100% coverage from signatures or rules alone. BOLA does not work that way.
Testing an API security claim
Question 04
Do you detect attacks or only report on them — and where can you actually enforce?
Posture and inventory tools are useful, but they aren't attack detection, and detection isn't enforcement. A significant share of the API security market finds problems and hands them to some other product to act on — meaning your blocking depends on whatever firewall or gateway you already own.
Then there's where enforcement can happen at all. A CDN-based control only protects traffic routed through its own network. Internal APIs, private data centers, and anything that doesn't traverse that network are invisible to it — which matters if you're regulated and need protection inside your own environment.
Solid
A concrete example of an attack — credential stuffing against an API, a BOLA exploit chain, an abuse pattern — that the platform stopped itself, in its own engine, with the detection logic explained. Plus a clear statement of supported deployment models: on-premises, any cloud, hybrid, or network-only.
Hedge
"We give you full visibility into your API attack surface" as the entire answer. Or "we integrate with your existing enforcement point" — that's someone else's product doing the blocking.
Red Flag
Every example is a schema change or a new endpoint appearing, not an attack. Or the platform can only enforce on traffic routed through the vendor's network, and you have APIs that don't.
Question 05
How do you tell two of my users apart?
This separates request-level tooling from identity-aware tooling. If the vendor can't explain how they distinguish User A making 10,000 calls from User A and User B each making 5,000, they can't catch account takeover, credential stuffing spread across accounts, or scraping that rotates identities.
Solid
Token and session correlation, plus the ability to block a specific misbehaving user or session — not just an IP, since attackers rotate IPs constantly and IP-based grouping breaks down against NAT, mobile carriers, and any distributed attack.
Hedge
"We use advanced machine learning to identify anomalies," with no mention of what identity signal feeds the model.
Red Flag
The detection is IP-based only, or the only available response is an IP block.
Question 06
What share of your customers run you in blocking mode, and how long did it take them to get there?
A platform that's brilliant in "detect only" and unusable in "block" isn't a security control — it's an expensive alert feed someone has to babysit. This question also tells you how much tuning time to budget, and it's a proxy for false positive rate: teams don't leave a noisy engine in blocking mode.
Solid
A real percentage, plus a typical timeline from onboarding to confident blocking (weeks, not "it depends" with nothing after it). Ask what the false positive rate looks like at that point, and whether the platform was designed to sit in the traffic path and block from day one or was retrofitted to.
Hedge
"Most of our customers eventually move to blocking mode," with no number attached.
Red Flag
The vendor won't give a number, or can't name a single customer blocking across their whole estate.
AI security vendor evaluation: testing an AI or agentic AI claim
Worth asking even if "AI security" is one line item on the RFP: the category moves fast enough that a vendor's answer six months ago may already be out of date. Gartner predicts the average Fortune 500 enterprise will have more than 150,000 AI agents by 2028, while only 13% of organizations believe they have the right governance in place today.
Question 07
Which of the AI security jobs are you selling me?
"AI security" collapses several distinct problems into one term: securing the AI models and infrastructure you run, securing applications built on LLMs (prompt injection, output handling), securing agents acting autonomously with tool access and credentials, and securing the APIs that connect all of it to the business. A vendor selling one should say so, not imply they cover all of them.
Solid
A direct answer naming which they address, an honest "we don't do that" for the rest, and a clear answer on whether AI coverage and API coverage share one platform or are two products sold together.
Hedge
"We secure the full AI stack," with no breakdown when pressed.
Red Flag
The vendor conflates all of them under one pitch and can't distinguish them when asked directly.
Question 08
Show me the AI in my environment that I didn't deploy.
Shadow AI is shadow IT's sequel — a team wiring an LLM API key into a workflow, a SaaS tool that quietly added an AI feature, an internal script calling a model endpoint. If the vendor can only inventory what you tell them about, this is the same gap as Q1.
Solid
Discovery based on observed traffic to known model provider endpoints and API patterns, not a questionnaire sent to engineering. Ask whether AI systems, APIs, and supporting infrastructure land in one continuously updated inventory or three separate ones.
Hedge
"We help you govern your AI usage," with no discovery mechanism behind it.
Red Flag
Discovery requires you to already know what to look for.
Question 09
An agent authenticates with a valid token and then does something it shouldn't. What happens?
This is the core agentic risk, and it's different from anything a traditional WAF was built for: the credential is legitimate, the session is legitimate, and the action is still wrong — an agent pulling every record in a table when its task needed one, or calling a tool outside its intended scope.
Solid
Behavioral or intent-based controls — scope enforcement, action-level anomaly detection, rate or blast-radius limits — that operate after authentication, not just at the login gate. Ask which product in their portfolio delivers that control, and whether it's included in the platform you're being quoted.
Hedge
"Our platform secures the full AI lifecycle," with nothing about behavior after a valid token is issued.
Red Flag
The answer is entirely authentication and access control, with no mechanism for catching legitimate-credential misuse.
Question 10
What can I hand an auditor?
AI governance frameworks are arriving faster than most security stacks can produce evidence for them — only 13% of organizations believe they have the right governance in place today. A platform that can't generate an artifact — not a dashboard you screenshot, an actual exportable record — isn't compliance-ready no matter what the datasheet says.
Solid
A named, exportable report (AI and API inventory, access mapped to policy, incident history) the vendor can produce on request, ideally mapped to a framework you recognize — NIST AI RMF, ISO 42001, or the regime you already report against, such as PCI DSS, HIPAA, or SOC 2.
Hedge
"We support compliance reporting," with no example artifact offered.
Red Flag
"We're working on that," for a product already sold as compliance-ready. Or governance that defines policies with no technical control behind them to enforce them.
Testing the commercial story
Question 11
What exactly is metered, and what happens when agent traffic triples next quarter?
Agentic workflows generate call volume that doesn't resemble human traffic — one agent can make more API calls in an hour than a human team makes in a week. Gartner expects more than 30% of the increase in API demand to come from AI and LLM-enabled applications by 2026. If pricing is metered on calls, requests, or "AI transactions," ask what a 3x spike does to the bill before you're the one finding out.
Solid
A named unit of measurement, a worked example at your current volume, and a stated answer for 2x or 3x — a cap, a tier change, an overage rate.
Hedge
"Our pricing scales with your needs," with no unit or example attached.
Red Flag
The vendor can't explain their own metering model, or the honest answer is uncapped overage billing with no ceiling.
Question 12
Which of these products share a code base, and which arrived through an acquisition?
Consolidation is real, but a unified pitch deck doesn't mean a unified product. Two acquired tools on separate backends, stitched together with a shared login page, behave like two products — different update cadences, support queues, and failure modes — even when sold as one platform. The tell is a long SKU list and a thin integration story: ask how many separate licenses full coverage actually requires.
Solid
A direct answer about architecture — which modules were built together, which were acquired, and whether policy, logging, and detection are genuinely shared or run in parallel. Ask them to demonstrate discovery and blocking in one product, in one console.
Hedge
"It's all one unified platform," with no willingness to describe the underlying architecture.
Red Flag
The vendor won't answer, or support ownership turns out to be split across two teams for products sold as one.
Score your vendor.
After the call, tally three numbers. Write the last one down before you move to reference calls — it tells you exactly what to verify.
Red flags raised (out of 12)
Questions the vendor couldn't answer at all (out of 12)
The single answer that would most change your decision if it turned out to be wrong
If you want a version built for scoring live on the call, the Audit Kit has a fillable rubric for all 12 questions plus space for follow-up notes.
FAQ
Frequently asked questions
What are the most important questions to ask a WAF vendor specifically?
Start with Q1–Q3 — undocumented API discovery, how much of a request body actually gets inspected, and item-level OWASP API Top 10 coverage. Those three tell you how far a WAF vendor's coverage extends past traditional HTTP pattern matching.
What's the difference between a WAF and an API security vendor?
A legacy WAF scans HTTP traffic for known bad patterns and filters on matches — often only at a CDN edge, and only for traffic routed through that network. API security adds identity-aware behavioral analysis, tracking what a specific authenticated user or agent does over time, which a signature-based WAF was never built to do. A modern WAAP platform does both in one engine, parsing and decoding the full request rather than scanning it for patterns, and deploys wherever your applications actually run. The questions above help you find out how much of each you're getting.
How many vendor calls should I run before renewal?
Enough to compare at least two answers to every question above, including your incumbent. A renewal conversation with only the current vendor in the room has no benchmark to grade their answers against.
Do I need a separate AI security vendor, or does my WAAP vendor already cover it?
It depends which of the AI security jobs (Q7) you need covered. Some WAAP platforms have extended runtime detection and enforcement to AI and agent traffic; others are relabeling existing API security features; others are AI-native startups without application and API security depth underneath. Ask Q7 through Q10 of your current vendor before assuming you need a new line item — and ask whether AI and API coverage share one platform or arrive as two contracts.
What's a reasonable timeline to move from POV to blocking mode?
Ask the vendor for their actual customer data (Q6) rather than relying on a rule of thumb — the honest range varies widely by traffic complexity and how much tuning the platform needs first. The better question is why it needs tuning at all: platforms that confirm an attack is real before acting, rather than guessing from patterns, generate fewer false alarms and reach confident blocking faster.
Should smaller security teams skip this and just trust the vendor's RFP answers?
The opposite — a smaller team has less capacity to recover from a bad platform choice, which makes live, verifiable answers more important, not less. A 20-minute call using these questions is cheaper than a year on the wrong platform.
Ready to see how these answers hold up in practice?
Put Wallarm through all 12 questions on a live call.