When James Governor, analyst and co-founder of RedMonk, sat down with Marek Poliks, Head of AI at LaunchDarkly, he promised the conversation would be “a little bit spicy.” It delivered.
Poliks started building and governing agents long before it became the industry’s obsession, and he joined LaunchDarkly to help define what runtime control looks like in practice. Governor came in with genuine curiosity: How does AI agent governance build on the core concepts of feature management?
What followed was one of the more honest conversations we’ve heard about where agent governance is actually falling short, why the gateway model has real limitations, and why observability shouldn’t be the last line of defense when agents are running in production.
Below is an excerpt that’s been edited for clarity.
—
James Governor: You’ve got some views on why the gateway approach doesn’t fully make sense. What’s wrong with the endpoint approach?
Marek Poliks: There’s nothing in principle wrong with a gateway. And in fact, I think every mature enterprise AI body should have a gateway. That’s a critical control point. Some of my best friends are gateways.
But they also introduce a lot of issues. Especially if you’re using a third-party gateway, you’ve introduced a serious level of vulnerability, a serious level of dependency—a critical juncture point within your system. This is how a lot of AI observability and AI tooling, especially around governance, gets instrumented—including guardrails. You’re introducing a third-party dependency that adds latency and single-point-of-failure logic right at the API call itself to the model provider, which is already such an infrastructurally contingent moment.
And the bigger question is: If you’re sending critical information—the enforcement of whether or not someone has access to a model, or whether a guardrail should be imposed—if you’re sending that to a third party, you’re sending everything the customer sends in the form of a user prompt, the model’s response, all of this business-critical, PII-forward, security-rich information through a brittle third point of failure.
The majority of people I see—especially the advanced folks working in highly regulated industries—when they’re building gateways, they’re confronting this impossible problem: How do I regulate what’s going into and out of these models without looking into what’s actually being said, without storing any of that information anywhere, because I’m not allowed to?
Centralized administration of AI is a good thing. But if that centralized administration doesn’t have an understanding of the constituent components of the harness of a given agent, it can be toothless. Most gateways are just: Have access to this model, you don’t have access to this model … maybe if the model starts to underperform, we’ll switch to this model. But they’re not a highly active control point, because the amount of context being handled there isn’t very rich. You don’t have the full harness information. You don’t have a tools registry or a skills registry that you can actually supervise. You’re just working with an application that is a client that’s somewhat invisible to you. You have an API call that you’re handling. And that’s it.
So you’re limited in terms of what you can control, you’re limited in terms of your governance, and you’re sitting at the most contingent, the most brittle, the most security-complex point of the entire architecture.
And so for us, it’s cooler to be inside the application, where we can provide guardrails and even online evals and other kinds of metrics without necessarily revealing any context back to LaunchDarkly at all. Our online evals work by sending you a harness and saying, “Do an online eval.” They don’t return any information to LaunchDarkly. There’s no API call to LaunchDarkly being made in the middle of the run—no added latency, no requirement to pass back customer context or customer query. And you still get your eval.
James Governor: You’ve talked quite a lot about instrumentation. Will observability save us?
Marek Poliks: It will not save us. Observability won’t save us.
Can you think of a worse word? Who wants to observe a dynamic, incredibly contingent, powerful system? Observability to me means passivity—looking at a giant log of every bad experience my customer’s ever had. And those experiences have happened. That’s what it means. It’s like living testimony that something bad occurred.
And the goal is to get ahead of that. That’s even more important in the agentic era, because real bad things can happen. The more useful a system is, the more critical, contingent, complicated information it has access to—the more agency it has to do things that are potentially bad. The blast radius is large already.
That doesn’t mean information is bad. Information is great—it’s super important to have information. And logs are great. But what it means is that you need more. You need the ability to actually intervene. You need the ability to get actually active inside of runtime. You need the ability to keep problems from actually happening. And that is more useful than information about a thing that’s happened that may or may not be reproducible ever again.
Watch the full MonkCast episode below.
FAQs
1. Is observability enough to govern AI agents in production?
No. Observability is retrospective by design: it tells you what already went wrong, after a customer experienced it. Logs and traces matter, but governing agents requires the ability to intervene during runtime and prevent failures, not just document them. In agentic systems, where the blast radius is wider, detection after the fact is insufficient.
2. What is the gateway approach to AI governance, and what are its limits?
A gateway centralizes AI access at the API call to the model provider. It works as an access control point, deciding which models a team can use and failing over when one underperforms. Its limit is context: a gateway sees the API call, not the agent's full harness, tools registry, or skills registry, so its enforcement stays shallow.
3. Why is a third-party AI gateway a security risk?
Because every user prompt and model response passes through it. That means business-critical, PII-heavy data routed through an external dependency that also adds latency and a single point of failure at the most brittle point in the architecture. Regulated teams face a harder version: enforce policy on model traffic without inspecting or storing it.
4. What does runtime control mean for AI agents?
Runtime control means enforcing policy from inside the application while an agent is executing, rather than intercepting traffic at the network edge. Because the control point sits next to the harness, it can see which tools and skills an agent has access to and apply guardrails against those components, not just the model endpoint.
5. Can you run evals on an agent without sending prompt data to a vendor?
Yes. LaunchDarkly pushes online eval instructions to the harness and execute locally, returning no prompt or response data to LaunchDarkly. There is no mid-run API call back to the vendor, so there is no added latency and no requirement to pass customer context outside your own environment.
