As AI tools move from proof of concept to production, development teams face the challenge of balancing speed, safety, flexibility, and cost. We recently spoke with representatives from Poka—Marc-Alexandre Bérubé, Director of Engineering, and Edmund Lam, Staff Software Developer—to learn about how their team is embracing this work.
Poka builds a connected worker platform, designed by manufacturers for manufacturers. Think Nestlé, Mars, Bosch; teams that rely on Poka to streamline training, communication, and day-to-day operations on the factory floor. In managing this platform, there’s a critical need to change dense legacy documentation (such as physical books and dated PDFs) into usable, real time work instructions.
“Our talented team figured out that we could leverage GenAI power to convert those documents into Poka work instructions,” said Bérubé. “To facilitate this, we’ve used the joint power of LaunchDarkly and AWS Bedrock.”
GenAI in production, in real time
Shipping GenAI features to production is different from building them in a sandbox. Poka doesn’t integrate only one model; they test multiple foundational models, from providers like Anthropic and Cohere, using AWS Bedrock’s model marketplace. Each model has trade-offs: some are faster, some are more accurate, and some are more costly.
“We use feature flags to control which model we’re going to use,” said Lam. “We can do side-by-side comparisons. For example, when the second version of Anthropic Claude Sonnet 3.5 launched last October, it was available in Bedrock right away. We ran it in production the next day.”
When performance issues surfaced, they flipped a feature flag and rolled back, without needing code changes or redeployment.
Prompt updates are no problem
Prompt tuning can be the difference between delightful and disastrous GenAI output. Poka uses LaunchDarkly AI Configs to dynamically manage prompt variations.
“It’s definitely easier to change prompts on the fly within a UI as opposed to changing them in our code and then having to commit and redeploy each time,” said Lam. “It gives us the opportunity to let our product managers and designers test out changes in prompts and see the results immediately, without affecting production.”
This empowers non-engineers to contribute directly to GenAI workflows, reducing development cycles. Enabling other team members to be involved in the release process has also created faster iterations.
The Guillotine incident: speed and safety in action
Early in beta testing, a customer uploaded a file called “Clean and inspect the guillotine.” The term “guillotine” triggered Poka’s internal guardrails as violent content, even though, in this case, it was referring to standard manufacturing equipment.
The fix was a simple prompt tweak, and the deployment was instant.
“With the AI Configs of LaunchDarkly, we could apply it very quickly into production without a redeploy,” said Lam.
Marc-Alexandre added, “It’s a perfect example of how unpredictable LLMs can be and how we must, as developers, think differently than regular software development.”
Cost awareness without sacrificing experience
Across all organizations using AI tools, managing cost and quality is a top requirement
“You know, prompt tweaks can swing the cost or quality,” Bérubé said. “It’s all about finding the sweet spot. And that’s why we monitor both to make sure that we are optimizing from both ends, without throwing things off balance.”
LaunchDarkly and Amazon Bedrock enable teams to target specific user segments (e.g., premium users receive more powerful models) and roll out updates gradually or experiment side-by-side with full observability.
What’s working (and could also work for you)
Poka’s success hinges on three things:
- Isolation: Changes are decoupled from code. Models and prompts are configurable and dynamic.
- Targeting: Teams can serve different models or prompts to different user segments.
- Control: Rollbacks are instant; rollouts are measurable. “The combination of feature flags, configs, and the flexibility of using Bedrock,” said Lam, “lets us isolate changes, target specific segments, and roll out gradually. If there’s a problem, we roll back fast.”
For teams just getting started with GenAI, his advice is simple: “Move fast but safely. You don’t have to get everything perfect up front. Look for the value you’re bringing to customers and iterate along the way.”
The Bottom Line
Poka has gone beyond experimenting with GenAI; they’re now scaling it reliably in production. Their setup, AWS Bedrock for model access and LaunchDarkly AI Configs for feature control, is a model any engineering team can follow.
Ready to build like Poka? Learn more about AI Configs.
Like what you read?
Get a demo
