Amazon Alexa Enterprise • 2024
Teaching Alexa's AI to Talk
Product Designer , Oct 2024 – Mar 2025
Design Lead · PMs · Engineers · Applied Scientists · Solution Architects
AI/LLM Design · Conversational UX · Systems Design
TLDR;
When Alexa+ launched, third-party developers and enterprise customers needed a way to build reliable voice integrations. But without the right guidance, even a capable LLM couldn't reliably route a customer request to the correct enterprise API.
As one of two designers on the Category SDK team, I owned the end-to-end design of scenario-based exemplars across multiple enterprise categories from the conversation flows, expert instructions, to the APL (Alexa Presentation Layer) templates that teach Alexa's LLM how each third-party API should behave.
The result: 90%+ successful API routing accuracy across categories, faster developer onboarding, and a scalable framework now used across new enterprise categories.
HOW EXEMPLARS GUIDE AI BEHAVIOR
Alexa+ uses a large language model to interpret what users say and route requests to the right service. But LLMs don't come pre-programmed, they learn from examples.
An exemplar is a structured scenario that shows the LLM exactly how a conversation should go: what a user might say, how Alexa should respond, and how to handle edge cases gracefully.
Think of exemplars as behavioral blueprints. My job was to design them. They were grounded in real customer scenarios, aligned with Product, Science, and Engineering. I ensured they were scalable across every new category we onboarded.
CHALLENGE
Alexa+ was new and our partners' APIs weren't ready for an LLM experience.
Alexa+ introduced a large language model that could understand a wider range of natural language requests. But LLMs are probabilistic. The same request could be interpreted in different ways, with no guarantee it would reach the right API. Partner APIs lacked the context an LLM needs to understand intent.
The goal: give the LLM a comprehensive set of examples covering what customers might ask and how Alexa should respond — consistently, across every partner and category.
APPROACH & STRATEGY
The initial approach focused on volume.
We started with 20+ utterances per API — variations like "Sign me up for yoga" or "Book the yoga class." Simple and fast, but after testing with the Science team, routing accuracy wasn't meeting the bar. Utterances created too much noise and didn't reflect how people actually talk.
We needed scenarios, not just phrases.
We shifted to a scenario-driven approach complete interaction flows capturing user intent, expected responses, expert instructions, and edge cases. Fewer examples, but each one grounded in real conversations.
DESIGN PROCESS
After seeing the old approach fall short, it was clear the LLM needed it needed complete scenarios.
For each exemplar I defined a real user request, an expected Alexa response, an expected action, and expert instructions for handling edge cases like no available times, scheduling conflicts, multiple matches, and contextual substitutions.
I partnered with Science to validate routing accuracy, collaborated with Engineering to ensure flows aligned with API constraints, and refined documentation to reduce onboarding time as new categories were added.
FINAL SOLUTION
A scalable LLM framework built on clarity, consistency, and real scenarios.
The final framework gave each service category three components that worked together:
Expert instructions — behavioral rules governing when to clarify, paginate, confirm, or suggest alternatives. These told the LLM exactly how to behave across every state.
Annotated exemplars — scenario scripts distinct enough for the LLM to accurately route between categories and edge cases. Each exemplar covered a real interaction arc from request to resolution.
APL templates — screen layouts for Echo Show devices paired to voice responses, ensuring voice and visual worked together consistently across every partner integration.
Modular framework by design — any new service category could adopt the same structure without starting from scratch.
IMPACT
The framework resulted in 90%+ successful API routing accuracy across categories during experimentation.
Shifting to scenario-based exemplars improved API routing accuracy to ~90% during testing. Applied scientists used the exemplars directly to evaluate LLM behavior making design documentation the shared language between design, product, and science.
The reusable framework reduced ramp-up time across new categories. Same structure, no starting from scratch, and a foundation that scales as the platform expands.
REFLECTION
LLM experiences need interaction patterns, not just prompts.
The scenario-based approach gave the LLM something it could learn from and gave the team something they could evaluate against.
Define success criteria before you start building.
Aligning with science and product on what "good" looked like before writing a single exemplar would have saved iteration cycles and reduced back-and-forth mid-project.