Teaching Alexa's AI to Talk
A M A ZO N A L E X A E N T E R P R I S E · 2 0 2 4 - 2025
ROLE
Product Designer — with the design lead, PMs, Engineers, Applied Scientists
CRAFT
AI/LLM behavior design · conversational UX · systems design
OUTCOMES
90%+ LLM routing accuracy · framework now used across every new enterprise category
"Alexa, sign me up for hot yoga."
Easy for a person. Hard for an LLM — because Alexa has to decide whether that means the class downstairs or a studio across town. Same words. Two completely different answers.
Alexa+ gave Alexa a large language model that could finally understand how people actually talk. But LLMs are probabilistic. They guess. And nothing in our partners' APIs told the model how to guess right.
I designed how the AI should behave. One of two designers on the Category SDK team, I partnered with the design lead on the framework and owned half the enterprise categories.
BEFORE — WRONG DESTINATION; ROUTED TO LOCAL SEARCH
One request. Two valid destinations. No way for the model to choose.
Multiply that across every partner API in every category, and you get an assistant that works most of the time and fails unpredictably. For an enterprise customer, unpredictable is worse than limited.
AFTER — THE CLASS DOWNSTAIRS; ROUTED TO ACTIVITY BOOKING
The model wasn't broken. Our examples were.
We started with volume — twenty-plus utterances per API, every phrasing we could imagine. I worked closely with a data scientist who ran the evaluations and reported results back to me. Accuracy missed the bar.
More phrases just meant more noise. None of them told the model what a request actually meant. So we stopped writing phrases and started writing behavior.
One exemplar. Four components.
What a customer says. How Alexa should answer. The rules that get it there. And what happens when it all goes wrong.
Fewer examples than we started with. Each one a complete interaction arc, from request to resolution.
Not prompts. Behavioral blueprints.
Every blueprint became a screen.
Voice was only half of it. Every exemplar paired to an APL template — the screen layout an Echo Show renders while Alexa is talking. Same content, two channels, consistent across every partner integration.
The instructions covered tone, not just logic. A data scientist ran the evaluations against my exemplars and brought results back each round, which meant my design documentation wasn't a handoff artifact — it was the thing being tested. Engineering built against the same doc.
The design doc became the spec.
LLM experiences need patterns, not prompts.
A prompt tells a model what to say once. A pattern tells it how to behave — and gives your team something to measure it against.
Writing is model behavior.
y expert instructions covered tone, not just routing — when a resident says they have a headache, Alexa doesn't sound delighted. Change the words, change how the product treats people.