Lucy is an integrated data application platform for the American Express ecosystem, providing developers with data and application abstractions to simplify product development effort while satisfying complex enterprise requirements.
Data Application
Voice-First Shopping Assistant
Design conversational checkout for embedded device. Designing, writing, and testing with live AI prototypes.
(To comply with my non-disclosure agreement, some images presented in this project have been treated as unreadable on purpose. The information in this project does not necessarily reflect the view of Visa)
Overview
Role: Lead product designer
Scope: Drive the end-to-end UX strategy to build consumer trust and confidence in AI-driven transactions.
Timeline: Jan 2026 - June 2026
Responsibility:
-
Drove and led the agentic non-text commerce framework, where I helped shape the concept approach to explore voice-based agentic shopping behaviors
-
Partnered with cross-functional team to develop new design and research workflow with AI
-
Informed Visa's overall strategies towards non-text based agentic transactions with research insights
Early Impact:
-
Gained business buy-in and secured funding for future research.
-
Established and shared UX guidelines for voice-first agentic commerce with key stakeholders.
-
Scaled the AI-enabled workflow through design and testing iterations across design, content and research team.
-
Shared research findings with external OEMs to inform and enable new payment use cases.
Challenges
Design Challenges
-
Maintaining User Control: Users need clear visibility into what the agent is doing, why it is taking action, and when their confirmation is required.
-
Risk of Incorrect Purchases: AI models can hallucinate, misinterpret user intent, or complete purchases that do not align with the user’s expectations.
-
Lack of Trust in AI Agents: Users may not yet trust an AI agent to make purchases or manage payments on their behalf.
-
Unfamiliar Interaction Patterns: Voice-based purchasing is not yet widely adopted, which can make the experience feel unfamiliar and less predictable.
-
Ambiguity in an Emerging Space: Agentic commerce is still evolving, with few established patterns, expectations, or interaction models.
Prototype Video Demo
Demo Notes:
-
Subtitles were added to the demo video for accessibility and are not part of the product design. Audio is required to hear the conversation with the in-car AI assistant.
-
In the intended experience, the fingerprint sensor would be integrated into the steering wheel; the on-screen version is shown for demonstration purposes only.
-
The authentication interface was still a work in progress when this case study was created.
Click the music icon to listen to the conversation
The Design Journey
Drove Vision Through System Thinking
Mapping Out the Vision
Because agentic, non-text shopping is a brand-new concept, I collaborated closely with product leaders to map out the long-term vision for agentic transactions. Ideally, consumer inputs will not be limited to voice or text, but will extend to images, gestures, and other modalities. Depending on the device type and its capabilities, we aim to explore various interactions between humans, devices, and artificial intelligence to optimize the purchasing and authentication experience.
Proposed use cases for design and research:
-
Purchase with agent in vehicle
-
Smart glasses or VR sets
-
Smart home devices
Key area for testing:
-
Consumers' perception of various authentication methods
-
Visa brand
Proposed project planning and scope:
-
Phase I: Concept test drive
-
Phase II: Explorations of devices such as smart glasses and/or VR sets.
Design Strategy
Design and Research in an Ambiguous Space
Discovery
-
Desk research
-
Concept visualization
Concept Alignment
-
Alignments across 3 product group
-
Key authentication methods
Concept Validation
-
Experience mapping
-
Role-play interviews
-
Comfort survey
Live Prototyping
-
Visual assets
-
Conversation design
-
Authentication
In-car Testing
-
3 Authentication
-
2 Conversation flows
-
Driving VS Stationary
Strategy
-
Guideline for partners
-
Voice agentic APIs
-
Authentication framework
Resource and Planning
Assembling a Cross-Functional Team
After we received business buy-in for the initial proposal, we were able to get support from research, content, and design engineer team to execute the project.
We kicked off this project in close collaboration with the research and product team. Our goal is to deploy product strategies in this space by 2027.
Understand the Space
Initial Desk Research
Research report credits to Mariah and Halima
Early Design Exploration
Visualize the Initial Concept to Socialize with Stakeholders
In order to visualize the concept and socialize it with cross-functional teams, the first draft was designed in Figma. This helped drives alignment between research, content, products leads from three different departments.
After sharing the initial concept and conducting desk research, we achieved alignment and recieved business buy-in among three different product group: data tokens, agentic commerce, and in-car payments.
Designing, Writing, and Testing with AI
Prototyping with AI Tools after Mapping Out Key Moments
Later, research team and I decided to work on different components separately to maximize productivity. With the help of AI, we're able to quickly build a voice prototype without a user interface to test voice interactions. Meanwhile, I had been working on developing a working prototype with detailed visuals.
(Snapshot of the user journey map we created collaboratively)
The 2nd round of research includes experience mapping, comfort survey, and role play interviews. Based on the key learning from these three research, we were able to map out the step by step journey for next research iteration (See the screenshot above for reference).
From the role-play interviews, we have learned that:
-
Users want the entire experience to stay as hands free as possible
-
Authentication methods should be as unobtrusive to the driving experience as possible
-
The AI should guide, not force, user decisions
-
Users need to hear the entire order total, including taxes and fees, before getting prompted for consent and authentication.
From the survey we learned
-
73.3% of the participants would feel comfortable asking AI agent to purchase food for pickup/delivery
-
47.2% of participants feel comfortable with a maximum amount of $50
-
Participants felt that the most secure methods for authentication were: facial recognition, knowledge based confirmation, finger printer sensor, and a spoken PIN
From the experience mapping study we learned
-
The beginning and the end steps of an agentic voice experience were pretty locked down for participants.
-
However, the middle, especially regarding providing consent and authentication for the order (The Visa Moments), came out in two distinct ways: doing these steps All At Once or Step By Step.
With the assistants of AI, we were able to quickly build 6 working prototypes in 2 week
Designing, Writing, and Testing with AI
Using Replit, Figma Make, and Adobe Firefly to Rapidly Prototyping once the Design Architecture Was Established.
Our team shifted from the Figma prototype to a live, interactive prototype for next round of testing. We utilized a suite of advanced AI tools—including Replit, Adobe Firefly, and Claude Code—to build a function AI assistant.
Because this was our team's first time transitioning directly from UX design to AI-driven code development, we adopted a continuous, "learn-as-we-go" mindset. Over time, this experimentation solidified into a structured, repeatable workflow focus on three core areas: Visual Assets, Conversation Guidelines, Micro-Interactions.
Designing and Writing with AI
Part One: Developing the Visuals Assets
Generating, refining, and implementing dynamic UI elements and product imagery using AI tools like adobe firefly and Figma AI.
1.1 Utilizing Adobe Illustrator and Firefly to refine the visuals for the car assets on the dashboard.
1.2 Utilizing Figma AI and Replit to refine the design of the dashboard
Designing and Writing with AI
Part Two: Collaboration with Content Designer in Replit
Training and prompt-engineering the assistant to ensure helpful, accurate, and context-aware user interactions.
2.1 AI generates language easily. But tone takes craft.
Condescending
Direct but not blunt
Overly instructional
Helpful but not patronizing
Confident in the wrong moments
Conversational but not casual or robotic
Designing and Writing with AI
Part Three: Developing the Micro-Interactions and Trust-Centric Moments
Leveraging Claude Code and Replit to code fine-grained animations, feedback loops, and state changes that makes the experience feel smooth.
3.1 Prototyping different authentication methods
Provisioning flow: enable car as secure payment device
Authentication: Fingerprint + device token
Authentication: Passphrase + device token
Testing in Real World
We tested six AI prototypes across three different authentication methods under both driving and stationary conditions.
Map of driving loop
Prototype setup in the vehicle
(Above are some selected sections from the research report. Credit to: Mariah and Ipso researchers)
Testing the prototypes in both driving and stationary conditions provided deep insights into user preferences and behaviors. Ultimately, the system's success relies on three core pillars: earning trust, building confidence through clear confirmation, and demonstrating immediate value."
From designing screens to crafting conversations
One thing we have learned from live testing is that we are not just prototyping flows and logic, we're now prototyping conversations and micro-interactions.
Designing the foundational building blocks, not just individual flows and screens
Designing Building Block
Identify Key Design Moments: Shifting from Click-Thru Checkout to Conversational Checkout
Key moments we need to design for in each phase:
1. Discovery
-
Does this agent knows me and my preferences?
2. Checkout
-
What did I ordered?
-
What's the total price including taxes and fees?
3. Payment
-
Can I trust this agent with my payment information?
-
Which do I use?
-
Can I trust the merchant?
-
Can I edit my order or even get my money back if something went wrong?
4. Share/Abandon
-
Will I have consistent experience with this new technology?
-
Can I get my money back if something went wrong with my order?
-
How about dispute?
Designing Building Block
Developing Conversation Model: Consent and Authentication Happens Step-by-Step Reduce Cognitive Load
Experience 1: Consent and Authentication happen at the same time - ALL AT ONCE
-
You decide to order food for delivery to your home
-
You start the conversation with the agent
-
The agent confirms your identity
-
You tell the agent what you want to order
-
You and agent go back and forth to get your specific order
-
The agent asks you if the order is correct
-
You confirm the items in the order
-
The agent shares the total price including tax and delivery fees, then the agent asks for explicit consent to place the order via your payment card by prompting you for authentication
-
You confirm with the agent to place the order for $X amount by completing an authentication step
-
The agent confirms that the order is placed and provides the delivery estimated time of arrival
-
You end the interaction with the agent
Experience 2: Consent and Authentication happen in separate steps - STEP BY STEP
-
You decide to order food for delivery to your home
-
You start the conversation with the agent
-
The agent confirms your identity
-
You tell the agent what you want to order
-
You and agent go back and forth to get your specific order
-
The agent asks you if the order is correct
-
You confirm the items in the order
-
The agent shares the total price including tax and delivery fees
-
The agent asks for explicit consent to place the order for $X amount
-
You confirm with the agent to place the order for $X amount
-
The agent prompts you for authentication to use your payment card
-
You complete an authentication step so the agent can use your payment card
-
The agent confirms that the order is placed and provides the delivery estimated time of arrival
-
You end the interaction with the agent
We tested 2 conversation structures in our live research. One interesting finding is that people prefer when consent and authentication happened all-in-once when given the option. But their driving capability was impacted when handling both together.
Designing Building Block
Lead with Voice, Use Concise Audio Cues, Include Minimal Visuals
Discovery: Find merchant based on people preference and route
Payment: Utilized card art as visual cue for payment info.
Authentication: Allow consumer choose their preferred authentication during setup. (The fingerprint authentication UI in the demo is still working in progress)
Order review: Both visual and voice reviews of the order.
Order completion: Once order is confirmed and completed, user will be able to view it in the wallet and email.
Strategies and Next Step
Contextual awareness is the foundation: Integrating AI with driving behavior and real-time traffic data, and users' shopping preference maximizes system value.
Seamless authentication: Establish a default authentication method during the provisioning flow, and provide a reliable fallback option if the default fails.
Risk-mitigated deployment: To address critical in-car purchasing safety concerns, the initial pilot should be restricted to stationary vehicle states.
Multi-modal ecosystem expansion: Investigate wearable and VR hardware integrations to scale agentic commerce beyond traditional in-car screens.