Shape
Multimodal Interaction
Designing interactions that combine speech, touch, gesture or images into one clear exchange.
What is Multimodal Interaction?
Designing interactions that combine speech, touch, gesture or images into one clear exchange.
What you learn
When speech and pointing disagree, confirm instead of guessing.
When it helps
people combine speech, touch or gestures, and the system must work out what they mean.
Use it at work
- List the ways people can give input.
- Decide how to combine them when they agree.
- When they disagree, ask instead of guessing.
- Keep the context when people switch from one to another.
Your output
List the moments where your product combines inputs, such as voice, touch and gaze. Decide what happens when they disagree.
Watch for
Silently picking one input when two disagree.
Related practices
- Multimodal Alternatives
- Conversational Interaction
- State Modeling
- Uncertainty Communication
- Spatial Interaction
Sources
Enable JavaScript to try the interactive activity.