Multimodal AI Will Change How We Design Digital Experiences

๐Ÿ“… March 21, 2024

Most software interfaces still assume that users will type, click and navigate screens. Advances in multimodal AI point toward a broader model in which people can communicate through combinations of language, images and voice depending on what is most natural for the task.

A useful way to evaluate any emerging technology is to ask what it changes in the operating model, not only what it can demonstrate.

The interface can move closer to the problem

A technician may want to show a device rather than describe it. A customer may prefer to speak. A designer may want to combine a sketch with written instructions. Multimodal systems can reduce the translation work users perform before software can help them.

The important point is not to maximize novelty. It is to create a system that people can understand, operate and improve. That is how emerging technology becomes durable business capability.

More modalities create more responsibility

Voice, image and video introduce additional privacy, consent, accessibility and authenticity concerns. Product teams should think about data capture and retention before adding a modality simply because it is technically possible.

The important point is not to maximize novelty. It is to create a system that people can understand, operate and improve. That is how emerging technology becomes durable business capability.

Design should remain intentional

Natural interaction does not mean the disappearance of structure. Users still need confirmation, control and clear boundaries. The best multimodal experiences will combine flexibility with predictable interaction patterns.

The important point is not to maximize novelty. It is to create a system that people can understand, operate and improve. That is how emerging technology becomes durable business capability.

Three Questions for Leaders

  • Which modality is most natural for the userโ€™s task?
  • What new privacy or consent issues appear?
  • How will the interface preserve control and predictability?

Multimodal AI is not only a model capability. It is a product design opportunity. The challenge is to make software more natural without making it less understandable.

References & Sources:

  • Evergreen leadership perspective; no single product announcement is required for context.
  • Platform Engineering: The Quiet Discipline Behind Faster Software Teams