AI UX PlaygroundNewsletterJoin 2K+ AI designers and PMs on Substack. New teardowns, patterns, and prompts as they drop.

Inputs

Multimodal Input

Send images, files, or audio with text in one turn. Show what is attached, how it will be used, and any limits, so people are not stuck in a separate upload flow.

Interactive demo

signup-screen.pngImage · ready

Overview

How might we let people combine text with images or files in one turn so the model sees both, without a separate upload-only mode?

When to use

  • ChatGPT, Gemini, Claude, and Grok composers where screenshots, photos, or docs ride with the prompt.
  • Critique, debugging, and research turns that need the model to see the artifact, not a description of it.
  • Products that attach media in the same composer as the question, with preview before send.
  • Flows that must show type, size, and remove so people know what context will ship.

When to skip

  • Text-only products with no vision, document, or audio capability.
  • Tiny mobile composers where attachment UI crowds send and voice controls.
  • Strict offline tools that cannot process media and would only show dead ends.

States

Design attach → preview → caption → send as one composer, not two apps.

  1. 01

    Empty

    Composer is text-only. Attach lives beside the input (+ or paperclip) without forcing a mode switch.

  2. 02

    Attaching

    The person picks an image or file. Upload or camera capture may still be in progress.

  3. 03

    Previewed

    A thumbnail or type chip shows what will be sent, with filename and remove. Text can be typed beside it.

  4. 04

    Composing

    Prompt and media coexist. Limits (type, size, count) are visible before send fails.

  5. 05

    Sent

    The turn lands with media and text together in the thread so history matches what the model received.

  6. 06

    Failed

    Unsupported type, size, or upload error surfaces on the attachment, not as a silent drop after send.

Key UX elements

The parts that must be present for media-plus-text turns to feel trustworthy.

Attach

Keep media entry in the composer.

A + or paperclip next to the input beats a separate upload screen that breaks the prompt.

Preview

Show what the model will see.

Thumbnail for images, type icon and name for files. Mystery attachments invite wrong questions.

Caption

Let text and media share one turn.

The prompt field stays available so people can ask about the attachment without leaving compose.

Remove

Make discard obvious on the preview.

Clearing media must not wipe the typed question unless the person clears that too.

Limits

Disclose type, size, and count early.

Reject unsupported media before send with a clear reason, not after a long failed turn.

Thread

Keep media visible in history.

Sent turns should show the attachment with the question so later readers know the grounding.

Rules

  • Accepting files with no preview, type label, or remove control.

  • Silent truncation or resizing with no size or format limits shown.

  • Treating attachments as optional chrome when the model cannot see them.

  • Forcing a separate “upload mode” instead of composing media with the prompt.

Evidence

ProductImplementation
ChatGPTImage and file attach in the composer with removable chips before send.
GeminiPhotos, Drive, and camera alongside text; multimodal turns in the thread.
ClaudeDocument and image attach with project/context files for longer briefs.
PerplexityUpload or paste media into search/ask flows with source-aware answers.

Real-world examples

See all

FAQ

What is multimodal input in AI UX?

Multimodal input lets users combine text with images, files, audio, or other media in a single request so the model can reason across formats instead of text alone.

How should attachments appear in the composer?

Show a preview or type chip, filename, and remove control for every attachment before send. Users must see exactly what context the model will receive.

Is voice a multimodal input pattern?

Voice can be, when speech is one channel among others in the same turn. Dedicated voice-only mode is usually a separate input-mode pattern rather than multimodal attach.

What limits should products disclose?

File types, count, size, and whether vision or OCR is available. Hidden rejections after send feel like product failures, not user mistakes.