Gesture Input

Gesture input is an AI UX pattern that interprets drawn shapes, strokes, or hand gestures as commands or search queries. It suits touch and pen interfaces where sketching is faster than menus for spatial or creative tasks, and needs visible feedback plus undo when recognition misfires.

Share

Interactive demo

What’s confusing on this signup?
Circle what feels unclear and I’ll explain it. Scribble to rewrite.
Circle here

Overview

The design problem

How might we design gesture input so people can trust and act on AI output?

Use this pattern

When this pattern fits

  • Ideal for touch interfaces, creative applications, and mobile tools where gesture-based input provides intuitive, natural interaction methods.

Avoid this pattern

When to skip or lighten it

  • Precision tasks better served by keyboard shortcuts.
  • Accessibility requirements mandate non-gesture alternatives.
  • Gesture recognition latency breaks drawing flow.

States

State model coming soon

Key UX elements

Key UX elements coming soon

Anti-patterns to avoid

  • Gestures with no on-screen legend or tutorial.

  • Conflicts with OS-level system gestures.

  • No undo when gesture misfires.

  • Requiring perfect shapes with no tolerance.

How products use it

ProductImplementation
Apple PencilScribble and lasso selection in Notes and markup.
Samsung S PenAir actions and circle-to-search gestures.
Surface PenInk to shape and gesture shortcuts in Windows ink.
Creative appsCircle lasso for AI edit regions on canvas.

Implementation

Copy this prompt to generate a production-ready implementation in Cursor, Claude Code, Lovable, or any AI coding agent.

Generate a production-ready implementation of the "Gesture Input" AI interface design pattern.

Pattern Definition:

Frequently asked questions

What gestures work for AI actions?

Circle to search, lasso to inpaint, swipe to dismiss suggestions, pinch to zoom canvas regions.

Feedback during gesture?

Show live stroke, recognized intent label, and cancel zone.

Gesture vs voice commands?

Gestures are spatial on touch or pen. Voice commands are spoken action triggers.

Discoverability?

First-run overlay, settings cheatsheet, and hover hints on pen hover where supported.

Weekly AI UX in your inbox

Weekly AI interface UX notes and resources on Substack, no spam, unsubscribe anytime.

Subscribe on Substack