AI UX PlaygroundNewsletterJoin 2K+ AI designers and PMs on Substack. New teardowns, patterns, and prompts as they drop.

Marketing

Write A/B test hypotheses

Copy-paste experimentation prompt for clear hypotheses, metrics, and test designs.

Prompt

Create an A/B test hypothesis for [feature/change].

Context:
- What you want to test: [describe]
- Current behavior: [describe]
- Proposed change: [describe]
- Business goal: [describe]

Provide:

1. Hypothesis Statement
   - Clear hypothesis format:
     "We believe that [change] will [expected outcome] for [user segment] because [rationale]"
   - Alternative hypothesis
   - Null hypothesis

2. Test Design
   - Control variant (current)
   - Treatment variant (proposed change)
   - Key differences between variants
   - What's being tested

3. Success Metrics
   - Primary metric (what determines success)
   - Secondary metrics
   - Guardrail metrics (what shouldn't change)
   - Statistical significance threshold

4. User Segmentation
   - Who will see the test
   - Segmentation criteria
   - Sample size requirements
   - Exclusion criteria

5. Implementation Plan
   - Technical requirements
   - Design requirements
   - Content requirements
   - Timeline and duration

6. Risk Assessment
   - Potential risks
   - Mitigation strategies
   - Rollback plan
   - Monitoring approach

7. Success Criteria
   - What constitutes a win
   - What constitutes a loss
   - What requires further testing
   - Decision framework

8. Analysis Plan
   - How to analyze results
   - Statistical methods
   - When to end the test
   - Reporting structure

Format as a complete A/B test plan ready for implementation.

How to use

  1. 01

    Replace [feature/change] and [describe] placeholders with your test details

  2. 02

    Describe what you want to test: Replace [What you want to test] with your test idea (e.g., "New checkout button color" or "Simplified form layout")

  3. 03

    Describe current behavior: Replace [Current behavior] with current design/behavior

  4. 04

    Describe proposed change: Replace [Proposed change] with the new design/behavior you want to test

  5. 05

    Describe business goal: Replace [Business goal] with what you want to achieve (e.g., "Increase checkout completion rate by 15%")

  6. 06

    Add context before the prompt: Describe your product and target users. Example: "We're an e-commerce site. Target users: Mobile shoppers. Goal: Reduce cart abandonment."

  7. 07

    Paste the modified prompt into your preferred AI tool, like ChatGPT or Claude

  8. 08

    Review the test plan: Check hypothesis statement, test design, success metrics, and implementation plan

  9. 09

    Verify success criteria: Ensure primary metric aligns with your business goal

  10. 10

    Export to your tool: Copy the A/B test plan to Optimizely, VWO, or your experimentation platform

Tips

  • Include baseline metrics: Mention current metrics (e.g., "Current checkout rate: 45%") so AI can set appropriate targets
  • Specify user segment: Mention target user segment (e.g., "Mobile users" or "First-time visitors") for targeted testing
  • Request sample size: Ask "Calculate required sample size for 80% power and 95% confidence" if needed
  • For complex tests: Break down complex tests into simpler variants: "Create hypothesis for checkout button test only"
  • Save test plan: Reuse the test plan structure for future A/B tests