name: experiment-designer description: Designs A/B tests from hypotheses and interprets experiment results with statistical rigour. Use when user says "run an experiment", "design an A/B test", "test this feature", "interpret these results", "was this experiment successful", or "what sample size do I need". metadata: author: Mohit Aggarwal version: 1.0.0 category: data-and-metrics tags: [experimentation, data, analytics, ab-testing] documentation: https://github.com/mohitagw15856/pm-claude-skills

Experiment Designer Skill

Purpose

Produce rigorous experiment designs from product hypotheses, and interpret results with statistical and practical significance — so you can defend every decision to a sceptical engineering lead or data scientist.

Two-Phase Process

Phase 1: Experiment Design

Required inputs: hypothesis, primary metric, current baseline, minimum detectable effect (MDE), available sample size per day.

Output:

Hypothesis restated as: "If we [change], we expect [metric] to [move by X%] because [reason]"
Control and variant definitions
Primary metric (one only)
Secondary guardrail metrics (2-3 max)
Required sample size (calculated from MDE and baseline)
Estimated run time in days
Pre-defined success criteria (before the test runs — no moving goalposts)
Design risk flags: novelty effects, seasonal confounds, multiple testing issues, network effects, sample ratio mismatch risks

Phase 2: Results Interpretation

Required inputs: control results, variant results, p-value or raw numbers, run duration, any anomalies observed.