v1.0
This commit is contained in:
Pawel Huryn
@@ -0,0 +1,99 @@
|
||||
---
|
||||
description: Perform cohort analysis on user data — retention curves, feature adoption, and engagement trends
|
||||
argument-hint: "<data file or description of what to analyze>"
|
||||
---
|
||||
|
||||
# /analyze-cohorts -- Cohort Analysis
|
||||
|
||||
Analyze user retention and engagement patterns by cohort. Upload your data or describe what you need, and get retention curves, feature adoption trends, and actionable insights.
|
||||
|
||||
## Invocation
|
||||
|
||||
```
|
||||
/analyze-cohorts [upload a CSV of user activity data]
|
||||
/analyze-cohorts Monthly retention for users who signed up in Jan-Jun, grouped by acquisition channel
|
||||
/analyze-cohorts Help me set up a cohort analysis for our onboarding redesign
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Accept Data or Define Analysis
|
||||
|
||||
Two paths:
|
||||
- **With data**: User uploads a CSV/spreadsheet with user-level data (user_id, signup_date, activity_date, event_type, etc.)
|
||||
- **Without data**: User describes the analysis they need → generate the SQL query and analysis framework
|
||||
|
||||
### Step 2: Define Cohorts
|
||||
|
||||
Ask:
|
||||
- What defines a cohort? (signup week/month, acquisition channel, plan tier, first feature used)
|
||||
- What is the retention event? (login, core action, any activity, purchase)
|
||||
- What time granularity? (daily, weekly, monthly)
|
||||
- What time range?
|
||||
|
||||
### Step 3: Analyze
|
||||
|
||||
Apply the **cohort-analysis** skill:
|
||||
|
||||
**If data is provided:**
|
||||
- Process the data using Python (pandas) to create cohort tables
|
||||
- Calculate retention rates per cohort per period
|
||||
- Generate retention curves
|
||||
- Identify patterns: improving/declining cohorts, seasonal effects, anomalies
|
||||
- Compare feature adoption across cohorts
|
||||
|
||||
**If describing an analysis:**
|
||||
- Design the cohort analysis framework
|
||||
- Generate SQL queries to extract the data
|
||||
- Create a template spreadsheet for the analysis
|
||||
- Define the metrics and visualization approach
|
||||
|
||||
### Step 4: Generate Report
|
||||
|
||||
```
|
||||
## Cohort Analysis: [Description]
|
||||
|
||||
**Date**: [today]
|
||||
**Cohort definition**: [e.g., signup month]
|
||||
**Retention event**: [e.g., completed a project]
|
||||
**Granularity**: [weekly/monthly]
|
||||
|
||||
### Retention Table
|
||||
| Cohort | Size | Week 1 | Week 2 | Week 3 | ... | Week 12 |
|
||||
|--------|------|--------|--------|--------|-----|---------|
|
||||
|
||||
### Key Findings
|
||||
1. **[Finding]** — [supporting data]
|
||||
2. ...
|
||||
|
||||
### Cohort Comparison
|
||||
- **Best-performing cohort**: [which, why]
|
||||
- **Worst-performing cohort**: [which, why]
|
||||
- **Trend**: [improving/declining/stable over time]
|
||||
|
||||
### Retention Benchmarks
|
||||
| Period | Your Rate | Industry Benchmark | Gap |
|
||||
|--------|----------|-------------------|-----|
|
||||
|
||||
### Recommendations
|
||||
1. [What to investigate or change based on findings]
|
||||
2. ...
|
||||
|
||||
### Follow-Up Queries
|
||||
[SQL queries for deeper investigation]
|
||||
```
|
||||
|
||||
If data was provided, save analysis as both markdown report and CSV/spreadsheet.
|
||||
|
||||
### Step 5: Offer Next Steps
|
||||
|
||||
- "Want me to **segment this further** by another dimension?"
|
||||
- "Should I **set up metrics alerts** based on these retention thresholds?"
|
||||
- "Want me to **design experiments** to improve retention for the weakest cohort?"
|
||||
|
||||
## Notes
|
||||
|
||||
- Cohort analysis is only as good as the retention event definition — push for a meaningful action, not just "logged in"
|
||||
- Early cohorts often look different due to founding user bias — note this when comparing
|
||||
- If retention is calculated using a Python script, save the script so the user can re-run with new data
|
||||
- Seasonal effects can masquerade as trends — flag if cohort differences might be calendar-driven
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
description: Analyze A/B test results — statistical significance, sample size validation, and ship/extend/stop recommendations
|
||||
argument-hint: "<test results as data, screenshot, or description>"
|
||||
---
|
||||
|
||||
# /analyze-test -- A/B Test Analysis
|
||||
|
||||
Evaluate experiment results with statistical rigor and translate findings into a clear product decision: ship, extend, or stop.
|
||||
|
||||
## Invocation
|
||||
|
||||
```
|
||||
/analyze-test Control: 4.2% conversion (n=5000), Variant: 4.8% conversion (n=5100)
|
||||
/analyze-test [upload a CSV of test results]
|
||||
/analyze-test [screenshot from your experimentation platform]
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Accept Test Data
|
||||
|
||||
Accept in any format:
|
||||
- Summary statistics (conversion rates, sample sizes per variant)
|
||||
- Raw event data (CSV with user_id, variant, converted, timestamp)
|
||||
- Screenshot from an experimentation platform (Optimizely, LaunchDarkly, etc.)
|
||||
- Description of the experiment and results
|
||||
|
||||
### Step 2: Validate Test Design
|
||||
|
||||
Before analyzing results, check:
|
||||
- Was sample size sufficient? (run a power analysis)
|
||||
- Was the test run long enough? (capture weekly cycles, minimum 1-2 business cycles)
|
||||
- Was randomization clean? (check for sample ratio mismatch)
|
||||
- Were there any external factors during the test period?
|
||||
|
||||
Flag issues if found — results from a flawed test can be misleading.
|
||||
|
||||
### Step 3: Analyze Results
|
||||
|
||||
Apply the **ab-test-analysis** skill:
|
||||
|
||||
- **Statistical significance**: Calculate p-value and confidence interval
|
||||
- **Effect size**: Absolute and relative difference between variants
|
||||
- **Practical significance**: Is the effect large enough to matter for the business?
|
||||
- **Confidence interval**: What's the range of plausible true effects?
|
||||
- **Segment analysis**: If data allows, check for differential effects by user segment
|
||||
|
||||
### Step 4: Generate Analysis
|
||||
|
||||
```
|
||||
## A/B Test Analysis: [Test Name]
|
||||
|
||||
**Date**: [today]
|
||||
**Test duration**: [X days/weeks]
|
||||
**Total sample**: [N users]
|
||||
|
||||
### Results Summary
|
||||
| Variant | Sample | Metric | Rate | 95% CI |
|
||||
|---------|--------|--------|------|--------|
|
||||
| Control | [n] | [metric] | [X%] | [X% - Y%] |
|
||||
| Variant | [n] | [metric] | [X%] | [X% - Y%] |
|
||||
|
||||
### Statistical Analysis
|
||||
- **Relative lift**: [+X%] ([CI range])
|
||||
- **P-value**: [X]
|
||||
- **Statistically significant**: [Yes/No] at 95% confidence
|
||||
- **Minimum detectable effect**: [X%] (what the test was powered to detect)
|
||||
|
||||
### Sample Size Check
|
||||
- **Required sample**: [N] per variant (for [X%] MDE at 80% power)
|
||||
- **Actual sample**: [N] per variant
|
||||
- **Verdict**: [Sufficiently powered / Underpowered / Overpowered]
|
||||
|
||||
### Decision
|
||||
|
||||
**Recommendation: [SHIP / EXTEND / STOP]**
|
||||
|
||||
[Clear explanation of why, considering both statistical and practical significance]
|
||||
|
||||
### Business Impact Estimate
|
||||
If shipped to 100% of users:
|
||||
- **Expected impact**: [metric change per month/quarter]
|
||||
- **Revenue impact**: [if applicable]
|
||||
- **Confidence**: [How certain we are about this estimate]
|
||||
|
||||
### Caveats
|
||||
- [Any concerns about the test validity]
|
||||
- [Segments where results differ]
|
||||
- [Novelty effects or other biases to consider]
|
||||
|
||||
### Follow-Up
|
||||
- [What to test next based on learnings]
|
||||
- [Monitoring plan if shipping the variant]
|
||||
```
|
||||
|
||||
### Step 5: Offer Next Steps
|
||||
|
||||
- "Want me to **design a follow-up experiment** based on these findings?"
|
||||
- "Should I **run the analysis for specific segments**?"
|
||||
- "Want me to **generate the SQL** to monitor this metric post-launch?"
|
||||
|
||||
## Notes
|
||||
|
||||
- Statistical significance ≠ practical significance — a 0.1% lift can be significant with enough data but not worth shipping
|
||||
- Always check for sample ratio mismatch before trusting results
|
||||
- Novelty effects can inflate short-term results — recommend monitoring for 2-4 weeks post-launch
|
||||
- If the test is underpowered, the right answer is usually "extend" not "no effect"
|
||||
- For revenue metrics, use confidence intervals to estimate best-case and worst-case business impact
|
||||
- If data is provided as CSV, generate the full analysis using Python with scipy.stats
|
||||
@@ -0,0 +1,84 @@
|
||||
---
|
||||
description: Generate SQL queries from natural language — supports BigQuery, PostgreSQL, MySQL, and more
|
||||
argument-hint: "<what you want to know, in plain English>"
|
||||
---
|
||||
|
||||
# /write-query -- SQL Query Generator
|
||||
|
||||
Describe what data you need in plain English and get an optimized SQL query. Supports multiple dialects and can read your schema from uploaded files.
|
||||
|
||||
## Invocation
|
||||
|
||||
```
|
||||
/write-query Show me daily active users for the last 30 days, broken down by plan tier
|
||||
/write-query Find users who signed up last month but never completed onboarding
|
||||
/write-query [upload a schema diagram] What's the conversion rate from trial to paid by cohort?
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Understand the Question
|
||||
|
||||
Parse the user's natural language request to identify:
|
||||
- What data is being requested (metrics, dimensions, filters)
|
||||
- Time range and granularity
|
||||
- Grouping and ordering preferences
|
||||
- Output expectations (raw data, aggregated, ranked)
|
||||
|
||||
### Step 2: Determine Schema
|
||||
|
||||
If a schema is available (uploaded diagram, DDL, or description):
|
||||
- Map the request to specific tables and columns
|
||||
- Identify necessary joins
|
||||
|
||||
If no schema is provided:
|
||||
- Ask for the database type (BigQuery, PostgreSQL, MySQL, etc.)
|
||||
- Infer a reasonable schema from the question and ask the user to confirm
|
||||
- Use common SaaS data model conventions as defaults
|
||||
|
||||
### Step 3: Generate Query
|
||||
|
||||
Apply the **sql-queries** skill:
|
||||
|
||||
- Write the SQL query in the correct dialect
|
||||
- Optimize for readability and performance
|
||||
- Include comments explaining key logic
|
||||
- Add CTEs for complex queries to improve readability
|
||||
- Handle edge cases (NULLs, timezone considerations, duplicate handling)
|
||||
|
||||
### Step 4: Present and Iterate
|
||||
|
||||
```
|
||||
## SQL Query: [What It Does]
|
||||
|
||||
**Dialect**: [BigQuery / PostgreSQL / MySQL / etc.]
|
||||
**Tables used**: [list]
|
||||
|
||||
### Query
|
||||
[SQL code block with comments]
|
||||
|
||||
### What This Returns
|
||||
[Description of the output: columns, rows, expected result shape]
|
||||
|
||||
### Assumptions
|
||||
- [Schema assumptions made]
|
||||
- [Business logic assumptions]
|
||||
|
||||
### Notes
|
||||
- [Performance considerations for large datasets]
|
||||
- [Edge cases handled or flagged]
|
||||
```
|
||||
|
||||
Offer:
|
||||
- "Want me to **modify this** — add filters, change grouping, extend the time range?"
|
||||
- "Should I **create a companion query** for a related metric?"
|
||||
- "Want me to **build a dashboard** around this query?"
|
||||
- "Need a **cohort analysis** version of this?"
|
||||
|
||||
## Notes
|
||||
|
||||
- Always include comments in the SQL — PMs share queries with analysts who need to understand intent
|
||||
- Default to readable over clever — CTEs over nested subqueries
|
||||
- Flag queries that might be slow on large datasets and suggest optimization
|
||||
- If the request is ambiguous (e.g., "active users"), ask the user to define the metric precisely
|
||||
- Offer to generate the query in multiple dialects if the user is unsure which database they're using
|
||||
Reference in New Issue
Block a user