Windsurf + Aider targets, MCP server, and demo placement (#33)
Broadens both reach (more tools) and content types (an MCP server), continuing the multi-platform story. Windsurf + Aider: - build-exports.mjs gains two platforms: exports/windsurf/*.md (workspace rules, trigger: model_decision) and exports/aider/*.md (conventions for `aider --read`). Now 5 platforms (ChatGPT, Gemini, Cursor, Windsurf, Aider). - install.sh + bin/cli.mjs install both (windsurf -> .windsurf/rules, aider -> .aider/skills with a --read hint); generated README index is excluded from copies. - One-line windsurf-install.sh / aider-install.sh wrappers for parity. MCP server (new content type): - mcp/server.mjs — zero-dependency stdio MCP server exposing list_skills, search_skills, get_skill. Published as a second bin (pm-claude-skills-mcp). Logs to stderr; reads bundled skills/ at startup. mcp/README.md documents client config. Also: README hero "See it in action" demo placement (ready to swap in a GIF; recording guide in web/docs-assets/README.md), Works-With table + exports + install docs updated, CHANGELOG Unreleased. package.json files/bin updated. Claude-Session: https://claude.ai/code/session_016JWn5jRD5tcEFKrubjQ6Px Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
mohitagw15856
GitHub
@@ -0,0 +1,210 @@
|
||||
# AI Ethics Review Skill
|
||||
|
||||
This skill produces a structured ethical review of an AI or machine learning feature, model, or product. Output covers fairness, transparency, privacy, safety, accountability, and societal impact — with risk scoring, prioritised mitigations, and a checklist suitable for governance review or responsible AI documentation.
|
||||
|
||||
> ⚠️ This skill provides a structured framework for identifying and documenting ethical risks. It is not a substitute for legal advice, regulated algorithmic impact assessments, or specialist ethics review required in specific jurisdictions (e.g. EU AI Act, UK AI regulation).
|
||||
|
||||
## Required Inputs
|
||||
|
||||
Ask the user for these if not provided:
|
||||
- **Feature or model name** and what it does
|
||||
- **Who it affects** — which users or people does the AI interact with, make decisions about, or collect data from?
|
||||
- **What decisions or outputs it produces** — recommendations, predictions, classifications, generation, automation?
|
||||
- **Consequentiality** — how significant are the AI's decisions? (low-stakes suggestions vs decisions that affect employment, credit, health, safety, etc.)
|
||||
- **Data used** — what training data, user data, or third-party data is used?
|
||||
- **Human oversight** — is there a human in the loop, and at what stage?
|
||||
- **Deployment context** — who will use this and how? (internal tool / consumer-facing / automated pipeline)
|
||||
|
||||
## Output Structure
|
||||
|
||||
---
|
||||
|
||||
# AI Ethics Review: [Feature / Model Name]
|
||||
|
||||
**Product / system:** [Name and brief description]
|
||||
**Review type:** [Pre-deployment review / Post-deployment audit / Change review]
|
||||
**Risk tier:** [High / Medium / Low — based on consequentiality, scale, and affected population]
|
||||
**Reviewer:** [Name / Team]
|
||||
**Date:** [Date]
|
||||
**Status:** [Draft / Approved / Requires escalation]
|
||||
|
||||
---
|
||||
|
||||
## 1. Feature Summary
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **What it does** | [1–2 sentences — plain English description of the AI feature and its purpose] |
|
||||
| **Who uses it** | [End users / internal teams / automated system] |
|
||||
| **Who is affected by its outputs** | [May be different from who uses it — e.g. an AI hiring tool is used by HR but affects candidates] |
|
||||
| **Output type** | [Recommendation / Classification / Prediction / Generation / Automation / Scoring] |
|
||||
| **Scale** | [How many people affected per day/month?] |
|
||||
| **Consequentiality** | [High: affects access to services, employment, credit, health, safety / Medium: influences decisions / Low: suggestions with easy override] |
|
||||
| **Human oversight level** | [Full automation / Human review before action / Human can override after action / Advisory only] |
|
||||
|
||||
---
|
||||
|
||||
## 2. Risk Tier Assessment
|
||||
|
||||
| Factor | Score (1–3) | Rationale |
|
||||
|---|---|---|
|
||||
| **Consequentiality** (impact on individuals) | [1=low, 3=high] | [e.g. 3 — model output influences hiring decisions] |
|
||||
| **Scale** (number of people affected) | [1=few, 3=many] | [e.g. 2 — internal tool used for ~500 candidates/year] |
|
||||
| **Reversibility** (can harm be undone?) | [1=reversible, 3=irreversible] | [e.g. 2 — unfair rejection can be appealed but may not be caught] |
|
||||
| **Vulnerability of affected group** | [1=general population, 3=protected or vulnerable group] | [e.g. 2 — includes protected characteristics in the decision context] |
|
||||
| **Transparency** (do affected people know?) | [1=informed, 3=opaque] | [e.g. 3 — candidates are not told AI is used in screening] |
|
||||
|
||||
**Composite risk tier:** [High (12–15) / Medium (7–11) / Low (3–6)]
|
||||
|
||||
**Risk tier implications:**
|
||||
- **High:** Mandatory senior ethics review, DPA/DPIA required, human-in-loop for all consequential decisions, ongoing monitoring required
|
||||
- **Medium:** Ethics review recommended, document mitigations, quarterly monitoring
|
||||
- **Low:** Standard review, document assumptions, annual review
|
||||
|
||||
---
|
||||
|
||||
## 3. Fairness & Bias
|
||||
|
||||
*Does the AI treat people equitably across groups?*
|
||||
|
||||
**Protected characteristics relevant to this feature:**
|
||||
[List applicable protected characteristics — age, gender, race/ethnicity, disability, religion, national origin, etc.]
|
||||
|
||||
| Risk | Analysis | Mitigation |
|
||||
|---|---|---|
|
||||
| **Training data bias** | [Does the training data reflect historical discrimination? e.g. hiring data that reflects past biases in who was hired] | [Audit training data for demographic representation / use debiasing techniques / document data lineage] |
|
||||
| **Proxy discrimination** | [Could the model use a proxy for a protected characteristic? e.g. using postcode as a proxy for race] | [Identify proxy features / test for disparate impact using adversarial debiasing] |
|
||||
| **Differential performance** | [Does the model perform differently across demographic groups? — e.g. lower accuracy for underrepresented groups] | [Disaggregate performance metrics by group / set minimum performance thresholds per group] |
|
||||
| **Feedback loops** | [Does the model's output reinforce existing disparities? e.g. recommending content that keeps disadvantaged groups in lower-engagement patterns] | [Monitor outcome distributions over time / implement feedback loop detection] |
|
||||
|
||||
**Fairness evaluation method:** [What method will be used to measure fairness — statistical parity / equalised odds / individual fairness? Who is responsible for running it and how often?]
|
||||
|
||||
---
|
||||
|
||||
## 4. Transparency & Explainability
|
||||
|
||||
*Can affected people understand how the AI makes decisions?*
|
||||
|
||||
| Dimension | Current state | Required state | Gap |
|
||||
|---|---|---|---|
|
||||
| **User disclosure** | [Are users told they're interacting with AI?] | [Yes — required for trust and regulation] | [e.g. No disclosure on current UI] |
|
||||
| **Decision explanation** | [Can the system explain why it reached a conclusion?] | [For high-stakes decisions: yes] | [e.g. Black-box model — no feature attribution available] |
|
||||
| **Right to know** | [Can affected people ask how a decision was made?] | [Yes — required under GDPR Art. 22 for automated decisions] | [e.g. No process exists] |
|
||||
| **Confidence calibration** | [Does the model express appropriate uncertainty?] | [Yes — overconfident models cause over-reliance] | [e.g. Model outputs binary label without confidence score] |
|
||||
|
||||
**Explainability approach:** [LIME / SHAP / rule-based surrogate / LLM-generated rationale / none — and why]
|
||||
|
||||
---
|
||||
|
||||
## 5. Privacy & Data
|
||||
|
||||
*Is personal data used responsibly and lawfully?*
|
||||
|
||||
| Risk | Analysis | Mitigation |
|
||||
|---|---|---|
|
||||
| **Data minimisation** | [Does the model use more personal data than necessary?] | [Audit input features — remove any that don't improve performance and involve unnecessary data collection] |
|
||||
| **Data retention** | [How long is personal data retained for training and inference?] | [Define retention policy aligned to GDPR / CCPA / sector requirements] |
|
||||
| **Re-identification risk** | [Could model outputs or training data be used to identify individuals?] | [Differential privacy / k-anonymity / output rate limiting] |
|
||||
| **Third-party data** | [Is data from third parties used? Is it licensed for this use?] | [Audit data licensing / get legal sign-off on each third-party source] |
|
||||
| **Cross-border data transfer** | [Is personal data transferred across jurisdictions?] | [Legal review — Standard Contractual Clauses or equivalent] |
|
||||
|
||||
**DPIA required?** [Yes / No / Uncertain — for High tier or whenever processing is likely to result in high risk to individuals under GDPR Art. 35]
|
||||
|
||||
---
|
||||
|
||||
## 6. Safety & Reliability
|
||||
|
||||
*What happens when the AI gets it wrong?*
|
||||
|
||||
| Failure mode | Likelihood | Impact | Mitigation |
|
||||
|---|---|---|---|
|
||||
| **False positives** | [H/M/L] | [e.g. Flagging a legitimate transaction as fraud — customer locked out] | [Set threshold conservatively; human review for edge cases] |
|
||||
| **False negatives** | [H/M/L] | [e.g. Missing a real fraud case — financial loss] | [Monitor false negative rate; set minimum recall threshold] |
|
||||
| **Out-of-distribution inputs** | [H/M/L] | [Model behaves unpredictably on inputs outside training distribution] | [Input validation; confidence thresholding — route uncertain inputs to human review] |
|
||||
| **Model degradation** | [M] | [Performance degrades as data distributions shift post-deployment] | [Scheduled performance monitoring; drift detection alerts] |
|
||||
| **Adversarial inputs** | [L/M] | [Deliberate manipulation of inputs to game the model] | [Adversarial testing; rate limiting; anomaly detection on inputs] |
|
||||
| **Single point of failure** | [L/M] | [Model outage causes downstream system failure] | [Graceful degradation — define fallback behaviour when model is unavailable] |
|
||||
|
||||
**Fallback behaviour:** [What happens if the AI is unavailable or returns low-confidence output? — e.g. route to human review / use rule-based fallback / block the action]
|
||||
|
||||
---
|
||||
|
||||
## 7. Accountability & Governance
|
||||
|
||||
*Who is responsible when things go wrong?*
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| **Who owns this AI feature?** | [Team or individual with end-to-end accountability] |
|
||||
| **Who approved deployment?** | [Name and role — must be documented] |
|
||||
| **Who is responsible for ongoing monitoring?** | [Team and cadence] |
|
||||
| **Who can shut it down?** | [Who has kill-switch authority and under what conditions?] |
|
||||
| **How are incidents reported?** | [Internal escalation path + external disclosure process if required] |
|
||||
| **Is this subject to regulation?** | [EU AI Act / UK AI regulation / sector-specific rules — FINRA, FDA, FCA, etc.] |
|
||||
|
||||
**Incident response plan:** [Link to or describe what happens if the model causes harm — detection, escalation, remediation, disclosure]
|
||||
|
||||
---
|
||||
|
||||
## 8. Societal Impact
|
||||
|
||||
*Beyond individual users — what are the broader effects?*
|
||||
|
||||
| Impact area | Risk | Mitigation |
|
||||
|---|---|---|
|
||||
| **Labour displacement** | [Does this AI automate tasks that currently employ people?] | [Transition plan / human-AI collaboration framing / skills retraining commitment] |
|
||||
| **Environmental impact** | [What is the carbon cost of training and inference?] | [Measure and offset; prefer efficient architectures; use renewable-energy infrastructure where possible] |
|
||||
| **Power concentration** | [Does this AI give the deploying organisation disproportionate power over individuals?] | [Ensure right to opt out; avoid lock-in; consider open alternatives] |
|
||||
| **Information ecosystem** | [Could this AI contribute to misinformation, filter bubbles, or manipulation?] | [Provenance labelling / content policies / algorithmic diversity requirements] |
|
||||
|
||||
---
|
||||
|
||||
## 9. Mitigation Priorities
|
||||
|
||||
| # | Risk | Severity | Action | Owner | Deadline |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | [Highest risk — e.g. No disclosure to affected candidates] | Critical | [Add AI disclosure to UI and candidate-facing documentation] | [PM + Legal] | [Before launch] |
|
||||
| 2 | [e.g. No fairness evaluation across demographic groups] | High | [Commission third-party fairness audit using [method]] | [ML team + external auditor] | [Within 30 days of launch] |
|
||||
| 3 | [e.g. No model monitoring in place] | High | [Deploy performance and drift monitoring dashboard] | [ML Ops] | [Launch day] |
|
||||
| 4 | [e.g. DPIA not completed] | High | [Complete DPIA with DPO before deployment] | [Legal / DPO] | [Before launch] |
|
||||
|
||||
---
|
||||
|
||||
## 10. Pre-Deployment Checklist
|
||||
|
||||
- [ ] Ethics review completed and approved by required reviewers
|
||||
- [ ] DPIA completed (if required)
|
||||
- [ ] Fairness evaluation completed and results documented
|
||||
- [ ] AI disclosure is in place wherever required
|
||||
- [ ] Human oversight mechanism is defined and tested
|
||||
- [ ] Kill-switch and escalation path is documented and tested
|
||||
- [ ] Model monitoring is deployed and alerting is configured
|
||||
- [ ] Data lineage and training data audit documented
|
||||
- [ ] Legal sign-off obtained on data licensing and cross-border transfers
|
||||
- [ ] Incident response plan in place
|
||||
|
||||
---
|
||||
|
||||
## Quality Checks
|
||||
|
||||
- [ ] "Who is affected" includes people the AI makes decisions *about*, not just who uses the product
|
||||
- [ ] Fairness analysis names specific protected characteristics, not just "diverse groups"
|
||||
- [ ] Safety section covers both false positive and false negative failure modes
|
||||
- [ ] Accountability section names real people, not teams or roles
|
||||
- [ ] Mitigations are specific and time-bound — not "monitor and review"
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- [ ] Do not limit the affected-population analysis to users of the product — AI that makes decisions about people (hiring, credit, content moderation) affects non-users who have no opt-out
|
||||
- [ ] Do not accept "we will monitor" as a mitigation without specifying what is monitored, at what threshold, and who acts
|
||||
- [ ] Do not assign fairness analysis to the model team alone — protected characteristic analysis requires input from legal, HR, or a subject-matter expert
|
||||
- [ ] Do not defer the DPIA to post-launch — for high-risk tier systems, a DPIA is a pre-requisite for lawful deployment under GDPR
|
||||
- [ ] Do not conflate statistical accuracy with fairness — a model can be 95% accurate overall while performing significantly worse for a protected group
|
||||
|
||||
## Example Trigger Phrases
|
||||
|
||||
- "Run an AI ethics review for [feature]"
|
||||
- "Conduct an ethical impact assessment for our new ML model"
|
||||
- "Review the AI risks for our hiring / credit / recommendation system"
|
||||
- "Build a responsible AI checklist for our product"
|
||||
- "What are the ethical risks of using AI for [use case]?"
|
||||
@@ -0,0 +1,164 @@
|
||||
# AI Product Canvas Skill
|
||||
|
||||
Define AI products with the same rigour as any product decision — but with additional layers for data, model, evaluation, and responsible AI. This canvas prevents the most common AI product failure: building a technically impressive feature that doesn't solve a real problem.
|
||||
|
||||
## AI Product Anti-Patterns to Check First
|
||||
|
||||
Before building, flag if any of these apply:
|
||||
- ❌ "We should add AI to [existing feature]" — with no user problem defined
|
||||
- ❌ Accuracy target undefined before build begins
|
||||
- ❌ No plan for what happens when the model is wrong
|
||||
- ❌ User-facing AI output with no human review or fallback
|
||||
- ❌ Training data not audited for bias or quality
|
||||
- ❌ No evaluation metric — "we'll know it when we see it"
|
||||
|
||||
---
|
||||
|
||||
## AI Product Canvas Output Format
|
||||
|
||||
### AI Product Canvas — [Feature Name] — [Date]
|
||||
|
||||
**PM Owner:** [Name]
|
||||
**ML/AI Lead:** [Name]
|
||||
**Status:** Discovery / Design / Build / Evaluation / Live
|
||||
|
||||
---
|
||||
|
||||
#### 1. Problem Definition
|
||||
**User problem being solved:**
|
||||
> [What specific situation is the user in? What job are they trying to get done?]
|
||||
|
||||
**Why AI?**
|
||||
> [What makes this problem require AI vs a deterministic solution? If the answer is "because we can," stop here.]
|
||||
|
||||
**Success for the user looks like:**
|
||||
> [What outcome does the user experience when the AI feature is working well?]
|
||||
|
||||
---
|
||||
|
||||
#### 2. AI Approach
|
||||
|
||||
**Task type:**
|
||||
- [ ] Classification
|
||||
- [ ] Generation (text, image, code)
|
||||
- [ ] Summarisation / extraction
|
||||
- [ ] Recommendation
|
||||
- [ ] Search / retrieval
|
||||
- [ ] Prediction / forecasting
|
||||
- [ ] Conversation / agent
|
||||
|
||||
**Model approach:**
|
||||
- [ ] LLM API (GPT-4, Claude, Gemini, etc.) — specify: [Model name + version]
|
||||
- [ ] Fine-tuned model on own data
|
||||
- [ ] Custom model trained from scratch
|
||||
- [ ] RAG (retrieval-augmented generation)
|
||||
- [ ] Embedding + vector search
|
||||
|
||||
**Rationale for chosen approach:** [Why this, not alternatives]
|
||||
|
||||
---
|
||||
|
||||
#### 3. Data Requirements
|
||||
|
||||
| Data Type | Source | Volume | Quality Status | Bias Risk |
|
||||
|---|---|---|---|---|
|
||||
| [Training data] | [Where it comes from] | [Volume] | [Audit status] | H/M/L |
|
||||
| [Evaluation data] | [Where it comes from] | [Volume] | [Audit status] | H/M/L |
|
||||
|
||||
**Data gaps:** [What's missing and plan to get it]
|
||||
**Privacy considerations:** [Any PII in training or inference data]
|
||||
**Data ownership:** [Do we own this data? Can we use it for training?]
|
||||
|
||||
---
|
||||
|
||||
#### 4. Evaluation Framework
|
||||
|
||||
**Primary metric:** [The number that defines success — accuracy, F1, BLEU, user rating, task completion rate]
|
||||
**Minimum acceptable threshold:** [Below X, the feature does not ship]
|
||||
**Human evaluation plan:** [How will humans review model outputs? Sampling rate? Review panel?]
|
||||
|
||||
| Evaluation Type | Method | Cadence | Owner |
|
||||
|---|---|---|---|
|
||||
| Offline (pre-launch) | [Test set, benchmark] | Pre-launch | ML Lead |
|
||||
| Online (post-launch) | [A/B test, user feedback] | Weekly | PM + ML |
|
||||
| Adversarial | [Red-team, edge cases] | Pre-launch | Safety reviewer |
|
||||
|
||||
---
|
||||
|
||||
#### 5. User Experience Design
|
||||
|
||||
**How is AI output presented?**
|
||||
- [ ] Direct output shown to user (high trust required)
|
||||
- [ ] AI-assisted with user confirmation
|
||||
- [ ] Suggestion user can accept/reject
|
||||
- [ ] Background action with audit log
|
||||
|
||||
**Confidence and uncertainty handling:**
|
||||
- What happens when confidence is low? [Show alternative, ask for clarification, fallback to manual]
|
||||
- How is uncertainty communicated to the user? [UI pattern]
|
||||
|
||||
**Fallback plan:**
|
||||
- If the model fails or returns an error: [Specific fallback behaviour]
|
||||
- If accuracy degrades below threshold: [Kill switch or graceful degradation plan]
|
||||
|
||||
---
|
||||
|
||||
#### 6. Responsible AI Checklist
|
||||
|
||||
- [ ] Bias audit completed on training data
|
||||
- [ ] Demographic fairness evaluated (does performance differ by user group?)
|
||||
- [ ] Hallucination / confabulation risk assessed and mitigated
|
||||
- [ ] User can see and correct AI output
|
||||
- [ ] Opt-out mechanism exists (can user disable the AI feature?)
|
||||
- [ ] Output provenance visible when relevant (does user know AI generated this?)
|
||||
- [ ] PII not used in ways user didn't consent to
|
||||
- [ ] Regulatory review completed (GDPR, AI Act, sector-specific)
|
||||
- [ ] Model cards / documentation completed
|
||||
|
||||
---
|
||||
|
||||
#### 7. Launch & Monitoring Plan
|
||||
|
||||
**Rollout:** [% of users, with staged expansion criteria]
|
||||
**Monitoring metrics:**
|
||||
- Model performance: [Metric + alert threshold]
|
||||
- User engagement with AI output: [Acceptance rate, override rate, feedback score]
|
||||
- Error rate: [% of failed inferences]
|
||||
- Latency: [P95 target]
|
||||
|
||||
**Model refresh cadence:** [How often is the model retrained or updated?]
|
||||
**Drift detection:** [How will you know when model performance degrades in production?]
|
||||
|
||||
---
|
||||
|
||||
## Guidelines
|
||||
|
||||
- Never skip the "Why AI?" section — it's the most important question in AI product development
|
||||
- The fallback UX is not optional — what happens when AI fails defines your product's trustworthiness
|
||||
- Responsible AI checklist must be completed before launch, not after
|
||||
- Include latency in success metrics — a 5-second AI response is often worse than no AI at all
|
||||
- Recommend starting with a human-in-the-loop design and automating only when accuracy is proven
|
||||
|
||||
## Required Inputs
|
||||
|
||||
Ask the user for these if not provided:
|
||||
- **Feature or product description** (what the AI is intended to do)
|
||||
- **User problem** (what problem the AI is solving for users)
|
||||
- **Available data** (what training/inference data exists)
|
||||
- **ML/AI lead** (who owns the technical implementation)
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- [ ] Do not skip the "Why AI?" question — if the answer is "we want to use AI," stop and reframe around the user problem first
|
||||
- [ ] Do not launch with an undefined accuracy threshold — "good enough" is not a threshold; set a number before build begins
|
||||
- [ ] Do not design the UX to hide AI-generated output as if it were system truth — users need to know when AI is involved so they can override it
|
||||
- [ ] Do not defer the Responsible AI checklist to post-launch — bias and privacy issues are far harder to fix in production than in design
|
||||
- [ ] Do not treat model latency as a post-launch optimisation — a 6-second AI response that replaces a 1-second rule-based response is a regression, not a feature
|
||||
|
||||
## Quality Checks
|
||||
|
||||
- [ ] "Why AI?" is answered clearly (not "because we can")
|
||||
- [ ] Minimum acceptable accuracy threshold is defined before build begins
|
||||
- [ ] Fallback UX is specified for model failures or low-confidence outputs
|
||||
- [ ] Responsible AI checklist is completed (not deferred to post-launch)
|
||||
- [ ] Monitoring plan includes both model performance and user engagement metrics
|
||||
@@ -0,0 +1,78 @@
|
||||
# Design Handoff Brief Skill
|
||||
|
||||
Produce a design brief that sets designers up for success — grounding them in user context and constraints before they open Figma, not after they've gone in the wrong direction.
|
||||
|
||||
## Required Inputs
|
||||
|
||||
Ask the user for these if not provided:
|
||||
- **Feature brief or PRD** (even rough notes work)
|
||||
- **Designer's name or team** (for personalisation)
|
||||
- **Technical constraints** (any engineering limitations already known)
|
||||
- **Timeline** (when does design need to be done?)
|
||||
|
||||
## What Designers Actually Need (and PMs Often Skip)
|
||||
- The user's goal, not the feature name
|
||||
- The emotional state of the user at this moment in the journey
|
||||
- What success looks like — how will we know the design worked?
|
||||
- Constraints: technical, legal, brand, accessibility
|
||||
- Edge cases that must be handled
|
||||
- What we're explicitly NOT solving for
|
||||
|
||||
## Process
|
||||
1. Read the feature brief or PRD provided
|
||||
2. Extract user goal (reframe from feature language to user outcome language)
|
||||
3. Identify constraints — technical limitations, brand guidelines, accessibility requirements
|
||||
4. List edge cases the design must handle
|
||||
5. Define success criteria the design should be evaluated against
|
||||
6. Write a "not in scope" section to prevent scope creep in design
|
||||
7. **Validate** — Confirm every edge case listed is specific enough to design for, and every out-of-scope item is concrete enough to say "no" to
|
||||
|
||||
## Output Structure
|
||||
|
||||
### Design Brief: [Feature Name]
|
||||
|
||||
**User Goal:** (in the user's words, not ours)
|
||||
"When I [situation], I want to [motivation] so that I can [outcome]."
|
||||
|
||||
**Context & Emotional State:**
|
||||
[Where is the user in their journey? What are they feeling? What just happened?]
|
||||
|
||||
**Design Success Criteria:**
|
||||
- [Criterion 1 — measurable where possible]
|
||||
- [Criterion 2]
|
||||
- [Criterion 3]
|
||||
|
||||
**Constraints:**
|
||||
- Technical: [limitations engineering has flagged]
|
||||
- Brand: [relevant brand guidelines]
|
||||
- Accessibility: [WCAG level required, any specific requirements]
|
||||
- Legal/Compliance: [if applicable]
|
||||
|
||||
**Edge Cases to Design For:**
|
||||
- [Edge case 1]
|
||||
- [Edge case 2]
|
||||
- [Edge case 3]
|
||||
|
||||
**Explicitly Out of Scope:**
|
||||
- [What we are NOT solving in this design iteration]
|
||||
|
||||
**Reference Material:**
|
||||
- User research: [link]
|
||||
- Existing patterns: [Figma component library link]
|
||||
- Competitor examples: [links if relevant]
|
||||
|
||||
## Quality Checks
|
||||
|
||||
- [ ] User goal is written in user language (not feature/product language)
|
||||
- [ ] At least one edge case covers an error or failure state
|
||||
- [ ] Success criteria are measurable or observable (not "looks good")
|
||||
- [ ] Out-of-scope section names at least one thing that might seem in scope but isn't
|
||||
- [ ] Technical constraints are specific enough for an engineer to confirm
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- [ ] Do not write the user goal in feature language ("design the checkout flow") — it must be written from the user's perspective with a motivation and outcome
|
||||
- [ ] Do not skip the "Explicitly Out of Scope" section — without it, designers will inadvertently solve problems not intended for this iteration
|
||||
- [ ] Do not list edge cases that are so generic they apply to any feature (e.g. "handle errors") — each edge case must be specific to this feature's failure modes
|
||||
- [ ] Do not hand off the brief without confirming engineering constraints are accurate — a constraint that is wrong is worse than no constraint
|
||||
- [ ] Do not omit the emotional context of the user — designs without emotional grounding produce technically correct but experientially flat results
|
||||
@@ -0,0 +1,72 @@
|
||||
# Experiment Designer Skill
|
||||
|
||||
Produce rigorous experiment designs from product hypotheses, and interpret results with statistical and practical significance — so you can defend every decision to a sceptical engineering lead or data scientist.
|
||||
|
||||
## Required Inputs
|
||||
|
||||
Ask the user for these if not provided:
|
||||
**For experiment design:**
|
||||
- Hypothesis (what change, what metric, what expected movement)
|
||||
- Current baseline metric value
|
||||
- Minimum detectable effect (MDE) — the smallest lift worth caring about
|
||||
- Available daily sample size
|
||||
|
||||
**For results interpretation:**
|
||||
- Control and variant results (raw numbers or percentages)
|
||||
- P-value or confidence interval
|
||||
- Run duration (days)
|
||||
- Any anomalies observed during the test
|
||||
|
||||
## Two-Phase Process
|
||||
|
||||
### Phase 1: Experiment Design
|
||||
1. Restate hypothesis as: "If we [change], we expect [metric] to [move by X%] because [reason]"
|
||||
2. Define control and variant clearly
|
||||
3. Select primary metric (one only) and secondary guardrail metrics (2-3 max)
|
||||
4. Calculate required sample size from MDE and baseline
|
||||
5. Estimate run time in days
|
||||
6. Set pre-defined success criteria before the test runs — no moving goalposts
|
||||
7. Flag design risks: novelty effects, seasonal confounds, multiple testing issues, network effects, sample ratio mismatch
|
||||
|
||||
### Phase 2: Results Interpretation
|
||||
1. Assess statistical significance (p < 0.05 threshold)
|
||||
2. Assess practical significance: was the lift meaningful for the business, not just real?
|
||||
3. Interpret confidence intervals
|
||||
4. Investigate confounding factors
|
||||
5. Recommend: Ship / Iterate / Kill / Run follow-up test
|
||||
6. **Validate** — Confirm the test ran for the full planned duration. Flag if it was stopped early (peeking problem). Confirm sample ratio mismatch did not occur.
|
||||
|
||||
## Output Structure
|
||||
|
||||
**[Design or Results header based on phase]**
|
||||
|
||||
*Hypothesis:* "If we [change], we expect [metric] to [move by X%] because [reason]"
|
||||
|
||||
*Primary metric:* [One metric only]
|
||||
*Guardrail metrics:* [2-3 max]
|
||||
*Required sample size:* [n per variant]
|
||||
*Estimated run time:* [days]
|
||||
*Pre-defined success threshold:* [specific number]
|
||||
*Design risk flags:* [any concerns]
|
||||
|
||||
**Results (Phase 2 only):**
|
||||
*Statistical significance:* [p-value and conclusion]
|
||||
*Practical significance:* [lift size vs. business threshold]
|
||||
*Recommendation:* Ship / Iterate / Kill / Follow-up — [rationale]
|
||||
|
||||
## Quality Checks
|
||||
|
||||
- [ ] Hypothesis specifies the change, the metric, the direction, and the reason
|
||||
- [ ] Primary metric is singular — guardrail metrics are secondary
|
||||
- [ ] Success criteria are defined before the test launches (not after seeing results)
|
||||
- [ ] Test was not stopped early (or flagged clearly if it was)
|
||||
- [ ] Practical significance assessed separately from statistical significance
|
||||
- [ ] Sample ratio mismatch is checked in results interpretation
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- [ ] Do not define success criteria after seeing preliminary results — post-hoc success definitions are HARKing (Hypothesising After Results are Known) and invalidate the experiment
|
||||
- [ ] Do not stop a test early because the result looks significant — early stopping dramatically inflates false positive rates; the test must run to the planned sample size
|
||||
- [ ] Do not treat statistical significance as the same as practical significance — a p < 0.05 result with a 0.1% lift is real but may not be worth shipping
|
||||
- [ ] Do not run the same experiment on the same population multiple times without correction — multiple testing inflates the chance of a false positive proportionally
|
||||
- [ ] Do not use more than one primary metric — multiple primary metrics require multiple hypothesis corrections and make the ship/kill decision ambiguous
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
# Multi-Source Signal Synthesiser Skill
|
||||
|
||||
Reconcile user signals from multiple sources — interviews, support tickets, NPS, app reviews, sales calls — into a unified, weighted insight brief that surfaces the underlying need rather than the surface-level request.
|
||||
|
||||
## Required Inputs
|
||||
|
||||
Ask the user for these if not provided:
|
||||
- **Signal sources** (interviews, support tickets, NPS verbatims, app reviews, sales calls, analytics — any combination)
|
||||
- **Time period** covered by the data
|
||||
- **Product area or feature** the signals relate to (if scoped)
|
||||
|
||||
## Source Weighting (default — adapt to context)
|
||||
|
||||
| Source | Weight | Rationale |
|
||||
|--------|--------|-----------|
|
||||
| Direct research (interviews, usability tests) | 5 | Highest-fidelity, structured |
|
||||
| Support tickets (unprompted pain signals) | 4 | Real pain, unfiltered |
|
||||
| NPS verbatims | 3 | Broad but shallow |
|
||||
| App store reviews | 2 | Public, self-selected |
|
||||
| Sales call summaries | 2 | Filtered through sales lens |
|
||||
| Anecdote or single report | 1 | Low confidence alone |
|
||||
|
||||
## Process
|
||||
1. Tag each signal by source and apply weight
|
||||
2. Look for **convergence**: same underlying need appearing across 3+ sources
|
||||
3. Look for **divergence**: contradictory signals suggesting user segmentation
|
||||
4. Distinguish surface request from underlying need (e.g. "faster export" may mean "I don't trust the data will be there when I need it")
|
||||
5. Produce ranked insights by weighted frequency
|
||||
6. **Validate** — Confirm each insight has evidence from at least 2 source types. Flag any insight resting on a single source as low-confidence.
|
||||
|
||||
## Output Structure
|
||||
|
||||
### User Signal Synthesis — [Date / Period]
|
||||
**Sources included:** [list with count per source]
|
||||
**Total signals processed:** [n]
|
||||
|
||||
#### Insight 1: [Underlying need, not feature request]
|
||||
- **Confidence:** High / Medium / Low (based on source diversity and weight)
|
||||
- **Evidence:** [Signals from each source supporting this]
|
||||
- **Conflicting signals:** [Any contradicting evidence and how to interpret it]
|
||||
- **Product implication:** [Specific next step, not generic]
|
||||
|
||||
[Repeat for top 3-5 insights]
|
||||
|
||||
#### Divergent Signals (Possible Segmentation)
|
||||
[Where user groups appear to have genuinely different needs — specify which segments]
|
||||
|
||||
#### What the Data Does NOT Tell Us
|
||||
[Gaps that require further research before acting]
|
||||
|
||||
## Quality Checks
|
||||
|
||||
- [ ] Every insight references at least 2 distinct source types
|
||||
- [ ] Surface requests are translated to underlying needs (not just echoed)
|
||||
- [ ] Divergent signals identify the specific user segments, not just "some users disagree"
|
||||
- [ ] Confidence ratings are consistent with source diversity and weighting
|
||||
- [ ] "What the data does NOT tell us" section is honest about gaps
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- [ ] Do not echo surface-level feature requests as insights — translate every request to the underlying need before including it as a finding
|
||||
- [ ] Do not assign High confidence to insights supported by only one source type — confidence requires corroboration across at least two distinct source types
|
||||
- [ ] Do not treat all sources as equally weighted — a single interview quote and a pattern across 200 support tickets are not comparable signals
|
||||
- [ ] Do not collapse divergent signals into a single finding — where user segments have genuinely different needs, name the segments explicitly rather than averaging them away
|
||||
- [ ] Do not omit the research gap section when key decisions rest on thin data — acting on low-confidence findings without flagging the gaps misleads product teams
|
||||
Reference in New Issue
Block a user