@caoscompanybr/merlin 3.5.0

package/.merlin-core/skills/domain/marketing-ops/references/activate/ab-testing.md

@@ -0,0 +1,379 @@
+---
+source: coreyhaines31/marketingskills (MIT) v2.0.1
+imported: 2026-05-20
+stage: activate
+upstream_path: skills/ab-testing/SKILL.md
+---
+
+> **Adjacency:** This skill is part of the [`marketing-ops`](../../SKILL.md) umbrella, organized by funnel stage (activate). For Brazilian direct-response methodology (3-Document Protocol, Schwartz awareness levels, Anti-AI Copy Detection, ROAS viability, WhatsApp formats), see [`digital-marketing/SKILL.md`](../../../digital-marketing/SKILL.md). For page-level SaaS copy, see [`general/copywriting/SKILL.md`](../../../../general/copywriting/SKILL.md). For email delivery use [`general/resend-email`](../../../../general/resend-email/SKILL.md), for visuals [`general/image-gen`](../../../../general/image-gen/SKILL.md), for decks [`general/blkskrn`](../../../../general/blkskrn/SKILL.md).
+
+---
+
+# A/B Test Setup
+
+You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.
+
+## Initial Assessment
+
+**Check for product marketing context first:**
+If `.agents/product-marketing.md` exists (or `.claude/product-marketing.md`, or the legacy `product-marketing-context.md` filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.
+
+Before designing a test, understand:
+
+1. **Test Context** - What are you trying to improve? What change are you considering?
+2. **Current State** - Baseline conversion rate? Current traffic volume?
+3. **Constraints** - Technical complexity? Timeline? Tools available?
+
+---
+
+## Core Principles
+
+### 1. Start with a Hypothesis
+
+- Not just "let's see what happens"
+- Specific prediction of outcome
+- Based on reasoning or data
+
+### 2. Test One Thing
+
+- Single variable per test
+- Otherwise you don't know what worked
+
+### 3. Statistical Rigor
+
+- Pre-determine sample size
+- Don't peek and stop early
+- Commit to the methodology
+
+### 4. Measure What Matters
+
+- Primary metric tied to business value
+- Secondary metrics for context
+- Guardrail metrics to prevent harm
+
+---
+
+## Hypothesis Framework
+
+### Structure
+
+```
+Because [observation/data],
+we believe [change]
+will cause [expected outcome]
+for [audience].
+We'll know this is true when [metrics].
+```
+
+### Example
+
+**Weak**: "Changing the button color might increase clicks."
+
+**Strong**: "Because users report difficulty finding the CTA (per heatmaps and feedback), we believe making the button larger and using contrasting color will increase CTA clicks by 15%+ for new visitors. We'll measure click-through rate from page view to signup start."
+
+---
+
+## Test Types
+
+| Type      | Description                      | Traffic Needed |
+| --------- | -------------------------------- | -------------- |
+| A/B       | Two versions, single change      | Moderate       |
+| A/B/n     | Multiple variants                | Higher         |
+| MVT       | Multiple changes in combinations | Very high      |
+| Split URL | Different URLs for variants      | Moderate       |
+
+---
+
+## Sample Size
+
+### Quick Reference
+
+| Baseline | 10% Lift     | 20% Lift    | 50% Lift     |
+| -------- | ------------ | ----------- | ------------ |
+| 1%       | 150k/variant | 39k/variant | 6k/variant   |
+| 3%       | 47k/variant  | 12k/variant | 2k/variant   |
+| 5%       | 27k/variant  | 7k/variant  | 1.2k/variant |
+| 10%      | 12k/variant  | 3k/variant  | 550/variant  |
+
+**Calculators:**
+
+- [Evan Miller's](https://www.evanmiller.org/ab-testing/sample-size.html)
+- [Optimizely's](https://www.optimizely.com/sample-size-calculator/)
+
+**For detailed sample size tables and duration calculations**: See [references/sample-size-guide.md](references/sample-size-guide.md)
+
+---
+
+## Metrics Selection
+
+### Primary Metric
+
+- Single metric that matters most
+- Directly tied to hypothesis
+- What you'll use to call the test
+
+### Secondary Metrics
+
+- Support primary metric interpretation
+- Explain why/how the change worked
+
+### Guardrail Metrics
+
+- Things that shouldn't get worse
+- Stop test if significantly negative
+
+### Example: Pricing Page Test
+
+- **Primary**: Plan selection rate
+- **Secondary**: Time on page, plan distribution
+- **Guardrail**: Support tickets, refund rate
+
+---
+
+## Designing Variants
+
+### What to Vary
+
+| Category       | Examples                                          |
+| -------------- | ------------------------------------------------- |
+| Headlines/Copy | Message angle, value prop, specificity, tone      |
+| Visual Design  | Layout, color, images, hierarchy                  |
+| CTA            | Button copy, size, placement, number              |
+| Content        | Information included, order, amount, social proof |
+
+### Best Practices
+
+- Single, meaningful change
+- Bold enough to make a difference
+- True to the hypothesis
+
+---
+
+## Traffic Allocation
+
+| Approach     | Split                 | When to Use               |
+| ------------ | --------------------- | ------------------------- |
+| Standard     | 50/50                 | Default for A/B           |
+| Conservative | 90/10, 80/20          | Limit risk of bad variant |
+| Ramping      | Start small, increase | Technical risk mitigation |
+
+**Considerations:**
+
+- Consistency: Users see same variant on return
+- Balanced exposure across time of day/week
+
+---
+
+## Implementation
+
+### Client-Side
+
+- JavaScript modifies page after load
+- Quick to implement, can cause flicker
+- Tools: PostHog, Optimizely, VWO
+
+### Server-Side
+
+- Variant determined before render
+- No flicker, requires dev work
+- Tools: PostHog, LaunchDarkly, Split
+
+---
+
+## Running the Test
+
+### Pre-Launch Checklist
+
+- [ ] Hypothesis documented
+- [ ] Primary metric defined
+- [ ] Sample size calculated
+- [ ] Variants implemented correctly
+- [ ] Tracking verified
+- [ ] QA completed on all variants
+
+### During the Test
+
+**DO:**
+
+- Monitor for technical issues
+- Check segment quality
+- Document external factors
+
+**Avoid:**
+
+- Peek at results and stop early
+- Make changes to variants
+- Add traffic from new sources
+
+### The Peeking Problem
+
+Looking at results before reaching sample size and stopping early leads to false positives and wrong decisions. Pre-commit to sample size and trust the process.
+
+---
+
+## Analyzing Results
+
+### Statistical Significance
+
+- 95% confidence = p-value < 0.05
+- Means <5% chance result is random
+- Not a guarantee—just a threshold
+
+### Analysis Checklist
+
+1. **Reach sample size?** If not, result is preliminary
+2. **Statistically significant?** Check confidence intervals
+3. **Effect size meaningful?** Compare to MDE, project impact
+4. **Secondary metrics consistent?** Support the primary?
+5. **Guardrail concerns?** Anything get worse?
+6. **Segment differences?** Mobile vs. desktop? New vs. returning?
+
+### Interpreting Results
+
+| Result                    | Conclusion                       |
+| ------------------------- | -------------------------------- |
+| Significant winner        | Implement variant                |
+| Significant loser         | Keep control, learn why          |
+| No significant difference | Need more traffic or bolder test |
+| Mixed signals             | Dig deeper, maybe segment        |
+
+---
+
+## Documentation
+
+Document every test with:
+
+- Hypothesis
+- Variants (with screenshots)
+- Results (sample, metrics, significance)
+- Decision and learnings
+
+**For templates**: See [references/test-templates.md](references/test-templates.md)
+
+---
+
+## Growth Experimentation Program
+
+Individual tests are valuable. A continuous experimentation program is a compounding asset. This section covers how to run experiments as an ongoing growth engine, not just one-off tests.
+
+### The Experiment Loop
+
+```
+1. Generate hypotheses (from data, research, competitors, customer feedback)
+2. Prioritize with ICE scoring
+3. Design and run the test
+4. Analyze results with statistical rigor
+5. Promote winners to a playbook
+6. Generate new hypotheses from learnings
+→ Repeat
+```
+
+### Hypothesis Generation
+
+Feed your experiment backlog from multiple sources:
+
+| Source              | What to Look For                                                |
+| ------------------- | --------------------------------------------------------------- |
+| Analytics           | Drop-off points, low-converting pages, underperforming segments |
+| Customer research   | Pain points, confusion, unmet expectations                      |
+| Competitor analysis | Features, messaging, or UX patterns they use that you don't     |
+| Support tickets     | Recurring questions or complaints about conversion flows        |
+| Heatmaps/recordings | Where users hesitate, rage-click, or abandon                    |
+| Past experiments    | "Significant loser" tests often reveal new angles to try        |
+
+### ICE Prioritization
+
+Score each hypothesis 1-10 on three dimensions:
+
+| Dimension      | Question                                                  |
+| -------------- | --------------------------------------------------------- |
+| **Impact**     | If this works, how much will it move the primary metric?  |
+| **Confidence** | How sure are we this will work? (Based on data, not gut.) |
+| **Ease**       | How fast and cheap can we ship and measure this?          |
+
+**ICE Score** = (Impact + Confidence + Ease) / 3
+
+Run highest-scoring experiments first. Re-score monthly as context changes.
+
+### Experiment Velocity
+
+Track your experimentation rate as a leading indicator of growth:
+
+| Metric                         | Target                                                                                             |
+| ------------------------------ | -------------------------------------------------------------------------------------------------- |
+| Experiments launched per month | 4-8 for most teams                                                                                 |
+| Win rate                       | 20-30% is common for mature programs (sustained higher rates may indicate conservative hypotheses) |
+| Average test duration          | 2-4 weeks                                                                                          |
+| Backlog depth                  | 20+ hypotheses queued                                                                              |
+| Cumulative lift                | Compound gains from all winners                                                                    |
+
+### The Experiment Playbook
+
+When a test wins, don't just implement it — document the pattern:
+
+```
+## [Experiment Name]
+**Date**: [date]
+**Hypothesis**: [the hypothesis]
+**Sample size**: [n per variant]
+**Result**: [winner/loser/inconclusive] — [primary metric] changed by [X%] (95% CI: [range], p=[value])
+**Guardrails**: [any guardrail metrics and their outcomes]
+**Segment deltas**: [notable differences by device, segment, or cohort]
+**Why it worked/failed**: [analysis]
+**Pattern**: [the reusable insight — e.g., "social proof near pricing CTAs increases plan selection"]
+**Apply to**: [other pages/flows where this pattern might work]
+**Status**: [implemented / parked / needs follow-up test]
+```
+
+Over time, your playbook becomes a library of proven growth patterns specific to your product and audience.
+
+### Experiment Cadence
+
+**Weekly (30 min)**: Review running experiments for technical issues and guardrail metrics. Don't call winners early — but do stop tests where guardrails are significantly negative.
+
+**Bi-weekly**: Conclude completed experiments. Analyze results, update playbook, launch next experiment from backlog.
+
+**Monthly (1 hour)**: Review experiment velocity, win rate, cumulative lift. Replenish hypothesis backlog. Re-prioritize with ICE.
+
+**Quarterly**: Audit the playbook. Which patterns have been applied broadly? Which winning patterns haven't been scaled yet? What areas of the funnel are under-tested?
+
+---
+
+## Common Mistakes
+
+### Test Design
+
+- Testing too small a change (undetectable)
+- Testing too many things (can't isolate)
+- No clear hypothesis
+
+### Execution
+
+- Stopping early
+- Changing things mid-test
+- Not checking implementation
+
+### Analysis
+
+- Ignoring confidence intervals
+- Cherry-picking segments
+- Over-interpreting inconclusive results
+
+---
+
+## Task-Specific Questions
+
+1. What's your current conversion rate?
+2. How much traffic does this page get?
+3. What change are you considering and why?
+4. What's the smallest improvement worth detecting?
+5. What tools do you have for testing?
+6. Have you tested this area before?
+
+---
+
+## Related Skills
+
+- **cro**: For generating test ideas based on CRO principles
+- **analytics**: For setting up test measurement
+- **copywriting**: For creating variant copy

package/.merlin-core/skills/domain/marketing-ops/references/activate/analytics/references/event-library.md

@@ -0,0 +1,19 @@
+---
+source: coreyhaines31/marketingskills (MIT) v2.0.1
+imported: 2026-05-20 (stub) — full body deferred to follow-up cleanup
+stage: activate
+skill: analytics
+upstream_path: skills/analytics/references/event-library.md
+---
+
+# event-library — reference stub
+
+This is a thin pointer to an upstream reference. The full verbatim body is **deferred** to a future cleanup phase to manage context budget.
+
+**Upstream URL (canonical body):** https://github.com/coreyhaines31/marketingskills/blob/main/skills/analytics/references/event-library.md
+
+**Raw fetch URL:** https://raw.githubusercontent.com/coreyhaines31/marketingskills/main/skills/analytics/references/event-library.md
+
+**Parent skill:** [`../analytics.md`](../analytics.md) (stage: activate)
+
+**To import the full body** (future task): `curl -sSL <raw URL> -o references/activate/analytics/references/event-library.md`, then prepend the standard Adjacency header.

package/.merlin-core/skills/domain/marketing-ops/references/activate/analytics/references/ga4-implementation.md

@@ -0,0 +1,19 @@
+---
+source: coreyhaines31/marketingskills (MIT) v2.0.1
+imported: 2026-05-20 (stub) — full body deferred to follow-up cleanup
+stage: activate
+skill: analytics
+upstream_path: skills/analytics/references/ga4-implementation.md
+---
+
+# ga4-implementation — reference stub
+
+This is a thin pointer to an upstream reference. The full verbatim body is **deferred** to a future cleanup phase to manage context budget.
+
+**Upstream URL (canonical body):** https://github.com/coreyhaines31/marketingskills/blob/main/skills/analytics/references/ga4-implementation.md
+
+**Raw fetch URL:** https://raw.githubusercontent.com/coreyhaines31/marketingskills/main/skills/analytics/references/ga4-implementation.md
+
+**Parent skill:** [`../analytics.md`](../analytics.md) (stage: activate)
+
+**To import the full body** (future task): `curl -sSL <raw URL> -o references/activate/analytics/references/ga4-implementation.md`, then prepend the standard Adjacency header.

package/.merlin-core/skills/domain/marketing-ops/references/activate/analytics/references/gtm-implementation.md

@@ -0,0 +1,19 @@
+---
+source: coreyhaines31/marketingskills (MIT) v2.0.1
+imported: 2026-05-20 (stub) — full body deferred to follow-up cleanup
+stage: activate
+skill: analytics
+upstream_path: skills/analytics/references/gtm-implementation.md
+---
+
+# gtm-implementation — reference stub
+
+This is a thin pointer to an upstream reference. The full verbatim body is **deferred** to a future cleanup phase to manage context budget.
+
+**Upstream URL (canonical body):** https://github.com/coreyhaines31/marketingskills/blob/main/skills/analytics/references/gtm-implementation.md
+
+**Raw fetch URL:** https://raw.githubusercontent.com/coreyhaines31/marketingskills/main/skills/analytics/references/gtm-implementation.md
+
+**Parent skill:** [`../analytics.md`](../analytics.md) (stage: activate)
+
+**To import the full body** (future task): `curl -sSL <raw URL> -o references/activate/analytics/references/gtm-implementation.md`, then prepend the standard Adjacency header.