convoke-agents 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +920 -0
- package/INSTALLATION.md +230 -0
- package/LICENSE +21 -0
- package/README.md +330 -0
- package/UPDATE-GUIDE.md +220 -0
- package/_bmad/bme/_vortex/README.md +150 -0
- package/_bmad/bme/_vortex/agents/contextualization-expert.md +100 -0
- package/_bmad/bme/_vortex/agents/discovery-empathy-expert.md +117 -0
- package/_bmad/bme/_vortex/agents/hypothesis-engineer.md +117 -0
- package/_bmad/bme/_vortex/agents/lean-experiments-specialist.md +118 -0
- package/_bmad/bme/_vortex/agents/learning-decision-expert.md +117 -0
- package/_bmad/bme/_vortex/agents/production-intelligence-specialist.md +117 -0
- package/_bmad/bme/_vortex/agents/research-convergence-specialist.md +117 -0
- package/_bmad/bme/_vortex/compass-routing-reference.md +312 -0
- package/_bmad/bme/_vortex/config.yaml +46 -0
- package/_bmad/bme/_vortex/contracts/hc1-empathy-artifacts.md +152 -0
- package/_bmad/bme/_vortex/contracts/hc2-problem-definition.md +125 -0
- package/_bmad/bme/_vortex/contracts/hc3-hypothesis-contract.md +112 -0
- package/_bmad/bme/_vortex/contracts/hc4-experiment-context.md +140 -0
- package/_bmad/bme/_vortex/contracts/hc5-signal-report.md +130 -0
- package/_bmad/bme/_vortex/examples/hc2-example-problem-definition.md +85 -0
- package/_bmad/bme/_vortex/examples/hc3-example-hypothesis-contract.md +103 -0
- package/_bmad/bme/_vortex/examples/hc5-example-signal-report.md +76 -0
- package/_bmad/bme/_vortex/guides/EMMA-USER-GUIDE.md +232 -0
- package/_bmad/bme/_vortex/guides/ISLA-USER-GUIDE.md +208 -0
- package/_bmad/bme/_vortex/guides/LIAM-USER-GUIDE.md +255 -0
- package/_bmad/bme/_vortex/guides/MAX-USER-GUIDE.md +213 -0
- package/_bmad/bme/_vortex/guides/MILA-USER-GUIDE.md +235 -0
- package/_bmad/bme/_vortex/guides/NOAH-USER-GUIDE.md +258 -0
- package/_bmad/bme/_vortex/guides/WADE-USER-GUIDE.md +245 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/empathy-map.template.md +143 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/steps/step-01-define-user.md +60 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/steps/step-02-says-thinks.md +67 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/steps/step-03-does-feels.md +79 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/steps/step-04-pain-points.md +87 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/steps/step-05-gains.md +103 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/steps/step-06-synthesize.md +104 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/validate.md +117 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/empathy-map/workflow.md +44 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/steps/step-01-define-requirements.md +85 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/steps/step-02-user-flows.md +59 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/steps/step-03-information-architecture.md +68 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/steps/step-04-wireframe-sketch.md +97 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/steps/step-05-components.md +128 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/steps/step-06-synthesize.md +83 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/wireframe.template.md +287 -0
- package/_bmad/bme/_vortex/workflows/_deprecated/wireframe/workflow.md +44 -0
- package/_bmad/bme/_vortex/workflows/assumption-mapping/steps/step-01-setup.md +66 -0
- package/_bmad/bme/_vortex/workflows/assumption-mapping/steps/step-02-context.md +93 -0
- package/_bmad/bme/_vortex/workflows/assumption-mapping/steps/step-03-risk-mapping.md +103 -0
- package/_bmad/bme/_vortex/workflows/assumption-mapping/steps/step-04-synthesize.md +101 -0
- package/_bmad/bme/_vortex/workflows/assumption-mapping/workflow.md +49 -0
- package/_bmad/bme/_vortex/workflows/behavior-analysis/steps/step-01-setup.md +81 -0
- package/_bmad/bme/_vortex/workflows/behavior-analysis/steps/step-02-context.md +67 -0
- package/_bmad/bme/_vortex/workflows/behavior-analysis/steps/step-03-classification.md +98 -0
- package/_bmad/bme/_vortex/workflows/behavior-analysis/steps/step-04-evidence.md +100 -0
- package/_bmad/bme/_vortex/workflows/behavior-analysis/steps/step-05-synthesize.md +174 -0
- package/_bmad/bme/_vortex/workflows/behavior-analysis/workflow.md +52 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/contextualize-scope.template.md +67 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/steps/step-01-list-opportunities.md +47 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/steps/step-02-define-criteria.md +36 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/steps/step-03-evaluate-opportunities.md +30 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/steps/step-04-define-boundaries.md +32 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/steps/step-05-validate-fit.md +28 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/steps/step-06-synthesize.md +36 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/validate.md +30 -0
- package/_bmad/bme/_vortex/workflows/contextualize-scope/workflow.md +59 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/empathy-map.template.md +143 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/steps/step-01-define-user.md +60 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/steps/step-02-says-thinks.md +67 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/steps/step-03-does-feels.md +79 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/steps/step-04-pain-points.md +87 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/steps/step-05-gains.md +103 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/steps/step-06-synthesize.md +107 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/validate.md +117 -0
- package/_bmad/bme/_vortex/workflows/empathy-map/workflow.md +45 -0
- package/_bmad/bme/_vortex/workflows/experiment-design/steps/step-01-setup.md +66 -0
- package/_bmad/bme/_vortex/workflows/experiment-design/steps/step-02-context.md +77 -0
- package/_bmad/bme/_vortex/workflows/experiment-design/steps/step-03-design.md +114 -0
- package/_bmad/bme/_vortex/workflows/experiment-design/steps/step-04-synthesize.md +128 -0
- package/_bmad/bme/_vortex/workflows/experiment-design/workflow.md +51 -0
- package/_bmad/bme/_vortex/workflows/hypothesis-engineering/steps/step-01-setup.md +66 -0
- package/_bmad/bme/_vortex/workflows/hypothesis-engineering/steps/step-02-context.md +80 -0
- package/_bmad/bme/_vortex/workflows/hypothesis-engineering/steps/step-03-brainwriting.md +79 -0
- package/_bmad/bme/_vortex/workflows/hypothesis-engineering/steps/step-04-assumption-mapping.md +102 -0
- package/_bmad/bme/_vortex/workflows/hypothesis-engineering/steps/step-05-synthesize.md +130 -0
- package/_bmad/bme/_vortex/workflows/hypothesis-engineering/workflow.md +52 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/lean-experiment.template.md +29 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/steps/step-01-hypothesis.md +58 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/steps/step-02-design.md +68 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/steps/step-03-metrics.md +73 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/steps/step-04-run.md +75 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/steps/step-05-analyze.md +84 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/steps/step-06-decide.md +111 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/validate.md +30 -0
- package/_bmad/bme/_vortex/workflows/lean-experiment/workflow.md +26 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/lean-persona.template.md +163 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/steps/step-01-define-job.md +72 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/steps/step-02-current-solution.md +83 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/steps/step-03-problem-contexts.md +90 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/steps/step-04-forces-anxieties.md +98 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/steps/step-05-success-criteria.md +103 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/steps/step-06-synthesize.md +129 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/validate.md +30 -0
- package/_bmad/bme/_vortex/workflows/lean-persona/workflow.md +50 -0
- package/_bmad/bme/_vortex/workflows/learning-card/learning-card.template.md +179 -0
- package/_bmad/bme/_vortex/workflows/learning-card/steps/step-01-experiment-context.md +100 -0
- package/_bmad/bme/_vortex/workflows/learning-card/steps/step-02-raw-results.md +125 -0
- package/_bmad/bme/_vortex/workflows/learning-card/steps/step-03-analysis.md +125 -0
- package/_bmad/bme/_vortex/workflows/learning-card/steps/step-04-validated-learning.md +139 -0
- package/_bmad/bme/_vortex/workflows/learning-card/steps/step-05-implications.md +134 -0
- package/_bmad/bme/_vortex/workflows/learning-card/steps/step-06-synthesize.md +121 -0
- package/_bmad/bme/_vortex/workflows/learning-card/validate.md +134 -0
- package/_bmad/bme/_vortex/workflows/learning-card/workflow.md +51 -0
- package/_bmad/bme/_vortex/workflows/mvp/mvp.template.md +40 -0
- package/_bmad/bme/_vortex/workflows/mvp/steps/step-01-riskiest-assumption.md +17 -0
- package/_bmad/bme/_vortex/workflows/mvp/steps/step-02-success-criteria.md +13 -0
- package/_bmad/bme/_vortex/workflows/mvp/steps/step-03-smallest-test.md +13 -0
- package/_bmad/bme/_vortex/workflows/mvp/steps/step-04-scope-features.md +13 -0
- package/_bmad/bme/_vortex/workflows/mvp/steps/step-05-build-measure-learn.md +13 -0
- package/_bmad/bme/_vortex/workflows/mvp/steps/step-06-synthesize.md +28 -0
- package/_bmad/bme/_vortex/workflows/mvp/validate.md +30 -0
- package/_bmad/bme/_vortex/workflows/mvp/workflow.md +36 -0
- package/_bmad/bme/_vortex/workflows/pattern-mapping/steps/step-01-setup.md +102 -0
- package/_bmad/bme/_vortex/workflows/pattern-mapping/steps/step-02-context.md +81 -0
- package/_bmad/bme/_vortex/workflows/pattern-mapping/steps/step-03-pattern-identification.md +88 -0
- package/_bmad/bme/_vortex/workflows/pattern-mapping/steps/step-04-theme-clustering.md +100 -0
- package/_bmad/bme/_vortex/workflows/pattern-mapping/steps/step-05-synthesize.md +135 -0
- package/_bmad/bme/_vortex/workflows/pattern-mapping/workflow.md +58 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/pivot-patch-persevere.template.md +201 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/steps/step-01-evidence-review.md +125 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/steps/step-02-hypothesis-assessment.md +132 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/steps/step-03-option-analysis.md +167 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/steps/step-04-stakeholder-input.md +141 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/steps/step-05-decision.md +161 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/steps/step-06-action-plan.md +188 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/validate.md +159 -0
- package/_bmad/bme/_vortex/workflows/pivot-patch-persevere/workflow.md +51 -0
- package/_bmad/bme/_vortex/workflows/pivot-resynthesis/steps/step-01-setup.md +97 -0
- package/_bmad/bme/_vortex/workflows/pivot-resynthesis/steps/step-02-context.md +86 -0
- package/_bmad/bme/_vortex/workflows/pivot-resynthesis/steps/step-03-jtbd-reframing.md +88 -0
- package/_bmad/bme/_vortex/workflows/pivot-resynthesis/steps/step-04-pains-gains-revision.md +76 -0
- package/_bmad/bme/_vortex/workflows/pivot-resynthesis/steps/step-05-synthesize.md +158 -0
- package/_bmad/bme/_vortex/workflows/pivot-resynthesis/workflow.md +52 -0
- package/_bmad/bme/_vortex/workflows/product-vision/product-vision.template.md +147 -0
- package/_bmad/bme/_vortex/workflows/product-vision/steps/step-01-define-problem.md +89 -0
- package/_bmad/bme/_vortex/workflows/product-vision/steps/step-02-target-market.md +91 -0
- package/_bmad/bme/_vortex/workflows/product-vision/steps/step-03-unique-approach.md +87 -0
- package/_bmad/bme/_vortex/workflows/product-vision/steps/step-04-future-state.md +100 -0
- package/_bmad/bme/_vortex/workflows/product-vision/steps/step-05-principles.md +92 -0
- package/_bmad/bme/_vortex/workflows/product-vision/steps/step-06-synthesize.md +170 -0
- package/_bmad/bme/_vortex/workflows/product-vision/validate.md +30 -0
- package/_bmad/bme/_vortex/workflows/product-vision/workflow.md +55 -0
- package/_bmad/bme/_vortex/workflows/production-monitoring/steps/step-01-setup.md +84 -0
- package/_bmad/bme/_vortex/workflows/production-monitoring/steps/step-02-context.md +66 -0
- package/_bmad/bme/_vortex/workflows/production-monitoring/steps/step-03-monitoring.md +74 -0
- package/_bmad/bme/_vortex/workflows/production-monitoring/steps/step-04-prioritization.md +97 -0
- package/_bmad/bme/_vortex/workflows/production-monitoring/steps/step-05-synthesize.md +183 -0
- package/_bmad/bme/_vortex/workflows/production-monitoring/workflow.md +52 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/proof-of-concept.template.md +25 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/steps/step-01-risk.md +79 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/steps/step-02-scope.md +105 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/steps/step-03-build.md +92 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/steps/step-04-test.md +103 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/steps/step-05-evaluate.md +114 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/steps/step-06-document.md +125 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/validate.md +30 -0
- package/_bmad/bme/_vortex/workflows/proof-of-concept/workflow.md +26 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/proof-of-value.template.md +29 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/steps/step-01-value-hypothesis.md +75 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/steps/step-02-validation-design.md +94 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/steps/step-03-willingness.md +96 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/steps/step-04-test.md +107 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/steps/step-05-analyze.md +116 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/steps/step-06-document.md +147 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/validate.md +30 -0
- package/_bmad/bme/_vortex/workflows/proof-of-value/workflow.md +26 -0
- package/_bmad/bme/_vortex/workflows/research-convergence/steps/step-01-setup.md +69 -0
- package/_bmad/bme/_vortex/workflows/research-convergence/steps/step-02-context.md +70 -0
- package/_bmad/bme/_vortex/workflows/research-convergence/steps/step-03-jtbd-framing.md +81 -0
- package/_bmad/bme/_vortex/workflows/research-convergence/steps/step-04-pains-gains.md +77 -0
- package/_bmad/bme/_vortex/workflows/research-convergence/steps/step-05-synthesize.md +147 -0
- package/_bmad/bme/_vortex/workflows/research-convergence/workflow.md +50 -0
- package/_bmad/bme/_vortex/workflows/signal-interpretation/steps/step-01-setup.md +68 -0
- package/_bmad/bme/_vortex/workflows/signal-interpretation/steps/step-02-context.md +67 -0
- package/_bmad/bme/_vortex/workflows/signal-interpretation/steps/step-03-signal-analysis.md +85 -0
- package/_bmad/bme/_vortex/workflows/signal-interpretation/steps/step-04-anomaly-detection.md +93 -0
- package/_bmad/bme/_vortex/workflows/signal-interpretation/steps/step-05-synthesize.md +163 -0
- package/_bmad/bme/_vortex/workflows/signal-interpretation/workflow.md +52 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/steps/step-01-discovery-scope.md +77 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/steps/step-02-research-methods.md +152 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/steps/step-03-research-plan.md +159 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/steps/step-04-execute.md +169 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/steps/step-05-organize-data.md +149 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/steps/step-06-synthesize.md +159 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/user-discovery.template.md +231 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/validate.md +153 -0
- package/_bmad/bme/_vortex/workflows/user-discovery/workflow.md +45 -0
- package/_bmad/bme/_vortex/workflows/user-interview/steps/step-01-research-goals.md +100 -0
- package/_bmad/bme/_vortex/workflows/user-interview/steps/step-02-interview-script.md +123 -0
- package/_bmad/bme/_vortex/workflows/user-interview/steps/step-03-recruitment.md +144 -0
- package/_bmad/bme/_vortex/workflows/user-interview/steps/step-04-conduct.md +154 -0
- package/_bmad/bme/_vortex/workflows/user-interview/steps/step-05-findings.md +163 -0
- package/_bmad/bme/_vortex/workflows/user-interview/steps/step-06-synthesize.md +171 -0
- package/_bmad/bme/_vortex/workflows/user-interview/user-interview.template.md +250 -0
- package/_bmad/bme/_vortex/workflows/user-interview/validate.md +142 -0
- package/_bmad/bme/_vortex/workflows/user-interview/workflow.md +51 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/steps/step-01-current-state.md +56 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/steps/step-02-evidence-inventory.md +70 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/steps/step-03-gap-analysis.md +76 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/steps/step-04-stream-evaluation.md +57 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/steps/step-05-recommendation.md +65 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/steps/step-06-navigation-plan.md +72 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/validate.md +75 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/vortex-navigation.template.md +105 -0
- package/_bmad/bme/_vortex/workflows/vortex-navigation/workflow.md +54 -0
- package/index.js +56 -0
- package/package.json +77 -0
- package/scripts/README.md +226 -0
- package/scripts/convoke-doctor.js +322 -0
- package/scripts/docs-audit.js +584 -0
- package/scripts/install-all-agents.js +9 -0
- package/scripts/install-vortex-agents.js +208 -0
- package/scripts/postinstall.js +104 -0
- package/scripts/update/convoke-migrate.js +169 -0
- package/scripts/update/convoke-update.js +272 -0
- package/scripts/update/convoke-version.js +134 -0
- package/scripts/update/lib/agent-registry.js +144 -0
- package/scripts/update/lib/backup-manager.js +243 -0
- package/scripts/update/lib/config-merger.js +242 -0
- package/scripts/update/lib/migration-runner.js +367 -0
- package/scripts/update/lib/refresh-installation.js +171 -0
- package/scripts/update/lib/utils.js +96 -0
- package/scripts/update/lib/validator.js +360 -0
- package/scripts/update/lib/version-detector.js +241 -0
- package/scripts/update/migrations/1.0.x-to-1.3.0.js +128 -0
- package/scripts/update/migrations/1.1.x-to-1.3.0.js +29 -0
- package/scripts/update/migrations/1.2.x-to-1.3.0.js +29 -0
- package/scripts/update/migrations/1.3.x-to-1.5.0.js +29 -0
- package/scripts/update/migrations/1.4.x-to-1.5.0.js +29 -0
- package/scripts/update/migrations/1.5.x-to-1.6.0.js +95 -0
- package/scripts/update/migrations/1.6.x-to-1.7.0.js +29 -0
- package/scripts/update/migrations/1.7.x-to-2.0.0.js +31 -0
- package/scripts/update/migrations/registry.js +194 -0
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Learning Card: {experiment-name}"
|
|
3
|
+
date: {date}
|
|
4
|
+
created-by: {user-name} with Max (learning-decision-expert)
|
|
5
|
+
experiment-name: {experiment-name}
|
|
6
|
+
hypothesis: {hypothesis-summary}
|
|
7
|
+
status: DRAFT
|
|
8
|
+
confidence: {HIGH|MEDIUM|LOW|EXPLORATORY}
|
|
9
|
+
outcome: {validated|invalidated|partially-validated|inconclusive}
|
|
10
|
+
tags: [{tags}]
|
|
11
|
+
version: 1.0
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Learning Card: {experiment-name}
|
|
15
|
+
|
|
16
|
+
> **Confidence:** {confidence-rating} | **Outcome:** {outcome} | **Date:** {date}
|
|
17
|
+
|
|
18
|
+
## One-Sentence Summary
|
|
19
|
+
|
|
20
|
+
{one-sentence-summary}
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## Experiment Context
|
|
25
|
+
|
|
26
|
+
### Hypothesis
|
|
27
|
+
{hypothesis-statement}
|
|
28
|
+
|
|
29
|
+
### Method
|
|
30
|
+
- **Type:** {experiment-type}
|
|
31
|
+
- **Sample:** {sample-size-and-description}
|
|
32
|
+
- **Duration:** {experiment-duration}
|
|
33
|
+
- **Recruitment:** {how-participants-selected}
|
|
34
|
+
|
|
35
|
+
### Pre-Defined Success Criteria
|
|
36
|
+
|
|
37
|
+
| Metric | Target | Rationale |
|
|
38
|
+
|--------|--------|-----------|
|
|
39
|
+
| {metric-1} | {target-1} | {why-this-threshold} |
|
|
40
|
+
| {metric-2} | {target-2} | {why-this-threshold} |
|
|
41
|
+
| {metric-3} | {target-3} | {why-this-threshold} |
|
|
42
|
+
|
|
43
|
+
### Strategic Context
|
|
44
|
+
{why-this-experiment-mattered}
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Raw Results
|
|
49
|
+
|
|
50
|
+
### Quantitative
|
|
51
|
+
|
|
52
|
+
| Metric | Target | Actual | Met? |
|
|
53
|
+
|--------|--------|--------|------|
|
|
54
|
+
| {metric-1} | {target-1} | {actual-1} | {yes/no/partially} |
|
|
55
|
+
| {metric-2} | {target-2} | {actual-2} | {yes/no/partially} |
|
|
56
|
+
| {metric-3} | {target-3} | {actual-3} | {yes/no/partially} |
|
|
57
|
+
|
|
58
|
+
### Qualitative Highlights
|
|
59
|
+
|
|
60
|
+
**Key Quotes:**
|
|
61
|
+
{direct-quotes-with-context}
|
|
62
|
+
|
|
63
|
+
**Observed Behaviors:**
|
|
64
|
+
{notable-behaviors}
|
|
65
|
+
|
|
66
|
+
### Unexpected Findings
|
|
67
|
+
{unexpected-findings}
|
|
68
|
+
|
|
69
|
+
### Data Quality Assessment
|
|
70
|
+
- **Sample representativeness:** {assessment}
|
|
71
|
+
- **Duration adequacy:** {assessment}
|
|
72
|
+
- **Confounding factors:** {factors}
|
|
73
|
+
- **Missing data:** {gaps}
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Analysis
|
|
78
|
+
|
|
79
|
+
### Success Criteria Assessment
|
|
80
|
+
|
|
81
|
+
| Criterion | Result | Rating | Reasoning |
|
|
82
|
+
|-----------|--------|--------|-----------|
|
|
83
|
+
| {criterion-1} | {result-1} | {CLEAR PASS/MARGINAL PASS/MARGINAL FAIL/CLEAR FAIL/INCONCLUSIVE} | {why} |
|
|
84
|
+
| {criterion-2} | {result-2} | {rating} | {why} |
|
|
85
|
+
|
|
86
|
+
### Key Patterns
|
|
87
|
+
{pattern-analysis}
|
|
88
|
+
|
|
89
|
+
### Qualitative-Quantitative Alignment
|
|
90
|
+
{alignment-analysis}
|
|
91
|
+
|
|
92
|
+
### Alternative Explanations Considered
|
|
93
|
+
{alternative-explanations}
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Validated Learnings
|
|
98
|
+
|
|
99
|
+
### Assumption Status
|
|
100
|
+
|
|
101
|
+
| Assumption | Status | Confidence | Key Evidence |
|
|
102
|
+
|-----------|--------|------------|--------------|
|
|
103
|
+
| {assumption-1} | {VALIDATED/INVALIDATED/PARTIALLY/UNRESOLVED} | {HIGH/MEDIUM/LOW} | {evidence-summary} |
|
|
104
|
+
| {assumption-2} | {status} | {confidence} | {evidence} |
|
|
105
|
+
|
|
106
|
+
### Core Learning Statements
|
|
107
|
+
|
|
108
|
+
1. **Learning:** {learning-statement-1}
|
|
109
|
+
**Evidence:** {supporting-evidence}
|
|
110
|
+
**Conditions:** {when-this-holds}
|
|
111
|
+
|
|
112
|
+
2. **Learning:** {learning-statement-2}
|
|
113
|
+
**Evidence:** {supporting-evidence}
|
|
114
|
+
**Conditions:** {when-this-holds}
|
|
115
|
+
|
|
116
|
+
3. **Learning:** {learning-statement-3}
|
|
117
|
+
**Evidence:** {supporting-evidence}
|
|
118
|
+
**Conditions:** {when-this-holds}
|
|
119
|
+
|
|
120
|
+
### New Questions Raised
|
|
121
|
+
|
|
122
|
+
| Question | Why It Matters | How to Test | Priority |
|
|
123
|
+
|----------|---------------|-------------|----------|
|
|
124
|
+
| {question-1} | {importance} | {experiment-idea} | {HIGH/MEDIUM/LOW} |
|
|
125
|
+
| {question-2} | {importance} | {experiment-idea} | {priority} |
|
|
126
|
+
|
|
127
|
+
### Evidence Quality Rating
|
|
128
|
+
**Overall:** {HIGH/MODERATE/WEAK/ANECDOTAL}
|
|
129
|
+
**Justification:** {why-this-rating}
|
|
130
|
+
|
|
131
|
+
---
|
|
132
|
+
|
|
133
|
+
## Implications
|
|
134
|
+
|
|
135
|
+
### Product Implications
|
|
136
|
+
{product-implications-with-build-dont-build-modify-investigate}
|
|
137
|
+
|
|
138
|
+
### Strategy Implications
|
|
139
|
+
{strategy-implications}
|
|
140
|
+
|
|
141
|
+
### Assumption Cascade
|
|
142
|
+
**Now stronger:** {reinforced-assumptions}
|
|
143
|
+
**Now weaker:** {undermined-assumptions}
|
|
144
|
+
**New to test:** {new-assumptions}
|
|
145
|
+
|
|
146
|
+
### Decision Trigger
|
|
147
|
+
{does-this-trigger-a-major-decision}
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Recommended Next Actions
|
|
152
|
+
|
|
153
|
+
| Timeframe | Action | Owner | Rationale |
|
|
154
|
+
|-----------|--------|-------|-----------|
|
|
155
|
+
| Immediately | {action} | {who} | {why} |
|
|
156
|
+
| This sprint | {action} | {who} | {why} |
|
|
157
|
+
| This quarter | {action} | {who} | {why} |
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## Connections
|
|
162
|
+
|
|
163
|
+
- **Builds on:** {related-learning-cards}
|
|
164
|
+
- **Contradicts:** {conflicting-findings}
|
|
165
|
+
- **Enables:** {future-experiments}
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Revision History
|
|
170
|
+
|
|
171
|
+
| Date | What Changed | Why | Updated By |
|
|
172
|
+
|------|--------------|-----|------------|
|
|
173
|
+
| {date} | Initial creation | First synthesis | {user-name} |
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
**Created with:** Convoke v2.0.0 - Vortex Pattern (Systematize Stream)
|
|
178
|
+
**Agent:** Max (Learning & Decision Expert)
|
|
179
|
+
**Workflow:** learning-card
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
---
|
|
2
|
+
step: 1
|
|
3
|
+
workflow: learning-card
|
|
4
|
+
title: Experiment Context
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Step 1: Experiment Context
|
|
8
|
+
|
|
9
|
+
Before we analyze results, we need to clearly establish what was tested and why. This context ensures the learning card is useful months from now when you've forgotten the details.
|
|
10
|
+
|
|
11
|
+
## Why This Matters
|
|
12
|
+
|
|
13
|
+
A learning without context is just a random fact. Context establishes:
|
|
14
|
+
- The original hypothesis and its importance to your strategy
|
|
15
|
+
- The method chosen and why it was appropriate
|
|
16
|
+
- The timeline and conditions under which the experiment ran
|
|
17
|
+
- The success criteria that were defined BEFORE seeing results (this prevents moving goalposts)
|
|
18
|
+
|
|
19
|
+
## Your Task
|
|
20
|
+
|
|
21
|
+
Answer these questions to establish the experiment context:
|
|
22
|
+
|
|
23
|
+
### 1. What experiment did you run?
|
|
24
|
+
|
|
25
|
+
Give it a clear name and a one-sentence description.
|
|
26
|
+
- **Name:** (e.g., "Landing Page Smoke Test", "Concierge MVP Week 3", "Price Sensitivity Survey")
|
|
27
|
+
- **Description:** What did you do, in plain language?
|
|
28
|
+
|
|
29
|
+
### 2. What hypothesis were you testing?
|
|
30
|
+
|
|
31
|
+
Use the format: "We believe that [target users] will [expected behavior] because [rationale]."
|
|
32
|
+
|
|
33
|
+
Be specific:
|
|
34
|
+
- **Weak:** "We believe users will like our product"
|
|
35
|
+
- **Strong:** "We believe that remote engineering managers with 5+ direct reports will complete weekly status reviews 40% faster using async video updates because they currently spend 3+ hours/week in synchronous standup meetings"
|
|
36
|
+
|
|
37
|
+
### 3. What method did you use?
|
|
38
|
+
|
|
39
|
+
Describe the experiment type and setup:
|
|
40
|
+
- **Type:** (Interview, Survey, A/B test, Landing page, Concierge MVP, Wizard of Oz, Prototype test, etc.)
|
|
41
|
+
- **Sample size:** How many participants/users/responses?
|
|
42
|
+
- **Duration:** How long did the experiment run?
|
|
43
|
+
- **Recruitment:** How were participants selected? (This matters for bias assessment)
|
|
44
|
+
|
|
45
|
+
### 4. What were the pre-defined success criteria?
|
|
46
|
+
|
|
47
|
+
What thresholds did you set BEFORE running the experiment?
|
|
48
|
+
|
|
49
|
+
Examples:
|
|
50
|
+
- "At least 30% of visitors click the CTA"
|
|
51
|
+
- "7 out of 10 interview participants mention this pain point unprompted"
|
|
52
|
+
- "Completion time under 5 minutes for 80% of participants"
|
|
53
|
+
- "Net Promoter Score above 40"
|
|
54
|
+
|
|
55
|
+
### 5. What was the strategic context?
|
|
56
|
+
|
|
57
|
+
Why did this experiment matter? What decision was it supposed to inform?
|
|
58
|
+
- What stream of the Vortex was this part of?
|
|
59
|
+
- What assumption in your business model does this test?
|
|
60
|
+
- What would you do differently depending on the outcome?
|
|
61
|
+
|
|
62
|
+
## Example
|
|
63
|
+
|
|
64
|
+
**Experiment Name:** Async Status Update Smoke Test
|
|
65
|
+
|
|
66
|
+
**Description:** We created a simple Loom-based workflow where team members recorded 2-minute daily status videos instead of attending standup meetings. Ran with 3 pilot teams for 2 weeks.
|
|
67
|
+
|
|
68
|
+
**Hypothesis:** We believe that remote engineering teams (5-8 people) will maintain alignment without daily standup meetings if they record 2-minute async video updates, because team members report standups are disruptive and poorly timed across time zones.
|
|
69
|
+
|
|
70
|
+
**Method:**
|
|
71
|
+
- **Type:** Concierge MVP (manual process using Loom + Slack channel)
|
|
72
|
+
- **Sample size:** 3 teams, 19 total participants
|
|
73
|
+
- **Duration:** 2 weeks (10 business days)
|
|
74
|
+
- **Recruitment:** Volunteered from teams that had complained about standup meetings in last retrospective
|
|
75
|
+
|
|
76
|
+
**Success Criteria (pre-defined):**
|
|
77
|
+
- 80%+ daily participation rate (at least 8 of 10 days per person)
|
|
78
|
+
- Team alignment score maintained or improved (weekly survey, 1-5 scale)
|
|
79
|
+
- Average video length under 3 minutes
|
|
80
|
+
- Managers report spending less time on status tracking (time log comparison)
|
|
81
|
+
|
|
82
|
+
**Strategic Context:**
|
|
83
|
+
- Part of Externalize stream: testing our value proposition before building
|
|
84
|
+
- Tests core assumption: "Teams will actually do async updates consistently"
|
|
85
|
+
- If validated: proceed to build MVP with recording + dashboard features
|
|
86
|
+
- If invalidated: explore hybrid model (fewer meetings, not zero meetings)
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Your Turn
|
|
91
|
+
|
|
92
|
+
Please provide the experiment context using the structure above.
|
|
93
|
+
|
|
94
|
+
**Important:** If you didn't define success criteria before the experiment, acknowledge that. Post-hoc criteria are a red flag for confirmation bias -- we'll note that in the learning card so future readers know the evidence quality.
|
|
95
|
+
|
|
96
|
+
## Next Step
|
|
97
|
+
|
|
98
|
+
When you've provided complete experiment context, I'll load:
|
|
99
|
+
|
|
100
|
+
{project-root}/_bmad/bme/_vortex/workflows/learning-card/steps/step-02-raw-results.md
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
---
|
|
2
|
+
step: 2
|
|
3
|
+
workflow: learning-card
|
|
4
|
+
title: Raw Results
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Step 2: Raw Results
|
|
8
|
+
|
|
9
|
+
Now we capture the raw data and observations -- exactly what happened, without interpretation. This step is deliberately separated from analysis to prevent confirmation bias.
|
|
10
|
+
|
|
11
|
+
## Why This Matters
|
|
12
|
+
|
|
13
|
+
The most common mistake in experiment analysis is jumping straight to conclusions. Separating raw data from interpretation:
|
|
14
|
+
- **Prevents cherry-picking** - You record ALL results, not just ones that support your hypothesis
|
|
15
|
+
- **Enables re-analysis** - Others can draw different conclusions from the same data
|
|
16
|
+
- **Catches surprises** - The most valuable learnings are often the unexpected ones
|
|
17
|
+
- **Builds credibility** - Stakeholders trust conclusions grounded in visible data
|
|
18
|
+
|
|
19
|
+
## Your Task
|
|
20
|
+
|
|
21
|
+
Capture the raw results without analysis or interpretation. Just the facts.
|
|
22
|
+
|
|
23
|
+
### 1. Quantitative Results
|
|
24
|
+
|
|
25
|
+
Report the numbers against each success criterion:
|
|
26
|
+
|
|
27
|
+
| Metric | Target | Actual | Met? |
|
|
28
|
+
|--------|--------|--------|------|
|
|
29
|
+
| {metric-1} | {target-1} | {actual-1} | Yes/No |
|
|
30
|
+
| {metric-2} | {target-2} | {actual-2} | Yes/No |
|
|
31
|
+
|
|
32
|
+
Include ALL metrics you tracked, even ones that seem irrelevant now.
|
|
33
|
+
|
|
34
|
+
### 2. Qualitative Observations
|
|
35
|
+
|
|
36
|
+
Capture direct quotes, observed behaviors, and notable moments:
|
|
37
|
+
|
|
38
|
+
**Direct Quotes (verbatim where possible):**
|
|
39
|
+
- "..." - Participant 1, context
|
|
40
|
+
- "..." - Participant 2, context
|
|
41
|
+
|
|
42
|
+
**Observed Behaviors:**
|
|
43
|
+
- What did participants actually do (vs. what they said they'd do)?
|
|
44
|
+
- Where did they get stuck, confused, or frustrated?
|
|
45
|
+
- What surprised you?
|
|
46
|
+
|
|
47
|
+
**Notable Moments:**
|
|
48
|
+
- Any "aha" moments?
|
|
49
|
+
- Any emotional reactions?
|
|
50
|
+
- Any participants who used the product/solution in unexpected ways?
|
|
51
|
+
|
|
52
|
+
### 3. Unexpected Findings
|
|
53
|
+
|
|
54
|
+
What happened that you DIDN'T expect? These are often the most valuable data points.
|
|
55
|
+
|
|
56
|
+
- Anything participants mentioned that wasn't in your hypothesis?
|
|
57
|
+
- Any patterns you didn't anticipate?
|
|
58
|
+
- Any metrics that moved in surprising directions?
|
|
59
|
+
- Any segments that behaved differently from others?
|
|
60
|
+
|
|
61
|
+
### 4. Data Quality Assessment
|
|
62
|
+
|
|
63
|
+
Be honest about the limitations:
|
|
64
|
+
|
|
65
|
+
- **Sample bias:** Were participants representative of your target user?
|
|
66
|
+
- **Duration:** Was the experiment long enough to see real behavior?
|
|
67
|
+
- **Confounding factors:** What else was happening that might have affected results?
|
|
68
|
+
- **Missing data:** What couldn't you measure that you wish you had?
|
|
69
|
+
- **Hawthorne effect:** Did participants behave differently because they knew they were being observed?
|
|
70
|
+
|
|
71
|
+
## Example
|
|
72
|
+
|
|
73
|
+
**Quantitative Results:**
|
|
74
|
+
|
|
75
|
+
| Metric | Target | Actual | Met? |
|
|
76
|
+
|--------|--------|--------|------|
|
|
77
|
+
| Daily participation rate | 80%+ | 73% overall (Team A: 91%, Team B: 68%, Team C: 62%) | No |
|
|
78
|
+
| Alignment score (1-5) | Maintain at 3.8+ | Week 1: 3.2, Week 2: 3.9 | Partially (Week 2 only) |
|
|
79
|
+
| Average video length | Under 3 min | 2 min 14 sec | Yes |
|
|
80
|
+
| Manager time on status | Reduce by 30%+ | Manager A: -45%, Manager B: -20%, Manager C: +10% | Mixed |
|
|
81
|
+
|
|
82
|
+
**Qualitative Observations:**
|
|
83
|
+
|
|
84
|
+
Direct quotes:
|
|
85
|
+
- "I actually liked knowing I could watch updates on MY schedule, not theirs" - Manager A
|
|
86
|
+
- "By day 4 I forgot to record. It's just not in my routine yet." - Dev, Team C
|
|
87
|
+
- "The videos were better than standups because people showed their screens instead of just talking" - Tech Lead, Team A
|
|
88
|
+
- "I don't want to be on camera. Can I just do audio?" - Dev, Team B
|
|
89
|
+
|
|
90
|
+
Observed behaviors:
|
|
91
|
+
- Team A adopted a pattern of recording first thing in morning (habit-stacking with coffee)
|
|
92
|
+
- Team B participation dropped steadily over 2 weeks
|
|
93
|
+
- Team C had one champion who recorded every day; others were inconsistent
|
|
94
|
+
- 4 participants started watching videos at 2x speed by day 3
|
|
95
|
+
|
|
96
|
+
Notable moments:
|
|
97
|
+
- Manager A discovered a blocker through a video that had existed for 3 days unreported
|
|
98
|
+
- Team B tried to game it by recording 15-second "nothing to report" videos
|
|
99
|
+
|
|
100
|
+
**Unexpected Findings:**
|
|
101
|
+
- Screen-sharing in videos was MORE informative than verbal standup updates
|
|
102
|
+
- Async worked better for cross-timezone teams (Team A) than co-located teams (Team C)
|
|
103
|
+
- Junior developers participated more consistently than senior developers
|
|
104
|
+
- Two participants started using videos for non-status purposes (design reviews, bug reports)
|
|
105
|
+
|
|
106
|
+
**Data Quality:**
|
|
107
|
+
- Sample bias: All volunteer teams who already disliked standups (positive selection bias)
|
|
108
|
+
- Duration: 2 weeks may not show long-term habit formation
|
|
109
|
+
- Confounding: Team A was starting a new sprint (high motivation), Team C was in maintenance mode (low energy)
|
|
110
|
+
- Missing: No customer impact data, no quality metrics
|
|
111
|
+
- Hawthorne: Likely -- teams knew managers were tracking participation
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Your Turn
|
|
116
|
+
|
|
117
|
+
Please share the raw results of your experiment. Resist the urge to interpret -- just give me the data. We'll make sense of it in the next step.
|
|
118
|
+
|
|
119
|
+
**Reminder:** Include results that contradict your hypothesis too. Those are the most valuable learning opportunities.
|
|
120
|
+
|
|
121
|
+
## Next Step
|
|
122
|
+
|
|
123
|
+
When you've captured the raw results, I'll load:
|
|
124
|
+
|
|
125
|
+
{project-root}/_bmad/bme/_vortex/workflows/learning-card/steps/step-03-analysis.md
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
---
|
|
2
|
+
step: 3
|
|
3
|
+
workflow: learning-card
|
|
4
|
+
title: Analysis
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Step 3: Analysis
|
|
8
|
+
|
|
9
|
+
Now we analyze the raw results. What does the data actually say? This is where we look for patterns, assess significance, and separate signal from noise.
|
|
10
|
+
|
|
11
|
+
## Why This Matters
|
|
12
|
+
|
|
13
|
+
Raw data doesn't speak for itself. Analysis transforms data into insight by:
|
|
14
|
+
- Distinguishing between statistically meaningful results and random variation
|
|
15
|
+
- Identifying patterns across different segments and metrics
|
|
16
|
+
- Spotting contradictions between quantitative and qualitative data
|
|
17
|
+
- Assessing whether results are strong enough to act on
|
|
18
|
+
|
|
19
|
+
## Your Task
|
|
20
|
+
|
|
21
|
+
Work through these analysis lenses systematically:
|
|
22
|
+
|
|
23
|
+
### 1. Success Criteria Assessment
|
|
24
|
+
|
|
25
|
+
For each pre-defined success criterion, assess the result:
|
|
26
|
+
|
|
27
|
+
**Use this framework:**
|
|
28
|
+
- **CLEAR PASS:** Met or exceeded the threshold with margin
|
|
29
|
+
- **MARGINAL PASS:** Met the threshold but barely -- could go either way with more data
|
|
30
|
+
- **MARGINAL FAIL:** Close to threshold but didn't reach it -- worth investigating why
|
|
31
|
+
- **CLEAR FAIL:** Clearly did not meet the threshold
|
|
32
|
+
- **INCONCLUSIVE:** Not enough data or too many confounding factors to judge
|
|
33
|
+
|
|
34
|
+
For each: explain WHY you're assigning that rating.
|
|
35
|
+
|
|
36
|
+
### 2. Pattern Analysis
|
|
37
|
+
|
|
38
|
+
Look for patterns in the data:
|
|
39
|
+
|
|
40
|
+
**Segment Differences:**
|
|
41
|
+
- Did different user segments behave differently?
|
|
42
|
+
- Was there a "best case" group vs. a "worst case" group?
|
|
43
|
+
- What distinguishes users who succeeded from those who didn't?
|
|
44
|
+
|
|
45
|
+
**Time-based Patterns:**
|
|
46
|
+
- Did behavior change over the experiment duration?
|
|
47
|
+
- Was there a learning curve? A novelty effect? A drop-off?
|
|
48
|
+
- At what point did behavior stabilize (or didn't it)?
|
|
49
|
+
|
|
50
|
+
**Correlation Patterns:**
|
|
51
|
+
- Do any metrics correlate with each other?
|
|
52
|
+
- Does high engagement in one area predict success in another?
|
|
53
|
+
- Are there leading indicators of success or failure?
|
|
54
|
+
|
|
55
|
+
### 3. Qualitative-Quantitative Alignment
|
|
56
|
+
|
|
57
|
+
Compare what people SAID with what they DID:
|
|
58
|
+
|
|
59
|
+
- Do the numbers match the quotes?
|
|
60
|
+
- Are there contradictions? (e.g., "I loved it" but usage dropped)
|
|
61
|
+
- Which should you trust more in this context -- the quantitative or qualitative data?
|
|
62
|
+
|
|
63
|
+
### 4. Statistical Rigor Check
|
|
64
|
+
|
|
65
|
+
Be honest about confidence levels:
|
|
66
|
+
|
|
67
|
+
- **Sample size:** Was it large enough to draw conclusions? (Rule of thumb: for surveys, 30+ responses; for A/B tests, use a significance calculator; for interviews, 5-8 per segment for qualitative patterns)
|
|
68
|
+
- **Confidence level:** If quantitative, what's the statistical significance? (p-value, confidence interval)
|
|
69
|
+
- **Effect size:** Even if statistically significant, is the effect large enough to matter practically?
|
|
70
|
+
- **Replication:** Would you expect the same results if you ran this again?
|
|
71
|
+
|
|
72
|
+
### 5. Alternative Explanations
|
|
73
|
+
|
|
74
|
+
For each finding, consider: is there another explanation?
|
|
75
|
+
|
|
76
|
+
**Common alternative explanations:**
|
|
77
|
+
- **Selection bias:** Did you only test with people predisposed to succeed?
|
|
78
|
+
- **Novelty effect:** Did they engage because it was new, not because it was better?
|
|
79
|
+
- **Hawthorne effect:** Did observation itself change behavior?
|
|
80
|
+
- **Regression to the mean:** Were your baseline measurements unusually high/low?
|
|
81
|
+
- **Confounding variables:** Was something else changing at the same time?
|
|
82
|
+
|
|
83
|
+
## Example
|
|
84
|
+
|
|
85
|
+
**Success Criteria Assessment:**
|
|
86
|
+
|
|
87
|
+
| Criterion | Result | Rating | Reasoning |
|
|
88
|
+
|-----------|--------|--------|-----------|
|
|
89
|
+
| 80%+ daily participation | 73% overall | MARGINAL FAIL | 7 points short, but Team A hit 91%. Suggests it's achievable with right conditions, not universally. |
|
|
90
|
+
| Alignment score 3.8+ | 3.2 -> 3.9 | MARGINAL PASS | Only met in Week 2. Learning curve effect likely. Need longer experiment to know if this holds. |
|
|
91
|
+
| Video under 3 min | 2:14 avg | CLEAR PASS | Well under threshold with no outliers above 4 min. |
|
|
92
|
+
| Manager time -30% | -45%, -20%, +10% | INCONCLUSIVE | Too small a sample (n=3) and too varied. Manager C's increase may be due to monitoring the new process. |
|
|
93
|
+
|
|
94
|
+
**Pattern Analysis:**
|
|
95
|
+
- Clear segment difference: Cross-timezone teams benefited most (Team A: +91% participation, highest alignment improvement). Co-located teams saw least benefit (Team C: 62% participation).
|
|
96
|
+
- Time pattern: Week 1 participation was 80%, Week 2 dropped to 67%. Novelty effect likely.
|
|
97
|
+
- Correlation: Teams with a "recording champion" (someone who recorded every day) had 25% higher overall participation.
|
|
98
|
+
|
|
99
|
+
**Qualitative-Quantitative Alignment:**
|
|
100
|
+
- CONTRADICTION: Team B members said they "liked" async updates in the mid-point survey, but their participation dropped from 78% (Week 1) to 58% (Week 2). Words didn't match behavior.
|
|
101
|
+
- ALIGNMENT: Manager A's quote about discovering blockers matched the quantitative finding that screen-sharing revealed issues faster.
|
|
102
|
+
|
|
103
|
+
**Statistical Rigor:**
|
|
104
|
+
- Sample: n=19 across 3 teams is too small for quantitative conclusions. Treat all findings as directional signals, not proof.
|
|
105
|
+
- Confidence: Cannot calculate meaningful p-values with this sample size.
|
|
106
|
+
- Effect size: Alignment score improvement (0.7 points on 5-point scale) is practically meaningful IF it holds.
|
|
107
|
+
|
|
108
|
+
**Alternative Explanations:**
|
|
109
|
+
- Volunteer bias is the biggest threat: teams who already hated standups were predisposed to prefer any alternative
|
|
110
|
+
- Novelty effect likely contributed to Week 1's higher participation
|
|
111
|
+
- Manager monitoring likely boosted participation (Hawthorne effect)
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Your Turn
|
|
116
|
+
|
|
117
|
+
Please analyze your raw results using the lenses above. Be rigorous and honest -- a learning card that says "inconclusive" is more valuable than one that pretends certainty.
|
|
118
|
+
|
|
119
|
+
**Key principle:** It's better to learn that you can't conclude anything than to draw false conclusions. Uncertainty is data too.
|
|
120
|
+
|
|
121
|
+
## Next Step
|
|
122
|
+
|
|
123
|
+
When you've completed the analysis, I'll load:
|
|
124
|
+
|
|
125
|
+
{project-root}/_bmad/bme/_vortex/workflows/learning-card/steps/step-04-validated-learning.md
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
---
|
|
2
|
+
step: 4
|
|
3
|
+
workflow: learning-card
|
|
4
|
+
title: Validated Learning
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Step 4: Validated Learning
|
|
8
|
+
|
|
9
|
+
Now we extract the validated learnings -- the actual knowledge gained from this experiment. What assumptions were confirmed? What was invalidated? What new questions emerged?
|
|
10
|
+
|
|
11
|
+
## Why This Matters
|
|
12
|
+
|
|
13
|
+
Validated learning is the currency of Lean Startup. It's the difference between "we tried stuff" and "we know something we didn't know before." Validated learnings:
|
|
14
|
+
- Convert experiment effort into permanent organizational knowledge
|
|
15
|
+
- Reduce uncertainty for future decisions
|
|
16
|
+
- Prevent repeating the same experiments
|
|
17
|
+
- Build confidence (or appropriate caution) for next steps
|
|
18
|
+
|
|
19
|
+
## Your Task
|
|
20
|
+
|
|
21
|
+
### 1. Classify Each Original Assumption
|
|
22
|
+
|
|
23
|
+
For every assumption that was tested, categorize it:
|
|
24
|
+
|
|
25
|
+
**VALIDATED** - Evidence strongly supports this assumption
|
|
26
|
+
- What specific evidence validates it?
|
|
27
|
+
- How confident are you? (High / Medium / Low)
|
|
28
|
+
- What conditions are required for this to hold?
|
|
29
|
+
|
|
30
|
+
**INVALIDATED** - Evidence contradicts this assumption
|
|
31
|
+
- What specific evidence invalidates it?
|
|
32
|
+
- Was it completely wrong or just partially wrong?
|
|
33
|
+
- What's the corrected understanding?
|
|
34
|
+
|
|
35
|
+
**PARTIALLY VALIDATED** - Evidence supports it in some contexts but not others
|
|
36
|
+
- Under what conditions does it hold?
|
|
37
|
+
- Under what conditions does it fail?
|
|
38
|
+
- What's the refined hypothesis?
|
|
39
|
+
|
|
40
|
+
**UNRESOLVED** - The experiment didn't generate enough evidence to decide
|
|
41
|
+
- Why couldn't you resolve it? (Sample too small? Wrong method? Confounding factors?)
|
|
42
|
+
- What would you need to resolve it?
|
|
43
|
+
|
|
44
|
+
### 2. Extract the Core Learnings
|
|
45
|
+
|
|
46
|
+
Distill your findings into clear learning statements. A good learning statement:
|
|
47
|
+
- Is specific and actionable
|
|
48
|
+
- States what you now KNOW (not what you hope)
|
|
49
|
+
- Includes the conditions under which it's true
|
|
50
|
+
- Is falsifiable by future experiments
|
|
51
|
+
|
|
52
|
+
**Format:** "We learned that [specific finding] when [conditions], based on [evidence]."
|
|
53
|
+
|
|
54
|
+
**Examples of strong learning statements:**
|
|
55
|
+
- "We learned that cross-timezone teams adopt async video updates at 90%+ participation rates when a team champion models the behavior daily, based on Team A's 91% participation vs. Team C's 62% without a champion."
|
|
56
|
+
- "We learned that novelty drives initial adoption but doesn't sustain it, based on participation dropping from 80% (Week 1) to 67% (Week 2) across all teams."
|
|
57
|
+
- "We learned that we CANNOT conclude whether async updates improve team alignment, because our sample size (n=3 teams) and duration (2 weeks) were insufficient."
|
|
58
|
+
|
|
59
|
+
**Examples of weak learning statements (avoid these):**
|
|
60
|
+
- "Users like async updates" (too vague, no conditions, no evidence cited)
|
|
61
|
+
- "Our product is better than standups" (not tested, opinion masquerading as learning)
|
|
62
|
+
- "The experiment was successful" (binary thinking, misses nuance)
|
|
63
|
+
|
|
64
|
+
### 3. Identify New Questions
|
|
65
|
+
|
|
66
|
+
Every good experiment generates more questions than it answers. What new questions emerged?
|
|
67
|
+
|
|
68
|
+
**Format for each:**
|
|
69
|
+
- **Question:** What do you now need to know?
|
|
70
|
+
- **Why it matters:** What decision does this inform?
|
|
71
|
+
- **How to test:** What experiment would answer this?
|
|
72
|
+
- **Priority:** How urgently do you need this answer?
|
|
73
|
+
|
|
74
|
+
### 4. Rate the Overall Learning Quality
|
|
75
|
+
|
|
76
|
+
Honestly assess the quality of evidence behind your learnings:
|
|
77
|
+
|
|
78
|
+
**STRONG EVIDENCE:**
|
|
79
|
+
- Adequate sample size
|
|
80
|
+
- Pre-defined success criteria
|
|
81
|
+
- Controlled for major biases
|
|
82
|
+
- Results are replicable
|
|
83
|
+
|
|
84
|
+
**MODERATE EVIDENCE:**
|
|
85
|
+
- Directional but not definitive
|
|
86
|
+
- Some bias concerns
|
|
87
|
+
- Small sample but consistent patterns
|
|
88
|
+
- Would benefit from replication
|
|
89
|
+
|
|
90
|
+
**WEAK EVIDENCE:**
|
|
91
|
+
- Very small sample
|
|
92
|
+
- Post-hoc criteria
|
|
93
|
+
- Significant bias concerns
|
|
94
|
+
- Contradictory signals
|
|
95
|
+
|
|
96
|
+
**ANECDOTAL:**
|
|
97
|
+
- Based on individual stories, not patterns
|
|
98
|
+
- No quantitative support
|
|
99
|
+
- High risk of confirmation bias
|
|
100
|
+
|
|
101
|
+
## Example
|
|
102
|
+
|
|
103
|
+
**Assumption Classification:**
|
|
104
|
+
|
|
105
|
+
| Assumption | Status | Confidence | Evidence |
|
|
106
|
+
|-----------|--------|------------|----------|
|
|
107
|
+
| Teams will do async updates consistently | PARTIALLY VALIDATED | Medium | Works with champions (91%) but fails without (62-68%) |
|
|
108
|
+
| Async maintains team alignment | UNRESOLVED | Low | Alignment improved Week 2 but sample too small |
|
|
109
|
+
| Managers will save time | PARTIALLY VALIDATED | Low | 1 of 3 saved significant time; 1 actually spent more |
|
|
110
|
+
| Screen-sharing adds value | VALIDATED | Medium | Unprompted adoption + blocker discovery incident |
|
|
111
|
+
|
|
112
|
+
**Core Learnings:**
|
|
113
|
+
|
|
114
|
+
1. "We learned that async video updates CAN achieve 90%+ adoption when a team champion records daily and the team is distributed across time zones, based on Team A's results. However, without a champion and with co-located teams, adoption drops below 70%."
|
|
115
|
+
|
|
116
|
+
2. "We learned that video screen-sharing is more information-dense than verbal standup reports, based on the unprompted adoption by 12/19 participants and the discovery of a 3-day-old blocker that verbal standups had missed."
|
|
117
|
+
|
|
118
|
+
3. "We learned that initial enthusiasm does NOT predict sustained usage, based on the 13-point drop in participation between Week 1 and Week 2, despite positive survey responses throughout."
|
|
119
|
+
|
|
120
|
+
**New Questions:**
|
|
121
|
+
- Can we design a habit-forming mechanism to sustain participation beyond the novelty period? (HIGH PRIORITY -- blocks MVP decision)
|
|
122
|
+
- Does async work for teams larger than 8 people? (MEDIUM -- scoping question)
|
|
123
|
+
- Would audio-only options increase participation for camera-shy team members? (MEDIUM -- 2 participants raised this)
|
|
124
|
+
|
|
125
|
+
**Overall Evidence Quality:** MODERATE - Directional findings with clear patterns, but sample too small for definitive conclusions. Should replicate with 8-10 teams over 4+ weeks before committing to building.
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## Your Turn
|
|
130
|
+
|
|
131
|
+
Please extract your validated learnings using the structure above. Remember: acknowledging what you DON'T know is as valuable as stating what you do know.
|
|
132
|
+
|
|
133
|
+
**Key principle:** A validated learning is not "we were right." It's "we now understand this specific thing about how our users behave under these conditions, and here's the evidence."
|
|
134
|
+
|
|
135
|
+
## Next Step
|
|
136
|
+
|
|
137
|
+
When you've extracted your validated learnings, I'll load:
|
|
138
|
+
|
|
139
|
+
{project-root}/_bmad/bme/_vortex/workflows/learning-card/steps/step-05-implications.md
|