latticerb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE +176 -0
- data/README.md +121 -0
- data/exe/lattice +7 -0
- data/lib/lattice/agent.rb +153 -0
- data/lib/lattice/cli.rb +469 -0
- data/lib/lattice/constraints.rb +357 -0
- data/lib/lattice/formatters.rb +126 -0
- data/lib/lattice/ipog.rb +222 -0
- data/lib/lattice/mersenne_twister.rb +102 -0
- data/lib/lattice/model.rb +83 -0
- data/lib/lattice/parser.rb +387 -0
- data/lib/lattice/resources/claude/lattice-workflow/SKILL.md +142 -0
- data/lib/lattice/resources/claude/lattice-workflow/references/modeling-rules.md +52 -0
- data/lib/lattice/resources/claude/lattice-workflow/references/worked-examples.md +59 -0
- data/lib/lattice/resources/codex/lattice-workflow/SKILL.md +142 -0
- data/lib/lattice/resources/codex/lattice-workflow/agents/openai.yaml +7 -0
- data/lib/lattice/resources/codex/lattice-workflow/references/modeling-rules.md +52 -0
- data/lib/lattice/resources/codex/lattice-workflow/references/worked-examples.md +59 -0
- data/lib/lattice/resources/generic/lattice-workflow/SKILL.md +150 -0
- data/lib/lattice/resources/generic/lattice-workflow/references/modeling-rules.md +52 -0
- data/lib/lattice/resources/generic/lattice-workflow/references/worked-examples.md +59 -0
- data/lib/lattice/version.rb +5 -0
- data/lib/lattice.rb +87 -0
- data/lib/latticerb.rb +4 -0
- metadata +86 -0
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lattice-workflow
|
|
3
|
+
description: Use when planning or testing behavior or variant surfaces with interacting states, roles, permissions, feature flags, modes, providers, optional fields, component props, rendering branches, template variants, config matrices, lifecycle states, combinatorial edge cases, pairwise coverage, or t-way coverage. Extract a Lattice schema, run the local Lattice CLI, and use generated rows to strengthen plans, tests, fixtures, rendered variants, or review matrices.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Lattice Workflow
|
|
7
|
+
|
|
8
|
+
Use this skill to keep Claude on extraction and interpretation while Lattice handles the combinatorics.
|
|
9
|
+
|
|
10
|
+
## When To Use It
|
|
11
|
+
|
|
12
|
+
- derive a combinatorial model from a feature or PRD
|
|
13
|
+
- derive a test matrix from existing code
|
|
14
|
+
- derive a variant matrix from a component, partial, template, design-system story, or config surface
|
|
15
|
+
- convert edge cases into explicit parameter/value constraints
|
|
16
|
+
- generate pairwise or 3-way scenarios without hand-enumerating combinations
|
|
17
|
+
- render a compact visual review matrix instead of every possible variant
|
|
18
|
+
- review a plan involving roles, permissions, modes, providers, feature flags, optional fields, component props, rendering branches, template variants, config matrices, or lifecycle states
|
|
19
|
+
- find coverage gaps where bugs hide in interactions rather than single branches
|
|
20
|
+
|
|
21
|
+
## When NOT To Use It
|
|
22
|
+
|
|
23
|
+
- **Small enumerable spaces.** Fewer than ~10 total combinations — enumerate by hand.
|
|
24
|
+
- **Independent dimensions.** No interactions can surface a bug; each parameter affects behavior independently.
|
|
25
|
+
- **Non-deterministic systems.** The same input row won't produce repeatable output (e.g., testing LLM responses).
|
|
26
|
+
|
|
27
|
+
## Minimal Example
|
|
28
|
+
|
|
29
|
+
A schema (YAML on stdin):
|
|
30
|
+
|
|
31
|
+
```yaml
|
|
32
|
+
model_name: checkout
|
|
33
|
+
parameters:
|
|
34
|
+
user: [signed_in, guest]
|
|
35
|
+
payment: [card, apple_pay, paypal]
|
|
36
|
+
cart: [empty, single, multi]
|
|
37
|
+
constraints:
|
|
38
|
+
- invalid_pair: {payment: apple_pay, cart: empty}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
One generated row (JSON):
|
|
42
|
+
|
|
43
|
+
```json
|
|
44
|
+
{"user": "guest", "payment": "card", "cart": "multi"}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
The schema is the contract. The generated rows are the source of truth. The schema file is optional transport.
|
|
48
|
+
|
|
49
|
+
## Preflight
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
command -v lattice >/dev/null 2>&1 && lattice --help >/dev/null
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
If it fails, install the Ruby gem from RubyGems (or add `gem "latticerb"` to the project Gemfile):
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
gem install latticerb
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The Python implementation (`pipx install lattice-cli` from PyPI) is interchangeable — both produce byte-identical output for the same schema and seed.
|
|
62
|
+
|
|
63
|
+
## Workflow
|
|
64
|
+
|
|
65
|
+
1. **Extract the schema.** Choose one coherent interaction surface — one feature, one workflow, one service behavior, one endpoint family, one component's variant space, one template family, or one config matrix. Parameters represent independent decisions or state partitions. Values are meaningful partitions, not every literal in the codebase.
|
|
66
|
+
|
|
67
|
+
2. **Encode constraints.** Use the smallest constraint type that matches the rule (see [Constraint Selection](#constraint-selection) below). Encode true impossibilities — do not strip them and mark invalid rows downstream.
|
|
68
|
+
|
|
69
|
+
3. **Validate.**
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
cat schema.json | lattice validate
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
4. **Generate.** Default to pairwise unless explicitly asked for higher strength.
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
cat schema.json | lattice generate
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
5. **Interpret the output.** Plan mode: rows become plan revisions, missing decisions, review scenarios. Test mode: rows become assertions (see [Test Mode Discipline](#test-mode-discipline) below). Variant mode: rows become fixtures, rendered variants, screenshots, visual-diff inputs.
|
|
82
|
+
|
|
83
|
+
## Constraint Selection
|
|
84
|
+
|
|
85
|
+
| Rule shape | Use |
|
|
86
|
+
|---|---|
|
|
87
|
+
| Two assignments cannot coexist | `invalid_pair` |
|
|
88
|
+
| One-to-one pairing (if A then B and if B then A) | `bidirectional` |
|
|
89
|
+
| A requires B, but B may appear elsewhere | `forward_dep` |
|
|
90
|
+
| Child parameter only applies for some parent values | `conditional` |
|
|
91
|
+
| This assignment set must appear in at least one row | `forced` |
|
|
92
|
+
| Rule only breaks under three or more antecedents | `higher_order` |
|
|
93
|
+
|
|
94
|
+
Prefer the smallest constraint type that matches. Use `conditional` instead of hand-writing `N/A` values.
|
|
95
|
+
|
|
96
|
+
## Test Mode Discipline
|
|
97
|
+
|
|
98
|
+
Source expected behavior from intent — the user's request, a spec, a PRD, a ticket, or explicit reasoning about what the system *should* do. **The code under test is evidence, not truth.** If intent is ambiguous for a row, stop and ask the user.
|
|
99
|
+
|
|
100
|
+
Do not silently encode current behavior and hedge with `bug_signal:`-style annotations. That produces characterization tests, which lock in the implementation (bugs included) instead of catching divergence from intent. The asymmetries Lattice is designed to expose disappear when the test suite mirrors the same mental model as the code.
|
|
101
|
+
|
|
102
|
+
## Common Pitfalls
|
|
103
|
+
|
|
104
|
+
❌ Strip constraints to make generation easier, then mark impossible rows invalid after the fact.
|
|
105
|
+
✅ Encode the constraint in the schema, regenerate. Encoding constraints changes the coverage universe to valid interactions — that's the point.
|
|
106
|
+
|
|
107
|
+
❌ Write tests that assert what the code currently does.
|
|
108
|
+
✅ Write tests that assert what the spec, PRD, or user requested. If intent is ambiguous, ask before writing.
|
|
109
|
+
|
|
110
|
+
❌ Separate parameters for two fields where one determines the other (e.g., `currency` and `currency_symbol`).
|
|
111
|
+
✅ One parameter plus a constraint expressing the dependency.
|
|
112
|
+
|
|
113
|
+
❌ Brittle, prose-y value names (`payment_with_apple_pay_using_new_token_flow`).
|
|
114
|
+
✅ Compact partitions: `apple_pay`, `apple_pay_new_token`.
|
|
115
|
+
|
|
116
|
+
❌ Manually add or remove rows after `lattice generate`.
|
|
117
|
+
✅ If the generated set is wrong, fix the schema. Hand-editing rows breaks the coverage guarantee.
|
|
118
|
+
|
|
119
|
+
## References
|
|
120
|
+
|
|
121
|
+
- [references/modeling-rules.md](references/modeling-rules.md) — extraction heuristics, parameter selection, value partitioning, constraint guidance
|
|
122
|
+
- [references/worked-examples.md](references/worked-examples.md) — concrete examples in this repo
|
|
123
|
+
|
|
124
|
+
## Development Fallback
|
|
125
|
+
|
|
126
|
+
In a checkout of the Lattice source where the package is not installed:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
# Ruby checkout
|
|
130
|
+
cat schema.json | ruby -Ilib exe/lattice validate
|
|
131
|
+
cat schema.json | ruby -Ilib exe/lattice generate
|
|
132
|
+
|
|
133
|
+
# Python checkout
|
|
134
|
+
cat schema.json | PYTHONPATH=src python3 -m lattice validate
|
|
135
|
+
cat schema.json | PYTHONPATH=src python3 -m lattice generate
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
To install or refresh this skill from the installed package:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
lattice agent install-claude-skill
|
|
142
|
+
```
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Modeling Rules
|
|
2
|
+
|
|
3
|
+
Use this file when you are extracting a schema from code or prose and need tighter guardrails than the main skill body provides.
|
|
4
|
+
|
|
5
|
+
## Parameter Selection
|
|
6
|
+
|
|
7
|
+
- Scope one schema to one coherent behavior or rendering surface.
|
|
8
|
+
Good scopes: one feature, one service behavior, one workflow, one endpoint family, one state-machine slice, one component's variant space, one template family.
|
|
9
|
+
Bad default scope: the whole app.
|
|
10
|
+
- Model the decisions that change behavior, not every field in a payload.
|
|
11
|
+
- Prefer parameters that are visible in tests, UI branches, rendering branches, validation logic, template variants, component props, or feature flags.
|
|
12
|
+
- Split large raw domains into behavioral partitions.
|
|
13
|
+
Good: `pay_range = present|absent`
|
|
14
|
+
Bad: every exact salary string
|
|
15
|
+
- Avoid separate parameters for values that are already implied by another parameter unless the implication is partial and needs a constraint.
|
|
16
|
+
|
|
17
|
+
## Value Selection
|
|
18
|
+
|
|
19
|
+
- Values should be mutually exclusive and collectively useful for testing.
|
|
20
|
+
- Keep domains small and meaningful. Most parameters should land in the 2-5 value range.
|
|
21
|
+
- Normalize booleans as `true` and `false`.
|
|
22
|
+
- For optional subordinate parameters, let `conditional` introduce `N/A`. Do not hand-add `N/A` unless the schema genuinely needs it as a first-class value.
|
|
23
|
+
|
|
24
|
+
## Constraint Choice
|
|
25
|
+
|
|
26
|
+
- `invalid_pair`: two assignments cannot coexist
|
|
27
|
+
- `bidirectional`: one-to-one pairing; if A then B and if B then A
|
|
28
|
+
- `forward_dep`: A requires B, but B may appear elsewhere
|
|
29
|
+
- `conditional`: a child parameter only applies for some parent values
|
|
30
|
+
- `forced`: this assignment set must appear in at least one generated row
|
|
31
|
+
- `higher_order`: the invalid rule only appears when two or more antecedents are true
|
|
32
|
+
|
|
33
|
+
Choose the smallest expressive type. If a rule can be represented as `invalid_pair`, do not use `higher_order`.
|
|
34
|
+
|
|
35
|
+
Do not strip true constraints to get a smaller or easier unconstrained run. Do not generate nonsensical rows and mark them invalid downstream. Lattice's coverage accounting should run over valid interactions, so impossible combinations belong in the schema as `invalid_pair`, `forward_dep`, `bidirectional`, `conditional`, or `higher_order` constraints.
|
|
36
|
+
|
|
37
|
+
## Extraction Process
|
|
38
|
+
|
|
39
|
+
1. Choose the behavior or rendering surface you are modeling.
|
|
40
|
+
2. Start from branches, validations, and business rules inside that scope.
|
|
41
|
+
3. Convert prose like "only when", "must", "cannot", and "except" into candidate constraints.
|
|
42
|
+
4. Remove cosmetic parameters that do not affect behavior.
|
|
43
|
+
5. Check whether a missing value is really a separate partition or just noise.
|
|
44
|
+
6. Validate the schema.
|
|
45
|
+
7. Only after validation, generate scenarios.
|
|
46
|
+
|
|
47
|
+
## Interpretation
|
|
48
|
+
|
|
49
|
+
- Plan mode: each row becomes a review scenario or a missing decision in the plan.
|
|
50
|
+
- Test mode: each row becomes a candidate setup matrix. Source the expected behavior from intent — user request, spec, PRD, or ticket — not from the code under test. If you cannot determine intent for a row, stop and surface the ambiguity to the user. Do not encode the current behavior and label it `bug_signal:` to hedge: characterization tests pass for the wrong reasons and hide the asymmetries Lattice is designed to expose. Example: if a mailbox accepts a spoofed `From:` header today, the assertion is still "rejects spoofed sender" — let the test fail and fix the code, do not invert the assertion to match the bug.
|
|
51
|
+
- Variant mode: each row becomes a rendered variant, fixture, screenshot, visual diff input, contact-sheet item, or review checklist item.
|
|
52
|
+
- Generated rows are not test names by themselves. The agent still needs to map them to application-specific assertions.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Worked Examples
|
|
2
|
+
|
|
3
|
+
These examples live in the repo-level `examples/` directory.
|
|
4
|
+
|
|
5
|
+
## `examples/plan-mode-saved-search`
|
|
6
|
+
|
|
7
|
+
Use when the agent starts from a feature discussion or PRD and needs a review checklist.
|
|
8
|
+
|
|
9
|
+
- Input: `feature.md`
|
|
10
|
+
- Schema transport: `model.yaml`
|
|
11
|
+
- Output: `scenarios.json`
|
|
12
|
+
|
|
13
|
+
## `examples/test-mode-checkout`
|
|
14
|
+
|
|
15
|
+
Use when the agent starts from code and an existing test surface and needs missing cases.
|
|
16
|
+
|
|
17
|
+
- Input: `code-notes.md`
|
|
18
|
+
- Schema transport: `model.json`
|
|
19
|
+
- Output: `scenarios.json`
|
|
20
|
+
|
|
21
|
+
## `examples/three-way-notifications`
|
|
22
|
+
|
|
23
|
+
Use when pairwise is not enough and the user explicitly wants 3-way coverage.
|
|
24
|
+
|
|
25
|
+
- Input: `brief.md`
|
|
26
|
+
- Schema transport: `model.yaml`
|
|
27
|
+
- Output: `scenarios.json`
|
|
28
|
+
|
|
29
|
+
## `examples/component-variant-matrix`
|
|
30
|
+
|
|
31
|
+
Use when the agent starts from a visual component, rendering surface, template, or design-system story and needs compact review variants instead of every possible combination.
|
|
32
|
+
|
|
33
|
+
- Input: `brief.md`
|
|
34
|
+
- Schema transport: `model.yaml`
|
|
35
|
+
- Output: `scenarios.json`
|
|
36
|
+
|
|
37
|
+
## `examples/lattice-self-test`
|
|
38
|
+
|
|
39
|
+
Use when you want a meta example of the harness extracting a schema for Lattice's own CLI surface and then using generated rows to drive CLI smoke tests.
|
|
40
|
+
|
|
41
|
+
- Input: `brief.md`
|
|
42
|
+
- Schema transport: `model.yaml`
|
|
43
|
+
- Output: `scenarios.json`
|
|
44
|
+
|
|
45
|
+
## `examples/agent-bootstrap-matrix`
|
|
46
|
+
|
|
47
|
+
Use when you want a meta example of Lattice testing its own agent skill bootstrap behavior across harness detection, force/skip directives, target modes, and output formats.
|
|
48
|
+
|
|
49
|
+
- Input: `brief.md`
|
|
50
|
+
- Schema transport: `model.yaml`
|
|
51
|
+
- Output: `scenarios.json`
|
|
52
|
+
|
|
53
|
+
## `examples/schema-guidance-matrix`
|
|
54
|
+
|
|
55
|
+
Use when you want a meta example of Lattice testing its own schema-discoverability guidance across help text, agent instructions, installed skill files, and validation errors.
|
|
56
|
+
|
|
57
|
+
- Input: `brief.md`
|
|
58
|
+
- Schema transport: `model.yaml`
|
|
59
|
+
- Output: `scenarios.json`
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lattice-workflow
|
|
3
|
+
description: Use when planning or testing behavior or variant surfaces with interacting states, roles, permissions, feature flags, modes, providers, optional fields, component props, rendering branches, template variants, config matrices, lifecycle states, combinatorial edge cases, pairwise coverage, or t-way coverage. Extract a Lattice schema, run the local Lattice CLI, and use generated rows to strengthen plans, tests, fixtures, rendered variants, or review matrices.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Lattice Workflow
|
|
7
|
+
|
|
8
|
+
Use this skill to keep the agent on extraction and interpretation while Lattice handles the combinatorics.
|
|
9
|
+
|
|
10
|
+
## When To Use It
|
|
11
|
+
|
|
12
|
+
- derive a combinatorial model from a feature or PRD
|
|
13
|
+
- derive a test matrix from existing code
|
|
14
|
+
- derive a variant matrix from a component, partial, template, design-system story, or config surface
|
|
15
|
+
- convert edge cases into explicit parameter/value constraints
|
|
16
|
+
- generate pairwise or 3-way scenarios without hand-enumerating combinations
|
|
17
|
+
- render a compact visual review matrix instead of every possible variant
|
|
18
|
+
- review a plan involving roles, permissions, modes, providers, feature flags, optional fields, component props, rendering branches, template variants, config matrices, or lifecycle states
|
|
19
|
+
- find coverage gaps where bugs hide in interactions rather than single branches
|
|
20
|
+
|
|
21
|
+
## When NOT To Use It
|
|
22
|
+
|
|
23
|
+
- **Small enumerable spaces.** Fewer than ~10 total combinations — enumerate by hand.
|
|
24
|
+
- **Independent dimensions.** No interactions can surface a bug; each parameter affects behavior independently.
|
|
25
|
+
- **Non-deterministic systems.** The same input row won't produce repeatable output (e.g., testing LLM responses).
|
|
26
|
+
|
|
27
|
+
## Minimal Example
|
|
28
|
+
|
|
29
|
+
A schema (YAML on stdin):
|
|
30
|
+
|
|
31
|
+
```yaml
|
|
32
|
+
model_name: checkout
|
|
33
|
+
parameters:
|
|
34
|
+
user: [signed_in, guest]
|
|
35
|
+
payment: [card, apple_pay, paypal]
|
|
36
|
+
cart: [empty, single, multi]
|
|
37
|
+
constraints:
|
|
38
|
+
- invalid_pair: {payment: apple_pay, cart: empty}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
One generated row (JSON):
|
|
42
|
+
|
|
43
|
+
```json
|
|
44
|
+
{"user": "guest", "payment": "card", "cart": "multi"}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
The schema is the contract. The generated rows are the source of truth. The schema file is optional transport.
|
|
48
|
+
|
|
49
|
+
## Preflight
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
command -v lattice >/dev/null 2>&1 && lattice --help >/dev/null
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
If it fails, install the Ruby gem from RubyGems (or add `gem "latticerb"` to the project Gemfile):
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
gem install latticerb
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The Python implementation (`pipx install lattice-cli` from PyPI) is interchangeable — both produce byte-identical output for the same schema and seed.
|
|
62
|
+
|
|
63
|
+
## Workflow
|
|
64
|
+
|
|
65
|
+
1. **Extract the schema.** Choose one coherent interaction surface — one feature, one workflow, one service behavior, one endpoint family, one component's variant space, one template family, or one config matrix. Parameters represent independent decisions or state partitions. Values are meaningful partitions, not every literal in the codebase.
|
|
66
|
+
|
|
67
|
+
2. **Encode constraints.** Use the smallest constraint type that matches the rule (see [Constraint Selection](#constraint-selection) below). Encode true impossibilities — do not strip them and mark invalid rows downstream.
|
|
68
|
+
|
|
69
|
+
3. **Validate.**
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
cat schema.json | lattice validate
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
4. **Generate.** Default to pairwise unless explicitly asked for higher strength.
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
cat schema.json | lattice generate
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
5. **Interpret the output.** Plan mode: rows become plan revisions, missing decisions, review scenarios. Test mode: rows become assertions (see [Test Mode Discipline](#test-mode-discipline) below). Variant mode: rows become fixtures, rendered variants, screenshots, visual-diff inputs.
|
|
82
|
+
|
|
83
|
+
## Constraint Selection
|
|
84
|
+
|
|
85
|
+
| Rule shape | Use |
|
|
86
|
+
|---|---|
|
|
87
|
+
| Two assignments cannot coexist | `invalid_pair` |
|
|
88
|
+
| One-to-one pairing (if A then B and if B then A) | `bidirectional` |
|
|
89
|
+
| A requires B, but B may appear elsewhere | `forward_dep` |
|
|
90
|
+
| Child parameter only applies for some parent values | `conditional` |
|
|
91
|
+
| This assignment set must appear in at least one row | `forced` |
|
|
92
|
+
| Rule only breaks under three or more antecedents | `higher_order` |
|
|
93
|
+
|
|
94
|
+
Prefer the smallest constraint type that matches. Use `conditional` instead of hand-writing `N/A` values.
|
|
95
|
+
|
|
96
|
+
## Test Mode Discipline
|
|
97
|
+
|
|
98
|
+
Source expected behavior from intent — the user's request, a spec, a PRD, a ticket, or explicit reasoning about what the system *should* do. **The code under test is evidence, not truth.** If intent is ambiguous for a row, stop and ask the user.
|
|
99
|
+
|
|
100
|
+
Do not silently encode current behavior and hedge with `bug_signal:`-style annotations. That produces characterization tests, which lock in the implementation (bugs included) instead of catching divergence from intent. The asymmetries Lattice is designed to expose disappear when the test suite mirrors the same mental model as the code.
|
|
101
|
+
|
|
102
|
+
## Common Pitfalls
|
|
103
|
+
|
|
104
|
+
❌ Strip constraints to make generation easier, then mark impossible rows invalid after the fact.
|
|
105
|
+
✅ Encode the constraint in the schema, regenerate. Encoding constraints changes the coverage universe to valid interactions — that's the point.
|
|
106
|
+
|
|
107
|
+
❌ Write tests that assert what the code currently does.
|
|
108
|
+
✅ Write tests that assert what the spec, PRD, or user requested. If intent is ambiguous, ask before writing.
|
|
109
|
+
|
|
110
|
+
❌ Separate parameters for two fields where one determines the other (e.g., `currency` and `currency_symbol`).
|
|
111
|
+
✅ One parameter plus a constraint expressing the dependency.
|
|
112
|
+
|
|
113
|
+
❌ Brittle, prose-y value names (`payment_with_apple_pay_using_new_token_flow`).
|
|
114
|
+
✅ Compact partitions: `apple_pay`, `apple_pay_new_token`.
|
|
115
|
+
|
|
116
|
+
❌ Manually add or remove rows after `lattice generate`.
|
|
117
|
+
✅ If the generated set is wrong, fix the schema. Hand-editing rows breaks the coverage guarantee.
|
|
118
|
+
|
|
119
|
+
## References
|
|
120
|
+
|
|
121
|
+
- [references/modeling-rules.md](references/modeling-rules.md) — extraction heuristics, parameter selection, value partitioning, constraint guidance
|
|
122
|
+
- [references/worked-examples.md](references/worked-examples.md) — concrete examples in this repo
|
|
123
|
+
|
|
124
|
+
## Development Fallback
|
|
125
|
+
|
|
126
|
+
In a checkout of the Lattice source where the package is not installed:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
# Ruby checkout
|
|
130
|
+
cat schema.json | ruby -Ilib exe/lattice validate
|
|
131
|
+
cat schema.json | ruby -Ilib exe/lattice generate
|
|
132
|
+
|
|
133
|
+
# Python checkout
|
|
134
|
+
cat schema.json | PYTHONPATH=src python3 -m lattice validate
|
|
135
|
+
cat schema.json | PYTHONPATH=src python3 -m lattice generate
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
To install or refresh this skill from the installed package:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
lattice agent install-codex-skill
|
|
142
|
+
```
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
interface:
|
|
2
|
+
display_name: "Lattice Workflow"
|
|
3
|
+
short_description: "Generate pairwise coverage for plans, tests, and variants"
|
|
4
|
+
default_prompt: "Use $lattice-workflow to model interacting states or variant dimensions, extract a Lattice schema, validate it, generate pairwise rows, and use the rows to strengthen the plan, tests, fixtures, or visual review."
|
|
5
|
+
|
|
6
|
+
policy:
|
|
7
|
+
allow_implicit_invocation: true
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Modeling Rules
|
|
2
|
+
|
|
3
|
+
Use this file when you are extracting a schema from code or prose and need tighter guardrails than the main skill body provides.
|
|
4
|
+
|
|
5
|
+
## Parameter Selection
|
|
6
|
+
|
|
7
|
+
- Scope one schema to one coherent behavior or rendering surface.
|
|
8
|
+
Good scopes: one feature, one service behavior, one workflow, one endpoint family, one state-machine slice, one component's variant space, one template family.
|
|
9
|
+
Bad default scope: the whole app.
|
|
10
|
+
- Model the decisions that change behavior, not every field in a payload.
|
|
11
|
+
- Prefer parameters that are visible in tests, UI branches, rendering branches, validation logic, template variants, component props, or feature flags.
|
|
12
|
+
- Split large raw domains into behavioral partitions.
|
|
13
|
+
Good: `pay_range = present|absent`
|
|
14
|
+
Bad: every exact salary string
|
|
15
|
+
- Avoid separate parameters for values that are already implied by another parameter unless the implication is partial and needs a constraint.
|
|
16
|
+
|
|
17
|
+
## Value Selection
|
|
18
|
+
|
|
19
|
+
- Values should be mutually exclusive and collectively useful for testing.
|
|
20
|
+
- Keep domains small and meaningful. Most parameters should land in the 2-5 value range.
|
|
21
|
+
- Normalize booleans as `true` and `false`.
|
|
22
|
+
- For optional subordinate parameters, let `conditional` introduce `N/A`. Do not hand-add `N/A` unless the schema genuinely needs it as a first-class value.
|
|
23
|
+
|
|
24
|
+
## Constraint Choice
|
|
25
|
+
|
|
26
|
+
- `invalid_pair`: two assignments cannot coexist
|
|
27
|
+
- `bidirectional`: one-to-one pairing; if A then B and if B then A
|
|
28
|
+
- `forward_dep`: A requires B, but B may appear elsewhere
|
|
29
|
+
- `conditional`: a child parameter only applies for some parent values
|
|
30
|
+
- `forced`: this assignment set must appear in at least one generated row
|
|
31
|
+
- `higher_order`: the invalid rule only appears when two or more antecedents are true
|
|
32
|
+
|
|
33
|
+
Choose the smallest expressive type. If a rule can be represented as `invalid_pair`, do not use `higher_order`.
|
|
34
|
+
|
|
35
|
+
Do not strip true constraints to get a smaller or easier unconstrained run. Do not generate nonsensical rows and mark them invalid downstream. Lattice's coverage accounting should run over valid interactions, so impossible combinations belong in the schema as `invalid_pair`, `forward_dep`, `bidirectional`, `conditional`, or `higher_order` constraints.
|
|
36
|
+
|
|
37
|
+
## Extraction Process
|
|
38
|
+
|
|
39
|
+
1. Choose the behavior or rendering surface you are modeling.
|
|
40
|
+
2. Start from branches, validations, and business rules inside that scope.
|
|
41
|
+
3. Convert prose like "only when", "must", "cannot", and "except" into candidate constraints.
|
|
42
|
+
4. Remove cosmetic parameters that do not affect behavior.
|
|
43
|
+
5. Check whether a missing value is really a separate partition or just noise.
|
|
44
|
+
6. Validate the schema.
|
|
45
|
+
7. Only after validation, generate scenarios.
|
|
46
|
+
|
|
47
|
+
## Interpretation
|
|
48
|
+
|
|
49
|
+
- Plan mode: each row becomes a review scenario or a missing decision in the plan.
|
|
50
|
+
- Test mode: each row becomes a candidate setup matrix. Source the expected behavior from intent — user request, spec, PRD, or ticket — not from the code under test. If you cannot determine intent for a row, stop and surface the ambiguity to the user. Do not encode the current behavior and label it `bug_signal:` to hedge: characterization tests pass for the wrong reasons and hide the asymmetries Lattice is designed to expose. Example: if a mailbox accepts a spoofed `From:` header today, the assertion is still "rejects spoofed sender" — let the test fail and fix the code, do not invert the assertion to match the bug.
|
|
51
|
+
- Variant mode: each row becomes a rendered variant, fixture, screenshot, visual diff input, contact-sheet item, or review checklist item.
|
|
52
|
+
- Generated rows are not test names by themselves. The agent still needs to map them to application-specific assertions.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Worked Examples
|
|
2
|
+
|
|
3
|
+
These examples live in the repo-level `examples/` directory.
|
|
4
|
+
|
|
5
|
+
## `examples/plan-mode-saved-search`
|
|
6
|
+
|
|
7
|
+
Use when the agent starts from a feature discussion or PRD and needs a review checklist.
|
|
8
|
+
|
|
9
|
+
- Input: `feature.md`
|
|
10
|
+
- Schema transport: `model.yaml`
|
|
11
|
+
- Output: `scenarios.json`
|
|
12
|
+
|
|
13
|
+
## `examples/test-mode-checkout`
|
|
14
|
+
|
|
15
|
+
Use when the agent starts from code and an existing test surface and needs missing cases.
|
|
16
|
+
|
|
17
|
+
- Input: `code-notes.md`
|
|
18
|
+
- Schema transport: `model.json`
|
|
19
|
+
- Output: `scenarios.json`
|
|
20
|
+
|
|
21
|
+
## `examples/three-way-notifications`
|
|
22
|
+
|
|
23
|
+
Use when pairwise is not enough and the user explicitly wants 3-way coverage.
|
|
24
|
+
|
|
25
|
+
- Input: `brief.md`
|
|
26
|
+
- Schema transport: `model.yaml`
|
|
27
|
+
- Output: `scenarios.json`
|
|
28
|
+
|
|
29
|
+
## `examples/component-variant-matrix`
|
|
30
|
+
|
|
31
|
+
Use when the agent starts from a visual component, rendering surface, template, or design-system story and needs compact review variants instead of every possible combination.
|
|
32
|
+
|
|
33
|
+
- Input: `brief.md`
|
|
34
|
+
- Schema transport: `model.yaml`
|
|
35
|
+
- Output: `scenarios.json`
|
|
36
|
+
|
|
37
|
+
## `examples/lattice-self-test`
|
|
38
|
+
|
|
39
|
+
Use when you want a meta example of the harness extracting a schema for Lattice's own CLI surface and then using generated rows to drive CLI smoke tests.
|
|
40
|
+
|
|
41
|
+
- Input: `brief.md`
|
|
42
|
+
- Schema transport: `model.yaml`
|
|
43
|
+
- Output: `scenarios.json`
|
|
44
|
+
|
|
45
|
+
## `examples/agent-bootstrap-matrix`
|
|
46
|
+
|
|
47
|
+
Use when you want a meta example of Lattice testing its own agent skill bootstrap behavior across harness detection, force/skip directives, target modes, and output formats.
|
|
48
|
+
|
|
49
|
+
- Input: `brief.md`
|
|
50
|
+
- Schema transport: `model.yaml`
|
|
51
|
+
- Output: `scenarios.json`
|
|
52
|
+
|
|
53
|
+
## `examples/schema-guidance-matrix`
|
|
54
|
+
|
|
55
|
+
Use when you want a meta example of Lattice testing its own schema-discoverability guidance across help text, agent instructions, installed skill files, and validation errors.
|
|
56
|
+
|
|
57
|
+
- Input: `brief.md`
|
|
58
|
+
- Schema transport: `model.yaml`
|
|
59
|
+
- Output: `scenarios.json`
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: lattice-workflow
|
|
3
|
+
description: Use when planning or testing behavior or variant surfaces with interacting states, roles, permissions, feature flags, modes, providers, optional fields, component props, rendering branches, template variants, config matrices, lifecycle states, combinatorial edge cases, pairwise coverage, or t-way coverage. Extract a Lattice schema, run the local Lattice CLI, and use generated rows to strengthen plans, tests, fixtures, rendered variants, or review matrices.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Lattice Workflow
|
|
7
|
+
|
|
8
|
+
Use this skill to keep the agent on extraction and interpretation while Lattice handles the combinatorics.
|
|
9
|
+
|
|
10
|
+
## When To Use It
|
|
11
|
+
|
|
12
|
+
- derive a combinatorial model from a feature or PRD
|
|
13
|
+
- derive a test matrix from existing code
|
|
14
|
+
- derive a variant matrix from a component, partial, template, design-system story, or config surface
|
|
15
|
+
- convert edge cases into explicit parameter/value constraints
|
|
16
|
+
- generate pairwise or 3-way scenarios without hand-enumerating combinations
|
|
17
|
+
- render a compact visual review matrix instead of every possible variant
|
|
18
|
+
- review a plan involving roles, permissions, modes, providers, feature flags, optional fields, component props, rendering branches, template variants, config matrices, or lifecycle states
|
|
19
|
+
- find coverage gaps where bugs hide in interactions rather than single branches
|
|
20
|
+
|
|
21
|
+
## When NOT To Use It
|
|
22
|
+
|
|
23
|
+
- **Small enumerable spaces.** Fewer than ~10 total combinations — enumerate by hand.
|
|
24
|
+
- **Independent dimensions.** No interactions can surface a bug; each parameter affects behavior independently.
|
|
25
|
+
- **Non-deterministic systems.** The same input row won't produce repeatable output (e.g., testing LLM responses).
|
|
26
|
+
|
|
27
|
+
## Minimal Example
|
|
28
|
+
|
|
29
|
+
A schema (YAML on stdin):
|
|
30
|
+
|
|
31
|
+
```yaml
|
|
32
|
+
model_name: checkout
|
|
33
|
+
parameters:
|
|
34
|
+
user: [signed_in, guest]
|
|
35
|
+
payment: [card, apple_pay, paypal]
|
|
36
|
+
cart: [empty, single, multi]
|
|
37
|
+
constraints:
|
|
38
|
+
- invalid_pair: {payment: apple_pay, cart: empty}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
One generated row (JSON):
|
|
42
|
+
|
|
43
|
+
```json
|
|
44
|
+
{"user": "guest", "payment": "card", "cart": "multi"}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
The schema is the contract. The generated rows are the source of truth. The schema file is optional transport.
|
|
48
|
+
|
|
49
|
+
## Preflight
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
command -v lattice >/dev/null 2>&1 && lattice --help >/dev/null
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
If it fails, install the Ruby gem from RubyGems (or add `gem "latticerb"` to the project Gemfile):
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
gem install latticerb
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The Python implementation (`pipx install lattice-cli` from PyPI) is interchangeable — both produce byte-identical output for the same schema and seed.
|
|
62
|
+
|
|
63
|
+
## Workflow
|
|
64
|
+
|
|
65
|
+
1. **Extract the schema.** Choose one coherent interaction surface — one feature, one workflow, one service behavior, one endpoint family, one component's variant space, one template family, or one config matrix. Parameters represent independent decisions or state partitions. Values are meaningful partitions, not every literal in the codebase.
|
|
66
|
+
|
|
67
|
+
2. **Encode constraints.** Use the smallest constraint type that matches the rule (see [Constraint Selection](#constraint-selection) below). Encode true impossibilities — do not strip them and mark invalid rows downstream.
|
|
68
|
+
|
|
69
|
+
3. **Validate.**
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
cat schema.json | lattice validate
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
4. **Generate.** Default to pairwise unless explicitly asked for higher strength.
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
cat schema.json | lattice generate
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
5. **Interpret the output.** Plan mode: rows become plan revisions, missing decisions, review scenarios. Test mode: rows become assertions (see [Test Mode Discipline](#test-mode-discipline) below). Variant mode: rows become fixtures, rendered variants, screenshots, visual-diff inputs.
|
|
82
|
+
|
|
83
|
+
## Constraint Selection
|
|
84
|
+
|
|
85
|
+
| Rule shape | Use |
|
|
86
|
+
|---|---|
|
|
87
|
+
| Two assignments cannot coexist | `invalid_pair` |
|
|
88
|
+
| One-to-one pairing (if A then B and if B then A) | `bidirectional` |
|
|
89
|
+
| A requires B, but B may appear elsewhere | `forward_dep` |
|
|
90
|
+
| Child parameter only applies for some parent values | `conditional` |
|
|
91
|
+
| This assignment set must appear in at least one row | `forced` |
|
|
92
|
+
| Rule only breaks under three or more antecedents | `higher_order` |
|
|
93
|
+
|
|
94
|
+
Prefer the smallest constraint type that matches. Use `conditional` instead of hand-writing `N/A` values.
|
|
95
|
+
|
|
96
|
+
## Test Mode Discipline
|
|
97
|
+
|
|
98
|
+
Source expected behavior from intent — the user's request, a spec, a PRD, a ticket, or explicit reasoning about what the system *should* do. **The code under test is evidence, not truth.** If intent is ambiguous for a row, stop and ask the user.
|
|
99
|
+
|
|
100
|
+
Do not silently encode current behavior and hedge with `bug_signal:`-style annotations. That produces characterization tests, which lock in the implementation (bugs included) instead of catching divergence from intent. The asymmetries Lattice is designed to expose disappear when the test suite mirrors the same mental model as the code.
|
|
101
|
+
|
|
102
|
+
## Common Pitfalls
|
|
103
|
+
|
|
104
|
+
❌ Strip constraints to make generation easier, then mark impossible rows invalid after the fact.
|
|
105
|
+
✅ Encode the constraint in the schema, regenerate. Encoding constraints changes the coverage universe to valid interactions — that's the point.
|
|
106
|
+
|
|
107
|
+
❌ Write tests that assert what the code currently does.
|
|
108
|
+
✅ Write tests that assert what the spec, PRD, or user requested. If intent is ambiguous, ask before writing.
|
|
109
|
+
|
|
110
|
+
❌ Separate parameters for two fields where one determines the other (e.g., `currency` and `currency_symbol`).
|
|
111
|
+
✅ One parameter plus a constraint expressing the dependency.
|
|
112
|
+
|
|
113
|
+
❌ Brittle, prose-y value names (`payment_with_apple_pay_using_new_token_flow`).
|
|
114
|
+
✅ Compact partitions: `apple_pay`, `apple_pay_new_token`.
|
|
115
|
+
|
|
116
|
+
❌ Manually add or remove rows after `lattice generate`.
|
|
117
|
+
✅ If the generated set is wrong, fix the schema. Hand-editing rows breaks the coverage guarantee.
|
|
118
|
+
|
|
119
|
+
## References
|
|
120
|
+
|
|
121
|
+
- [references/modeling-rules.md](references/modeling-rules.md) — extraction heuristics, parameter selection, value partitioning, constraint guidance
|
|
122
|
+
- [references/worked-examples.md](references/worked-examples.md) — concrete examples in this repo
|
|
123
|
+
|
|
124
|
+
## Development Fallback
|
|
125
|
+
|
|
126
|
+
In a checkout of the Lattice source where the package is not installed:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
# Ruby checkout
|
|
130
|
+
cat schema.json | ruby -Ilib exe/lattice validate
|
|
131
|
+
cat schema.json | ruby -Ilib exe/lattice generate
|
|
132
|
+
|
|
133
|
+
# Python checkout
|
|
134
|
+
cat schema.json | PYTHONPATH=src python3 -m lattice validate
|
|
135
|
+
cat schema.json | PYTHONPATH=src python3 -m lattice generate
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
To install or refresh a copy of this skill from the installed package:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
lattice agent install-skill /path/to/harness/skills/lattice-workflow
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
For harnesses that don't use skill directories, copy the output of:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
lattice agent instructions
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
into the harness memory, system instruction, rule file, or project instruction mechanism.
|