@assertrx/dino 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (89) hide show
  1. package/README.md +293 -0
  2. package/extensions/dino.ts +70 -0
  3. package/extensions/mabl-mcp.ts +306 -0
  4. package/extensions/playwright-mcp.ts +142 -0
  5. package/package.json +47 -0
  6. package/prompts/create-pr.md +110 -0
  7. package/prompts/generate-bdd-from-url.md +65 -0
  8. package/prompts/generate-pom-from-url-cli.md +48 -0
  9. package/prompts/generate-pom-from-url.md +45 -0
  10. package/prompts/generate-spec-from-feature.md +64 -0
  11. package/prompts/heal-playwright-failures.md +106 -0
  12. package/prompts/setup-playwright.md +81 -0
  13. package/scripts/start-dino.cmd +8 -0
  14. package/scripts/start-dino.mjs +226 -0
  15. package/scripts/start-dino.ps1 +9 -0
  16. package/scripts/start-dino.sh +10 -0
  17. package/skills/acli/SKILL.md +292 -0
  18. package/skills/bdd-gherkin/SKILL.md +552 -0
  19. package/skills/build-review-interface/SKILL.md +96 -0
  20. package/skills/error-analysis/SKILL.md +164 -0
  21. package/skills/eval-audit/SKILL.md +183 -0
  22. package/skills/evaluate-rag/SKILL.md +177 -0
  23. package/skills/generate-synthetic-data/SKILL.md +131 -0
  24. package/skills/mabl-mcp/SKILL.md +79 -0
  25. package/skills/playwright-cli/SKILL.md +246 -0
  26. package/skills/playwright-cli/advanced-workflows.md +679 -0
  27. package/skills/playwright-cli/core-commands.md +375 -0
  28. package/skills/playwright-cli/device-emulation.md +475 -0
  29. package/skills/playwright-cli/request-mocking.md +373 -0
  30. package/skills/playwright-cli/running-custom-code.md +553 -0
  31. package/skills/playwright-cli/screenshots-and-media.md +457 -0
  32. package/skills/playwright-cli/session-management.md +425 -0
  33. package/skills/playwright-cli/storage-and-auth.md +487 -0
  34. package/skills/playwright-cli/test-generation.md +336 -0
  35. package/skills/playwright-cli/tracing-and-debugging.md +501 -0
  36. package/skills/playwright-core/SKILL.md +101 -0
  37. package/skills/playwright-core/accessibility.md +1517 -0
  38. package/skills/playwright-core/angular.md +1086 -0
  39. package/skills/playwright-core/api-testing.md +1623 -0
  40. package/skills/playwright-core/assertions-and-waiting.md +703 -0
  41. package/skills/playwright-core/auth-flows.md +1081 -0
  42. package/skills/playwright-core/authentication.md +1409 -0
  43. package/skills/playwright-core/browser-apis.md +673 -0
  44. package/skills/playwright-core/browser-extensions.md +319 -0
  45. package/skills/playwright-core/canvas-and-webgl.md +494 -0
  46. package/skills/playwright-core/clock-and-time-mocking.md +427 -0
  47. package/skills/playwright-core/common-pitfalls.md +1318 -0
  48. package/skills/playwright-core/component-testing.md +1179 -0
  49. package/skills/playwright-core/configuration.md +731 -0
  50. package/skills/playwright-core/crud-testing.md +945 -0
  51. package/skills/playwright-core/debugging.md +827 -0
  52. package/skills/playwright-core/drag-and-drop.md +992 -0
  53. package/skills/playwright-core/electron-testing.md +622 -0
  54. package/skills/playwright-core/error-and-edge-cases.md +1181 -0
  55. package/skills/playwright-core/error-index.md +1903 -0
  56. package/skills/playwright-core/file-operations.md +749 -0
  57. package/skills/playwright-core/file-upload-download.md +1012 -0
  58. package/skills/playwright-core/fixtures-and-hooks.md +1072 -0
  59. package/skills/playwright-core/flaky-tests.md +883 -0
  60. package/skills/playwright-core/forms-and-validation.md +1056 -0
  61. package/skills/playwright-core/i18n-and-localization.md +622 -0
  62. package/skills/playwright-core/iframes-and-shadow-dom.md +488 -0
  63. package/skills/playwright-core/locator-strategy.md +588 -0
  64. package/skills/playwright-core/locators.md +743 -0
  65. package/skills/playwright-core/mobile-and-responsive.md +1669 -0
  66. package/skills/playwright-core/multi-context-and-popups.md +581 -0
  67. package/skills/playwright-core/multi-user-and-collaboration.md +477 -0
  68. package/skills/playwright-core/network-mocking.md +1395 -0
  69. package/skills/playwright-core/nextjs.md +1013 -0
  70. package/skills/playwright-core/performance-testing.md +608 -0
  71. package/skills/playwright-core/react.md +1082 -0
  72. package/skills/playwright-core/search-and-filter.md +1366 -0
  73. package/skills/playwright-core/security-testing.md +624 -0
  74. package/skills/playwright-core/service-workers-and-pwa.md +524 -0
  75. package/skills/playwright-core/test-architecture.md +569 -0
  76. package/skills/playwright-core/test-data-management.md +1209 -0
  77. package/skills/playwright-core/test-organization.md +947 -0
  78. package/skills/playwright-core/third-party-integrations.md +754 -0
  79. package/skills/playwright-core/trace-analysis.md +161 -0
  80. package/skills/playwright-core/visual-regression.md +1027 -0
  81. package/skills/playwright-core/vue.md +1162 -0
  82. package/skills/playwright-core/websockets-and-realtime.md +623 -0
  83. package/skills/playwright-core/when-to-mock.md +827 -0
  84. package/skills/playwright-pom/SKILL.md +94 -0
  85. package/skills/playwright-pom/page-object-model.md +921 -0
  86. package/skills/playwright-pom/pom-vs-fixtures-vs-helpers.md +937 -0
  87. package/skills/validate-evaluator/SKILL.md +215 -0
  88. package/skills/write-judge-prompt/SKILL.md +144 -0
  89. package/themes/dino.json +66 -0
@@ -0,0 +1,552 @@
1
+ ---
2
+ name: bdd-gherkin
3
+ description: Generate domain-driven Gherkin/Cucumber BDD scenarios focused on business requirements with minimal technical details when necessary for clarity
4
+ author: Anand Gopalakrishnan
5
+ metadata:
6
+ focus: business-domain
7
+ format: gherkin
8
+ version: cucumber-7+
9
+ ---
10
+
11
+ # Skill: bdd-gherkin
12
+
13
+ ## What I do
14
+
15
+ - Generate Gherkin feature files with business-focused scenarios
16
+ - Create domain-driven BDD scenarios using ubiquitous language
17
+ - Write scenarios in Given/When/Then format prioritizing business language
18
+ - Parse Jira-style ticket descriptions (Summary, Observed Behavior, Expected Behavior, Conditions/Context, Replication Steps, Acceptance Criteria) and convert them into Gherkin feature files
19
+ - Include Scenario Outlines with Examples tables when explicitly beneficial
20
+ - Add Background sections liberally when scenarios share common preconditions
21
+ - Use minimal tagging - only when explicitly needed
22
+ - Support data tables and doc strings where appropriate
23
+ - Focus on business behavior with minimal technical context when it aids clarity
24
+ - Always ask clarifying questions before generating scenarios, unless the user provides a complete Jira ticket description
25
+
26
+ ## When to use me
27
+
28
+ Use this skill when:
29
+ - User asks to create BDD scenarios or feature files
30
+ - User mentions Gherkin, Cucumber, or behavior-driven development
31
+ - User wants to document business requirements as executable specifications
32
+ - User needs acceptance criteria written in Given/When/Then format
33
+ - User requests feature files for user stories
34
+ - User pastes a Jira ticket description containing fields such as Summary, Observed Behavior, Expected Behavior, Conditions/Context, Replication Steps, or Acceptance Criteria
35
+
36
+ ## Handling Jira ticket descriptions
37
+
38
+ When the user provides a Jira-style ticket description, treat it as structured input and generate BDD feature files directly from the relevant sections.
39
+
40
+ ### Recognized Jira fields
41
+
42
+ | Jira field | How to use it |
43
+ |---|---|
44
+ | **Summary / Title** | Derive the Feature name and the business value statement (`As a ... I want ... So that ...`). |
45
+ | **Observed Behavior** | Capture the current/problem outcome. Turn error messages into negative scenarios or assertion text in `Then` steps. |
46
+ | **Expected Behavior** | Capture the desired outcome. Use as the primary source for `Then` steps and happy-path scenarios. |
47
+ | **Conditions and Context** | Convert environment, scope, and setup details into `Background` or `Given` preconditions. |
48
+ | **Replication Steps** | Convert each meaningful user action into a single high-level `When` step. Move detailed setup into `Given` using data tables or Background. |
49
+ | **Acceptance Criteria** | Primary source of scenarios. If already in Given/When/Then form, adapt it to follow this skill's rules (single-focused scenarios, one `When` per scenario, business language). |
50
+ | **Links / References** | Use only for context; do not include URLs in scenarios unless explicitly needed. |
51
+
52
+ ### Rules for Jira input mode
53
+
54
+ 1. **Generate scenarios without asking clarifying questions** when the ticket contains at least a Summary and Acceptance Criteria (or clear Expected Behavior). Only ask questions if critical information is missing.
55
+ 2. **Produce one feature file per distinct business capability**. If the ticket covers multiple capabilities, split them into separate `.feature` files.
56
+ 3. **Create single-focused scenarios** from each acceptance criterion, observed error, and significant replication path.
57
+ 4. **Use exactly one `When` action per scenario**. Combine low-level replication steps into a single high-level business action, and move setup details into `Given`.
58
+ 5. **Use `Background` liberally** for preconditions shared across scenarios (e.g., project created, SSA enabled, permissions granted).
59
+ 6. **Keep business language**; avoid UI field names, API endpoints, or database details unless they are essential for clarity.
60
+ 7. **Represent exact error messages from Observed Behavior as `Then` assertions** in negative scenarios, but phrase the scenario itself from the business perspective.
61
+
62
+ ### Jira-to-Gherkin example
63
+
64
+ **Input ticket summary:** Publish fails with "User not authorized to access Autodesk project" on newly created projects despite confirmed SSA setup.
65
+
66
+ ```gherkin
67
+ Feature: Publish models to newly created Autodesk projects
68
+ As a project team member
69
+ I want to publish models from Revit or AutoCAD to a new project after SSA setup
70
+ So that project data is available in ACC without access errors
71
+
72
+ Background:
73
+ Given a newly created Autodesk/ACC project exists
74
+ And SSA access is enabled for the project
75
+ And the SSA email has View-Download-Upload-Edit permissions on Project Files
76
+
77
+ Scenario: Successful model publish to a new project
78
+ Given Stratus SSA Status confirms "SSA has access to this project"
79
+ And the user is publishing from Revit or AutoCAD
80
+ When the user publishes a model to the new project
81
+ Then the publish succeeds
82
+ And no "User not authorized to access Autodesk project" error occurs
83
+
84
+ Scenario: Publish fails when SSA lacks a project publishing role
85
+ Given the SSA email is added to the project
86
+ But the SSA does not have a project role that allows publishing
87
+ When the user publishes a model to the new project
88
+ Then the publish fails
89
+ And the user sees "User does not have a project role that allows publishing for '<project name>'"
90
+ ```
91
+
92
+ Save as:
93
+ ```
94
+ publish-to-new-autodesk-project.feature
95
+ ```
96
+
97
+ ## Core principles I follow
98
+
99
+ ### 1. Domain-Driven Language with Pragmatic Clarity
100
+ - Use the **ubiquitous language** of the business domain as primary focus
101
+ - Prioritize business actions, outcomes, and rules
102
+ - Express scenarios from the user's or business stakeholder's perspective
103
+ - Allow minimal technical context when it adds clarity (e.g., "API endpoint", "database record") but avoid implementation specifics
104
+ - Balance business readability with practical clarity
105
+
106
+ **Good (Business-focused):**
107
+ ```gherkin
108
+ Given a customer with a premium account
109
+ When they purchase a product worth $500
110
+ Then they receive a 10% loyalty discount
111
+ ```
112
+
113
+ **Bad (Too much technical detail):**
114
+ ```gherkin
115
+ Given the database has a user with premium_flag = true in users table
116
+ When a POST request is sent to /api/purchase with JSON body {"amount":500}
117
+ Then the HTTP response status code is 200
118
+ And the response contains discount_percentage: 10
119
+ ```
120
+
121
+ **Acceptable (Minimal technical context for clarity):**
122
+ ```gherkin
123
+ Given a customer record exists with premium status
124
+ When they submit a purchase request for $500
125
+ Then the purchase is confirmed
126
+ And a 10% loyalty discount is applied
127
+ ```
128
+
129
+ ### 2. Feature Structure
130
+ Every feature file should include:
131
+ - **Feature**: Clear business capability or epic
132
+ - **Business value statement** (recommended): As a [role], I want [feature], So that [benefit]
133
+ - **Background** (use liberally): Common preconditions when 2+ scenarios share setup
134
+ - **Scenarios**: Individual test cases (preferred by default)
135
+ - **Scenario Outlines** (when explicitly beneficial): Use only when truly needed for data variations
136
+ - **Tags** (minimal): Only when explicitly needed for organization
137
+
138
+ ### 3. Given/When/Then Best Practices
139
+
140
+ **Given** - Establish context:
141
+ - Describe the initial state
142
+ - Set up preconditions
143
+ - Use past tense or present state ("a customer has...", "the account is...")
144
+ - No actions or events
145
+
146
+ **When** - Trigger action:
147
+ - **CRITICAL**: Single action or event only - exactly ONE When step per scenario
148
+ - Use present tense ("customer places order", "user cancels subscription")
149
+ - Should be one clear action per When
150
+ - **Avoid multiple When/And steps** - consolidate into a single high-level action
151
+ - Put data/parameters in the Given context, not as separate When steps
152
+
153
+ **Then** - Verify outcome:
154
+ - Observable business outcome
155
+ - Use present tense ("order is confirmed", "balance is updated")
156
+ - Focus on business results, not technical state
157
+
158
+ **And/But** - Chain related steps:
159
+ - Use for additional context, actions, or assertions
160
+ - Keep scenarios readable
161
+
162
+ ### 4. Scenario Outlines and Examples
163
+ Use Scenario Outlines sparingly - prefer regular Scenarios by default. Only use Outlines when:
164
+ - The same business rule needs validation with multiple data points
165
+ - Data variations are the focus of the test
166
+ - It significantly improves readability over multiple similar scenarios
167
+
168
+ ```gherkin
169
+ Scenario Outline: Apply volume discounts based on purchase quantity
170
+ Given a customer is purchasing <quantity> units
171
+ When they complete the checkout
172
+ Then they receive a <discount>% discount
173
+ And the total price is reduced accordingly
174
+
175
+ Examples: Standard volume tiers
176
+ | quantity | discount |
177
+ | 1 | 0 |
178
+ | 10 | 5 |
179
+ | 50 | 10 |
180
+ | 100 | 15 |
181
+ ```
182
+
183
+ **When in doubt, use regular Scenarios instead.**
184
+
185
+ ### 5. Data Tables
186
+ Use data tables to express structured data in a business-readable way:
187
+
188
+ **For multiple records (horizontal headers):**
189
+ ```gherkin
190
+ Given the following products are in the catalog:
191
+ | Product Name | Category | Price |
192
+ | Laptop | Electronics | 1200 |
193
+ | Desk Chair | Furniture | 350 |
194
+ | Monitor | Electronics | 400 |
195
+ ```
196
+
197
+ **For single record with multiple fields (vertical key-value pairs):**
198
+ ```gherkin
199
+ Given a user wants to add a bookmark with the following details:
200
+ | URL | https://example.com |
201
+ | Title | Example Website |
202
+ | Description | A helpful example site |
203
+ ```
204
+
205
+ **When to use data tables in Given steps:**
206
+ - When you have 2+ fields/attributes to specify
207
+ - When the Given step would become too long with inline values
208
+ - When you want to clearly show what fields are present vs. absent
209
+ - For better readability and maintainability
210
+
211
+ **❌ Don't: Use long inline values in Given**
212
+ ```gherkin
213
+ Given a user wants to add a bookmark with URL "https://example.com", title "Example Website", and description "A helpful example site for testing"
214
+ ```
215
+
216
+ **✓ Do: Use vertical data table for cleaner readability**
217
+ ```gherkin
218
+ Given a user wants to add a bookmark with the following details:
219
+ | URL | https://example.com |
220
+ | Title | Example Website |
221
+ | Description | A helpful example site for testing |
222
+ ```
223
+
224
+ ### 6. Doc Strings
225
+ Use doc strings (triple quotes) for larger text content:
226
+
227
+ ```gherkin
228
+ Given a customer service representative receives this inquiry:
229
+ """
230
+ I ordered product #12345 two weeks ago but haven't
231
+ received any shipping confirmation. Can you help?
232
+ """
233
+ When they process the inquiry
234
+ Then a shipping status update is sent to the customer
235
+ ```
236
+
237
+ ### 7. Tags for Organization
238
+ Use tags minimally - only when they provide clear organizational value:
239
+
240
+ ```gherkin
241
+ Scenario: Complete purchase with valid payment
242
+
243
+ @wip
244
+ Scenario: Process return for defective item
245
+ ```
246
+
247
+ Only add tags when:
248
+ - Marking work in progress (@wip)
249
+ - Explicitly requested by the user
250
+ - Required for test execution filtering
251
+ - Necessary for team workflow
252
+
253
+ Avoid over-tagging with @smoke, @regression, @critical unless specifically needed.
254
+
255
+ ## Scenario Scope: Default to Single-Focused
256
+
257
+ **IMPORTANT: Always default to single-focused scenarios unless the user explicitly requests journey-based scenarios.**
258
+
259
+ ### Single-Focused Scenarios (DEFAULT)
260
+ - Each scenario tests **one specific business rule or behavior**
261
+ - Scenarios are independent and can run in isolation
262
+ - Easier to understand, maintain, and debug
263
+ - Clear pass/fail for specific capabilities
264
+ - Preferred approach for most situations
265
+
266
+ ### Journey-Based Scenarios (Only when explicitly requested)
267
+ - Test end-to-end user workflows across multiple capabilities
268
+ - Show how features work together in realistic user flows
269
+ - Used when the user specifically asks for:
270
+ - "End-to-end scenarios"
271
+ - "User journey scenarios"
272
+ - "Full workflow tests"
273
+ - "Integration scenarios"
274
+
275
+ **When in doubt, use single-focused scenarios.**
276
+
277
+ ## Questions to ask
278
+
279
+ **ALWAYS ask clarifying questions before generating scenarios, unless the user has provided a complete Jira ticket description.** When the user requests BDD scenarios without a structured Jira ticket, ask:
280
+
281
+ 1. **What business capability or feature is this testing?**
282
+ - Helps frame the Feature description
283
+
284
+ 2. **Who is the primary actor or user role?**
285
+ - Defines perspective for scenarios (customer, admin, agent, etc.)
286
+
287
+ 3. **What business value does this provide?**
288
+ - Helps decide if "As a/I want/So that" statement is appropriate
289
+
290
+ 4. **What are the specific business rules or constraints?**
291
+ - Helps identify scenarios and edge cases
292
+
293
+ 5. **Are there common preconditions across scenarios?**
294
+ - Determines Background section usage
295
+
296
+ **Note:** Do NOT ask about single-focused vs. journey scenarios - always default to single-focused unless the user explicitly requests journeys.
297
+
298
+ **Jira ticket exception:** If the input is a Jira description containing Summary plus Acceptance Criteria or clear Expected Behavior, generate scenarios directly using the rules in [Handling Jira ticket descriptions](#handling-jira-ticket-descriptions). Only ask questions if critical information (e.g., the action or expected outcome) is missing.
299
+
300
+ Only generate scenarios after receiving answers to relevant questions, or after parsing a complete Jira ticket description.
301
+
302
+ ## Standard feature file template
303
+
304
+ ```gherkin
305
+ Feature: [Business capability name]
306
+ As a [role/actor]
307
+ I want [feature/capability]
308
+ So that [business value/benefit]
309
+
310
+ Background:
311
+ Given [common precondition for all scenarios]
312
+ And [another common precondition]
313
+
314
+ Scenario: [Happy path scenario name]
315
+ Given [initial context]
316
+ And [additional context]
317
+ When [user action]
318
+ Then [expected business outcome]
319
+ And [additional verification]
320
+
321
+ Scenario: [Edge case or validation scenario]
322
+ Given [specific context for edge case]
323
+ When [action that triggers validation]
324
+ Then [expected business behavior]
325
+
326
+ Scenario: [Another regular scenario]
327
+ Given [context]
328
+ When [action]
329
+ Then [outcome]
330
+ ```
331
+
332
+ **Notes:**
333
+ - Business value statement is recommended but optional
334
+ - Use Background liberally when scenarios share setup
335
+ - Prefer regular Scenarios over Scenario Outlines
336
+ - Add tags only when explicitly needed
337
+ - **Default to single-focused scenarios** unless user explicitly requests journeys
338
+
339
+ ## Common anti-patterns to avoid
340
+
341
+ ### ❌ Don't: Include excessive technical implementation
342
+ ```gherkin
343
+ When the user clicks the "Submit" button with id="btn-submit" using Selenium driver
344
+ Then the API returns HTTP status code 200 with JSON schema validation
345
+ And the database transaction commits with ACID properties
346
+ ```
347
+
348
+ ### ✓ Do: Focus on business action and outcome (minimal tech context OK)
349
+ ```gherkin
350
+ When the user submits their application
351
+ Then the application is accepted for review
352
+ And a confirmation is sent
353
+ ```
354
+
355
+ ### ❌ Don't: Use Scenario Outline unnecessarily
356
+ ```gherkin
357
+ Scenario Outline: Process single order
358
+ Given a customer places an order
359
+ When they pay <amount>
360
+ Then order is confirmed
361
+
362
+ Examples:
363
+ | amount |
364
+ | 100 |
365
+ ```
366
+
367
+ ### ✓ Do: Use regular Scenario when data variation isn't the focus
368
+ ```gherkin
369
+ Scenario: Process customer order
370
+ Given a customer places an order
371
+ When they complete payment
372
+ Then the order is confirmed
373
+ ```
374
+
375
+ ### ❌ Don't: Create journey scenarios unless explicitly requested
376
+ ```gherkin
377
+ # Only use this pattern when user explicitly asks for journey/end-to-end scenarios
378
+ Scenario: Complete first-time purchase journey
379
+ Given a new customer browses the product catalog
380
+ When they select a product and add it to their cart
381
+ And complete the checkout process with payment
382
+ Then their order is confirmed
383
+ And they receive a welcome discount on their next purchase
384
+ ```
385
+
386
+ ### ✓ Do: Default to single-focused scenarios (preferred approach)
387
+ ```gherkin
388
+ Scenario: Add item to cart
389
+ Given a customer views a product
390
+ When they add it to cart
391
+ Then the cart contains 1 item
392
+ And the product appears in their cart
393
+
394
+ Scenario: Complete checkout with items in cart
395
+ Given a customer has 1 item in their cart
396
+ When they complete the checkout process
397
+ Then their order is confirmed
398
+ And they receive a confirmation email
399
+ ```
400
+
401
+ ### ❌ Don't: Break high-level actions into multiple When steps
402
+ ```gherkin
403
+ Scenario: Add a bookmark with all fields
404
+ Given a user is ready to save a new bookmark
405
+ When they enter "https://example.com" as the URL
406
+ And they enter "Example Website" as the title
407
+ And they enter "A description" as the description
408
+ And they submit the bookmark
409
+ Then the bookmark is saved successfully
410
+ ```
411
+
412
+ ### ✓ Do: Use a single high-level When action with data table in Given
413
+ ```gherkin
414
+ Scenario: Add a complete bookmark
415
+ Given a user wants to add a bookmark with the following details:
416
+ | URL | https://example.com |
417
+ | Title | Example Website |
418
+ | Description | A description |
419
+ When they add the bookmark
420
+ Then the bookmark is saved successfully
421
+ And the bookmark appears in their bookmark list
422
+ ```
423
+
424
+ ### ❌ Don't: Use long inline values in Given steps
425
+ ```gherkin
426
+ Scenario: Add a complete bookmark
427
+ Given a user wants to add a bookmark with URL "https://example.com", title "Example Website", and description "A helpful example site for testing"
428
+ When they add the bookmark
429
+ Then the bookmark is saved successfully
430
+ ```
431
+
432
+ ### ✓ Do: Use data table when Given has 2+ fields or gets too long
433
+ ```gherkin
434
+ Scenario: Add a complete bookmark
435
+ Given a user wants to add a bookmark with the following details:
436
+ | URL | https://example.com |
437
+ | Title | Example Website |
438
+ | Description | A helpful example site for testing |
439
+ When they add the bookmark
440
+ Then the bookmark is saved successfully
441
+ ```
442
+
443
+ ## Workflow example
444
+
445
+ **User:** "Create BDD scenarios for a customer loyalty program where customers earn points on purchases"
446
+
447
+ **Response:**
448
+
449
+ First, let me ask a few clarifying questions:
450
+ 1. What are the specific business rules for earning points? (e.g., 1 point per dollar, different rates for tiers?)
451
+ 2. Are there different customer tiers or segments to consider?
452
+ 3. What actions should customers be able to do with points? (redeem, expire, transfer?)
453
+
454
+ [After receiving answers:]
455
+
456
+ I'll create a feature file for the customer loyalty program:
457
+
458
+ ```gherkin
459
+ Feature: Customer Loyalty Points
460
+ As a customer
461
+ I want to earn points on my purchases
462
+ So that I can receive rewards and discounts on future orders
463
+
464
+ Background:
465
+ Given the loyalty program is active
466
+ And point values are set to 1 point per dollar spent
467
+
468
+ Scenario: Earn points on standard purchase
469
+ Given a customer with a basic loyalty account
470
+ And their current point balance is 0
471
+ When they complete a purchase of $100
472
+ Then they earn 100 loyalty points
473
+ And their new point balance is 100
474
+
475
+ Scenario: Silver tier earns bonus points
476
+ Given a customer with Silver membership
477
+ When they complete a purchase of $100
478
+ Then they earn 150 loyalty points
479
+
480
+ Scenario: Gold tier earns higher bonus points
481
+ Given a customer with Gold membership
482
+ When they complete a purchase of $100
483
+ Then they earn 200 loyalty points
484
+
485
+ Scenario: Redeem points for discount on purchase
486
+ Given a customer has 500 loyalty points
487
+ And they have items worth $80 in their cart
488
+ When they redeem 400 points for a discount
489
+ Then $40 is deducted from their order total
490
+ And their final amount is $40
491
+ And their remaining point balance is 100
492
+
493
+ Scenario: Points expire after 12 months of inactivity
494
+ Given a customer earned 300 points on January 1, 2025
495
+ And they have had no account activity since then
496
+ When the date reaches January 2, 2026
497
+ Then their 300 points are marked as expired
498
+ And their available point balance is 0
499
+ And they receive an expiration notification
500
+ ```
501
+
502
+ Would you like me to:
503
+ 1. Add more scenarios for edge cases?
504
+ 2. Include scenarios for points transfer or gifting?
505
+ 3. Add validation scenarios for fraud prevention?
506
+
507
+ ## Validation checklist
508
+
509
+ Before finalizing scenarios, verify:
510
+ - [ ] Prioritizes business language with minimal technical details for clarity
511
+ - [ ] Uses domain/business language throughout
512
+ - [ ] **Scenarios are single-focused (default) unless user explicitly requested journeys**
513
+ - [ ] Given/When/Then structure is followed correctly
514
+ - [ ] **Each scenario has exactly ONE When action (no multiple When/And action steps)**
515
+ - [ ] **Data and parameters are in Given context, not spread across multiple When steps**
516
+ - [ ] **Data tables used for Given steps with 2+ fields or long inline values**
517
+ - [ ] Regular Scenarios used by default; Scenario Outlines only when explicitly beneficial
518
+ - [ ] Tags used minimally - only when needed
519
+ - [ ] Background used liberally when scenarios share preconditions
520
+ - [ ] Outcomes are observable business results
521
+ - [ ] Business value statement included when it adds value (recommended but optional)
522
+ - [ ] Scenarios are readable by non-technical stakeholders
523
+ - [ ] **When input is a Jira ticket, Summary, Observed/Expected Behavior, Conditions/Context, Replication Steps, and Acceptance Criteria are mapped to appropriate scenarios**
524
+
525
+ ## Tips for success
526
+
527
+ 1. **Think like a business analyst**: What does the business care about?
528
+ 2. **Balance clarity with domain focus**: Use business language primarily, minimal tech terms for clarity
529
+ 3. **Default to single-focused scenarios**: Test one behavior per scenario unless explicitly asked for journeys
530
+ 4. **Keep scenarios independent**: Each should run in isolation
531
+ 5. **Make scenarios maintainable**: Avoid brittle test data dependencies
532
+ 6. **Use declarative style**: Say WHAT should happen, not HOW to do it
533
+ 7. **One action per scenario**: Keep exactly ONE When step - move data to Given context
534
+ 8. **High-level actions**: Use business-level actions like "add bookmark", not field-by-field steps
535
+ 9. **Use data tables liberally**: For 2+ fields or long values in Given steps - improves readability
536
+ 10. **Ask questions first**: Always clarify requirements before generating, unless the input is a complete Jira ticket description
537
+ 11. **Review with stakeholders**: Scenarios should be readable by product owners
538
+ 12. **Use Background liberally**: Share common setup across scenarios
539
+ 13. **Prefer regular Scenarios**: Use Outlines only when truly beneficial
540
+
541
+ ## File naming conventions
542
+
543
+ Feature files should use descriptive names:
544
+ - `customer-loyalty-program.feature`
545
+ - `order-checkout.feature`
546
+ - `inventory-management.feature`
547
+ - `user-registration.feature`
548
+
549
+ Not:
550
+ - `test1.feature`
551
+ - `scenarios.feature`
552
+ - `cucumber_test.feature`
@@ -0,0 +1,96 @@
1
+ ---
2
+ name: build-review-interface
3
+ description: >
4
+ Build a custom browser-based annotation interface tailored to your data for
5
+ reviewing LLM traces and collecting structured feedback. Use when you need to
6
+ build an annotation tool, review traces, or collect human labels.
7
+ ---
8
+
9
+ # Build a Custom Annotation Interface
10
+
11
+ ## Overview
12
+
13
+ Build an HTML page that loads traces from a data source (JSON/CSV file), displays one trace at a time with Pass/Fail buttons, a free-text notes field, and Next/Previous navigation. Save labels to a local file (CSV/SQLite/JSON). Then customize to the domain using the guidelines below.
14
+
15
+ ## Data Display
16
+
17
+ Format all data in the most human-readable representation for the domain. Emails should look like emails. Code should have syntax highlighting. Markdown should be rendered. Tables should be tables. JSON should be pretty-printed and collapsible.
18
+
19
+ - **Collapse repetitive elements.** If every trace shares the same system prompt, put it in a `<details>` toggle.
20
+ - **Extract and surface key metadata.** If traces contain a property name, client type, or session ID buried in the data, extract it and display it prominently as a header or badge.
21
+ - **Color-code by role or status.** Use left-border colors to distinguish user messages, assistant messages, tool calls, and system prompts at a glance.
22
+ - **Group related elements visually.** Tool calls and their responses should be visually linked (indentation, shared border).
23
+ - **Collapse what doesn't help judgment.** Verbose tool response JSON, intermediate reasoning steps, and debugging context go behind toggles.
24
+ - **Highlight what matters most.** Make the primary content reviewers judge visually dominant. Bold key entities (prices, dates, names). Use font size and spacing to create hierarchy.
25
+ - **Show the full trace.** Include all intermediate steps (tool calls, retrieved context, reasoning), not just the final output. Collapse them by default but keep them accessible.
26
+ - **Sanitize rendered content.** Strip raw HTML from LLM outputs before rendering. Disable images in rendered markdown if they could be tracking pixels.
27
+
28
+ ## Feedback Collection
29
+
30
+ Annotate at the trace level. The reviewer judges the whole trace, not individual spans.
31
+
32
+ - Binary Pass/Fail buttons as the primary action.
33
+ - Free-text notes field for the reviewer to describe what went wrong (or right).
34
+ - Defer button for uncertain cases.
35
+ - Auto-save on every action.
36
+
37
+ Once you have established failure categories from error analysis, you can later add predefined failure mode tags as clickable checkboxes, dropdowns or picklists so reviewers can select from known categories in addition to writing notes. But don't add these in the initial build.
38
+
39
+ ## Navigation and Status
40
+
41
+ - Next/Previous buttons and keyboard arrow keys.
42
+ - Trace counter showing position and progress ("12 of 87 remaining").
43
+ - Jump to specific trace by ID.
44
+ - Counts of labeled vs unlabeled traces.
45
+
46
+ ## Keyboard Shortcuts
47
+
48
+ ```
49
+ Arrow keys = Navigate traces
50
+ 1 = Pass 2 = Fail
51
+ D = Defer U = Undo last action
52
+ Cmd+S = Save Cmd+Enter = Save and next
53
+ ```
54
+
55
+ ## Selecting Traces to Load
56
+
57
+ Build the app to accept traces from any source (JSON/CSV file). Keep sampling logic outside the app in a separate script. Start with random sampling.
58
+
59
+ ## Additional Features
60
+
61
+ **Reference panel:** Toggle-able panel showing ground truth, expected answers, or rubric definitions alongside the trace.
62
+
63
+ **Filtering:** Filter traces by metadata dimensions relevant to the product (channel, user type, pipeline version).
64
+
65
+ **Clustering:** Group traces by metadata or semantic similarity. Show representative traces per cluster with drill-down.
66
+
67
+ ## Design Checklist
68
+
69
+ - [ ] Same layout, controls, and terminology on every trace
70
+ - [ ] Pass and Fail buttons are visually distinct (color, size)
71
+ - [ ] Keyboard shortcuts work for all primary actions
72
+ - [ ] Full trace accessible even when sections are collapsed
73
+ - [ ] Labels persist automatically without explicit save
74
+ - [ ] Trace-level annotation (not span-level) as the default
75
+ - [ ] All data rendered in its native format (markdown as HTML, code with highlighting, JSON pretty-printed, tables as HTML tables, URLs as clickable links)
76
+
77
+ ## Testing
78
+
79
+ After building the interface, verify it with Playwright.
80
+
81
+ **Visual review:** Take screenshots of the interface with representative trace data loaded. Review each screenshot for:
82
+ - Layout and spacing: is the visual hierarchy clear? Can you immediately see what matters?
83
+ - Readability: is all data rendered in its native format? Are there any raw JSON blobs, unrendered markdown, or unstyled content?
84
+ - Aesthetics: does the interface look professional and clean? Would a domain expert use this?
85
+ - Responsiveness: does the layout hold at different window sizes?
86
+
87
+ **Functional test:** Write a Playwright script that performs a full annotation workflow:
88
+ 1. Load the app and verify traces are displayed
89
+ 2. Click Pass on a trace, verify the label is saved
90
+ 3. Click Fail on a trace, add a note, verify both are saved
91
+ 4. Click Defer, verify it is recorded
92
+ 5. Navigate forward and backward with buttons and keyboard shortcuts
93
+ 6. Verify the trace counter updates correctly
94
+ 7. Verify auto-save by reloading the page and checking labels persist
95
+ 8. Expand collapsed sections (system prompts, tool calls) and verify content is accessible
96
+ 9. Test that all keyboard shortcuts trigger the correct actions