@justjoshn/pi-pstack 0.0.0-stage → 0.15.36-pi.1

package/automations/benny/skills/reproduce-and-fix-issues/references/control-adapter.md

@@ -0,0 +1,169 @@
+# Control-adapter contract
+
+Benny does not know how to start or drive every app. The user must configure one control skill or adapter that implements this contract for the target app.
+
+Set its skill name in `control.skill_name`.
+
+Set the completed user-facing feature map path in `control.feature_map_path`. Copy and fill [`feature-map.example.md`](./feature-map.example.md) outside `.pi/automations/benny/` instead of editing the copied example.
+
+If the skill, feature map, or a required capability is absent, ambiguous, or incomplete, repro and fix work must fail closed.
+
+## Required capabilities
+
+### Bring up
+
+Start the requested app revision in the requested test environment.
+
+Input:
+
+- Repository and revision
+- Build or start mode
+- Workspace, account, fixture, and feature-state requirements
+- Artifact directory
+- Completed feature-map path
+
+Return:
+
+- Session identifier
+- How the adapter confirmed the correct app and environment
+- Stable app markers
+- Running process or target details needed by later calls
+- Any missing capability
+
+The adapter must distinguish the target app from a similar window, shell, or production instance.
+
+### Drive UI
+
+Perform real user actions:
+
+- Click
+- Type
+- Press keys
+- Scroll
+- Drag
+- Resize
+- Navigate through app controls
+
+Prefer roles, labels, and stable selectors. Use coordinates only after a fresh screenshot.
+
+Return each action and the observed state change.
+
+Do not set internal state, call hidden app methods, write directly to storage, or inject DOM changes to create the symptom.
+
+### Drive mapped features and states
+
+Read the relevant feature-map section before driving the app.
+
+The adapter must expose ways to:
+
+- Navigate every mapped feature through the user-visible path.
+- Invoke the adapter action names listed for that feature.
+- Interact with default, hover, focus-visible, active, disabled, loading, empty, error, selected, open, expanded, and feature-specific states when they apply.
+- Arrange a state through safe fixture data, permissions, flags, service responses, or supported test controls.
+- Reset the feature for a second independent repro attempt.
+- Capture the screenshot, video, and read-only cross-check named by the feature map.
+
+Use roles, accessible names, ARIA relationships, stable component markers, and purpose-named data attributes. Never use generated CSS or StyleX classes, dynamic hashes, child indexes, or brittle DOM position.
+
+Arranging a precondition is not permission to inject the reported symptom. The repro itself must still come from real user interaction.
+
+### Inspect state
+
+Read state to confirm what the UI shows.
+
+Examples:
+
+- Accessibility tree
+- DOM or view hierarchy
+- Process state
+- Local logs
+- Network request status
+- App-exposed debug state
+
+Inspection is read-only. If a query changes state, it belongs in `drive UI` and must represent a real user action.
+
+### Screenshot
+
+Capture the current app state to a requested path.
+
+Return:
+
+- File path
+- Capture time
+- App marker or window title
+- Short description of what should be visible
+
+The screenshot must show enough app chrome to prove that the correct app is under test.
+
+### Recording
+
+Start and stop a screen recording around the full repro path.
+
+Return:
+
+- File path
+- Start and stop times
+- Captured window or region
+- Whether audio or sensitive overlays were omitted
+
+The recording must show the discriminating final state, not only setup or a loading screen.
+
+### Cleanup
+
+Stop processes and sessions created by the adapter.
+
+Remove disposable:
+
+- Browser or app profiles
+- Temporary workspaces
+- Test accounts or fixtures when the adapter created them
+- Debug ports and tunnels
+- Captures past their retention window
+
+Return what was stopped, removed, retained, or left for a person.
+
+Cleanup must not delete user work.
+
+## Adapter behavior
+
+The adapter must:
+
+- Report capabilities before the repro starts.
+- Report which feature-map sections it can drive and which are blocked.
+- Use the same environment inputs for baseline and patched builds.
+- Surface startup failures as failures.
+- Bound retries.
+- Keep secrets out of logs and artifacts.
+- Keep captures outside the repository.
+- Support a fresh or reset state between the two repro attempts.
+- Avoid production changes unless the user explicitly configured a safe test action.
+
+## Environment translation
+
+Before declaring an environment block, restate the defect without platform-specific nouns and ask whether the same behavior can be tested safely in the available environment.
+
+Examples:
+
+- A named browser may mean any external browser.
+- A named key may mean the configured shortcut.
+- A named remote host may mean a delayed or disconnected remote target.
+
+Use a translated attempt only when it tests the same underlying behavior. Label it as translated evidence. Do not call it an exact repro when the missing environment is part of the defect.
+
+Hardware prompts, operating-system permission dialogs, device-only APIs, and unavailable account states may be real blocks.
+
+## Setup check
+
+Before enabling the repro automation, run one harmless adapter check:
+
+1. Bring up the app.
+2. Confirm the stable app marker.
+3. Load one completed feature-map section.
+4. Navigate to that feature through its user path.
+5. Exercise one disposable state through mapped adapter actions.
+6. Inspect the resulting state.
+7. Capture a screenshot.
+8. Record a short clip.
+9. Clean up.
+
+Enable repro work only when all nine steps succeed and no source-channel Slack post is involved.

package/automations/benny/skills/reproduce-and-fix-issues/references/feature-map.example.md

@@ -0,0 +1,205 @@
+# Feature-map example
+
+Map every user-facing feature Benny may reproduce. Read the relevant section before driving the app. Keep this map at the user point of view. Discover internals and current code paths at runtime instead of freezing them here.
+
+Copy this file outside `.pi/automations/benny/`, for example to `.pi/benny/feature-map.md`, and set `control.feature_map_path` to the copy. Pack refreshes must not overwrite it.
+
+## Per-feature template
+
+### `<feature name>`
+
+`<one-line user-visible purpose>`
+
+#### How a user gets there
+
+- Click path: `<screen> -> <menu, tab, or panel> -> <control>`
+- Keyboard shortcut: `<shortcut or none>`
+
+#### How the control adapter drives it
+
+- `<adapter action>` with `<inputs>` should `<visible result>`.
+- Reset: `<how the adapter returns to a fresh state>`.
+
+#### Stable selectors
+
+- `<role and accessible name>`
+- `<ARIA relationship>`
+- `<data-component or purpose-named data attribute>`
+
+Never use generated CSS or StyleX classes, dynamic hashes, child indexes, or brittle DOM position.
+
+#### States to exercise
+
+- Default, hover, focus-visible, active, disabled
+- Loading, empty, error
+- Selected, open, expanded
+- `<relevant feature-specific variants>`
+
+Mark states that do not apply.
+
+#### Preconditions and setup
+
+- Auth: `<account state>`
+- Data: `<fixture>`
+- Permissions: `<role>`
+- Flags: `<flag or none>`
+- Services: `<required availability>`
+
+#### Evidence and cross-check
+
+- Screenshot: `<app identity, feature, and discriminating state>`
+- Video: `<entry path, interaction, and final state>`
+- Cross-check: `<read-only state or value that confirms the UI>`
+
+#### Gotchas
+
+- `<known dead end or wrong surface>`
+- `<safe environment translation>`
+
+## Fictional example
+
+These features belong to a fictional task app. They are examples, not required Benny features.
+
+### Sign in
+
+Lets a user enter the task app.
+
+#### How a user gets there
+
+- Open the app and choose `Sign in`. No shortcut.
+
+#### How the control adapter drives it
+
+- `open_app`, `click Sign in`, `fill credentials`, and `click Continue` should open the item list.
+- Reset by signing out and clearing the disposable session.
+
+#### Stable selectors
+
+- Button `Sign in`, textboxes `Email` and `Password`, `data-component="sign-in-form"`
+
+#### States to exercise
+
+- Default, focus-visible, submitting, disabled, loading, error
+
+#### Preconditions and setup
+
+- Disposable account and available authentication service
+
+#### Evidence and cross-check
+
+- Record landing page through item list. Check read-only session state.
+
+#### Gotchas
+
+- A marketing page is the wrong surface. A missing auth service is a block.
+
+### Item list and detail
+
+Lets a user browse items and open one.
+
+#### How a user gets there
+
+- Open the `Items` tab, then choose a row.
+
+#### How the control adapter drives it
+
+- `select_tab Items` and `click <fixture item>` should open its detail.
+- Reset by closing the detail and clearing selection.
+
+#### Stable selectors
+
+- Tab and list named `Items`, fixture-named row, `data-component="item-detail"`
+
+#### States to exercise
+
+- Loading, empty, error, selected, open, expanded
+
+#### Preconditions and setup
+
+- Named fixture items, read permission, available item service
+
+#### Evidence and cross-check
+
+- Show selection and matching detail title. Check selected-item ID.
+
+#### Gotchas
+
+- Search results may look similar but use a different path.
+
+### Item editor
+
+Lets a user create or edit an item.
+
+#### How a user gets there
+
+- Choose `Edit` from detail or `New item` from the list.
+
+#### How the control adapter drives it
+
+- `click Edit`, `fill <field>`, and `click Save` should update detail.
+- Reset by restoring the fixture.
+
+#### Stable selectors
+
+- Buttons `Edit`, `New item`, `Save`, form `Item editor`, label-linked fields
+
+#### States to exercise
+
+- Default, focus-visible, dirty, validating, disabled, saving, error, success
+
+#### Preconditions and setup
+
+- Editable fixture, write permission, available save service
+
+#### Evidence and cross-check
+
+- Show field change through updated detail. Check the stored item value read-only.
+
+#### Gotchas
+
+- Do not inject form state. A read-only detail field is not the editor.
+
+### Settings
+
+Lets a user change personal preferences.
+
+#### How a user gets there
+
+- Open the profile menu, then choose `Settings`.
+
+#### How the control adapter drives it
+
+- `open_menu Profile`, `click Settings`, and `toggle <preference>` should update the control.
+- Reset by restoring the starting preference.
+
+#### Stable selectors
+
+- Button `Profile`, menu item `Settings`, region `Settings`, purpose-named preference attribute
+
+#### States to exercise
+
+- Closed, open, selected, focus-visible, disabled, loading, error
+
+#### Preconditions and setup
+
+- Signed-in test account, known preferences, available preference service
+
+#### Evidence and cross-check
+
+- Show the menu path and final control state. Check the preference value read-only.
+
+#### Gotchas
+
+- Operating-system settings are a different surface.
+
+## Completeness checklist
+
+- Every reproducible user-facing feature has a section.
+- Every section names a user path, adapter actions, and reset.
+- Selectors use roles, names, ARIA, stable component markers, or purpose-named attributes.
+- No selector uses generated classes or DOM position.
+- Relevant interaction, loading, empty, error, selected, and expanded states are covered.
+- Auth, fixtures, permissions, flags, and services are explicit.
+- Screenshot, video, and underlying cross-check requirements are explicit.
+- Wrong surfaces, dead ends, and safe environment translations are listed.
+- Implementation details remain runtime discoveries.

package/automations/benny/skills/reproduce-and-fix-issues/references/verify-existing-fix.md

@@ -0,0 +1,93 @@
+# Verify an existing fix
+
+Use this mode when an open pull request or merged commit plausibly fixes the report.
+
+The existing artifact owns the fix. Verify it. Do not edit it, author a competing patch, or open another pull request.
+
+## Qualify the artifact
+
+Require one concrete artifact:
+
+- An open pull request with code changes that address the symptom
+- A merged pull request
+- A merged commit with matching code and intent
+
+A thread claim, tracker status, branch name, or cause hypothesis without a pull request or commit is not enough.
+
+When several artifacts exist, choose the one linked from the source thread or tracker. Otherwise choose the closest match to the affected code and state why.
+
+## Protect the working tree
+
+Use an isolated worktree or another clean checkout when the repository supports it. Do not overwrite user changes.
+
+Record:
+
+- Baseline revision
+- Patched revision
+- Pull request or commit URL
+- Build and environment inputs shared by both runs
+
+Use regular `github.com` pull request links.
+
+## Measure the baseline
+
+For an open pull request, use its base branch as the baseline.
+
+For a merged fix, use the revision immediately before the fix when that revision builds and represents the old behavior.
+
+Through the configured control adapter:
+
+1. Bring up the baseline app.
+2. Confirm the correct app and environment.
+3. Run the reported path through real UI actions.
+4. Observe the discriminating symptom.
+5. Reset and repeat it.
+6. Capture baseline recording, screenshot, and state check.
+
+If the symptom does not appear twice on the baseline, there is no baseline. Do not claim that the fix works.
+
+## Measure the patched build
+
+Build and run the pull request or fix commit with the same environment and data.
+
+1. Run the same UI path.
+2. Repeat it twice.
+3. Confirm that the broken state is gone.
+4. Confirm the expected state appears.
+5. Capture after recording, screenshot, and the same state check.
+
+Do not stop at compilation or tests. The after result must come from a running patched app.
+
+## Outcomes
+
+### Confirmed
+
+The baseline reproduces twice and the patched build resolves it twice.
+
+- Mark operations status as verified.
+- Link the artifact.
+- Post one concise source-thread reply after the source preflight.
+- Include the before and after result.
+- Open no pull request.
+
+### Insufficient fix
+
+The symptom appears on both baseline and patched builds.
+
+- Mark operations status as reproduced but not fixed.
+- Link the artifact and say it did not resolve the symptom.
+- Post the normal confirmed-repro source update if the run has not already used it.
+- Open no competing pull request.
+
+### Inconclusive
+
+The baseline does not reproduce, the patched app cannot run, or the evidence does not show the discriminating state.
+
+- Do not claim success.
+- State which half could not be measured.
+- Keep the result in the operations thread or the run's final message (its Task notification).
+- Post nothing in the source thread unless a direct question requires an answer.
+
+## Cleanup
+
+Stop both builds, remove temporary profiles and captures according to retention policy, and return the repository to its prior state without discarding user work.