agentme 0.35.2 → 0.37.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.filedist-package.yml +1 -1
- package/.xdrs/agentme/bdrs/index.md +2 -1
- package/.xdrs/agentme/bdrs/operations/{401-epic-feature-story-planning.md → 401-plan-epic-feature-story.md} +2 -2
- package/.xdrs/agentme/bdrs/operations/402-digital-product-roles.md +15 -19
- package/.xdrs/agentme/bdrs/operations/403-product-team-composition.md +3 -0
- package/.xdrs/agentme/bdrs/operations/404-team-roles-and-specialists.md +61 -82
- package/.xdrs/agentme/bdrs/operations/405-digital-product-roles-raci.md +131 -0
- package/.xdrs/agentme/bdrs/principles/002-okr-framework.md +1 -1
- package/.xdrs/agentme/bdrs/principles/003-product-delivery-value-flow.md +3 -2
- package/.xdrs/agentme/edrs/application/101-javascript-project-tooling.md +1 -1
- package/.xdrs/agentme/edrs/application/102-golang-project-tooling.md +1 -1
- package/.xdrs/agentme/edrs/application/103-python-project-tooling.md +1 -1
- package/.xdrs/agentme/edrs/application/127-external-system-adapter-skills.md +2 -0
- package/.xdrs/agentme/edrs/application/skills/azure-devops-connector/SKILL.md +202 -0
- package/.xdrs/agentme/edrs/application/skills/azure-devops-connector/SKILL.test.md +114 -0
- package/.xdrs/agentme/edrs/application/skills/{051-create-golang-project → create-golang-project}/SKILL.md +1 -1
- package/.xdrs/agentme/edrs/application/skills/create-golang-project/SKILL.test.md +72 -0
- package/.xdrs/agentme/edrs/application/skills/{050-create-javascript-project → create-javascript-project}/SKILL.md +23 -24
- package/.xdrs/agentme/edrs/application/skills/create-javascript-project/SKILL.test.md +72 -0
- package/.xdrs/agentme/edrs/application/skills/{052-create-python-project → create-python-project}/SKILL.md +1 -1
- package/.xdrs/agentme/edrs/application/skills/create-python-project/SKILL.test.md +79 -0
- package/.xdrs/agentme/edrs/application/skills/github-connector/SKILL.md +184 -0
- package/.xdrs/agentme/edrs/application/skills/github-connector/SKILL.test.md +118 -0
- package/.xdrs/agentme/edrs/application/skills/{200-run-skill-tests → run-skill-tests}/SKILL.md +4 -4
- package/.xdrs/agentme/edrs/application/skills/{200-run-skill-tests → run-skill-tests}/SKILL.test.md +7 -7
- package/.xdrs/agentme/edrs/application/skills/{010-select-relevant-xdrs → select-relevant-xdrs}/SKILL.md +1 -1
- package/.xdrs/agentme/edrs/application/skills/select-relevant-xdrs/SKILL.test.md +73 -0
- package/.xdrs/agentme/edrs/governance/502-contributing-guide-requirements.md +1 -1
- package/.xdrs/agentme/edrs/index.md +16 -8
- package/.xdrs/agentme/edrs/operations/402-dora-metrics-framework.md +82 -0
- package/.xdrs/agentme/edrs/operations/403-dora-deployment-frequency.md +37 -0
- package/.xdrs/agentme/edrs/operations/404-dora-lead-time-for-changes.md +41 -0
- package/.xdrs/agentme/edrs/operations/405-dora-change-failure-rate.md +42 -0
- package/.xdrs/agentme/edrs/operations/406-dora-time-to-restore-service.md +42 -0
- package/.xdrs/agentme/edrs/platform/301-monorepo-structure.md +1 -1
- package/.xdrs/agentme/edrs/platform/skills/{053-monorepo-setup → monorepo-setup}/SKILL.md +15 -15
- package/.xdrs/agentme/edrs/platform/skills/monorepo-setup/SKILL.test.md +80 -0
- package/.xdrs/agentme/edrs/principles/012-continuous-xdr-enrichment.md +1 -1
- package/.xdrs/agentme/edrs/principles/017-skill-testing.md +7 -4
- package/.xdrs/agentme/edrs/principles/articles/001-continuous-xdr-improvement.md +1 -1
- package/.xdrs/agentme/edrs/principles/skills/{150-refine-plan-mode → refine-plan-mode}/SKILL.md +27 -16
- package/.xdrs/agentme/edrs/principles/skills/{150-refine-plan-mode → refine-plan-mode}/SKILL.test.md +28 -4
- package/.xdrs/agentme/edrs/principles/skills/{151-refine-user-story → refine-user-story}/SKILL.md +4 -11
- package/.xdrs/agentme/edrs/principles/skills/{151-refine-user-story → refine-user-story}/SKILL.test.md +17 -17
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/Makefile +8 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/SKILL.md +662 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/SKILL.test.md +165 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/scripts/post-replies-azure-devops.js +219 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/scripts/post-replies-azure-devops.test.js +253 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/scripts/post-replies-github.js +237 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/scripts/post-replies-github.test.js +272 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/scripts/update-section.js +327 -0
- package/.xdrs/agentme/edrs/principles/skills/resolve-pr-comments/scripts/update-section.test.js +306 -0
- package/package.json +2 -2
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: github-connector
|
|
3
|
+
description: >
|
|
4
|
+
Base connector providing authentication, read access, and write access to GitHub pull
|
|
5
|
+
requests and their comments via the gh CLI. Pure I/O -- no triage logic, no business
|
|
6
|
+
decisions. Activate when an agent or skill (such as resolve-pr-comments) needs to fetch or
|
|
7
|
+
post PR comments, resolve review threads, or check out a PR branch on GitHub.
|
|
8
|
+
metadata:
|
|
9
|
+
author: flaviostutz
|
|
10
|
+
version: "1.1"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Overview
|
|
14
|
+
|
|
15
|
+
Reusable authentication and connection skill for GitHub pull requests, per
|
|
16
|
+
[agentme-edr-127](../../127-external-system-adapter-skills.md) rule 06 (connector naming).
|
|
17
|
+
Wraps the `gh` CLI -- GitHub's own supported API client -- so callers never construct raw
|
|
18
|
+
REST/GraphQL calls or handle GitHub-specific auth themselves. Contains no business logic (per
|
|
19
|
+
rule 05): it does not decide what a comment means, what action to take, or when to reply --
|
|
20
|
+
it only reads and writes data and normalizes it to the shape consumed by
|
|
21
|
+
[`resolve-pr-comments`](../../../principles/skills/resolve-pr-comments/SKILL.md).
|
|
22
|
+
|
|
23
|
+
## Instructions
|
|
24
|
+
|
|
25
|
+
### Authentication check
|
|
26
|
+
|
|
27
|
+
1. Verify the `gh` CLI is installed (`gh --version`); if missing, ask the human whether to
|
|
28
|
+
install it now via the appropriate package manager for their OS (e.g. `brew install gh`
|
|
29
|
+
on macOS). Only run the install command after explicit confirmation; if the human
|
|
30
|
+
declines or the install fails, report the install requirement and stop.
|
|
31
|
+
2. Run `gh auth status`. If authenticated, proceed -- this is the preferred path since `gh`
|
|
32
|
+
manages its own session token securely (per agentme-edr-124's least-exposure principle)
|
|
33
|
+
and needs no secret handling here.
|
|
34
|
+
3. If not authenticated, run
|
|
35
|
+
`gh auth login --hostname github.com --git-protocol https --web` instead of bare
|
|
36
|
+
`gh auth login` -- these flags answer the "Where do you use GitHub?", "preferred
|
|
37
|
+
protocol?", and "how would you like to authenticate?" prompts non-interactively so the
|
|
38
|
+
human is never asked them. `gh` still asks one local yes/no question ("Authenticate Git
|
|
39
|
+
with your GitHub credentials?"); accept its default (`Y`) automatically, since it only
|
|
40
|
+
wires the existing `gh` credential helper into git and needs no human input. The only
|
|
41
|
+
step that still requires the human is the one-time code / "Press Enter to open ... in
|
|
42
|
+
your browser" prompt that follows -- `gh` stores the resulting token in its own secure
|
|
43
|
+
storage once that completes. Never ask the human for a raw PAT and never read, request,
|
|
44
|
+
or feed a token to `gh` directly. Do not proceed with a write operation until
|
|
45
|
+
`gh auth status` reports an authenticated session.
|
|
46
|
+
4. Never substitute `gh` with a direct HTTP call (`curl`, `fetch`, or any other HTTP client)
|
|
47
|
+
against the GitHub REST/GraphQL API, and never scrape the PR's rendered HTML page as a
|
|
48
|
+
workaround -- this applies to reads as much as writes, and applies even when the target
|
|
49
|
+
data is public. If `gh` is missing or unauthenticated, stop and resolve that first (steps
|
|
50
|
+
1-3 above); do not degrade to an alternative retrieval method to route around it.
|
|
51
|
+
|
|
52
|
+
### Reading data
|
|
53
|
+
|
|
54
|
+
All read commands are plain `gh` invocations; none require confirmation (per
|
|
55
|
+
agentme-edr-127 rule 04, read-only operations are exempt from the HITL confirmation step
|
|
56
|
+
only -- they are not exempt from the CLI-only channel). Every read below MUST go through
|
|
57
|
+
`gh`; never construct the equivalent call with `curl`/another HTTP client, and never fetch
|
|
58
|
+
or scrape the PR's rendered web page as a substitute. When running `gh` from an automated
|
|
59
|
+
or non-interactive shell, prefix calls with `GH_PAGER=cat` (see Known Issues) so output is
|
|
60
|
+
never lost to a pager.
|
|
61
|
+
|
|
62
|
+
- PR metadata: `gh pr view <n> --json title,body,baseRefName,headRefName,url,state,isCrossRepository,headRepositoryOwner,headRepository`
|
|
63
|
+
- Issue-level (top) comments: `gh api repos/{owner}/{repo}/issues/{n}/comments`
|
|
64
|
+
- Review (file/line) comments: `gh api repos/{owner}/{repo}/pulls/{n}/comments`
|
|
65
|
+
- Review summaries: `gh api repos/{owner}/{repo}/pulls/{n}/reviews`
|
|
66
|
+
- Thread resolution state (REST does not expose this): `gh api graphql` with a
|
|
67
|
+
`reviewThreads` query on the PR, reading `isResolved` and each thread's comment node ids.
|
|
68
|
+
|
|
69
|
+
Normalize every fetched item to the shared record shape (`id`, `kind`, `status`, `can_reply`,
|
|
70
|
+
`can_resolve`, `path`, `line`, `content`, `author`, `in_reply_to`, `diff_hunk`, `url`):
|
|
71
|
+
- `kind` is `"issue-comment"`, `"review-comment"`, or `"review-summary"`.
|
|
72
|
+
- `status` is `"resolved"` when the GraphQL thread lookup marks it resolved, else `"open"`.
|
|
73
|
+
GitHub has no `wontfix`/`closed` state of its own -- `resolve-pr-comments` tracks those
|
|
74
|
+
locally.
|
|
75
|
+
- `can_resolve` is `true` only for `"review-comment"` items belonging to a resolvable
|
|
76
|
+
thread; `"issue-comment"` and `"review-summary"` are never resolvable -- set `false`.
|
|
77
|
+
- `in_reply_to` is resolved to the thread's top-level/root comment id, never an intermediate
|
|
78
|
+
reply, so replies always thread correctly.
|
|
79
|
+
- If the reply target is a reply-to-a-reply, resolve `in_reply_to` up to the root comment id
|
|
80
|
+
first (GitHub only allows replying to the root of a review thread).
|
|
81
|
+
- `diff_hunk` is taken verbatim from the `diff_hunk` field already present on each
|
|
82
|
+
`"review-comment"` item returned by the review-comments read command above -- no extra
|
|
83
|
+
fetch needed. Null for `"issue-comment"` and `"review-summary"` items, since neither is
|
|
84
|
+
file/line-scoped.
|
|
85
|
+
- `url` is taken verbatim from the `html_url` field already present on every issue-comment,
|
|
86
|
+
review-comment, and review object returned by the read commands above -- no extra fetch
|
|
87
|
+
needed for any `kind`.
|
|
88
|
+
|
|
89
|
+
### Writing data
|
|
90
|
+
|
|
91
|
+
Before any write below, show the mandatory confirmation (per agentme-edr-127 rule 04):
|
|
92
|
+
**System** (`owner/repo` + PR number), **Operation**, **Fields** (exact verbatim text to
|
|
93
|
+
post), **Estimated impact** (visible to PR participants, triggers notifications). Wait for
|
|
94
|
+
explicit confirmation; never proceed on an assumed "yes."
|
|
95
|
+
|
|
96
|
+
- Reply to an issue-level comment: `gh api repos/{owner}/{repo}/issues/{n}/comments -f body="..."`
|
|
97
|
+
- Reply to a review thread: `gh api repos/{owner}/{repo}/pulls/{n}/comments -f body="..." -F in_reply_to=<root-comment-id>`
|
|
98
|
+
- Post a new general PR comment: `gh pr comment <n> --body "..."`
|
|
99
|
+
- Resolve a review thread: `gh api graphql` with a `resolveReviewThread` mutation, passing
|
|
100
|
+
the thread's GraphQL node id (not the REST numeric id -- these are different identifier
|
|
101
|
+
spaces; see Known Issues).
|
|
102
|
+
- Check out the PR branch: `gh pr checkout <n>`.
|
|
103
|
+
|
|
104
|
+
### Constraints
|
|
105
|
+
|
|
106
|
+
- MUST use the `gh` CLI for every read and write handled by this connector -- never fall
|
|
107
|
+
back to `curl`, another raw HTTP client, or scraping the PR's HTML page, even when `gh` is
|
|
108
|
+
missing, unauthenticated, rate-limited, or erroring, and even when the target data is
|
|
109
|
+
public.
|
|
110
|
+
- MUST stop and follow the Authentication check steps above when `gh` cannot complete a
|
|
111
|
+
request, instead of silently degrading to an alternative retrieval method.
|
|
112
|
+
|
|
113
|
+
## Examples
|
|
114
|
+
|
|
115
|
+
**Input**: fetch all comments for `https://github.com/acme/widgets/pull/482`
|
|
116
|
+
|
|
117
|
+
Runs `gh pr view 482 --json ...` for metadata, then the three read commands above for
|
|
118
|
+
issue-comments, review-comments, and reviews, then one `gh api graphql` call for thread
|
|
119
|
+
resolution state, and returns a single normalized list.
|
|
120
|
+
|
|
121
|
+
**Input**: reply to review comment id `review-comment/91234` and mark it resolved
|
|
122
|
+
|
|
123
|
+
Shows the mandatory confirmation summary first. On explicit "yes," posts the reply via
|
|
124
|
+
`gh api repos/{owner}/{repo}/pulls/{n}/comments -F in_reply_to=91234`, then resolves the
|
|
125
|
+
thread via the GraphQL mutation using that comment's thread node id.
|
|
126
|
+
|
|
127
|
+
## Edge Cases
|
|
128
|
+
|
|
129
|
+
- **PR from a fork**: `headRepositoryOwner`/`headRepository` differ from the base repo;
|
|
130
|
+
checkout and branch comparisons must use the fork's remote, not the base repo's.
|
|
131
|
+
- **Review-summary comments**: never resolvable and never file/line-scoped; `path`/`line`
|
|
132
|
+
are always null and `can_resolve` is always `false`.
|
|
133
|
+
- **Reply-to-a-reply**: GitHub only supports replying to a thread's root comment; always
|
|
134
|
+
resolve `in_reply_to` up to the root before posting.
|
|
135
|
+
|
|
136
|
+
## Known Issues
|
|
137
|
+
|
|
138
|
+
- **Symptom:** `resolveReviewThread` mutation fails with a "could not resolve to a node"
|
|
139
|
+
error even though the comment id is valid.
|
|
140
|
+
**Cause:** the mutation requires the review thread's GraphQL node id, not the numeric
|
|
141
|
+
REST comment id or the numeric review id -- these are three different identifier spaces.
|
|
142
|
+
**Fix:** always fetch the thread's node id via the `reviewThreads` GraphQL query first,
|
|
143
|
+
and cache the numeric-id-to-node-id mapping per PR fetch. Verify the exact mutation shape
|
|
144
|
+
via `gh api graphql` introspection before first use in a new environment.
|
|
145
|
+
- **Symptom:** posting a reply or resolving a thread returns HTTP 403 despite `gh auth
|
|
146
|
+
status` showing a valid session.
|
|
147
|
+
**Cause:** the authenticated account lacks write/triage permission on the repository (for
|
|
148
|
+
example, an outside collaborator with read-only access).
|
|
149
|
+
**Fix:** degrade to reply-only where permitted, report the permission gap plainly, and
|
|
150
|
+
never retry the same call silently.
|
|
151
|
+
- **Symptom:** any `gh api` call returns HTTP 404 for a repo the human insists exists.
|
|
152
|
+
**Cause:** the authenticated token lacks the `repo` (or fine-grained equivalent) scope, so
|
|
153
|
+
GitHub reports a private resource as not found rather than as forbidden.
|
|
154
|
+
**Fix:** report this distinction explicitly and ask the human to re-run
|
|
155
|
+
`gh auth refresh -s repo` rather than assuming the PR truly does not exist.
|
|
156
|
+
- **Symptom:** repeated calls start failing with HTTP 403 and a rate-limit message.
|
|
157
|
+
**Cause:** GitHub's REST/GraphQL rate limits were exceeded, often from re-fetching the
|
|
158
|
+
full comment list too frequently in one session.
|
|
159
|
+
**Fix:** space out calls and reuse the already-fetched result within a session; never
|
|
160
|
+
silently retry in a tight loop.
|
|
161
|
+
- **Symptom:** an agent fetched PR metadata, comments, or diffs via `curl`/`api.github.com`
|
|
162
|
+
or by scraping the PR's rendered HTML page instead of using `gh`.
|
|
163
|
+
**Cause:** `gh` was missing or unauthenticated (e.g. `gh auth status` reported not logged
|
|
164
|
+
in), and the agent treated the unauthenticated public REST API or the rendered PR page as
|
|
165
|
+
an acceptable substitute since the target data was technically public.
|
|
166
|
+
**Fix:** never substitute `gh` with a direct HTTP call or a page scrape, regardless of
|
|
167
|
+
whether the data is public. Stop at the Authentication check step, report the missing or
|
|
168
|
+
failed `gh` session plainly, and wait for the human to install or authenticate `gh` before
|
|
169
|
+
retrying the same read or write through `gh`.
|
|
170
|
+
- **Symptom:** a `gh api`/`gh pr view` read command run from an automated shell appears to
|
|
171
|
+
hang or return no captured output at all, even though `gh` itself succeeded.
|
|
172
|
+
**Cause:** `gh` falls back to `$PAGER` (commonly `less`) for output it thinks may be
|
|
173
|
+
interactive; this switches the terminal to its alternate screen buffer, and content shown
|
|
174
|
+
there is not part of normal scrollback, so an automated caller never sees it -- piping
|
|
175
|
+
through `| cat` alone does not reliably prevent this.
|
|
176
|
+
**Fix:** prefix every `gh` invocation with `GH_PAGER=cat` (e.g.
|
|
177
|
+
`GH_PAGER=cat gh api repos/{owner}/{repo}/issues/{n}/comments`) when running non-
|
|
178
|
+
interactively, which disables `gh`'s pager unconditionally.
|
|
179
|
+
|
|
180
|
+
## References
|
|
181
|
+
|
|
182
|
+
- [`resolve-pr-comments`](../../../principles/skills/resolve-pr-comments/SKILL.md) -- consumes this connector's normalized output.
|
|
183
|
+
- [`agentme-edr-127`](../../127-external-system-adapter-skills.md) -- external system adapter skill rules (connector naming, Known Issues format, HITL-before-write).
|
|
184
|
+
- [`agentme-edr-124`](../../124-secrets-management.md) -- credential storage and retrieval.
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
---
|
|
2
|
+
skill: github-connector
|
|
3
|
+
skill-version: "1.0"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## Test Scenarios
|
|
7
|
+
|
|
8
|
+
### Scenario 1: Fetch and normalize all comment kinds for a PR, happy path
|
|
9
|
+
|
|
10
|
+
**Trigger / Input**
|
|
11
|
+
|
|
12
|
+
Fetch all comments for `https://github.com/acme/widgets/pull/482`, a PR with one issue-level
|
|
13
|
+
comment, two review (file/line) comments in one resolved thread, and one review-summary.
|
|
14
|
+
|
|
15
|
+
**Expected Behaviour**
|
|
16
|
+
|
|
17
|
+
The connector: (1) confirms `gh auth status` is authenticated; (2) runs the PR metadata
|
|
18
|
+
read; (3) runs the issue-comments, review-comments, and reviews read commands; (4) runs one
|
|
19
|
+
`gh api graphql` call to read thread resolution state; (5) returns a single normalized list
|
|
20
|
+
where the issue-level comment has `kind: "issue-comment"` and `can_resolve: false`, the two
|
|
21
|
+
review comments have `kind: "review-comment"`, `status: "resolved"`, and `can_resolve:
|
|
22
|
+
true`, and the review-summary has `kind: "review-summary"` and `can_resolve: false`.
|
|
23
|
+
|
|
24
|
+
**Assertions**
|
|
25
|
+
|
|
26
|
+
- [ ] Output normalizes every fetched item to the shared record shape (id, kind, status,
|
|
27
|
+
can_reply, can_resolve, path, line, content, author, in_reply_to).
|
|
28
|
+
- [ ] Output sets `can_resolve: false` for the issue-comment and the review-summary items.
|
|
29
|
+
- [ ] Output sets `status: "resolved"` for the two review comments in the resolved thread,
|
|
30
|
+
derived from the GraphQL thread lookup rather than the REST response alone.
|
|
31
|
+
- [ ] Connector never branches its own logic on business meaning of comment content.
|
|
32
|
+
|
|
33
|
+
### Scenario 2: Reply-to-a-reply resolves to the thread root before posting
|
|
34
|
+
|
|
35
|
+
**Trigger / Input**
|
|
36
|
+
|
|
37
|
+
Post a reply targeting a review comment that is itself a reply (not the root) within its
|
|
38
|
+
thread.
|
|
39
|
+
|
|
40
|
+
**Expected Behaviour**
|
|
41
|
+
|
|
42
|
+
Before posting, the connector resolves `in_reply_to` up to the thread's root comment id
|
|
43
|
+
(GitHub only accepts replies anchored to the root), shows the mandatory System/Operation/
|
|
44
|
+
Fields/Estimated impact confirmation using the resolved root id, and only posts via
|
|
45
|
+
`gh api repos/{owner}/{repo}/pulls/{n}/comments -F in_reply_to=<root-id>` after explicit
|
|
46
|
+
confirmation.
|
|
47
|
+
|
|
48
|
+
**Assertions**
|
|
49
|
+
|
|
50
|
+
- [ ] Connector resolves `in_reply_to` to the thread's root comment id, not the intermediate
|
|
51
|
+
reply id, before constructing the write call.
|
|
52
|
+
- [ ] Connector shows the mandatory confirmation (System, Operation, Fields, Estimated
|
|
53
|
+
impact) before posting.
|
|
54
|
+
- [ ] Connector does not post before receiving explicit human confirmation.
|
|
55
|
+
|
|
56
|
+
### Scenario 3: Permission-denied write degrades to reply-only, no silent retry
|
|
57
|
+
|
|
58
|
+
**Trigger / Input**
|
|
59
|
+
|
|
60
|
+
A resolve-thread write (`gh api graphql` `resolveReviewThread` mutation) returns HTTP 403
|
|
61
|
+
because the authenticated account lacks triage permission on the repository.
|
|
62
|
+
|
|
63
|
+
**Expected Behaviour**
|
|
64
|
+
|
|
65
|
+
Per the Known Issues entry for this symptom, the connector reports the permission gap
|
|
66
|
+
plainly to the caller, does not retry the same call, and continues to allow a reply-only
|
|
67
|
+
write path for that comment (an already-successful or subsequent reply post is unaffected).
|
|
68
|
+
|
|
69
|
+
**Assertions**
|
|
70
|
+
|
|
71
|
+
- [ ] Connector reports the permission error explicitly rather than failing silently.
|
|
72
|
+
- [ ] Connector does not silently retry the failed resolve call.
|
|
73
|
+
- [ ] Connector still allows a reply-only write for the same comment.
|
|
74
|
+
|
|
75
|
+
### Scenario 4: No GitHub session available halts before any write
|
|
76
|
+
|
|
77
|
+
**Trigger / Input**
|
|
78
|
+
|
|
79
|
+
`gh auth status` reports not logged in.
|
|
80
|
+
|
|
81
|
+
**Expected Behaviour**
|
|
82
|
+
|
|
83
|
+
The connector runs `gh auth login --hostname github.com --git-protocol https --web`
|
|
84
|
+
(non-interactive except for the one local git-credential yes/no, which the connector
|
|
85
|
+
auto-accepts on its default) so the human is only asked to complete the browser/device-code
|
|
86
|
+
step, and does not attempt any write operation until `gh auth status` reports an
|
|
87
|
+
authenticated session. Read-only operations that do not require authentication (if any) are
|
|
88
|
+
unaffected.
|
|
89
|
+
|
|
90
|
+
**Assertions**
|
|
91
|
+
|
|
92
|
+
- [ ] Connector does not attempt a write operation without an authenticated `gh` session.
|
|
93
|
+
- [ ] Connector runs `gh auth login` with `--hostname`/`--git-protocol`/`--web` rather than
|
|
94
|
+
bare `gh auth login`, so the human is not asked the host/protocol/method questions.
|
|
95
|
+
- [ ] Connector never asks for, reads, or feeds `gh` a raw PAT/token directly.
|
|
96
|
+
|
|
97
|
+
### Scenario 5: gh unavailable or unauthenticated never triggers a curl/scrape fallback
|
|
98
|
+
|
|
99
|
+
**Trigger / Input**
|
|
100
|
+
|
|
101
|
+
Fetch PR metadata, comments, and diff for `https://github.com/acme/widgets/pull/482` while
|
|
102
|
+
`gh auth status` reports not logged in (or `gh` is not installed).
|
|
103
|
+
|
|
104
|
+
**Expected Behaviour**
|
|
105
|
+
|
|
106
|
+
The connector does not issue any direct `curl`/HTTP request to `api.github.com`,
|
|
107
|
+
`github.com`, or a `.diff`/`.patch` endpoint, and does not fetch or parse the PR's rendered
|
|
108
|
+
HTML page as a substitute -- even though the target repository and PR are public. Instead it
|
|
109
|
+
follows the Authentication check steps, reports the missing/failed `gh` session plainly, and
|
|
110
|
+
halts every read and write until `gh auth status` reports an authenticated session.
|
|
111
|
+
|
|
112
|
+
**Assertions**
|
|
113
|
+
|
|
114
|
+
- [ ] Connector does not issue a `curl` or other raw HTTP request to any GitHub endpoint as
|
|
115
|
+
a substitute for `gh`.
|
|
116
|
+
- [ ] Connector does not fetch or parse the PR's HTML page as a scraping fallback.
|
|
117
|
+
- [ ] Connector halts reads and writes and prompts for `gh` installation/`gh auth login`
|
|
118
|
+
rather than degrading to an alternative retrieval method.
|
package/.xdrs/agentme/edrs/application/skills/{200-run-skill-tests → run-skill-tests}/SKILL.md
RENAMED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: run-skill-tests
|
|
3
3
|
description: >
|
|
4
4
|
Executes the SKILL.test.md test scenarios for a given skill, evaluates each assertion against the skill's
|
|
5
5
|
actual output, and reports a pass/fail result per scenario. Activate when the user asks to test, verify,
|
|
@@ -17,7 +17,7 @@ Loads `SKILL.test.md` from a skill directory, runs each scenario by invoking the
|
|
|
17
17
|
|
|
18
18
|
### Phase 1: Locate and Validate SKILL.test.md
|
|
19
19
|
|
|
20
|
-
1. Accept the skill path from the user (e.g., `.agents/skills/
|
|
20
|
+
1. Accept the skill path from the user (e.g., `.agents/skills/review` or `.xdrs/_local/adrs/principles/skills/005-my-skill`). If no path is given, ask: *"Which skill do you want to test? Provide the path to the skill directory."*
|
|
21
21
|
2. Resolve the path to the directory containing `SKILL.md`. If the path is a symlink, follow it to the real directory.
|
|
22
22
|
3. Check that `SKILL.test.md` exists in the same directory as `SKILL.md`. If it is missing, output:
|
|
23
23
|
|
|
@@ -91,9 +91,9 @@ A scenario passes only when every one of its assertions passes. The overall outc
|
|
|
91
91
|
|
|
92
92
|
## Examples
|
|
93
93
|
|
|
94
|
-
**Input**: "Test the skill at `.agents/skills/
|
|
94
|
+
**Input**: "Test the skill at `.agents/skills/review`"
|
|
95
95
|
|
|
96
|
-
- Phase 1 resolves the symlink → `.xdrs/_core/adrs/principles/skills/
|
|
96
|
+
- Phase 1 resolves the symlink → `.xdrs/_core/adrs/principles/skills/review/`
|
|
97
97
|
- Reads `SKILL.test.md`, parses 3 scenarios
|
|
98
98
|
- Executes each scenario with the target skill loaded
|
|
99
99
|
- Reports per-assertion PASS/FAIL and an overall outcome
|
package/.xdrs/agentme/edrs/application/skills/{200-run-skill-tests → run-skill-tests}/SKILL.test.md
RENAMED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
skill:
|
|
2
|
+
skill: run-skill-tests
|
|
3
3
|
skill-version: "1.0"
|
|
4
4
|
---
|
|
5
5
|
|
|
@@ -8,9 +8,9 @@ skill-version: "1.0"
|
|
|
8
8
|
### Scenario 1: Successfully executes scenarios and reports results
|
|
9
9
|
|
|
10
10
|
**Trigger / Input**
|
|
11
|
-
You are an agent with the `
|
|
11
|
+
You are an agent with the `run-skill-tests` skill loaded. The workspace has a skill at `.agents/skills/review` with both `SKILL.md` and `SKILL.test.md` present. The user says:
|
|
12
12
|
|
|
13
|
-
"Test the skill at `.agents/skills/
|
|
13
|
+
"Test the skill at `.agents/skills/review`"
|
|
14
14
|
|
|
15
15
|
**Expected Behaviour**
|
|
16
16
|
1. Skill resolves the path to the real directory (following the symlink).
|
|
@@ -22,7 +22,7 @@ You are an agent with the `200-run-skill-tests` skill loaded. The workspace has
|
|
|
22
22
|
7. Skill outputs a structured report using the mandated template with a final Outcome line.
|
|
23
23
|
|
|
24
24
|
**Assertions**
|
|
25
|
-
- [ ] Output contains `## Skill Test Report:
|
|
25
|
+
- [ ] Output contains `## Skill Test Report: review`.
|
|
26
26
|
- [ ] Output contains a `### Scenario` section for each scenario in `SKILL.test.md`.
|
|
27
27
|
- [ ] Each scenario section lists individual assertion results as `PASS` or `FAIL`.
|
|
28
28
|
- [ ] Output ends with a `## Summary` block containing `Scenarios:`, `Passed:`, `Failed:`, and `Outcome:`.
|
|
@@ -30,9 +30,9 @@ You are an agent with the `200-run-skill-tests` skill loaded. The workspace has
|
|
|
30
30
|
### Scenario 2: Halts with ERROR when SKILL.test.md is missing
|
|
31
31
|
|
|
32
32
|
**Trigger / Input**
|
|
33
|
-
You are an agent with the `
|
|
33
|
+
You are an agent with the `run-skill-tests` skill loaded. The workspace has a skill directory `.xdrs/agentme/edrs/application/skills/create-javascript-project` that contains `SKILL.md` but no `SKILL.test.md`. The user says:
|
|
34
34
|
|
|
35
|
-
"Run tests for `.xdrs/agentme/edrs/application/skills/
|
|
35
|
+
"Run tests for `.xdrs/agentme/edrs/application/skills/create-javascript-project`"
|
|
36
36
|
|
|
37
37
|
**Expected Behaviour**
|
|
38
38
|
1. Skill resolves the path to the directory.
|
|
@@ -49,7 +49,7 @@ You are an agent with the `200-run-skill-tests` skill loaded. The workspace has
|
|
|
49
49
|
### Scenario 3: Injects simulated human responses when present
|
|
50
50
|
|
|
51
51
|
**Trigger / Input**
|
|
52
|
-
You are an agent with the `
|
|
52
|
+
You are an agent with the `run-skill-tests` skill loaded. The workspace has a skill at `.xdrs/agentme/edrs/principles/skills/refine-plan-mode` with both `SKILL.md` and `SKILL.test.md` present. Scenario 1 in that `SKILL.test.md` includes a `**Simulated Human Responses**` section with 9 ordered responses. The user says:
|
|
53
53
|
|
|
54
54
|
"Test the skill at `.xdrs/agentme/edrs/principles/skills/refine-plan-mode`"
|
|
55
55
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: select-relevant-xdrs
|
|
3
3
|
description: >
|
|
4
4
|
Analyzes a client repository, extracts the full agentme XDR set, and excludes the records that do
|
|
5
5
|
not fit the project's structure and workflow needs. Activate this skill when the user asks to
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
---
|
|
2
|
+
skill: select-relevant-xdrs
|
|
3
|
+
skill-version: "1.0"
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## Test Scenarios
|
|
7
|
+
|
|
8
|
+
### Scenario 1: Install presets for a single-package Node.js library, happy path
|
|
9
|
+
|
|
10
|
+
**Trigger / Input**
|
|
11
|
+
|
|
12
|
+
"Install the right agentme XDR presets for this Node.js library." The repository is a single npm
|
|
13
|
+
package with a Makefile and no Go code, no monorepo layout, and no deployed/long-running service.
|
|
14
|
+
|
|
15
|
+
**Expected Behaviour**
|
|
16
|
+
|
|
17
|
+
1. Phase 1 inventories the shipped agentme XDR files.
|
|
18
|
+
2. Phase 2 analyzes the repository and detects a JavaScript/TypeScript single-package library with
|
|
19
|
+
Makefiles, no Go tooling, no monorepo structure, and no service surface.
|
|
20
|
+
3. Phase 3 selects exclusions with one-line rationale each — at minimum the Go project tooling XDR
|
|
21
|
+
and the service health-check XDR — while keeping baseline/broadly applicable guidance.
|
|
22
|
+
4. Phase 4 runs `npx -y agentme extract --output . --all --exclude <path> --exclude <path>` with
|
|
23
|
+
one `--exclude` flag per excluded XDR, then verifies `.xdrs/index.md`, `.xdrs/agentme/`, and
|
|
24
|
+
`AGENTS.md` exist and that every excluded path is absent from the output.
|
|
25
|
+
|
|
26
|
+
**Assertions**
|
|
27
|
+
|
|
28
|
+
- [ ] Output excludes `.xdrs/agentme/edrs/application/102-golang-project-tooling.md` and
|
|
29
|
+
`.xdrs/agentme/edrs/operations/401-service-health-check-endpoint.md` with a one-line
|
|
30
|
+
rationale for each.
|
|
31
|
+
- [ ] Output runs `agentme extract` with `--all` plus one `--exclude` flag per excluded XDR path,
|
|
32
|
+
rather than a hand-picked include list.
|
|
33
|
+
- [ ] Output verifies after extraction that `.xdrs/index.md`, `.xdrs/agentme/`, and `AGENTS.md`
|
|
34
|
+
exist and that the excluded paths are absent from the result.
|
|
35
|
+
|
|
36
|
+
### Scenario 2: Extraction blocked when the package exposes no inventory metadata
|
|
37
|
+
|
|
38
|
+
**Trigger / Input**
|
|
39
|
+
|
|
40
|
+
Run the skill in an environment where the CLI cannot enumerate the shipped XDRs, and the package
|
|
41
|
+
metadata and repository documentation also do not describe the shipped XDR set.
|
|
42
|
+
|
|
43
|
+
**Expected Behaviour**
|
|
44
|
+
|
|
45
|
+
Per Phase 1 step 4, since even the fallback inventory sources fail, the skill stops before
|
|
46
|
+
attempting any extraction and reports that automatic selection is blocked because the package does
|
|
47
|
+
not expose enough metadata in the current environment, rather than guessing an exclude list or
|
|
48
|
+
forcing a full installation.
|
|
49
|
+
|
|
50
|
+
**Assertions**
|
|
51
|
+
|
|
52
|
+
- [ ] Output stops before running any `agentme extract` command.
|
|
53
|
+
- [ ] Output reports that automatic selection is blocked due to insufficient package metadata,
|
|
54
|
+
rather than silently falling back to a default guess.
|
|
55
|
+
|
|
56
|
+
### Scenario 3: Debatable exclusion is kept, not excluded
|
|
57
|
+
|
|
58
|
+
**Trigger / Input**
|
|
59
|
+
|
|
60
|
+
Analyzing a currently single-package repository whose README states the team is planning to split
|
|
61
|
+
it into multiple applications "sometime in the future." Whether the monorepo-structure XDR still
|
|
62
|
+
applies is debatable.
|
|
63
|
+
|
|
64
|
+
**Expected Behaviour**
|
|
65
|
+
|
|
66
|
+
Per the Edge Cases entry on debatable exclusions, since the mismatch is not concrete today, the
|
|
67
|
+
skill keeps the monorepo-structure XDR in the installed set rather than excluding it on the basis
|
|
68
|
+
of a possible future restructuring.
|
|
69
|
+
|
|
70
|
+
**Assertions**
|
|
71
|
+
|
|
72
|
+
- [ ] Output keeps `.xdrs/agentme/edrs/platform/301-monorepo-structure.md` in the installed set.
|
|
73
|
+
- [ ] Output does not add it to the exclude list on the basis of a future, not-yet-concrete plan.
|
|
@@ -67,4 +67,4 @@ The content SHOULD stay concise and practical; do not turn `CONTRIBUTING.md` int
|
|
|
67
67
|
## References
|
|
68
68
|
|
|
69
69
|
- [agentme-edr-301 - Monorepo structure](../platform/301-monorepo-structure.md)
|
|
70
|
-
- [
|
|
70
|
+
- [monorepo-setup skill](../platform/skills/monorepo-setup/SKILL.md)
|
|
@@ -8,11 +8,12 @@ Propose changes via pull request. All changes must be verified for clarity and n
|
|
|
8
8
|
|
|
9
9
|
Foundational standards, principles, and guidelines.
|
|
10
10
|
|
|
11
|
-
- [
|
|
12
|
-
- [
|
|
11
|
+
- [refine-plan-mode](principles/skills/refine-plan-mode/SKILL.md) - **Refine plan mode** — MANDATORY skill for ANY planning activity (plan, design, propose, outline, draft, brainstorm, architect). Read and follow in full before any execution begins. Must be read from XDRS even when not in `.agents/skills`. *(skill)*
|
|
12
|
+
- [refine-user-story](principles/skills/refine-user-story/SKILL.md) - **Refine user stories** — Refine, elaborate, study or develop the contents of a user story used to create a unit of work for an agile team. Runs a structured 10-phase refinement process: understand the request, qualify requirements, research context, review consistency, validate visually, challenge from 9 user-perspective angles (Phase 6), challenge from 8 implementer-perspective angles (Phase 7), produce a ready-to-implement story (Phase 8), and run a final readiness double-check (Phase 9). *(skill)*
|
|
13
|
+
- [resolve-pr-comments](principles/skills/resolve-pr-comments/SKILL.md) - **Resolve PR comments** — Helps the OWNER of a pull request work through comments left by others: fetches every comment (GitHub or Azure DevOps) from its URL, tracks them in a local file, and walks through triaging each one (reply, won't-fix, fix) at a human-chosen automation level -- from fully automatic to fully guided, with fine-grained free-text control over the mix -- while syncing back to the provider always stays its own explicitly confirmed step. A hands-on, mutating workflow to answer feedback and land fixes -- not a code-review skill. Delegates provider-specific reads/writes to `github-connector` or `azure-devops-connector`. *(skill)*
|
|
13
14
|
- [agentme-edr-012](principles/012-continuous-xdr-enrichment.md) - **Continuous xdr improvement policy** - Promote recurring delivery lessons into reusable XDRs
|
|
14
15
|
- [agentme-edr-016](principles/016-cross-language-module-structure.md) - **Cross-language module structure** - Organize modules consistently across supported languages
|
|
15
|
-
- [agentme-edr-017](principles/017-skill-testing.md) - **skill testing** - Mandates a `SKILL.test.md` co-located with every skill in scopes that follow agentme; defines test scenario format (trigger, expected behaviour, assertions) and requires execution before merging any skill change *(includes skill: [
|
|
16
|
+
- [agentme-edr-017](principles/017-skill-testing.md) - **skill testing** - Mandates a `SKILL.test.md` co-located with every skill in scopes that follow agentme; defines test scenario format (trigger, expected behaviour, assertions) and requires execution before merging any skill change *(includes skill: [run-skill-tests](application/skills/run-skill-tests/SKILL.md))*
|
|
16
17
|
|
|
17
18
|
## Articles
|
|
18
19
|
|
|
@@ -30,15 +31,17 @@ Language and framework-specific tooling and project structure.
|
|
|
30
31
|
- [agentme-edr-124](application/124-secrets-management.md) - **Secrets management** - Handle secrets securely using native keychains and cloud secret managers
|
|
31
32
|
- [agentme-edr-125](application/125-coding-abstraction-practices.md) - **Coding abstraction practices** - Define when abstractions are justified and when they must be inlined
|
|
32
33
|
- [agentme-edr-127](application/127-external-system-adapter-skills.md) - **External system adapter skills** - Priority-ordered approach and adapter skill authoring standards for automating interactions with external systems
|
|
34
|
+
- [github-connector](application/skills/github-connector/SKILL.md) - **GitHub connector** — Base connector providing authentication, read access, and write access to GitHub pull requests and their comments via the `gh` CLI. *(skill)*
|
|
35
|
+
- [azure-devops-connector](application/skills/azure-devops-connector/SKILL.md) - **Azure DevOps connector** — Base connector providing authentication, read access, and write access to Azure DevOps pull requests and their comment threads via the `az` CLI. *(skill)*
|
|
33
36
|
|
|
34
37
|
### Language and framework tooling
|
|
35
38
|
|
|
36
|
-
- [agentme-edr-101](application/101-javascript-project-tooling.md) - **JavaScript project tooling and structure** - Scaffold JavaScript libraries with the standard toolchain *(includes skill: [
|
|
37
|
-
- [agentme-edr-102](application/102-golang-project-tooling.md) - **Go project tooling and structure** - Scaffold Go CLIs and libraries with the standard layout *(includes skill: [
|
|
38
|
-
- [agentme-edr-103](application/103-python-project-tooling.md) - **Python project tooling and structure** - Scaffold Python packages and CLIs with the standard layout *(includes skill: [
|
|
39
|
+
- [agentme-edr-101](application/101-javascript-project-tooling.md) - **JavaScript project tooling and structure** - Scaffold JavaScript libraries with the standard toolchain *(includes skill: [create-javascript-project](application/skills/create-javascript-project/SKILL.md))*
|
|
40
|
+
- [agentme-edr-102](application/102-golang-project-tooling.md) - **Go project tooling and structure** - Scaffold Go CLIs and libraries with the standard layout *(includes skill: [create-golang-project](application/skills/create-golang-project/SKILL.md))*
|
|
41
|
+
- [agentme-edr-103](application/103-python-project-tooling.md) - **Python project tooling and structure** - Scaffold Python packages and CLIs with the standard layout *(includes skill: [create-python-project](application/skills/create-python-project/SKILL.md))*
|
|
39
42
|
- [agentme-edr-104](application/104-cli-tool-standards.md) - **CLI tool standards** - Define command UX and behavior for CLI tools
|
|
40
43
|
- [agentme-edr-126](application/126-pragmatic-hexagonal-architecture.md) - **Pragmatic hexagonal architecture** - Organize application layers as External/Adapters/Application with practical coupling rules
|
|
41
|
-
- [
|
|
44
|
+
- [select-relevant-xdrs](application/skills/select-relevant-xdrs/SKILL.md) - **Select relevant XDRs**
|
|
42
45
|
|
|
43
46
|
### AI development
|
|
44
47
|
|
|
@@ -72,7 +75,7 @@ Data layer implementation and data management decisions.
|
|
|
72
75
|
|
|
73
76
|
Infrastructure implementation, delivery pipeline, and developer environment decisions.
|
|
74
77
|
|
|
75
|
-
- [agentme-edr-301](platform/301-monorepo-structure.md) - **Monorepo structure** - Standardize monorepo layout, tooling, and package boundaries *(includes skill: [
|
|
78
|
+
- [agentme-edr-301](platform/301-monorepo-structure.md) - **Monorepo structure** - Standardize monorepo layout, tooling, and package boundaries *(includes skill: [monorepo-setup](platform/skills/monorepo-setup/SKILL.md))*
|
|
76
79
|
- [agentme-edr-302](platform/302-github-pipelines.md) - **GitHub CI/CD pipelines** - Define required CI stages and workflow structure
|
|
77
80
|
- [agentme-edr-303](platform/303-common-targets.md) - **Common development script names** - Reuse standard build, lint, and test target names
|
|
78
81
|
- [agentme-edr-304](platform/304-tool-execution-and-scripting.md) - **Tool execution and scripting** - Run tools consistently across shells, Makefiles, and CI
|
|
@@ -90,3 +93,8 @@ Contribution and collaboration standards shared across projects.
|
|
|
90
93
|
Production behavior and operational response decisions.
|
|
91
94
|
|
|
92
95
|
- [agentme-edr-401](operations/401-service-health-check-endpoint.md) - **Service health check endpoint** - Expose a standard runtime health-check endpoint for services
|
|
96
|
+
- [agentme-edr-402](operations/402-dora-metrics-framework.md) - **DORA metrics framework** - Per-service/team measurement, sub-metric composition, maturity tiers, and adoption order for DORA's four key metrics
|
|
97
|
+
- [agentme-edr-403](operations/403-dora-deployment-frequency.md) - **DORA deployment frequency** - Calculation, examples, and challenges for deploys-per-week, batch size, and automation ratio
|
|
98
|
+
- [agentme-edr-404](operations/404-dora-lead-time-for-changes.md) - **DORA lead time for changes** - Calculation, examples, and challenges for PR cycle time, review latency, deploy-pipeline time, and total lead time
|
|
99
|
+
- [agentme-edr-405](operations/405-dora-change-failure-rate.md) - **DORA change failure rate** - Calculation, examples, and challenges for hotfix/revert rate, incident-linked deploy rate, and escaped defect rate, including cross-team attribution
|
|
100
|
+
- [agentme-edr-406](operations/406-dora-time-to-restore-service.md) - **DORA time to restore service** - Calculation, examples, and challenges for total restore time, detection time, acknowledgement time, and resolution time
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: agentme-edr-policy-402-dora-metrics-framework
|
|
3
|
+
description: Defines the framework for measuring DORA's four software delivery metrics per team and service, including sub-metric composition, maturity-tier definition, distribution reporting, adoption order, and daily usage by role. Use when implementing or reviewing DORA metrics measurement, dashboards, or reporting.
|
|
4
|
+
apply-to: Teams and organizations measuring, reporting, or improving software delivery performance using DORA's four key metrics
|
|
5
|
+
valid-from: 2026-09-18
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# agentme-edr-policy-402: DORA metrics framework
|
|
9
|
+
|
|
10
|
+
## Context and Problem Statement
|
|
11
|
+
|
|
12
|
+
Teams need objective signals of software delivery performance, but without a shared framework, teams measure different things, blend unlike services together, or reduce delivery health to one misleading number. How should DORA's four key metrics be scoped, combined into a maturity signal, and used across teams so results stay comparable, actionable, and resistant to gaming?
|
|
13
|
+
|
|
14
|
+
## Decision Outcome
|
|
15
|
+
|
|
16
|
+
**Adopt DORA's four key metrics, each measured per service and rolled up per team, decomposed into incrementally-adoptable sub-metrics defined in one companion policy per metric.**
|
|
17
|
+
|
|
18
|
+
Every team gets a maturity tier per metric plus one blended tier; company-wide reporting always shows a tier distribution, never a company-wide average.
|
|
19
|
+
|
|
20
|
+
### Details
|
|
21
|
+
|
|
22
|
+
#### 01-four-key-metrics
|
|
23
|
+
|
|
24
|
+
This framework covers DORA's four key metrics: Deployment Frequency ([agentme-edr-403](403-dora-deployment-frequency.md)), Lead Time for Changes ([agentme-edr-404](404-dora-lead-time-for-changes.md)), Change Failure Rate ([agentme-edr-405](405-dora-change-failure-rate.md)), and Time to Restore Service ([agentme-edr-406](406-dora-time-to-restore-service.md); DORA's current term is "Failed Deployment Recovery Time"). Sub-metrics MUST follow a hybrid composition model: Lead Time and Time to Restore sub-metrics are additive and sum to the parent value, while Deployment Frequency and Change Failure Rate sub-metrics are complementary signals that MUST NOT be summed. All time-based sub-metrics MUST use the median (p50) as the headline statistic instead of the mean, to resist outlier skew; percentile bands (p85/p95) MAY be added for distribution visibility.
|
|
25
|
+
|
|
26
|
+
#### 02-per-service-and-team-accounting
|
|
27
|
+
|
|
28
|
+
Every metric and sub-metric MUST be measured and reported per team, never only as an org-wide aggregate. The atomic unit of attribution MUST be the deployable service, not the team directly: a team's figures MUST be a rollup of its owned services, per [agentme-bdr-402](../../bdrs/operations/402-digital-product-roles.md)'s `01-product-team-purpose`/`02-product-team-scope-of-work` (a team owns its product's full lifecycle end-to-end). A change touching multiple services MUST be measured independently against each affected service's own timeline; it MUST NOT be flagged as cross-team or assigned to one majority owner.
|
|
29
|
+
|
|
30
|
+
#### 03-artifact-to-service-mapping
|
|
31
|
+
|
|
32
|
+
Code-path-based artifacts (pull requests, commits, deploy-workflow runs) MUST be attributed to a service using the repository's CODEOWNERS file, with path patterns defined at service-directory granularity. CODEOWNERS MUST NOT be used for GH Issues, since issues have no file path; issue-based sub-metrics MUST instead use an explicit label or component convention (for example a `service:<name>` label) cross-walked to the owning service.
|
|
33
|
+
|
|
34
|
+
#### 04-cross-team-incident-attribution
|
|
35
|
+
|
|
36
|
+
When an incident's root cause is traced to a different team's change, it MUST be counted against the root-cause team's Change Failure Rate ([agentme-edr-405](405-dora-change-failure-rate.md)) and, separately, tagged as a non-tiered visibility signal for the impacted team. It MUST NOT be double-counted into the impacted team's own numerator. Root-cause determination MUST follow [agentme-bdr-405](../../bdrs/operations/405-digital-product-roles-raci.md)'s `04-accountability-network-and-root-cause-tracing` methodology rather than a separate one defined here.
|
|
37
|
+
|
|
38
|
+
#### 05-maturity-level-definition
|
|
39
|
+
|
|
40
|
+
Each team MUST be assigned a named maturity tier (Elite, High, Medium, or Low, per DORA's current published bands, cited and dated rather than hardcoded here) for each of the four metrics. Each team MUST also receive one blended maturity label equal to the floor (worst) of its four per-metric tiers. This floor combination is this framework's own practical proxy, since DORA does not publish an official per-team combination formula; it is grounded in DORA's own finding that top performers tend to do well across all metrics. A Low tier on a metric with a genuine regulatory floor MUST NOT be treated as an excuse to stop improving the rest of that metric or any other metric within the team's control.
|
|
41
|
+
|
|
42
|
+
#### 06-maturity-level-distribution-reporting
|
|
43
|
+
|
|
44
|
+
Maturity level MUST NOT be averaged across teams into one company-wide figure. Company-wide reporting MUST show the percentage of teams at each blended tier as the headline view, with the four per-metric tier distributions available as a supporting drill-down. Distributions MUST NOT be used to rank or compete teams against each other, and individual team results MUST NOT be reported in a way that isolates one team's metrics from the others' — both patterns are documented by DORA as pitfalls that fuel finger-pointing and gaming.
|
|
45
|
+
|
|
46
|
+
#### 07-adoption-order
|
|
47
|
+
|
|
48
|
+
Teams SHOULD adopt the four metrics in this order: Deployment Frequency, then Lead Time for Changes, then Change Failure Rate, then Time to Restore Service. Each step needs one more data source or organizational capability than the last (deploy-workflow history only; then PR data; then a failure-labeling convention; then incident-management timestamps), and DORA's research shows that improving batch size and frequency first cascades into better lead time and stability. Each companion policy defines its own internal sub-metric adoption order using the same ease-plus-impact rationale.
|
|
49
|
+
|
|
50
|
+
#### 08-daily-usage-by-role
|
|
51
|
+
|
|
52
|
+
Roles SHOULD consult these metrics at the cadence below when carrying out their accountabilities as defined in [agentme-bdr-402](../../bdrs/operations/402-digital-product-roles.md), [agentme-bdr-404](../../bdrs/operations/404-team-roles-and-specialists.md), and [agentme-bdr-405](../../bdrs/operations/405-digital-product-roles-raci.md):
|
|
53
|
+
|
|
54
|
+
| Role | Cadence | Usage |
|
|
55
|
+
|---|---|---|
|
|
56
|
+
| Tech Lead / AI Lead | Daily | Primary consumer; accountable for CI/CD pipeline health and incident response; reads review latency ([agentme-edr-404](404-dora-lead-time-for-changes.md) `02`) jointly with Change Failure Rate to catch rubber-stamping |
|
|
57
|
+
| PO | Weekly | Reads Deployment Frequency and Lead Time alongside Change Failure Rate before go-live approvals |
|
|
58
|
+
| EM | Weekly/quarterly | Cross-checks a sustained Low tier against the team's `agentme-bdr-402` `03-workforce-allocation` operations-and-controls capacity |
|
|
59
|
+
| PjM | As-needed | Investigates cross-team blockers when Lead Time or Change Failure Rate issues trace to cross-team dependencies |
|
|
60
|
+
| PM / Leadership | Quarterly | Reads the tier distribution from rule `06`, never a company-wide average |
|
|
61
|
+
|
|
62
|
+
## Considered Options
|
|
63
|
+
|
|
64
|
+
* (REJECTED) **Single composite delivery score** — blend all four metrics into one number
|
|
65
|
+
* Reason: Hides which metric needs attention; contradicts DORA's own guidance against "one metric to rule them all"
|
|
66
|
+
* (REJECTED) **Company-wide averaged maturity score** — one blended number across all teams
|
|
67
|
+
* Reason: Masks teams that need help; contradicts DORA's guidance against disparate comparisons and siloed competition
|
|
68
|
+
* (REJECTED) **Strict-composition redesign for Deployment Frequency and Change Failure Rate** — force all sub-metrics to sum to the parent value
|
|
69
|
+
* Reason: Would require dropping batch size (an orthogonal dimension) and escaped-defect-rate (different attribution lag) to force a clean sum
|
|
70
|
+
|
|
71
|
+
## References
|
|
72
|
+
|
|
73
|
+
- [DORA's software delivery performance metrics](https://dora.dev/guides/dora-metrics-four-keys/)
|
|
74
|
+
- [agentme-edr-403](403-dora-deployment-frequency.md) — Deployment Frequency
|
|
75
|
+
- [agentme-edr-404](404-dora-lead-time-for-changes.md) — Lead Time for Changes
|
|
76
|
+
- [agentme-edr-405](405-dora-change-failure-rate.md) — Change Failure Rate
|
|
77
|
+
- [agentme-edr-406](406-dora-time-to-restore-service.md) — Time to Restore Service
|
|
78
|
+
- [agentme-edr-401](401-service-health-check-endpoint.md) — Service health check endpoint
|
|
79
|
+
- [agentme-edr-302](../platform/302-github-pipelines.md) — GitHub CI/CD pipelines
|
|
80
|
+
- [agentme-bdr-402](../../bdrs/operations/402-digital-product-roles.md) — Digital product roles: org & cross-team
|
|
81
|
+
- [agentme-bdr-404](../../bdrs/operations/404-team-roles-and-specialists.md) — Digital product roles: team & specialists
|
|
82
|
+
- [agentme-bdr-405](../../bdrs/operations/405-digital-product-roles-raci.md) — Digital product roles: RACI
|