@contextrail/assess 0.0.0-stage → 1.0.0-alpha.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -0
- package/README.md +82 -2
- package/assets/brand/icon-dark.png +0 -0
- package/assets/brand/icon-light.png +0 -0
- package/dist/assess.d.ts +109 -0
- package/dist/assess.js +1563 -0
- package/dist/assessment.d.ts +571 -0
- package/dist/assessment.js +404 -0
- package/dist/budget.d.ts +24 -0
- package/dist/budget.js +52 -0
- package/dist/candidate.d.ts +156 -0
- package/dist/candidate.js +361 -0
- package/dist/change-summary.d.ts +33 -0
- package/dist/change-summary.js +26 -0
- package/dist/cli.d.ts +2 -0
- package/dist/cli.js +406 -0
- package/dist/contracts.d.ts +99 -0
- package/dist/contracts.js +62 -0
- package/dist/diagnostics.d.ts +44 -0
- package/dist/diagnostics.js +412 -0
- package/dist/evidence.d.ts +73 -0
- package/dist/evidence.js +297 -0
- package/dist/feedback.d.ts +21 -0
- package/dist/feedback.js +154 -0
- package/dist/git.d.ts +40 -0
- package/dist/git.js +207 -0
- package/dist/github.d.ts +90 -0
- package/dist/github.js +1342 -0
- package/dist/index.d.ts +14 -0
- package/dist/index.js +14 -0
- package/dist/mcp.d.ts +28 -0
- package/dist/mcp.js +173 -0
- package/dist/pack.d.ts +160 -0
- package/dist/pack.js +377 -0
- package/dist/phases.d.ts +65 -0
- package/dist/phases.js +63 -0
- package/dist/planning.d.ts +64 -0
- package/dist/planning.js +135 -0
- package/dist/pr-context.d.ts +316 -0
- package/dist/pr-context.js +198 -0
- package/dist/prepare.d.ts +71 -0
- package/dist/prepare.js +335 -0
- package/dist/presentation.d.ts +310 -0
- package/dist/presentation.js +1076 -0
- package/dist/provider-diagnostics.d.ts +93 -0
- package/dist/provider-diagnostics.js +233 -0
- package/dist/providers.d.ts +45 -0
- package/dist/providers.js +291 -0
- package/dist/report-contract.d.ts +137 -0
- package/dist/report-contract.js +248 -0
- package/dist/results.d.ts +90 -0
- package/dist/results.js +128 -0
- package/dist/resume.d.ts +25 -0
- package/dist/resume.js +116 -0
- package/dist/source.d.ts +167 -0
- package/dist/source.js +675 -0
- package/dist/verification.d.ts +386 -0
- package/dist/verification.js +168 -0
- package/dist/version.d.ts +2 -0
- package/dist/version.js +9 -0
- package/docs/README.md +24 -0
- package/docs/action-reference.md +85 -0
- package/docs/architecture.md +158 -0
- package/docs/configuration.md +83 -0
- package/docs/contributing.md +46 -0
- package/docs/getting-started.md +92 -0
- package/docs/github-actions.md +141 -0
- package/docs/operations.md +211 -0
- package/docs/review-results.md +189 -0
- package/docs/security.md +68 -0
- package/docs/standards.md +62 -0
- package/examples/assess-pull-request.yml +109 -0
- package/package.json +56 -5
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Action inputs, outputs and delivery
|
|
2
|
+
|
|
3
|
+
[Documentation index](README.md) · [Assess overview](../README.md)
|
|
4
|
+
|
|
5
|
+
Look up a specific setting here. For a first integration, use [GitHub Actions setup](github-actions.md).
|
|
6
|
+
|
|
7
|
+
## Inputs and credentials
|
|
8
|
+
|
|
9
|
+
| Input | Contract |
|
|
10
|
+
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
11
|
+
| `package-version` | Required exact published npm SemVer. No tags, ranges, URLs, local builds or unavailable-version fallback. |
|
|
12
|
+
| `pr-number`, `base-branch` | Required PR number; optional expected target branch. |
|
|
13
|
+
| `reviewer` | Required reusable published `skill://` ID; never an execution or repository ID. |
|
|
14
|
+
| `mcp-url`, `mcp-org-id` | Required HTTPS endpoint and organization scope, using CLI connection vocabulary. |
|
|
15
|
+
| `provider`, `model` | Required `openai` or `openrouter` and explicit provider-native model. No fallback. |
|
|
16
|
+
| `enforcement` | `advisory` (default), `required`, or `strict`. See the [outcome table](review-results.md#interpret-judgments-and-exit-codes); execution failures always return 2. |
|
|
17
|
+
| `effort` | Optional CLI effort; empty delegates to provider default. |
|
|
18
|
+
| `max-time-ms` | Required positive workflow model deadline, at most 2147483647 ms. Does not bound total job time. |
|
|
19
|
+
| `feedback` | `send` (default) or `off`; CLI owns report submission and its status. |
|
|
20
|
+
| `repository-attribution` | `true` or `false` (default); explicitly includes repository identity in CLI retrieval/reporting. |
|
|
21
|
+
| `artifact-retention-days` | 1–90, default 7, within repository/organization retention policy. |
|
|
22
|
+
| `post-comment` | `false` by default; `true` creates/updates the bot PR summary. Requires `pull-requests: write`. |
|
|
23
|
+
| `pr-context` | `false` by default; collect bounded PR discussion/history for a matching CLI release supporting `--pr-context`. Sends that context to the selected model. |
|
|
24
|
+
| `github-token` | Defaults to `github.token`; Git/API reads, plus PR comment writes when enabled. |
|
|
25
|
+
| `mcp-auth-token` | Required bearer credential; map from `secrets.CONTEXTRAIL_MCP_AUTH_TOKEN`. |
|
|
26
|
+
| `openai-api-key`, `openrouter-api-key` | Supply the selected provider's credential from its named secret. |
|
|
27
|
+
|
|
28
|
+
Additional optional inputs: `phase-models` (JSON, default `{}`),
|
|
29
|
+
`log-level` (`quiet|info|debug`, default `info`) and `inline-comments` (default `false`,
|
|
30
|
+
requires `post-comment: 'true'`). The model deadline covers the complete model workflow,
|
|
31
|
+
not a separate allowance for each phase or continuation. PR title/body are prepared as intent; `pr-context`
|
|
32
|
+
controls the additional bounded discussions/history. Supply both existing provider secret
|
|
33
|
+
inputs only when explicitly configured phases use both providers.
|
|
34
|
+
|
|
35
|
+
Credentials are explicit Action inputs, never CLI arguments. Installation gets no
|
|
36
|
+
GitHub, MCP or provider credentials. Git/API read credentials reach only comparison
|
|
37
|
+
preparation, the final freshness read and the opt-in comment step; they are not written into Git configuration
|
|
38
|
+
or passed to model-execution commands. Trusted npm GitHub adapter commands receive only their
|
|
39
|
+
required GitHub credential. MCP and the explicitly configured phase providers' keys reach only the review subprocess;
|
|
40
|
+
ambient CLI policy/provider settings are excluded. Prefer secret inputs over caller
|
|
41
|
+
`env`: environment values on the calling step are inherited by every composite step.
|
|
42
|
+
Consumer steps can still access their job's GitHub token; composite steps are not a
|
|
43
|
+
security boundary against malicious trusted Action code.
|
|
44
|
+
|
|
45
|
+
## Outputs and artifacts
|
|
46
|
+
|
|
47
|
+
| Output | Meaning |
|
|
48
|
+
| ------------------- | --------------------------------------------------------------------------------------------------------------- |
|
|
49
|
+
| `assessment-exit` | Actual CLI 0/1/2, empty if not invoked or interrupted. Action/setup/upload failure can fail the job separately. |
|
|
50
|
+
| `run-id` | Unique CLI `runId` from confirmed state; no substituted workflow/check/reviewer identity. |
|
|
51
|
+
| `feedback-status` | `received`, `failed`, `pending`, `off` or `unavailable`. Feedback failure preserves CLI exit. |
|
|
52
|
+
| `comparison-status` | `current_at_finish`, `superseded` or `unknown`, when a comparison was prepared. |
|
|
53
|
+
| `artifact-url` | Evidence artifact URL when upload succeeds. |
|
|
54
|
+
| `comment-status` | `created`, `updated`, `skipped_superseded`, `skipped_newer` or `failed`; empty when disabled. |
|
|
55
|
+
|
|
56
|
+
Available state, reports, phase diagnostics, CLI diagnostics, comparison/setup status and npm lock
|
|
57
|
+
with resolved dependency integrity are preserved after failure. Installed dependencies
|
|
58
|
+
and fetched source objects are not uploaded. A separate post-publication artifact retains
|
|
59
|
+
validated write payloads and delivery outcomes, including unknown/partial writes;
|
|
60
|
+
`publication-artifact-url` exposes its URL. It uses the same retention setting. Review evidence includes cited source and generated assessments;
|
|
61
|
+
restrict artifact access and retention accordingly. Ordinary logs and summaries use
|
|
62
|
+
controlled statuses, identities, counts, bounded finding/correction text and unresolved reasons, not conversations, full source citations or raw upstream errors.
|
|
63
|
+
The summary confirms completion only from matching persisted comparison/reviewer
|
|
64
|
+
state. An ingestion receipt does not endorse a review.
|
|
65
|
+
|
|
66
|
+
The final API read labels a changed comparison (identity, selected target or base/head SHAs)
|
|
67
|
+
**SUPERSEDED; historical result only**. Closing a PR alone does not change its comparison. A failed freshness read means unknown. No result certifies future currency;
|
|
68
|
+
verify both SHAs when using it. A provider call may race a subsequent PR push, and
|
|
69
|
+
feedback can already have been sent for the accurately recorded fixed comparison.
|
|
70
|
+
Cancellation or job timeout can prevent later upload/summary steps; use a finite job
|
|
71
|
+
timeout with headroom beyond the model-session limit (for example, use 20 minutes for
|
|
72
|
+
600000 ms). Evidence preservation is attempted with `always()` after ordinary errors.
|
|
73
|
+
|
|
74
|
+
## Comment delivery and reruns
|
|
75
|
+
|
|
76
|
+
Comment creation/update is attempted after ordinary review failures with `always()`. A
|
|
77
|
+
permission/API failure is visible in the Action summary and fails that step independently of
|
|
78
|
+
`assessment-exit`; there is no automatic comment-write retry. Comments are scoped to the configured
|
|
79
|
+
PR/reviewer and current comparison, with workflow/run-attempt links for traceability. Cancellation
|
|
80
|
+
or job timeout can still prevent summary/comment execution. The caller must serialize publications per repository/PR/reviewer for rerun deduplication.
|
|
81
|
+
GitHub creation has no atomic compare-and-set; concurrent runs can race after absence checks.
|
|
82
|
+
The npm package owns presentation and GitHub adapter behavior; the Action invokes it.
|
|
83
|
+
|
|
84
|
+
After publishing this Action update, adopt its full public commit SHA as well as the intended
|
|
85
|
+
exact npm version. Re-running an old workflow retains its old Action code and inputs.
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
# Assessment lifecycle and contracts
|
|
2
|
+
|
|
3
|
+
[Documentation index](README.md) · [Assess overview](../README.md)
|
|
4
|
+
|
|
5
|
+
For maintainers and agents changing the harness. Setup and usage are covered in [getting started](getting-started.md).
|
|
6
|
+
|
|
7
|
+
## Lifecycle
|
|
8
|
+
|
|
9
|
+
1. **Prepare:** resolve fixed commits, retrieve the published reviewer and governing contexts, and write an evidence pack.
|
|
10
|
+
2. **Observe:** one coherent assessment reads pinned source and records independently answerable observations, possible standards concerns, assumptions and decisive unknowns for every selected context.
|
|
11
|
+
3. **Validate concerns:** independently examine each nominated concern against its original governing publication and cited source. Confirm a violation only when both applicability and contradiction are established. Keep missing evidence and ambiguous guidance distinct from violations.
|
|
12
|
+
4. **Report:** code derives outcomes, consolidates equivalent same-duty corrections, applies policy, persists artifacts and delivers configured feedback. An optional final pass checks the reader change summary separately.
|
|
13
|
+
|
|
14
|
+
The selected reviewer skill and original pinned contexts retain authority. Reviewer instructions may
|
|
15
|
+
narrow the investigation, but this is standards adherence auditing, not unrestricted bug hunting.
|
|
16
|
+
Each additional observation cites the original passage and pinned source; code supplies identities
|
|
17
|
+
and quotations from selected evidence segments. Accepted context groups are immutable and remain
|
|
18
|
+
available when a sibling group needs repair. Checkpoints retain observations and concern decisions.
|
|
19
|
+
An older judgment-based checkpoint cannot be silently interpreted as an observation-based one.
|
|
20
|
+
|
|
21
|
+
A no-concern observation is **observed**, not independently validated compliance. Structural coverage
|
|
22
|
+
means each selected context has observations; it does not prove every obligation was discovered.
|
|
23
|
+
Verification counters describe nominated concerns, separately from context coverage. Only confirmed
|
|
24
|
+
artifact concerns become findings. Conditional applicability remains a question. Actual execution of external operations is outside the audit; source inspection assesses the
|
|
25
|
+
mechanisms that implement applicable operational requirements.
|
|
26
|
+
|
|
27
|
+
The assessment allows 60 provider turns and 50 source-tool calls per response. Each focused validation
|
|
28
|
+
and optional summary session allows 12 provider turns. Excess reads receive
|
|
29
|
+
retry feedback rather than discarding accepted work. Each source tool retains its own path/range/size
|
|
30
|
+
bounds. The configured whole-review deadline is shared, across assessment and validation.
|
|
31
|
+
Global caller turn/token/time limits still apply. Validators receive a focused packet for their
|
|
32
|
+
concern and can read supporting code for specific missing facts; they do not receive every other
|
|
33
|
+
standard's assessment. The default transport, model settings, request retries and diagnostics are
|
|
34
|
+
shared with the production CLI and Action.
|
|
35
|
+
|
|
36
|
+
## Choose the reviewer and comparison
|
|
37
|
+
|
|
38
|
+
`--reviewer` takes a published `skill://...` ID, not a display name. Its instructions
|
|
39
|
+
provide the review lens; explicit context and domain selections plus required
|
|
40
|
+
context prerequisites determine the expected scope. The CLI retrieves exact
|
|
41
|
+
publication versions and reports missing, inaccessible or unsupported prerequisites.
|
|
42
|
+
Changing publications affects future runs, not the versions already recorded.
|
|
43
|
+
|
|
44
|
+
The reviewer contract consists of skill identity/metadata, review instructions,
|
|
45
|
+
`content.reviewer.selections` (explicit context IDs and domain selections with
|
|
46
|
+
`includeDescendants`), and optional supporting resources. Instruction-only reviewers
|
|
47
|
+
use the same contract with no assessment selections. Empty scope stays instruction-only.
|
|
48
|
+
|
|
49
|
+
`--repo DIR` defaults to the current directory. `--base REF` and `--head REF` are
|
|
50
|
+
required and resolve once to fixed commit and tree IDs. The comparison is base
|
|
51
|
+
commit → head commit; it does not automatically choose a merge base. Use exact
|
|
52
|
+
commit SHAs for automation. Fetch both histories and their blobs before running;
|
|
53
|
+
partial clones can leave supporting source unavailable. Later branch movement,
|
|
54
|
+
working-tree edits and index changes do not alter the review. The source tools read
|
|
55
|
+
both snapshots and unchanged supporting files, with bounded responses and
|
|
56
|
+
continuations. They do not execute repository scripts, tests, hooks or model shell
|
|
57
|
+
commands. Binary files, submodule contents and conventional credential files have
|
|
58
|
+
explicit source-access limits; this is a text/source review, not runtime testing.
|
|
59
|
+
|
|
60
|
+
To check retrieval and scope without calling a model, replace `run` with
|
|
61
|
+
`prepare` and omit provider/model arguments. Inspect `state.json` and its gaps.
|
|
62
|
+
Preparation exit 0 means the scope was prepared; entries remain **not evaluated**.
|
|
63
|
+
Preparation exits 2 for invalid configuration or incomplete retrieval and never 1.
|
|
64
|
+
|
|
65
|
+
## Evidence and final corrections
|
|
66
|
+
|
|
67
|
+
The assessment boundary is the proposed artifact, not general operational assurance. Each applicable
|
|
68
|
+
check relates a changed behavior to a governing obligation. Missing evidence makes that artifact
|
|
69
|
+
judgment unresolved only when it is decisive for the claim under review. Source-visible defects and
|
|
70
|
+
contradictions remain findings without requiring runtime reproduction. The current PR author's process, CI status and execution of external operations are not assessment subjects.
|
|
71
|
+
Independent validation separates irrelevant operational questions from relevant external evidence
|
|
72
|
+
limitations. The latter become nonblocking follow-ups with their governing version and boundary evidence,
|
|
73
|
+
not compliance scores. An assessor's scope label alone cannot clear a concern. Verification challenges applicability and proportionality
|
|
74
|
+
as well as supporting evidence, including whether a proposed follow-up actually conceals a code defect.
|
|
75
|
+
|
|
76
|
+
For configurable code, judge the source-controlled selector, validation, defaults and authority over
|
|
77
|
+
inputs. An unknown deployed value alone is outside the audit. Unavailable external dependency internals
|
|
78
|
+
are a recorded external limitation. Missing or unread implementation within the pinned repository remains
|
|
79
|
+
an unresolved artifact question; source-visible violations still require correction. Prior reviewer requests
|
|
80
|
+
are reassessed against this boundary; they do not become additional governing requirements.
|
|
81
|
+
|
|
82
|
+
Assessment produces observations and nominates concerns against the original pinned standards.
|
|
83
|
+
A focused validation session checks each nominated concern: its governing obligation, applicability,
|
|
84
|
+
source evidence and proposed correction. A decisive unknown must nominate a concern; it cannot
|
|
85
|
+
bypass validation by appearing only in an unknowns list. Final check records reflect the validator
|
|
86
|
+
decision while original proposals remain available for audit. Up to two concerns run concurrently. No-concern observations
|
|
87
|
+
remain observed; they are not independently certified as compliant. Missing decisive evidence remains
|
|
88
|
+
unresolved, without suppressing a separately supported violation in the same standard.
|
|
89
|
+
|
|
90
|
+
The assessment receives the evidence pack and can read permitted files in either pinned snapshot.
|
|
91
|
+
Current CI check status and the reviewer runtime identity are omitted from assessment and concern
|
|
92
|
+
validation inputs; the original PR context remains in the run audit. PR intent, discussion and
|
|
93
|
+
history remain available as context, not additional governing authority.
|
|
94
|
+
Each verifier receives the governing publication, reviewer constraints, nominated observation and cited
|
|
95
|
+
source fragments. It can read additional supporting code. Repository text cannot waive governing requirements.
|
|
96
|
+
Code extracts exact citations, generates identities and derives policy outcomes. Decisions and observations
|
|
97
|
+
remain in sealed local artifacts; metadata feedback identifies this method as `candidate-adherence-v3`.
|
|
98
|
+
|
|
99
|
+
If assessment supplies a change summary, a separate bounded summary session checks its evidence.
|
|
100
|
+
An unavailable or unsupported summary is omitted without invalidating completed standards results.
|
|
101
|
+
The summary explains changed behavior and review focus; it creates no additional requirements.
|
|
102
|
+
|
|
103
|
+
Model citations select `{ref, startSegment, endSegment}` from numbered evidence segments,
|
|
104
|
+
using inclusive indices relative to that evidence record. Code extracts the exact quotation,
|
|
105
|
+
character offsets and source location; models do not supply quotation text. An explicit range
|
|
106
|
+
also identifies an occurrence when text repeats. Canonical reports retain quotations and provenance.
|
|
107
|
+
Within each model session, short `context_N` and `evidence_N` handles identify the assigned
|
|
108
|
+
contexts and available evidence. Exact lookup restores canonical identities before validation;
|
|
109
|
+
unknown handles remain invalid. Audit artifacts retain canonical identities. Assessment result
|
|
110
|
+
citations are derived from the checks' governing passages and source references, so the model
|
|
111
|
+
does not have to repeat the same evidence in a separate result field.
|
|
112
|
+
Model inputs carry citation references and source/publication locations rather than repeated report
|
|
113
|
+
quotations; original evidence is supplied separately and remains available through the read tools. Code supplies revision
|
|
114
|
+
metadata from the current context state. The assessor supplies observations and nominated concerns. A focused verifier supplies the concern decision, applicability, duty strength and any accepted correction. Code derives aggregate outcomes without inventing confidence or treating absent concerns as compliance.
|
|
115
|
+
Canonical validators still check evidence, authority and consistency with findings.
|
|
116
|
+
Code also supplies actual package/Node/model identities and fixed comparison SHAs. When GitHub
|
|
117
|
+
context is collected, exact-head check/status receipts distinguish observed results from unavailable
|
|
118
|
+
API access. Neither runtime identity nor passing check metadata proves package safety, deployment
|
|
119
|
+
readiness or compliance with every governed requirement. Repository authority remains the pinned
|
|
120
|
+
repository's declared authority, not mutable PR discussion.
|
|
121
|
+
Selection failures identify the precise field and safe evidence identity in correction feedback
|
|
122
|
+
and diagnostics. Source bodies stay out of routine diagnostics. Selection bounds, exact text,
|
|
123
|
+
governing authority and independent verification remain enforced without extra correction rounds.
|
|
124
|
+
|
|
125
|
+
## Recovery and validation
|
|
126
|
+
|
|
127
|
+
The assessment can submit multiple ready context observation groups in one tool call. The advertised
|
|
128
|
+
schema describes the full contract; server-side validation handles each group independently so an
|
|
129
|
+
invalid group does not discard valid siblings. Accepted groups are immutable; identical replay is
|
|
130
|
+
acknowledged, while changed replay is rejected. Only remaining groups need correction. Retained
|
|
131
|
+
partial progress is evidence of an incomplete phase, not a verified assessment.
|
|
132
|
+
|
|
133
|
+
Each nominated concern has an independent focused validation record, with two workers sharing the
|
|
134
|
+
overall budget. A failure does not erase completed sibling decisions. Completion requires a decision
|
|
135
|
+
for every nominated concern. Resume verifies immutable source, publication and phase identities and
|
|
136
|
+
reuses accepted concern decisions; it never silently upgrades old judgment-based checkpoints.
|
|
137
|
+
|
|
138
|
+
Invalid submissions share a correction round; three invalid rounds without newly accepted work
|
|
139
|
+
stop the session. Accepted progress resets the allowance. Reads and duplicate submissions do not.
|
|
140
|
+
Every attempt still consumes the configured execution budget. Use one exact committed file for
|
|
141
|
+
scoped literal search, or omit the path for bounded whole-tree navigation.
|
|
142
|
+
|
|
143
|
+
Provider recovery allows at most two additional attempts per failed request, with 250ms then 500ms
|
|
144
|
+
backoff, for transport failures marked retryable by the SDK, HTTP 408/429/5xx and response
|
|
145
|
+
parsing/validation failures. It repeats only the failed model request with the existing conversation;
|
|
146
|
+
completed tools and accepted assessments are not replayed. A successful response resets the retry
|
|
147
|
+
allowance for the next request. Each request consumes the original
|
|
148
|
+
call/time/token budget, and cancellation interrupts the backoff. Authentication, credit-limit and
|
|
149
|
+
configuration failures stop immediately. Recovery is separate from submission correction and
|
|
150
|
+
unfinished-assessment continuation. `provider_retry`, `provider_recovered` and
|
|
151
|
+
`provider_retry_exhausted` events appear at info/debug verbosity. See [diagnostics](operations.md#health-logging-and-cost).
|
|
152
|
+
The observed token cutoff can overshoot by one call and is not a dollar ceiling.
|
|
153
|
+
Execution completion, assessment coverage, verification coverage and policy acceptance
|
|
154
|
+
are separate fields. Schema validity and exact citations do not prove a judgment correct.
|
|
155
|
+
|
|
156
|
+
Each phase persists input/output identities and evidence references. [Artifacts and resume](operations.md#inspect-evidence-and-resume) describes how these records support recovery.
|
|
157
|
+
|
|
158
|
+
The package exports `prepareAssessment`, `runAssessment`, `FixedSnapshotSource`, `McpPublishedReader`, provider constructors and policy helpers. Shipped type declarations describe their arguments; these APIs use the CLI's fixed-publication and source boundaries.
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# CLI configuration
|
|
2
|
+
|
|
3
|
+
[Documentation index](README.md) · [Assess overview](../README.md)
|
|
4
|
+
|
|
5
|
+
Use this reference for the standalone CLI. CLI flags override their environment aliases.
|
|
6
|
+
For GitHub Actions, start with [the workflow guide](github-actions.md); its inputs are separate from local environment aliases.
|
|
7
|
+
|
|
8
|
+
| Environment variable | Purpose |
|
|
9
|
+
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
10
|
+
| `CONTEXTRAIL_MCP_URL` | Authorized HTTP(S) MCP endpoint, including its configured path. Use HTTPS for remote credentials. URL credentials, queries and fragments are rejected. |
|
|
11
|
+
| `CONTEXTRAIL_MCP_AUTH_TOKEN` | Bearer token authorized to read the reviewer and selected contexts, and to submit feedback when enabled. |
|
|
12
|
+
| `CONTEXTRAIL_MCP_ORG_ID` | Organization scope, sent as `x-v2-org-id`. |
|
|
13
|
+
| `CONTEXTRAIL_MCP_TIMEOUT_MS` | Per connection/read timeout; default 30000 ms. `--mcp-timeout-ms N` overrides it. |
|
|
14
|
+
| `OPENAI_API_KEY` | Credential for `--provider openai`. |
|
|
15
|
+
| `OPENROUTER_API_KEY` | Credential for `--provider openrouter`. |
|
|
16
|
+
|
|
17
|
+
The MCP endpoint must support protocol **2026-07-28**. The process inherits exported
|
|
18
|
+
variables from its shell or the workflow step's `env`. Provider/model selection is
|
|
19
|
+
explicit, with no fallback. `--provider` and `--model` override
|
|
20
|
+
`CONTEXTRAIL_ASSESS_PROVIDER` and `CONTEXTRAIL_ASSESS_MODEL`.
|
|
21
|
+
|
|
22
|
+
OpenAI uses its Responses API. OpenRouter uses the OpenAI-compatible chat adapter at its own
|
|
23
|
+
endpoint. Submission tools request strict schema output; local validation remains mandatory because
|
|
24
|
+
provider enforcement can vary. Source tools retain their optional navigation arguments. Use a model ID
|
|
25
|
+
available to your provider account and capable of tool calling. For OpenRouter,
|
|
26
|
+
export `OPENROUTER_API_KEY` securely and replace the provider/model portion of the
|
|
27
|
+
quickstart with:
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
--provider openrouter --model YOUR_OPENROUTER_MODEL_ID
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
`--effort none|minimal|low|medium|high|xhigh` requests provider-specific reasoning
|
|
34
|
+
effort only when that model supports it. A valid CLI setting does not ensure that
|
|
35
|
+
a provider accepts it. Source excerpts and published guidance are sent to your
|
|
36
|
+
selected model provider; use an account and data-handling arrangement appropriate
|
|
37
|
+
for that material.
|
|
38
|
+
|
|
39
|
+
## Models by phase and logging
|
|
40
|
+
|
|
41
|
+
`--provider`, `--model` and `--effort` supply the explicit shared default. Optional
|
|
42
|
+
`--phase-models` JSON overrides `assess` or `verify`; an override includes provider
|
|
43
|
+
and model together, plus optional effort. Omitted effort uses that provider's default.
|
|
44
|
+
Unknown keys and unsupported configuration fail without model substitution.
|
|
45
|
+
|
|
46
|
+
```sh
|
|
47
|
+
--phase-models '{"verify":{"provider":"openrouter","model":"YOUR_VERIFICATION_MODEL"}}' \
|
|
48
|
+
--log-level info
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
The selected reviewer uses one coherent assessment session, followed by independent validation of each nominated concern (up to two concurrently). A separate optional session checks the change summary.
|
|
52
|
+
`--log-level quiet|info|debug` defaults to info; stdout retains machine JSON while safe progress
|
|
53
|
+
goes to stderr. All levels retain required diagnostics. Execution limits cover the invocation,
|
|
54
|
+
including bounded continuation of unfinished submissions. No stronger-model fallback is implicit.
|
|
55
|
+
|
|
56
|
+
## Limits and environment aliases
|
|
57
|
+
|
|
58
|
+
`--purpose TEXT` records your stated objective or stage for the reviewer.
|
|
59
|
+
Every option in this section, plus provider, model, effort, enforcement, feedback and repository-attribution, has the environment
|
|
60
|
+
form `CONTEXTRAIL_ASSESS_<UPPERCASE_FLAG_WITH_UNDERSCORES>`; CLI values override it.
|
|
61
|
+
For example, `--max-turns` maps to `CONTEXTRAIL_ASSESS_MAX_TURNS`.
|
|
62
|
+
`CONTEXTRAIL_ASSESS_ENFORCEMENT` accepts `advisory` (default), `required`, or `strict`.
|
|
63
|
+
Repository-attribution environment values must be `true` or `false`. Git arguments
|
|
64
|
+
and reviewer ID have no environment aliases. See `--help` for the current option list.
|
|
65
|
+
|
|
66
|
+
| Flag | Default / unit |
|
|
67
|
+
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
68
|
+
| `--max-turns N` | Unset; model calls/steps. |
|
|
69
|
+
| `--max-output-tokens N` | Unset; per-call output allowance, including reasoning tokens where applicable. |
|
|
70
|
+
| `--max-total-tokens N` | Unset; observed provider total-token cutoff between calls. Can overshoot by one call; missing usage prevents honest enforcement and stops the review. |
|
|
71
|
+
| `--max-time-ms N` | Unset; model-session deadline in ms, excluding preparation. |
|
|
72
|
+
| `--source-page-entries N` | 100 entries per list/change response. |
|
|
73
|
+
| `--source-search-matches N` / `--source-search-files N` | 100 each, per search page. |
|
|
74
|
+
| `--source-read-lines N` / `--source-read-bytes N` | 200 lines / 65536 bytes per source response; bytes >=4. |
|
|
75
|
+
| `--source-timeout-ms N` | 30000 ms per source operation. |
|
|
76
|
+
|
|
77
|
+
The per-call output allowance includes reasoning and structured tool arguments; a small cap can
|
|
78
|
+
truncate a submission even when source reads fit. Numeric limits are positive safe integers;
|
|
79
|
+
timers use 1..2147483647 ms. Unset execution limits still leave provider constraints in place.
|
|
80
|
+
The observed token limit can overshoot by one call and is not a dollar ceiling.
|
|
81
|
+
Workflow timeouts are separate; leave time for finalization and upload.
|
|
82
|
+
|
|
83
|
+
See [review policy](review-results.md#interpret-judgments-and-exit-codes) for the three enforcement modes, and [recovery](architecture.md#recovery-and-validation) for retry semantics.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Contributing and releases
|
|
2
|
+
|
|
3
|
+
[Documentation index](README.md) · [Assess overview](../README.md)
|
|
4
|
+
|
|
5
|
+
ContextRail maintains Assess. Development and release selection happen upstream; the public repository is a one-way distribution mirror.
|
|
6
|
+
|
|
7
|
+
## Where to make changes
|
|
8
|
+
|
|
9
|
+
Propose changes for upstream incorporation. Direct edits in the public mirror can block the next
|
|
10
|
+
export. The package, CLI, Action and documentation are maintained together.
|
|
11
|
+
|
|
12
|
+
The package root `README.md` is the canonical overview and is exported as the public README.
|
|
13
|
+
The shared `docs/` directory is the detailed documentation source. Keep task links and
|
|
14
|
+
examples valid in the exported repository and installed package. Public docs must not depend
|
|
15
|
+
on private repository paths, services, working notes or uncommitted plans.
|
|
16
|
+
|
|
17
|
+
## Validate a public checkout
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
pnpm install --frozen-lockfile
|
|
21
|
+
pnpm run typecheck
|
|
22
|
+
pnpm run lint
|
|
23
|
+
pnpm test
|
|
24
|
+
pnpm run build
|
|
25
|
+
npm pack --dry-run
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
In the upstream monorepo, follow its `AGENTS.md` and run `pnpm -w check`.
|
|
29
|
+
That gate includes export checks. Package-local builds remain available for normal development.
|
|
30
|
+
A built tarball or passing fixture does not prove that a version was published or that a live
|
|
31
|
+
provider integration works.
|
|
32
|
+
|
|
33
|
+
## Export and publication
|
|
34
|
+
|
|
35
|
+
Upstream semantic-release selects the public version. The development manifest's version is
|
|
36
|
+
not a registry release. Export uses an explicit file inventory; new public docs must be listed
|
|
37
|
+
there. The npm package's `files` list also includes docs so installed README links remain usable.
|
|
38
|
+
`release.json` binds the selected version and channel to the upstream commit; the public Action
|
|
39
|
+
commit is a separate immutable identity.
|
|
40
|
+
|
|
41
|
+
After successful publication, consumers update both the full Action SHA and exact npm version.
|
|
42
|
+
Verify registry availability before dispatching a review. The README badges track npm's
|
|
43
|
+
`latest` and `alpha` channels automatically; they do not choose either consumer pin.
|
|
44
|
+
|
|
45
|
+
See [Action qualification](github-actions.md#qualify-and-operate-a-consumer) for the live
|
|
46
|
+
integration check and [architecture](architecture.md) before changing the harness.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Run your first assessment
|
|
2
|
+
|
|
3
|
+
[Documentation index](README.md) · [Assess overview](../README.md)
|
|
4
|
+
|
|
5
|
+
This guide takes a Git comparison to a local report. Use [GitHub Actions](github-actions.md) if you
|
|
6
|
+
want PR comments and CI integration.
|
|
7
|
+
|
|
8
|
+
## Before you start
|
|
9
|
+
|
|
10
|
+
Have these ready:
|
|
11
|
+
|
|
12
|
+
- Node >=22.18.0 and Git >=2.45.
|
|
13
|
+
- A Git repository with both commits and their supporting objects available locally.
|
|
14
|
+
- Your organization's ContextRail MCP endpoint, organization ID and authorized token.
|
|
15
|
+
- A published reviewer skill ID such as `skill://engineering/your-reviewer` that selects published contexts.
|
|
16
|
+
- An OpenAI or OpenRouter API key and a tool-capable model available to that account.
|
|
17
|
+
|
|
18
|
+
If you do not yet have the ContextRail inputs, follow [standards setup](standards.md).
|
|
19
|
+
|
|
20
|
+
## Install an exact published version
|
|
21
|
+
|
|
22
|
+
Inspect available releases, choose a version, then replace `EXACT_VERSION` below:
|
|
23
|
+
|
|
24
|
+
```sh
|
|
25
|
+
npm view @contextrail/assess dist-tags
|
|
26
|
+
npm install --ignore-scripts @contextrail/assess@EXACT_VERSION
|
|
27
|
+
./node_modules/.bin/contextrail-assess --help
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Pin the version in your dependency lockfile. The badges show npm channels; they do not select a
|
|
31
|
+
version for your project. A source checkout's manifest can contain a development version rather
|
|
32
|
+
than a registry release. Public exports include `release.json` with the selected release and source commit.
|
|
33
|
+
|
|
34
|
+
## Check your guidance connection
|
|
35
|
+
|
|
36
|
+
Supply `CONTEXTRAIL_MCP_AUTH_TOKEN` securely through your secret manager or process environment.
|
|
37
|
+
The CLI does not load `.env` files. Set the nonsecret connection values and replace the remaining placeholders:
|
|
38
|
+
|
|
39
|
+
```sh
|
|
40
|
+
export CONTEXTRAIL_MCP_URL='https://YOUR_CONTEXTRAIL_HOST/mcp'
|
|
41
|
+
export CONTEXTRAIL_MCP_ORG_ID='YOUR_ORGANIZATION_ID'
|
|
42
|
+
|
|
43
|
+
./node_modules/.bin/contextrail-assess prepare \
|
|
44
|
+
--reviewer skill://engineering/your-reviewer \
|
|
45
|
+
--repo /path/to/repository \
|
|
46
|
+
--base BASE_COMMIT_SHA --head HEAD_COMMIT_SHA
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Preparation retrieves guidance and captures the comparison without a model call. Open the returned
|
|
50
|
+
run directory's `preparation.md`: confirm the commits, selected standards and any missing material.
|
|
51
|
+
A prepared scope is ready to assess; it is not a passing assessment.
|
|
52
|
+
|
|
53
|
+
The comparison is exactly base → head. The local CLI does not choose a merge base or include
|
|
54
|
+
uncommitted edits. For a PR comparison, use its merge base and head, or use the GitHub Action,
|
|
55
|
+
which prepares that comparison for you.
|
|
56
|
+
|
|
57
|
+
## Run the assessment
|
|
58
|
+
|
|
59
|
+
Supply `OPENAI_API_KEY` securely and choose a provider-native model ID:
|
|
60
|
+
|
|
61
|
+
```sh
|
|
62
|
+
./node_modules/.bin/contextrail-assess run \
|
|
63
|
+
--reviewer skill://engineering/your-reviewer \
|
|
64
|
+
--repo /path/to/repository \
|
|
65
|
+
--base BASE_COMMIT_SHA --head HEAD_COMMIT_SHA \
|
|
66
|
+
--provider openai --model YOUR_OPENAI_MODEL_ID \
|
|
67
|
+
--enforcement advisory --max-time-ms 600000
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
For OpenRouter, supply `OPENROUTER_API_KEY` and use
|
|
71
|
+
`--provider openrouter --model YOUR_OPENROUTER_MODEL_ID`. One model serves assessment and
|
|
72
|
+
validation by default. [Configuration](configuration.md) covers optional effort and phase overrides.
|
|
73
|
+
|
|
74
|
+
The deadline bounds model execution, not preparation or total process time. It is not a dollar cap.
|
|
75
|
+
Standard `info` logging shows progress on stderr; stdout remains machine-readable JSON.
|
|
76
|
+
|
|
77
|
+
## Read the result
|
|
78
|
+
|
|
79
|
+
The final JSON identifies the run directory and decision. Open
|
|
80
|
+
`.contextrail/assess/<run-id>/report.md` in the reviewed repository.
|
|
81
|
+
|
|
82
|
+
- Act on supported findings and their suggested corrections.
|
|
83
|
+
- Inspect unresolved questions for a specific missing implementation fact or ambiguous standard.
|
|
84
|
+
- Check completion and execution status before relying on the result.
|
|
85
|
+
- Keep the bundle if you need the exact work versions, guidance versions and cited evidence later.
|
|
86
|
+
|
|
87
|
+
Advisory exit 0 permits continuation; it does not mean every standard was satisfied. See
|
|
88
|
+
[results and next actions](review-results.md) for the outcome and exit-code table.
|
|
89
|
+
|
|
90
|
+
By default, structured assessment metadata is reported to ContextRail after local finalization.
|
|
91
|
+
Use `--feedback off` to disable that submission. Either way, source and guidance needed for the
|
|
92
|
+
assessment go to your model provider. Read [security and data handling](security.md) before sharing artifacts.
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
# Assess pull requests with GitHub Actions
|
|
2
|
+
|
|
3
|
+
[Documentation index](README.md) · [Assess overview](../README.md)
|
|
4
|
+
|
|
5
|
+
Start with a maintainer-triggered advisory assessment. It produces a PR comment and evidence bundle
|
|
6
|
+
without making standards findings a required merge gate. Execution failures still fail the job.
|
|
7
|
+
|
|
8
|
+
## Prepare the repository settings
|
|
9
|
+
|
|
10
|
+
Complete [standards setup](standards.md), then add these in **Settings → Secrets and variables → Actions**:
|
|
11
|
+
|
|
12
|
+
| Kind | Name | Value |
|
|
13
|
+
| -------- | ---------------------------- | ------------------------------------------------------------- |
|
|
14
|
+
| Variable | `CONTEXTRAIL_MCP_URL` | Your authorized HTTPS MCP endpoint |
|
|
15
|
+
| Variable | `CONTEXTRAIL_MCP_ORG_ID` | Your organization ID |
|
|
16
|
+
| Variable | `ASSESS_VERSION` | An exact published `@contextrail/assess` version |
|
|
17
|
+
| Variable | `ASSESS_MODEL` | A tool-capable OpenAI model ID available to your account |
|
|
18
|
+
| Secret | `CONTEXTRAIL_MCP_AUTH_TOKEN` | Token authorized for the selected skill/contexts and feedback |
|
|
19
|
+
| Secret | `OPENAI_API_KEY` | Your provider key |
|
|
20
|
+
|
|
21
|
+
Choose a version with `npm view @contextrail/assess dist-tags`. Use the resulting exact version,
|
|
22
|
+
not `alpha`, `latest` or a range. The public Action revision and installed npm version are separate
|
|
23
|
+
identities: review the Action source and copy its full 40-character commit SHA from
|
|
24
|
+
[the public repository](https://github.com/contextrail/assess). Neither identity automatically qualifies the other.
|
|
25
|
+
|
|
26
|
+
## Add the workflow
|
|
27
|
+
|
|
28
|
+
Save this as `.github/workflows/contextrail-assess.yml`. Replace `ACTION_COMMIT_SHA` and the
|
|
29
|
+
reviewer skill ID before committing it. No contribution checkout is needed.
|
|
30
|
+
|
|
31
|
+
```yaml
|
|
32
|
+
name: ContextRail Assess
|
|
33
|
+
on:
|
|
34
|
+
workflow_dispatch:
|
|
35
|
+
inputs:
|
|
36
|
+
pr-number:
|
|
37
|
+
description: Pull request number to assess
|
|
38
|
+
required: true
|
|
39
|
+
type: string
|
|
40
|
+
permissions: {}
|
|
41
|
+
concurrency:
|
|
42
|
+
group: contextrail-assess-${{ github.repository }}-${{ inputs.pr-number }}
|
|
43
|
+
cancel-in-progress: false
|
|
44
|
+
jobs:
|
|
45
|
+
assess:
|
|
46
|
+
runs-on: ubuntu-latest
|
|
47
|
+
timeout-minutes: 20
|
|
48
|
+
permissions:
|
|
49
|
+
contents: read
|
|
50
|
+
pull-requests: write
|
|
51
|
+
steps:
|
|
52
|
+
- name: Assess proposed changes
|
|
53
|
+
uses: contextrail/assess/action@ACTION_COMMIT_SHA
|
|
54
|
+
with:
|
|
55
|
+
package-version: ${{ vars.ASSESS_VERSION }}
|
|
56
|
+
pr-number: ${{ inputs.pr-number }}
|
|
57
|
+
reviewer: skill://engineering/your-reviewer
|
|
58
|
+
mcp-url: ${{ vars.CONTEXTRAIL_MCP_URL }}
|
|
59
|
+
mcp-org-id: ${{ vars.CONTEXTRAIL_MCP_ORG_ID }}
|
|
60
|
+
mcp-auth-token: ${{ secrets.CONTEXTRAIL_MCP_AUTH_TOKEN }}
|
|
61
|
+
provider: openai
|
|
62
|
+
model: ${{ vars.ASSESS_MODEL }}
|
|
63
|
+
openai-api-key: ${{ secrets.OPENAI_API_KEY }}
|
|
64
|
+
enforcement: advisory
|
|
65
|
+
max-time-ms: '600000'
|
|
66
|
+
post-comment: 'true'
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The workflow must be available on the repository's default branch for the manual Run workflow
|
|
70
|
+
control. Run a trusted workflow revision and enter the PR number. With the GitHub CLI:
|
|
71
|
+
|
|
72
|
+
```sh
|
|
73
|
+
gh workflow run contextrail-assess.yml --repo OWNER/REPOSITORY -f pr-number=123
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
For OpenRouter, change `provider` to `openrouter`, use `openrouter-api-key` with
|
|
77
|
+
`secrets.OPENROUTER_API_KEY`, and set `ASSESS_MODEL` to its provider-native model ID.
|
|
78
|
+
Do not include an unused provider secret.
|
|
79
|
+
|
|
80
|
+
After the first run, inspect the PR comment, Action summary and artifact bundle. Confirm the expected
|
|
81
|
+
commits and guidance versions. ContextRail feedback status is separate from execution: a receipt
|
|
82
|
+
confirms ingestion only. [Results and next actions](review-results.md) explains what to do with findings.
|
|
83
|
+
|
|
84
|
+
## Tune only what you need
|
|
85
|
+
|
|
86
|
+
- **Inline feedback:** add `inline-comments: 'true'` to publish eligible verified findings and code suggestions.
|
|
87
|
+
- **More progress detail:** add `log-level: debug`; routine logs still exclude full conversations and raw provider errors.
|
|
88
|
+
- **PR discussion/history:** add `pr-context: 'true'`. Read the [data implications](security.md) first.
|
|
89
|
+
- **Merge policy:** set `enforcement: required` for supported binding violations, or `strict` to also block unresolved evidence. Configure branch protection to require the job if desired. Do not use `continue-on-error` to create advisory behavior; it would hide execution failures.
|
|
90
|
+
- **Retention:** set `artifact-retention-days` to a permitted value from 1–90. The default is seven days.
|
|
91
|
+
- **Models:** one model serves both roles by default. Add `effort` only if the model supports it; [phase overrides](configuration.md#models-by-phase-and-logging) let you select a different verifier later.
|
|
92
|
+
|
|
93
|
+
Repository variable names belong to your caller workflow. In this example `ASSESS_MODEL` is passed
|
|
94
|
+
explicitly to the Action; setting it alone in a local shell does not configure the CLI. The maintained
|
|
95
|
+
monorepo caller additionally supports `ASSESS_EFFORT`, `ASSESS_VERIFY_MODEL`, `ASSESS_VERIFY_EFFORT`
|
|
96
|
+
and `ASSESS_ENFORCEMENT`. Standalone CLI aliases use `CONTEXTRAIL_ASSESS_*`.
|
|
97
|
+
|
|
98
|
+
## Triggers, permissions and trust
|
|
99
|
+
|
|
100
|
+
You decide which events and people may spend provider budget or access credentials. Manual dispatch
|
|
101
|
+
is a starting policy, not permission for untrusted PR authors to run credentialed code. If you add
|
|
102
|
+
labels, automatic PR events or schedules, define fork/bot/branch admission in the caller workflow.
|
|
103
|
+
The Action does not authorize a run from the PR author's association or account type.
|
|
104
|
+
|
|
105
|
+
The Action prepares a fixed PR comparison using GitHub and Git objects. PR events bind to event
|
|
106
|
+
SHAs; manual and scheduled invocations resolve current PR SHAs. The comparison uses the unique Git
|
|
107
|
+
merge base, excluding changes made only on the target branch. `comparison.json` records the analysis
|
|
108
|
+
base and observed target tip separately. Missing/ambiguous ancestry or stale event identities stop
|
|
109
|
+
preparation. Optional `base-branch` checks an expected target branch; it is not needed for the example.
|
|
110
|
+
|
|
111
|
+
Fork heads are fetched through the base repository's PR ref. The Action does not check out contribution
|
|
112
|
+
files or execute their scripts/hooks, and rechecks comparison identity before introducing provider
|
|
113
|
+
credentials. A final freshness check labels changed comparisons as superseded before publication.
|
|
114
|
+
|
|
115
|
+
A composite Action shares its job and runner. **Do not execute untrusted contribution code anywhere
|
|
116
|
+
in this credentialed job**, restore untrusted caches or run untrusted services alongside it. Only a
|
|
117
|
+
trusted Action revision and qualified npm artifact may receive credentials. Disabled npm lifecycle
|
|
118
|
+
scripts do not make runtime package code trusted. The Action requires Linux, Git >=2.45, access to
|
|
119
|
+
GitHub.com and the public npm registry; it installs Node 22.18.0. GHES/private registries are outside
|
|
120
|
+
this interface.
|
|
121
|
+
|
|
122
|
+
For read-only results, set `post-comment: 'false'` and reduce `pull-requests` to `read`; keep
|
|
123
|
+
`contents: read`. Optional `checks: read` and `statuses: read` permit capture of GitHub status metadata,
|
|
124
|
+
not logs. Those receipts are not the artifact assessment's subject. Use the default GitHub Actions
|
|
125
|
+
token for comments; it updates only the matching bot-owned comment. No labels or Slack messages are written.
|
|
126
|
+
|
|
127
|
+
## Qualify and operate a consumer
|
|
128
|
+
|
|
129
|
+
1. Check the installed package identity and the reviewed Action SHA.
|
|
130
|
+
2. Confirm the intended published reviewer and context versions are readable.
|
|
131
|
+
3. Run one small authorized PR and inspect its findings, source links, completion and evidence artifacts.
|
|
132
|
+
4. Check feedback delivery separately, and correct any endpoint/contract or credential problem.
|
|
133
|
+
5. Decide whether advisory behavior meets your needs before making the job a required check.
|
|
134
|
+
|
|
135
|
+
Use a fresh dispatch for current commits. Rerunning an old event can reject superseded comparisons;
|
|
136
|
+
removing/readding a trigger label is a new paid request. Duplicate execution requests are not deduplicated.
|
|
137
|
+
Disable the workflow to stop new runs. Rotate expired credentials through their issuing service and
|
|
138
|
+
GitHub secrets, then verify a small run without logging secret values.
|
|
139
|
+
|
|
140
|
+
See [Action reference](action-reference.md) for every input/output and failure-delivery behavior,
|
|
141
|
+
and [operations](operations.md) for diagnostics and recovery.
|