@contextrail/assess 1.0.0-alpha.2 → 1.0.0-alpha.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +115 -0
- package/dist/assess.d.ts +2 -0
- package/dist/assess.js +177 -136
- package/dist/assessment.d.ts +1 -14
- package/dist/assessment.js +0 -46
- package/dist/budget.d.ts +3 -0
- package/dist/budget.js +2 -0
- package/dist/candidate.js +135 -104
- package/dist/change-summary.d.ts +2 -1
- package/dist/change-summary.js +1 -1
- package/dist/cli.js +13 -1
- package/dist/contracts.d.ts +2 -8
- package/dist/contracts.js +2 -2
- package/dist/diagnostics.js +1 -0
- package/dist/evidence.d.ts +4 -0
- package/dist/evidence.js +21 -12
- package/dist/feedback.js +27 -23
- package/dist/git.d.ts +6 -0
- package/dist/git.js +30 -3
- package/dist/github.js +80 -53
- package/dist/index.d.ts +1 -0
- package/dist/index.js +1 -0
- package/dist/mcp.d.ts +2 -0
- package/dist/mcp.js +67 -56
- package/dist/pack.js +8 -1
- package/dist/pr-context.d.ts +2 -0
- package/dist/pr-context.js +27 -17
- package/dist/presentation.js +113 -71
- package/dist/provider-diagnostics.js +17 -12
- package/dist/providers.js +24 -20
- package/dist/report-contract.js +63 -57
- package/dist/results.js +5 -1
- package/dist/source.js +20 -11
- package/dist/speed.d.ts +13 -0
- package/dist/speed.js +72 -0
- package/docs/action-reference.md +6 -3
- package/docs/configuration.md +12 -1
- package/docs/contributing.md +10 -3
- package/docs/github-actions.md +22 -10
- package/docs/operations.md +4 -2
- package/examples/assess-pull-request.yml +1 -1
- package/package.json +1 -1
package/docs/action-reference.md
CHANGED
|
@@ -8,14 +8,16 @@ Look up a specific setting here. For a first integration, use [GitHub Actions se
|
|
|
8
8
|
|
|
9
9
|
| Input | Contract |
|
|
10
10
|
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
11
|
-
| `package-version` |
|
|
11
|
+
| `package-version` | Optional exact npm version override. Omit to install the matching version from the Action release manifest. No npm tags, ranges, URLs or local fallback. |
|
|
12
12
|
| `pr-number`, `base-branch` | Required PR number; optional expected target branch. |
|
|
13
13
|
| `reviewer` | Required reusable published `skill://` ID; never an execution or repository ID. |
|
|
14
14
|
| `mcp-url`, `mcp-org-id` | Required HTTPS endpoint and organization scope, using CLI connection vocabulary. |
|
|
15
15
|
| `provider`, `model` | Required `openai` or `openrouter` and explicit provider-native model. No fallback. |
|
|
16
16
|
| `enforcement` | `advisory` (default), `required`, or `strict`. See the [outcome table](review-results.md#interpret-judgments-and-exit-codes); execution failures always return 2. |
|
|
17
17
|
| `effort` | Optional CLI effort; empty delegates to provider default. |
|
|
18
|
+
| `speed` | `normal` (default), `fast` or `ultrafast` provider service tier. Unsupported tiers are logged and the run continues. |
|
|
18
19
|
| `max-time-ms` | Required positive workflow model deadline, at most 2147483647 ms. Does not bound total job time. |
|
|
20
|
+
| `max-verify-calls` | Optional model calls per finding verification; empty uses the package default (12). See [extended capacity](../README.md#give-large-pull-requests-more-capacity). |
|
|
19
21
|
| `feedback` | `send` (default) or `off`; CLI owns report submission and its status. |
|
|
20
22
|
| `repository-attribution` | `true` or `false` (default); explicitly includes repository identity in CLI retrieval/reporting. |
|
|
21
23
|
| `artifact-retention-days` | 1–90, default 7, within repository/organization retention policy. |
|
|
@@ -81,5 +83,6 @@ or job timeout can still prevent summary/comment execution. The caller must seri
|
|
|
81
83
|
GitHub creation has no atomic compare-and-set; concurrent runs can race after absence checks.
|
|
82
84
|
The npm package owns presentation and GitHub adapter behavior; the Action invokes it.
|
|
83
85
|
|
|
84
|
-
|
|
85
|
-
|
|
86
|
+
Select a major release channel, exact release tag or public commit SHA; the Action defaults to its
|
|
87
|
+
matching npm package. See [release selection](github-actions.md#choose-an-update-policy). Floating
|
|
88
|
+
channels can resolve differently on a later run; pin an exact commit to reproduce a prior selection.
|
package/docs/configuration.md
CHANGED
|
@@ -32,7 +32,17 @@ quickstart with:
|
|
|
32
32
|
|
|
33
33
|
`--effort none|minimal|low|medium|high|xhigh` requests provider-specific reasoning
|
|
34
34
|
effort only when that model supports it. A valid CLI setting does not ensure that
|
|
35
|
-
a provider accepts it.
|
|
35
|
+
a provider accepts it.
|
|
36
|
+
|
|
37
|
+
`--speed normal|fast|ultrafast` (default `normal`) requests the provider's service
|
|
38
|
+
tier for every call: `fast` sends `service_tier: "priority"` and `ultrafast` sends
|
|
39
|
+
`service_tier: "ultrafast"`. Both OpenAI and OpenRouter read the field, but only some
|
|
40
|
+
models serve each tier, and faster tiers cost more per token. The tier is best effort:
|
|
41
|
+
if the provider rejects it, Assess logs `provider_speed_unavailable`, resends that
|
|
42
|
+
request without the tier and stops requesting it for that model; if a response reports
|
|
43
|
+
a different tier, Assess logs the same event once. Neither case fails the run.
|
|
44
|
+
|
|
45
|
+
Source excerpts and published guidance are sent to your
|
|
36
46
|
selected model provider; use an account and data-handling arrangement appropriate
|
|
37
47
|
for that material.
|
|
38
48
|
|
|
@@ -69,6 +79,7 @@ and reviewer ID have no environment aliases. See `--help` for the current option
|
|
|
69
79
|
| `--max-output-tokens N` | Unset; per-call output allowance, including reasoning tokens where applicable. |
|
|
70
80
|
| `--max-total-tokens N` | Unset; observed provider total-token cutoff between calls. Can overshoot by one call; missing usage prevents honest enforcement and stops the review. |
|
|
71
81
|
| `--max-time-ms N` | Unset; model-session deadline in ms, excluding preparation. |
|
|
82
|
+
| `--max-verify-calls N` | 12; model calls per finding verification. Raise it for large changes. |
|
|
72
83
|
| `--source-page-entries N` | 100 entries per list/change response. |
|
|
73
84
|
| `--source-search-matches N` / `--source-search-files N` | 100 each, per search page. |
|
|
74
85
|
| `--source-read-lines N` / `--source-read-bytes N` | 200 lines / 65536 bytes per source response; bytes >=4. |
|
package/docs/contributing.md
CHANGED
|
@@ -38,9 +38,16 @@ there. The npm package's `files` list also includes docs so installed README lin
|
|
|
38
38
|
`release.json` binds the selected version and channel to the upstream commit; the public Action
|
|
39
39
|
commit is a separate immutable identity.
|
|
40
40
|
|
|
41
|
-
After
|
|
42
|
-
|
|
43
|
-
|
|
41
|
+
After publication, a separate job with Contents write permission waits for npm metadata and the
|
|
42
|
+
tarball, then advances `v<major>-alpha` or stable `v<major>`. It never moves a channel backward
|
|
43
|
+
when an older release is retried. These are lightweight tags, not GitHub Releases; exact version
|
|
44
|
+
tags remain immutable. Failed or pending publication leaves the channel unchanged.
|
|
45
|
+
|
|
46
|
+
If promotion times out while npm validates the upload, rerun the failed promotion job after the
|
|
47
|
+
package becomes available; do not republish an existing npm version. Publication jobs are serialized.
|
|
48
|
+
Tag rules must permit the intended floating channels while protecting exact release tags.
|
|
49
|
+
Consumers following a channel need no variable update. Exact tag/SHA consumers opt into each upgrade.
|
|
50
|
+
The README badges track npm's `latest` and `alpha` dist-tags, not the Git major channels.
|
|
44
51
|
|
|
45
52
|
See [Action qualification](github-actions.md#qualify-and-operate-a-consumer) for the live
|
|
46
53
|
integration check and [architecture](architecture.md) before changing the harness.
|
package/docs/github-actions.md
CHANGED
|
@@ -13,20 +13,30 @@ Complete [standards setup](standards.md), then add these in **Settings → Secre
|
|
|
13
13
|
| -------- | ---------------------------- | ------------------------------------------------------------- |
|
|
14
14
|
| Variable | `CONTEXTRAIL_MCP_URL` | Your authorized HTTPS MCP endpoint |
|
|
15
15
|
| Variable | `CONTEXTRAIL_MCP_ORG_ID` | Your organization ID |
|
|
16
|
-
| Variable | `ASSESS_VERSION` | An exact published `@contextrail/assess` version |
|
|
17
16
|
| Variable | `ASSESS_MODEL` | A tool-capable OpenAI model ID available to your account |
|
|
18
17
|
| Secret | `CONTEXTRAIL_MCP_AUTH_TOKEN` | Token authorized for the selected skill/contexts and feedback |
|
|
19
18
|
| Secret | `OPENAI_API_KEY` | Your provider key |
|
|
20
19
|
|
|
21
|
-
Choose
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
20
|
+
## Choose an update policy
|
|
21
|
+
|
|
22
|
+
Use `contextrail/assess/action@v1-alpha` during alpha. After a stable v1 release exists,
|
|
23
|
+
use `@v1` to receive compatible v1 updates. Moving to v2 is a deliberate workflow change.
|
|
24
|
+
Alpha is experimental and does not carry the stable compatibility promise.
|
|
25
|
+
|
|
26
|
+
Each channel advances only after the matching npm package is downloadable. The Action reads
|
|
27
|
+
its release manifest and installs that exact version; there is no separate package variable to maintain.
|
|
28
|
+
For controlled upgrades or rollback, use an exact release tag such as `@v1.0.0-alpha.3`,
|
|
29
|
+
or its full public commit SHA. Exact tags remain protected; floating tags have no GitHub Release attached.
|
|
30
|
+
Only select a release that has actually been published.
|
|
31
|
+
|
|
32
|
+
A channel opts into future releases by the publisher; it is not an immutable source pin.
|
|
33
|
+
A fresh run, including a rerun, can resolve a newer channel revision. Pin the recorded commit
|
|
34
|
+
when reproducing an earlier run. For an independently qualified package override, the low-level
|
|
35
|
+
`package-version` input still accepts an exact version; it is unnecessary for normal use.
|
|
25
36
|
|
|
26
37
|
## Add the workflow
|
|
27
38
|
|
|
28
|
-
Save this as `.github/workflows/contextrail-assess.yml`. Replace
|
|
29
|
-
reviewer skill ID before committing it. No contribution checkout is needed.
|
|
39
|
+
Save this as `.github/workflows/contextrail-assess.yml`. Replace the reviewer skill ID before committing it. No contribution checkout is needed.
|
|
30
40
|
|
|
31
41
|
```yaml
|
|
32
42
|
name: ContextRail Assess
|
|
@@ -50,9 +60,8 @@ jobs:
|
|
|
50
60
|
pull-requests: write
|
|
51
61
|
steps:
|
|
52
62
|
- name: Assess proposed changes
|
|
53
|
-
uses: contextrail/assess/action@
|
|
63
|
+
uses: contextrail/assess/action@v1-alpha
|
|
54
64
|
with:
|
|
55
|
-
package-version: ${{ vars.ASSESS_VERSION }}
|
|
56
65
|
pr-number: ${{ inputs.pr-number }}
|
|
57
66
|
reviewer: skill://engineering/your-reviewer
|
|
58
67
|
mcp-url: ${{ vars.CONTEXTRAIL_MCP_URL }}
|
|
@@ -88,11 +97,14 @@ confirms ingestion only. [Results and next actions](review-results.md) explains
|
|
|
88
97
|
- **PR discussion/history:** add `pr-context: 'true'`. Read the [data implications](security.md) first.
|
|
89
98
|
- **Merge policy:** set `enforcement: required` for supported binding violations, or `strict` to also block unresolved evidence. Configure branch protection to require the job if desired. Do not use `continue-on-error` to create advisory behavior; it would hide execution failures.
|
|
90
99
|
- **Retention:** set `artifact-retention-days` to a permitted value from 1–90. The default is seven days.
|
|
100
|
+
- **Large pull requests:** add a second label that re-runs an incomplete assessment with a faster tier, a longer time limit and more verification calls. See [give large pull requests more capacity](../README.md#give-large-pull-requests-more-capacity).
|
|
91
101
|
- **Models:** one model serves both roles by default. Add `effort` only if the model supports it; [phase overrides](configuration.md#models-by-phase-and-logging) let you select a different verifier later.
|
|
92
102
|
|
|
93
103
|
Repository variable names belong to your caller workflow. In this example `ASSESS_MODEL` is passed
|
|
94
104
|
explicitly to the Action; setting it alone in a local shell does not configure the CLI. The maintained
|
|
95
|
-
monorepo caller
|
|
105
|
+
monorepo caller uses one `ASSESS_RELEASE` variable (default `v1-alpha`) and records the resolved
|
|
106
|
+
public Action SHA and exact npm version in the summary and a `contextrail-assess-release-*` artifact.
|
|
107
|
+
It also supports `ASSESS_EFFORT`, `ASSESS_VERIFY_MODEL`, `ASSESS_VERIFY_EFFORT`
|
|
96
108
|
and `ASSESS_ENFORCEMENT`. Standalone CLI aliases use `CONTEXTRAIL_ASSESS_*`.
|
|
97
109
|
|
|
98
110
|
## Triggers, permissions and trust
|
package/docs/operations.md
CHANGED
|
@@ -95,8 +95,10 @@ syntax details remain unknown rather than being inferred from a generic exceptio
|
|
|
95
95
|
Start with the Action summary's terminal failure, then open `diagnostics.jsonl` in the
|
|
96
96
|
artifact. Correlate events by run ID, phase, attempt and call. The summary links the stop
|
|
97
97
|
reason to the last failing provider request in that phase; a recovered earlier error is
|
|
98
|
-
not the final cause.
|
|
99
|
-
|
|
98
|
+
not the final cause. Use the release identity in the summary to confirm which Action and package ran. The monorepo
|
|
99
|
+
retains `assess-release.json` in its release artifact. `ASSESS_RELEASE` selects both identities;
|
|
100
|
+
a floating channel advances after publication. For rollback, select the recorded public commit
|
|
101
|
+
SHA or exact release tag, then start a fresh dispatch.
|
|
100
102
|
|
|
101
103
|
At `info`, phase boundaries, provider request start/completion, assessment progress,
|
|
102
104
|
recovery, reporting and final outcome appear on stderr. `debug` also shows tool activity,
|
|
@@ -21,7 +21,7 @@ jobs:
|
|
|
21
21
|
runs-on: ubuntu-latest
|
|
22
22
|
timeout-minutes: 30
|
|
23
23
|
env:
|
|
24
|
-
ASSESS_PACKAGE_VERSION: '1.0.0-alpha.
|
|
24
|
+
ASSESS_PACKAGE_VERSION: '1.0.0-alpha.4'
|
|
25
25
|
REVIEWER_ID: skill://your-domain/your-reviewer
|
|
26
26
|
ASSESS_PROVIDER: openai
|
|
27
27
|
ASSESS_MODEL: REPLACE_WITH_PROVIDER_MODEL_ID
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@contextrail/assess",
|
|
3
|
-
"version": "1.0.0-alpha.
|
|
3
|
+
"version": "1.0.0-alpha.4",
|
|
4
4
|
"description": "Assess proposed Git changes against published organizational standards, with per-standard outcomes and retained evidence.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|