@grimoire-rs/indexer 0.2.2 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -18
- package/dist/ci.d.ts +115 -0
- package/dist/ci.d.ts.map +1 -0
- package/dist/ci.js +286 -0
- package/dist/ci.js.map +1 -0
- package/dist/cli/build.d.ts.map +1 -1
- package/dist/cli/build.js +3 -9
- package/dist/cli/build.js.map +1 -1
- package/dist/cli/ci.d.ts +6 -0
- package/dist/cli/ci.d.ts.map +1 -0
- package/dist/cli/ci.js +43 -0
- package/dist/cli/ci.js.map +1 -0
- package/dist/cli/dev.d.ts +7 -0
- package/dist/cli/dev.d.ts.map +1 -0
- package/dist/cli/dev.js +57 -0
- package/dist/cli/dev.js.map +1 -0
- package/dist/cli/init.d.ts +6 -1
- package/dist/cli/init.d.ts.map +1 -1
- package/dist/cli/init.js +294 -74
- package/dist/cli/init.js.map +1 -1
- package/dist/cli/main.d.ts.map +1 -1
- package/dist/cli/main.js +61 -8
- package/dist/cli/main.js.map +1 -1
- package/dist/cli/out_dir.d.ts +11 -0
- package/dist/cli/out_dir.d.ts.map +1 -0
- package/dist/cli/out_dir.js +30 -0
- package/dist/cli/out_dir.js.map +1 -0
- package/dist/cli/validate.d.ts +1 -0
- package/dist/cli/validate.d.ts.map +1 -1
- package/dist/cli/validate.js +47 -1
- package/dist/cli/validate.js.map +1 -1
- package/dist/templates.d.ts +12 -0
- package/dist/templates.d.ts.map +1 -0
- package/dist/templates.js +40 -0
- package/dist/templates.js.map +1 -0
- package/dist/validate/adapters/files.d.ts.map +1 -1
- package/dist/validate/adapters/files.js +13 -3
- package/dist/validate/adapters/files.js.map +1 -1
- package/dist/validate/index.js +17 -2
- package/dist/validate/index.js.map +1 -1
- package/package.json +1 -1
- package/templates/README.md +115 -19
- package/templates/ci/github-automerge.yml +36 -0
- package/templates/ci/github-enrich.yml +27 -0
- package/templates/ci/github-pages.yml +64 -0
- package/templates/ci/github-validate.yml +157 -0
- package/templates/ci/github-verify-ci.yml +47 -0
- package/templates/ci/gitlab-ci.yml +148 -0
- package/templates/ci/gitlab-enrich.sh +26 -0
- package/templates/ci/gitlab-verify.yml +16 -0
- package/templates/ci/header.txt +19 -0
- package/templates/gitattributes +4 -0
- package/templates/gitignore +7 -2
- package/templates/package.json +18 -0
- package/templates/publish.toml +2 -2
- package/templates/github-pages.yml +0 -24
- package/templates/github-validate.yml +0 -51
- package/templates/gitlab-ci.yml +0 -13
- package/templates/reusable/gitlab-index-ci.yml +0 -125
package/templates/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# {{title}}
|
|
2
2
|
|
|
3
|
-
A [Grimoire](https://github.com/grimoire-rs/grimoire) package index
|
|
3
|
+
A [Grimoire](https://github.com/grimoire-rs/grimoire) package index - a
|
|
4
4
|
static site listing the skills, rules, agents, mcp servers, and bundles
|
|
5
5
|
available in `{{registryHost}}`, served at {{baseUrl}}.
|
|
6
6
|
|
|
@@ -8,41 +8,137 @@ available in `{{registryHost}}`, served at {{baseUrl}}.
|
|
|
8
8
|
|
|
9
9
|
| Path | Purpose |
|
|
10
10
|
|---|---|
|
|
11
|
-
| `index/<namespace>/<package>/metadata.json` | One entry per package
|
|
12
|
-
| `index.config.json` | Site identity
|
|
11
|
+
| `index/<host>/<namespace>/<package>/metadata.json` | One entry per package - the source of truth |
|
|
12
|
+
| `index.config.json` | Site identity, branding, and the `ci` block the workflows are rendered from |
|
|
13
13
|
| `index-policy.json` | Committed allowlist of registry hosts contributions may point at |
|
|
14
|
-
| `
|
|
14
|
+
| `package.json` / `package-lock.json` | The renderer this index runs, pinned |
|
|
15
|
+
| `dist/` | Build output (`all.json`, per-path copies, the rendered site) - not committed |
|
|
16
|
+
|
|
17
|
+
## Getting set up
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
npm install
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
That is the whole toolchain. `package-lock.json` pins the exact renderer this
|
|
24
|
+
index builds, validates and deploys with - commit it, because CI installs from
|
|
25
|
+
it with `npm ci` and nothing is resolved on the runner.
|
|
26
|
+
|
|
27
|
+
## Everyday workflow
|
|
28
|
+
|
|
29
|
+
| Command | What it does |
|
|
30
|
+
|---|---|
|
|
31
|
+
| `npm run dev` | Serve the index locally with a live preview |
|
|
32
|
+
| `npm run build` | Compile `index/**` into `dist/` and render the site |
|
|
33
|
+
| `npm run validate` | Run the contribution gate over changed paths |
|
|
34
|
+
| `npm run enrich` | Refresh READMEs, logos and version lists from the registry (needs `grim`) |
|
|
35
|
+
| `npm run ci` | Re-render the CI workflows from `index.config.json` |
|
|
36
|
+
| `npm run ci:check` | Fail if the committed workflows have drifted from the config |
|
|
37
|
+
|
|
38
|
+
### Review a change locally
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
npm run dev
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
`dev` prints the URL to open once it is up - the site exactly as it deploys.
|
|
45
|
+
A domain-rooted `site` serves at `http://localhost:4321`; a project-Pages
|
|
46
|
+
`site` (`https://<owner>.github.io/<repo>`) serves under a `/<repo>` path
|
|
47
|
+
segment instead, because that is Astro's `base` for that layout too.
|
|
48
|
+
|
|
49
|
+
Entries under `index/` and the settings in `index.config.json` are read once,
|
|
50
|
+
at start-up, so restart after editing them. The dev server renders through the
|
|
51
|
+
same code path `npm run build` does: what you see is what deploys.
|
|
52
|
+
|
|
53
|
+
### Test a contribution before you open the pull request
|
|
54
|
+
|
|
55
|
+
`validate` is the same gate CI runs, and the same exit-code contract - 0
|
|
56
|
+
means eligible for auto-merge, non-zero means a maintainer takes a look -
|
|
57
|
+
but CI reads the forge identity and the PR tree off the platform, so a
|
|
58
|
+
local run has to supply them by hand:
|
|
59
|
+
|
|
60
|
+
```sh
|
|
61
|
+
npm run validate -- \
|
|
62
|
+
--root . --pr-tree . \
|
|
63
|
+
--forge github --author-login <your-github-login> --author-id <your-numeric-id> \
|
|
64
|
+
-- index/<host>/<your-namespace>/<your-package>/metadata.json
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
`--root` and `--pr-tree` both point at your own checkout here; CI points
|
|
68
|
+
them at two different checkouts (the trusted base and the PR head) so it
|
|
69
|
+
can also catch a re-registered login trying to inherit someone else's
|
|
70
|
+
package - fine to collapse for a solo sanity check, not something CI does.
|
|
71
|
+
`--author-id` is your numeric forge account id, not your login - find yours
|
|
72
|
+
at `https://api.github.com/users/<your-github-login>`. The gate makes real
|
|
73
|
+
calls to the forge API and to the registry, so it needs network access and
|
|
74
|
+
an entry that is actually yours to pass.
|
|
15
75
|
|
|
16
76
|
## Contributing a package
|
|
17
77
|
|
|
18
|
-
Add `index/<namespace>/<package>/metadata.json
|
|
19
|
-
|
|
20
|
-
|
|
78
|
+
Add `index/<host>/<your-namespace>/<your-package>/metadata.json` (`<host>`
|
|
79
|
+
is the forge this repo is hosted on, e.g. `github.com`), then open a pull
|
|
80
|
+
request.
|
|
21
81
|
|
|
22
82
|
```json
|
|
23
83
|
{
|
|
24
84
|
"schema": 1,
|
|
25
85
|
"name": "{{name}}-example",
|
|
26
86
|
"kind": "skill",
|
|
27
|
-
"ref": "{{registryHost}}/{{registryAlias}}/skills/{{name}}-example
|
|
87
|
+
"ref": "{{registryHost}}/{{registryAlias}}/skills/{{name}}-example",
|
|
28
88
|
"description": "One line describing what this package does.",
|
|
29
|
-
"owner": { "id":
|
|
89
|
+
"owner": { "id": 12345678, "github": "your-github-login" }
|
|
30
90
|
}
|
|
31
91
|
```
|
|
32
92
|
|
|
33
93
|
The directory name must equal `name`, and `kind` is one of `skill`,
|
|
34
|
-
`rule`, `agent`, `mcp`, `bundle`.
|
|
94
|
+
`rule`, `agent`, `mcp`, `bundle`. `ref` names a repository, not a tag or
|
|
95
|
+
digest - the gate resolves it to whatever is published. `owner.id` must be
|
|
96
|
+
your real numeric GitHub account id, not the placeholder above - find yours
|
|
97
|
+
at `https://api.github.com/users/your-github-login`; the gate checks it
|
|
98
|
+
against the forge, not the `github` login, since a login can be
|
|
99
|
+
re-registered by someone else.
|
|
35
100
|
|
|
36
|
-
##
|
|
101
|
+
## CI
|
|
102
|
+
|
|
103
|
+
This repo owns and runs its own CI: the workflow files are committed here, not
|
|
104
|
+
called from anywhere else. They are *rendered* from the `ci` block of
|
|
105
|
+
`index.config.json`, so the way to change what CI does is to change the config
|
|
106
|
+
and run `npm run ci`.
|
|
107
|
+
|
|
108
|
+
| Key | Default | What it changes |
|
|
109
|
+
|---|---|---|
|
|
110
|
+
| `forge` | `github` | Which pipeline is rendered - `github` or `gitlab` |
|
|
111
|
+
| `nodeVersion` | `22` | Node the jobs run on, and the GitLab image tag |
|
|
112
|
+
| `enrich` | `true` | Pull READMEs, logos and version lists from the registry before building |
|
|
113
|
+
| `grimVersion` | `latest` | grim release the enrichment step uses |
|
|
114
|
+
| `autoMerge` | `false` | Squash-merge a pull request the gate passed, then deploy (GitHub only) |
|
|
115
|
+
| `allowManualEdits` | `false` | Own the files by hand; drops the drift guard |
|
|
116
|
+
|
|
117
|
+
`autoMerge` is off until you turn it on: merging an untrusted contribution
|
|
118
|
+
unattended is your policy to set, not this package's to assume. It is GitHub
|
|
119
|
+
only, and that is not an oversight - GitHub always runs the base branch's copy
|
|
120
|
+
of a `pull_request_target` workflow, so the gate a pull request faces is one it
|
|
121
|
+
cannot edit. On GitLab the merge request supplies its own `.gitlab-ci.yml` and a
|
|
122
|
+
fork's pipeline runs in the fork, so an auto-merge there would act on a verdict
|
|
123
|
+
the contributor wrote. Setting `autoMerge` with `forge: gitlab` is refused
|
|
124
|
+
rather than rendered inert.
|
|
125
|
+
|
|
126
|
+
A `verify-ci` job re-renders and diffs on every change that can produce
|
|
127
|
+
drift - pushes and pull requests touching `index.config.json`,
|
|
128
|
+
`package.json`, `package-lock.json`, or `.github/workflows/**` - so a
|
|
129
|
+
hand-edit fails CI instead of silently forking. Action pins are excluded
|
|
130
|
+
from that diff, so Renovate may bump `uses: owner/action@<ref>` freely.
|
|
131
|
+
|
|
132
|
+
To pick up renderer fixes, bump `@grimoire-rs/indexer` the ordinary npm way and
|
|
133
|
+
re-render:
|
|
37
134
|
|
|
38
135
|
```sh
|
|
39
|
-
|
|
40
|
-
|
|
136
|
+
npm update @grimoire-rs/indexer
|
|
137
|
+
npm run ci
|
|
41
138
|
```
|
|
42
139
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
The
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
re-scaffolding, and your local edits are never overwritten.
|
|
140
|
+
No CI *definitions* are fetched from outside this repo - no reusable
|
|
141
|
+
workflow, no remote `include:` - so what you review in a diff is exactly
|
|
142
|
+
what runs. The jobs themselves still reach the network: `npm ci` from the
|
|
143
|
+
npm registry, `uses: actions/...@<sha>` from GitHub, and - with `enrich`
|
|
144
|
+
on - a grim release tarball.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
|
|
2
|
+
# `needs: validate` is the entire safety condition. The gate's exit code is
|
|
3
|
+
# the authorization - 0 means "eligible for auto-merge" - and `needs:` is
|
|
4
|
+
# what turns that into "this job does not start". A cancelled or skipped
|
|
5
|
+
# gate is not a success either, so both also stop here.
|
|
6
|
+
#
|
|
7
|
+
# This job checks nothing out. Contributor content never touches the runner
|
|
8
|
+
# that holds the write token; the only thing it does with the pull request is
|
|
9
|
+
# name its number to `gh`.
|
|
10
|
+
#
|
|
11
|
+
# `--match-head-commit` closes the window between the two jobs: the gate
|
|
12
|
+
# judged one head sha, and without this a push landing while it ran would be
|
|
13
|
+
# what got merged - the contribution reviewed and the contribution merged
|
|
14
|
+
# would be different commits. Mismatch fails the job, and the next
|
|
15
|
+
# `synchronize` event re-runs the gate against the new head anyway.
|
|
16
|
+
automerge:
|
|
17
|
+
needs: validate
|
|
18
|
+
runs-on: ubuntu-latest
|
|
19
|
+
permissions:
|
|
20
|
+
contents: write
|
|
21
|
+
pull-requests: write
|
|
22
|
+
actions: write # dispatch pages.yml after the merge
|
|
23
|
+
steps:
|
|
24
|
+
- name: Merge and deploy
|
|
25
|
+
env:
|
|
26
|
+
GH_TOKEN: ${{ github.token }}
|
|
27
|
+
REPO: ${{ github.repository }}
|
|
28
|
+
PR: ${{ github.event.pull_request.number }}
|
|
29
|
+
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
30
|
+
BRANCH: ${{ github.event.repository.default_branch }}
|
|
31
|
+
run: |
|
|
32
|
+
set -euo pipefail
|
|
33
|
+
gh pr merge "$PR" --repo "$REPO" --squash --match-head-commit "$HEAD_SHA"
|
|
34
|
+
# A push made with GITHUB_TOKEN triggers no workflow, so the merge
|
|
35
|
+
# this job just made would never build the site. Dispatch it.
|
|
36
|
+
gh workflow run pages.yml --repo "$REPO" --ref "$BRANCH"
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
|
|
2
|
+
# The index stores pointers; everything a reader looks at - README,
|
|
3
|
+
# changelog, logo, version list - lives in the registry, and grim is
|
|
4
|
+
# what reads a registry. `ci.grimVersion` pins the binary.
|
|
5
|
+
- name: Install grim
|
|
6
|
+
env:
|
|
7
|
+
# Resolved from `ci.grimVersion` when this file was rendered, so
|
|
8
|
+
# there is no version branch left to take at run time.
|
|
9
|
+
GRIM_RELEASE_BASE: "{{grimReleaseBase}}"
|
|
10
|
+
run: |
|
|
11
|
+
set -euo pipefail
|
|
12
|
+
tarball="grimoire-$(uname -m)-unknown-linux-gnu.tar.gz"
|
|
13
|
+
cd "$(mktemp -d)"
|
|
14
|
+
curl -fsSL --proto '=https' --tlsv1.2 \
|
|
15
|
+
-O "${GRIM_RELEASE_BASE}/${tarball}" -O "${GRIM_RELEASE_BASE}/${tarball}.sha256"
|
|
16
|
+
sha256sum -c "${tarball}.sha256"
|
|
17
|
+
tar -xzf "${tarball}" --strip-components=1 --wildcards -C /usr/local/bin '*/grim'
|
|
18
|
+
grim --version
|
|
19
|
+
|
|
20
|
+
# Enrichment is best-effort by design: a registry outage must degrade
|
|
21
|
+
# the site to pointers-only, never block a deploy that is otherwise
|
|
22
|
+
# ready. `enrich` itself keeps any sidecar it fails to refresh. Set
|
|
23
|
+
# `"enrich": false` in the `ci` block to drop these two steps entirely.
|
|
24
|
+
- name: Refresh package metadata
|
|
25
|
+
run: |
|
|
26
|
+
npm run enrich ||
|
|
27
|
+
echo "::warning::enrich failed - building without READMEs, logos and version lists"
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
{{header}}
|
|
2
|
+
# Builds this index and deploys it to GitHub Pages.
|
|
3
|
+
#
|
|
4
|
+
# Enable it once, in Settings -> Pages -> Source: "GitHub Actions". Until
|
|
5
|
+
# that is set, this workflow runs green and deploys nothing.
|
|
6
|
+
name: pages
|
|
7
|
+
|
|
8
|
+
on:
|
|
9
|
+
push:
|
|
10
|
+
branches: [main]
|
|
11
|
+
workflow_dispatch:
|
|
12
|
+
|
|
13
|
+
# Default-deny at the top; every job pays for exactly what it needs.
|
|
14
|
+
permissions: {}
|
|
15
|
+
|
|
16
|
+
concurrency:
|
|
17
|
+
group: grim-indexer-pages
|
|
18
|
+
cancel-in-progress: false
|
|
19
|
+
|
|
20
|
+
jobs:
|
|
21
|
+
build:
|
|
22
|
+
runs-on: ubuntu-latest
|
|
23
|
+
permissions:
|
|
24
|
+
contents: read
|
|
25
|
+
steps:
|
|
26
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
27
|
+
with:
|
|
28
|
+
persist-credentials: false
|
|
29
|
+
|
|
30
|
+
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
31
|
+
with:
|
|
32
|
+
node-version: "{{nodeVersion}}"
|
|
33
|
+
cache: npm
|
|
34
|
+
|
|
35
|
+
# `npm ci` installs exactly package-lock.json, so which renderer builds
|
|
36
|
+
# this site is a reviewed fact in the repository, not a resolution that
|
|
37
|
+
# happens on the runner. `--ignore-scripts` because this job runs
|
|
38
|
+
# MERGED contributor content: a `prepare` script or a lock entry that
|
|
39
|
+
# arrived through an auto-merged pull request would otherwise execute
|
|
40
|
+
# here, in the job that publishes the site.
|
|
41
|
+
- name: Install
|
|
42
|
+
run: npm ci --ignore-scripts
|
|
43
|
+
{{githubEnrichSteps}}
|
|
44
|
+
- name: Build index
|
|
45
|
+
run: npm run build
|
|
46
|
+
|
|
47
|
+
- uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
|
|
48
|
+
|
|
49
|
+
- uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
|
|
50
|
+
with:
|
|
51
|
+
path: dist
|
|
52
|
+
|
|
53
|
+
deploy:
|
|
54
|
+
needs: build
|
|
55
|
+
runs-on: ubuntu-latest
|
|
56
|
+
permissions:
|
|
57
|
+
pages: write
|
|
58
|
+
id-token: write
|
|
59
|
+
environment:
|
|
60
|
+
name: github-pages
|
|
61
|
+
url: ${{ steps.deployment.outputs.page_url }}
|
|
62
|
+
steps:
|
|
63
|
+
- id: deployment
|
|
64
|
+
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0
|
|
@@ -0,0 +1,157 @@
|
|
|
1
|
+
{{header}}
|
|
2
|
+
# Contribution gate for pull requests against this index.
|
|
3
|
+
#
|
|
4
|
+
# Exit 0 = eligible for auto-merge, non-zero = manual review required, so this
|
|
5
|
+
# job's success IS the gate. To enforce it, require the status check named
|
|
6
|
+
# `validate` in branch protection - the job key below is the context name, so
|
|
7
|
+
# renaming the job moves the required context with it, and requiring a context
|
|
8
|
+
# that never reports blocks every pull request forever.
|
|
9
|
+
#
|
|
10
|
+
# `pull_request_target` is deliberate and safe HERE, and only because of what
|
|
11
|
+
# this job does: it runs the renderer this repository has locked, against the
|
|
12
|
+
# TRUSTED base checkout, and reads the pull-request head as DATA ONLY in a
|
|
13
|
+
# separate directory - nothing in it is executed, imported, installed, or
|
|
14
|
+
# interpolated into a command. GitHub always runs the base branch's copy of a
|
|
15
|
+
# `pull_request_target` workflow, so a contributor editing this file in their
|
|
16
|
+
# pull request does not change the gate that judges it.
|
|
17
|
+
#
|
|
18
|
+
# The `npm ci` below installs the BASE checkout's package-lock.json, in the
|
|
19
|
+
# base directory. That lock is maintainer-committed and reviewed like any
|
|
20
|
+
# other file here. NEVER install, run, or `cd` into `pr-tree/`, and never add
|
|
21
|
+
# a `run:` step that touches it - that would execute contributor code with a
|
|
22
|
+
# token. `--ignore-scripts` holds the line one step further: the gate needs
|
|
23
|
+
# nothing a dependency lifecycle script provides.
|
|
24
|
+
#
|
|
25
|
+
# Do not widen `permissions:`.
|
|
26
|
+
#
|
|
27
|
+
# This gate covers third-party contributions. It does NOT cover an announce
|
|
28
|
+
# your own CI makes (the `--with-skills` layout, where this repo publishes into
|
|
29
|
+
# its own index): GitHub runs no workflows on a pull request opened with
|
|
30
|
+
# `secrets.GITHUB_TOKEN`, so `grim publish --announce` from CI lands an
|
|
31
|
+
# effectively ungated pull request - review it by hand. That pull request also
|
|
32
|
+
# needs "Allow GitHub Actions to create and approve pull requests"
|
|
33
|
+
# (Settings -> Actions) turned on, off by default; without it grim only pushes
|
|
34
|
+
# the branch and reports `outcome: "branch-pushed"`. Enabling it lets any
|
|
35
|
+
# workflow in this repo approve pull requests too, so weigh it against your
|
|
36
|
+
# branch protection.
|
|
37
|
+
name: validate
|
|
38
|
+
|
|
39
|
+
on:
|
|
40
|
+
pull_request_target:
|
|
41
|
+
paths:
|
|
42
|
+
- "index/**"
|
|
43
|
+
- index-policy.json
|
|
44
|
+
|
|
45
|
+
permissions: {}
|
|
46
|
+
|
|
47
|
+
# One gate run per pull request. Superseding a run cancels it, and a cancelled
|
|
48
|
+
# run is not a success - so nothing downstream of the gate can act on a verdict
|
|
49
|
+
# about a head that has already been replaced.
|
|
50
|
+
concurrency:
|
|
51
|
+
group: grim-indexer-validate-${{ github.event.pull_request.number }}
|
|
52
|
+
cancel-in-progress: true
|
|
53
|
+
|
|
54
|
+
jobs:
|
|
55
|
+
validate:
|
|
56
|
+
runs-on: ubuntu-latest
|
|
57
|
+
permissions:
|
|
58
|
+
contents: read
|
|
59
|
+
pull-requests: read
|
|
60
|
+
steps:
|
|
61
|
+
# Trusted: the committed state the pull request proposes to change.
|
|
62
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
63
|
+
with:
|
|
64
|
+
persist-credentials: false
|
|
65
|
+
|
|
66
|
+
# Untrusted: the pull-request head, as data. `persist-credentials: false`
|
|
67
|
+
# keeps a token out of the tree the contributor controls.
|
|
68
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
69
|
+
with:
|
|
70
|
+
ref: ${{ github.event.pull_request.head.sha }}
|
|
71
|
+
path: pr-tree
|
|
72
|
+
persist-credentials: false
|
|
73
|
+
|
|
74
|
+
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
75
|
+
with:
|
|
76
|
+
node-version: "{{nodeVersion}}"
|
|
77
|
+
cache: npm
|
|
78
|
+
|
|
79
|
+
- name: Install
|
|
80
|
+
run: npm ci --ignore-scripts
|
|
81
|
+
|
|
82
|
+
# Changed paths come from the API, not from a diff of the untrusted tree,
|
|
83
|
+
# and they reach the gate through a FILE - never through argv. They are
|
|
84
|
+
# attacker-controlled strings: appending them to the command line let a
|
|
85
|
+
# pull request that added an empty file named `-h` short-circuit the
|
|
86
|
+
# parser and exit 0, which this job's contract reads as "eligible for
|
|
87
|
+
# auto-merge". Do not reintroduce `xargs` here.
|
|
88
|
+
#
|
|
89
|
+
# EVERY changed path is passed through - no path filter, no status
|
|
90
|
+
# filter. Rejecting an out-of-bounds path is the gate's first job, and it
|
|
91
|
+
# ownership-checks a deletion before allowing it, so anything dropped
|
|
92
|
+
# here is dropped from the decision this job's exit code authorizes.
|
|
93
|
+
# Filtering to `^index/` would let a pull request that adds a valid entry
|
|
94
|
+
# and edits a workflow pass; filtering out `removed` would let one that
|
|
95
|
+
# adds its own entry and deletes a stranger's pass.
|
|
96
|
+
#
|
|
97
|
+
# The count cross-check is the same argument applied to pagination:
|
|
98
|
+
# `pulls/{n}/files` stops at 3000 files, so without it a large enough
|
|
99
|
+
# pull request pushes its interesting paths past the cap and the gate
|
|
100
|
+
# authorizes a change it never saw.
|
|
101
|
+
- name: Collect changed paths
|
|
102
|
+
env:
|
|
103
|
+
GH_TOKEN: ${{ github.token }}
|
|
104
|
+
REPO: ${{ github.repository }}
|
|
105
|
+
PR: ${{ github.event.pull_request.number }}
|
|
106
|
+
run: |
|
|
107
|
+
set -euo pipefail
|
|
108
|
+
files="repos/${REPO}/pulls/${PR}/files"
|
|
109
|
+
|
|
110
|
+
# ONE fetch, then every derivation from that one snapshot. Reading
|
|
111
|
+
# the endpoint once per derivation reads a mutable resource: a push
|
|
112
|
+
# landing between two calls left the count describing one head and
|
|
113
|
+
# the path list another, so the check certified a list it had never
|
|
114
|
+
# counted. `--paginate` emits one array per page; `-s add` makes them
|
|
115
|
+
# one array.
|
|
116
|
+
gh api --paginate "${files}" | jq -s 'add' > records.json
|
|
117
|
+
|
|
118
|
+
# Count RECORDS. The check exists because `pulls/{n}/files` stops at
|
|
119
|
+
# 3000 files - without it a large enough pull request pushes its
|
|
120
|
+
# interesting paths past the cap and the gate authorizes a change it
|
|
121
|
+
# never saw.
|
|
122
|
+
records=$(jq 'length' records.json)
|
|
123
|
+
declared=$(gh api "repos/${REPO}/pulls/${PR}" --jq '.changed_files')
|
|
124
|
+
if [ "${records}" -ne "${declared}" ]; then
|
|
125
|
+
echo "::error::collected ${records} of ${declared} changed files" \
|
|
126
|
+
"- the gate cannot judge what it cannot see"
|
|
127
|
+
exit 1
|
|
128
|
+
fi
|
|
129
|
+
|
|
130
|
+
# A rename must contribute BOTH names, so a record that claims to be
|
|
131
|
+
# one without naming its source is refused rather than silently
|
|
132
|
+
# contributing only its destination - which is precisely the hole
|
|
133
|
+
# below closes.
|
|
134
|
+
orphans=$(jq '[.[] | select((.status == "renamed" or .status == "copied")
|
|
135
|
+
and ((.previous_filename // "") == ""))] | length' records.json)
|
|
136
|
+
if [ "${orphans}" -ne 0 ]; then
|
|
137
|
+
echo "::error::${orphans} renamed file(s) arrived with no previous_filename" \
|
|
138
|
+
"- the source path cannot be checked"
|
|
139
|
+
exit 1
|
|
140
|
+
fi
|
|
141
|
+
|
|
142
|
+
# BOTH names of a rename. The API reports a rename as one record
|
|
143
|
+
# whose `filename` is the destination and whose `previous_filename`
|
|
144
|
+
# is the source; taking only the destination let a pull request copy
|
|
145
|
+
# a stranger's entry into its own namespace and delete the original,
|
|
146
|
+
# with the deletion never presented to the gate and no ownership
|
|
147
|
+
# check ever run on the victim's namespace.
|
|
148
|
+
jq -r '.[] | .filename, (.previous_filename // empty)' records.json > changed.txt
|
|
149
|
+
|
|
150
|
+
- name: Validate contribution
|
|
151
|
+
env:
|
|
152
|
+
GITHUB_TOKEN: ${{ github.token }}
|
|
153
|
+
run: |
|
|
154
|
+
set -euo pipefail
|
|
155
|
+
npm run validate -- \
|
|
156
|
+
--forge github --root . --pr-tree pr-tree --changed-from changed.txt
|
|
157
|
+
{{githubAutoMergeJob}}
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
{{header}}
|
|
2
|
+
# Fail closed when a generated CI file is hand-edited: this job re-renders from
|
|
3
|
+
# index.config.json and diffs against what is committed. Fix drift by running
|
|
4
|
+
# `npm run ci` locally and committing the result.
|
|
5
|
+
#
|
|
6
|
+
# It runs under `pull_request`, so it needs no secrets and never checks out a
|
|
7
|
+
# trusted token. That also means a fork's copy of this file is what runs on
|
|
8
|
+
# that fork's pull request - the guard is a maintenance check on the
|
|
9
|
+
# repository's own history, not a trust boundary. The contribution gate
|
|
10
|
+
# (validate.yml) is the trust boundary, and it rejects any pull request that
|
|
11
|
+
# touches a path outside `index/<host>/<namespace>/<package>/`.
|
|
12
|
+
name: verify-ci
|
|
13
|
+
|
|
14
|
+
on:
|
|
15
|
+
pull_request:
|
|
16
|
+
paths:
|
|
17
|
+
{{verifyPaths}}
|
|
18
|
+
push:
|
|
19
|
+
branches: [main]
|
|
20
|
+
paths:
|
|
21
|
+
{{verifyPaths}}
|
|
22
|
+
|
|
23
|
+
permissions: {}
|
|
24
|
+
|
|
25
|
+
jobs:
|
|
26
|
+
verify-ci:
|
|
27
|
+
runs-on: ubuntu-latest
|
|
28
|
+
permissions:
|
|
29
|
+
contents: read
|
|
30
|
+
steps:
|
|
31
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
32
|
+
with:
|
|
33
|
+
persist-credentials: false
|
|
34
|
+
|
|
35
|
+
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
36
|
+
with:
|
|
37
|
+
node-version: "{{nodeVersion}}"
|
|
38
|
+
cache: npm
|
|
39
|
+
|
|
40
|
+
# Rendering needs no build toolchain, so nothing here runs a dependency's
|
|
41
|
+
# lifecycle script.
|
|
42
|
+
- name: Install
|
|
43
|
+
run: npm ci --ignore-scripts
|
|
44
|
+
|
|
45
|
+
# Exits 65 on drift; the hint names the stale path and never its contents.
|
|
46
|
+
- name: Check the committed CI matches index.config.json
|
|
47
|
+
run: npm run ci:check
|
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
{{header}}
|
|
2
|
+
# Builds this index for GitLab Pages and gates merge requests.
|
|
3
|
+
|
|
4
|
+
stages:
|
|
5
|
+
- test
|
|
6
|
+
- deploy
|
|
7
|
+
|
|
8
|
+
# `npm ci` installs exactly package-lock.json, so which renderer runs is a
|
|
9
|
+
# reviewed fact in this repository rather than a resolution on the runner.
|
|
10
|
+
default:
|
|
11
|
+
image: node:{{nodeVersion}}-alpine
|
|
12
|
+
cache:
|
|
13
|
+
key:
|
|
14
|
+
files:
|
|
15
|
+
- package-lock.json
|
|
16
|
+
paths:
|
|
17
|
+
- .npm/
|
|
18
|
+
- .npm-gate/
|
|
19
|
+
|
|
20
|
+
# Contribution gate. Exit 0 = eligible for auto-merge, non-zero = manual
|
|
21
|
+
# review required, so this job's status IS the gate.
|
|
22
|
+
#
|
|
23
|
+
# The gate must judge the change against the COMMITTED policy, not the policy
|
|
24
|
+
# the merge request proposes - otherwise a contributor allowlists their own
|
|
25
|
+
# registry host in the same merge request. So the target branch is checked out
|
|
26
|
+
# into a sibling worktree and passed as `--root`, with the merge-request head
|
|
27
|
+
# passed as `--pr-tree` and read as data only.
|
|
28
|
+
#
|
|
29
|
+
# The gate installs and runs from `../.gate-base`, the target branch's worktree -
|
|
30
|
+
# never from the merge request's own checkout. Otherwise the contributor
|
|
31
|
+
# supplies the lockfile that decides which validator judges them, and
|
|
32
|
+
# `--ignore-scripts` alone does not fix that: the package it installs is still
|
|
33
|
+
# theirs to choose. `--ignore-scripts` stays anyway, one layer further in.
|
|
34
|
+
#
|
|
35
|
+
# RESIDUAL, and it is not fixable from this file. Two ways, both structural:
|
|
36
|
+
#
|
|
37
|
+
# 1. The merge request supplies its own `.gitlab-ci.yml`, so a contributor
|
|
38
|
+
# can delete this job outright.
|
|
39
|
+
# 2. GitLab runs a FORK merge request's pipeline in the source project, so
|
|
40
|
+
# `origin` is the fork. `origin/<target>` is then the fork's copy of the
|
|
41
|
+
# target branch, which means `../.gate-base` - and the `index-policy.json` and
|
|
42
|
+
# `package-lock.json` read from it - are contributor-controlled for a
|
|
43
|
+
# fork merge request. The `../.gate-base` install above is therefore defence in
|
|
44
|
+
# depth for same-project branches, not a trust boundary for forks.
|
|
45
|
+
#
|
|
46
|
+
# Enforce it instance-side - a compliance pipeline, a required merge check, or
|
|
47
|
+
# protected-branch approval rules - or treat the GitLab gate as advisory. The
|
|
48
|
+
# GitHub gate has no equivalent gap: `pull_request_target` always runs the base
|
|
49
|
+
# branch's copy of the workflow, against the base checkout.
|
|
50
|
+
grim-indexer:validate:
|
|
51
|
+
stage: test
|
|
52
|
+
variables:
|
|
53
|
+
GIT_DEPTH: 0
|
|
54
|
+
rules:
|
|
55
|
+
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
|
56
|
+
# The job image is picked for npm, and `node:*-alpine` ships no git - but
|
|
57
|
+
# this job diffs the merge request against its target branch, so git is not
|
|
58
|
+
# optional. GitLab's runner clones with a separate helper image, so nothing
|
|
59
|
+
# else puts git in here. Install it for whichever package manager the image
|
|
60
|
+
# has.
|
|
61
|
+
#
|
|
62
|
+
# The trailing check is the load-bearing part: the gate's contract is
|
|
63
|
+
# "exit 0 = eligible for auto-merge", so a missing or failed-to-install tool
|
|
64
|
+
# must fail the job, never silently skip validation.
|
|
65
|
+
before_script:
|
|
66
|
+
- |
|
|
67
|
+
set -eu
|
|
68
|
+
if ! command -v git >/dev/null 2>&1; then
|
|
69
|
+
if command -v apk >/dev/null 2>&1; then
|
|
70
|
+
apk add --no-cache git
|
|
71
|
+
elif command -v apt-get >/dev/null 2>&1; then
|
|
72
|
+
apt-get update -qq && apt-get install -y -qq --no-install-recommends git
|
|
73
|
+
fi
|
|
74
|
+
fi
|
|
75
|
+
command -v git >/dev/null 2>&1 || {
|
|
76
|
+
echo "grim-indexer: the gate needs git, and this image has none" >&2
|
|
77
|
+
exit 1
|
|
78
|
+
}
|
|
79
|
+
script:
|
|
80
|
+
- |
|
|
81
|
+
set -eu
|
|
82
|
+
git fetch --quiet origin "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
|
|
83
|
+
# `../.gate-base` sits outside $CI_PROJECT_DIR, which is the part GitLab
|
|
84
|
+
# cleans. A leftover from an earlier run makes `git worktree add` fail
|
|
85
|
+
# and, under `set -eu`, fails the job permanently - fail-closed, but the
|
|
86
|
+
# shape that gets a gate switched off rather than fixed.
|
|
87
|
+
#
|
|
88
|
+
# The leading dot is load-bearing. `..` is the NAMESPACE directory,
|
|
89
|
+
# shared with every sibling project on this runner, and a GitLab project
|
|
90
|
+
# path must start with a letter or a digit - so no project can be named
|
|
91
|
+
# `.gate-base` and this `rm -rf` can never reach somebody else's
|
|
92
|
+
# checkout. It was `../base` for one revision, which a group holding a
|
|
93
|
+
# project called `base` would have had wiped on every gate run.
|
|
94
|
+
rm -rf ../.gate-base
|
|
95
|
+
git worktree prune
|
|
96
|
+
git worktree add --detach ../.gate-base "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
|
|
97
|
+
# Every changed path, with no pathspec and no diff filter: rejecting an
|
|
98
|
+
# out-of-bounds path is the gate's first job, and it ownership-checks a
|
|
99
|
+
# deletion before allowing it. Narrowing to `-- index` would let a merge
|
|
100
|
+
# request that adds a valid entry and edits this very file pass; dropping
|
|
101
|
+
# `D` would let one that deletes a stranger's entry pass.
|
|
102
|
+
#
|
|
103
|
+
# `--no-renames` is load-bearing. Git reports a rename as the DESTINATION
|
|
104
|
+
# path only, so copying a stranger's metadata.json into your own
|
|
105
|
+
# namespace and deleting the original was detected as a rename, the
|
|
106
|
+
# deletion never reached the gate, and no ownership check ever ran on the
|
|
107
|
+
# victim's namespace. With this flag both sides are listed.
|
|
108
|
+
#
|
|
109
|
+
# NEVER add `-C`. Copy detection re-collapses the pair to the
|
|
110
|
+
# destination in either flag order, which puts the hole straight back.
|
|
111
|
+
#
|
|
112
|
+
# `-z` and `--changed-from` keep these attacker-controlled strings off
|
|
113
|
+
# the command line and out of the shell's hands entirely: a path named
|
|
114
|
+
# `-h` used to make the parser print help and exit 0, which this job's
|
|
115
|
+
# contract reads as "eligible for auto-merge".
|
|
116
|
+
git diff -z --name-only --no-renames \
|
|
117
|
+
"$CI_MERGE_REQUEST_DIFF_BASE_SHA" HEAD > "$CI_PROJECT_DIR/changed.txt"
|
|
118
|
+
# Installed and run from the trusted target branch, never from the
|
|
119
|
+
# merge request's own lockfile. Its own cache directory, because the
|
|
120
|
+
# shared one is a path the merge request can commit into.
|
|
121
|
+
cd ../.gate-base
|
|
122
|
+
rm -rf "$CI_PROJECT_DIR/.npm-gate"
|
|
123
|
+
npm ci --cache "$CI_PROJECT_DIR/.npm-gate" --prefer-offline --ignore-scripts
|
|
124
|
+
npm run validate -- \
|
|
125
|
+
--forge gitlab --root . --pr-tree "$CI_PROJECT_DIR" \
|
|
126
|
+
--changed-from "$CI_PROJECT_DIR/changed.txt"
|
|
127
|
+
{{gitlabVerifyJob}}
|
|
128
|
+
# GitLab Pages serves whatever the `pages` job leaves in `public/`.
|
|
129
|
+
pages:
|
|
130
|
+
stage: deploy
|
|
131
|
+
rules:
|
|
132
|
+
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
|
133
|
+
before_script:
|
|
134
|
+
- npm ci --cache .npm --prefer-offline --ignore-scripts
|
|
135
|
+
script:
|
|
136
|
+
- |
|
|
137
|
+
set -eu
|
|
138
|
+
{{gitlabEnrichScript}}
|
|
139
|
+
# Built into `dist/`, then moved. Building straight into `public/` was a
|
|
140
|
+
# data-loss bug: the compile step clears its output directory first, so it
|
|
141
|
+
# deleted this index's own committed asset layer (logo, favicon) before
|
|
142
|
+
# the renderer ever read it. `dist/` already contains those assets by the
|
|
143
|
+
# time this runs, so the replacement is complete.
|
|
144
|
+
- npm run build
|
|
145
|
+
- rm -rf public && mv dist public
|
|
146
|
+
artifacts:
|
|
147
|
+
paths:
|
|
148
|
+
- public
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# The index stores pointers; README, changelog, logo and version list all
|
|
2
|
+
# live in the registry, and grim is what reads a registry. Best-effort by
|
|
3
|
+
# design - a registry outage degrades the site to pointers-only rather
|
|
4
|
+
# than blocking a deploy, so the failure is logged and the build
|
|
5
|
+
# continues. Set `"enrich": false` in the `ci` block to drop this block.
|
|
6
|
+
#
|
|
7
|
+
# Nothing is committed and no token is involved: the sidecars live only
|
|
8
|
+
# in this job's workspace, which is why this needs no CI variables.
|
|
9
|
+
if command -v apk >/dev/null 2>&1; then
|
|
10
|
+
apk add --no-cache curl tar
|
|
11
|
+
elif command -v apt-get >/dev/null 2>&1; then
|
|
12
|
+
apt-get update -qq && apt-get install -y -qq --no-install-recommends curl tar ca-certificates
|
|
13
|
+
fi
|
|
14
|
+
# Alpine is musl; everything else here is glibc.
|
|
15
|
+
if [ -f /etc/alpine-release ]; then libc=musl; else libc=gnu; fi
|
|
16
|
+
# Resolved from `ci.grimVersion` when this file was rendered, so there is
|
|
17
|
+
# no version branch left to take at run time.
|
|
18
|
+
base="{{grimReleaseBase}}"
|
|
19
|
+
tarball="grimoire-$(uname -m)-unknown-linux-$libc.tar.gz"
|
|
20
|
+
tmp="$(mktemp -d)"
|
|
21
|
+
( cd "$tmp" \
|
|
22
|
+
&& curl -fsSL --proto '=https' --tlsv1.2 -O "$base/$tarball" -O "$base/$tarball.sha256" \
|
|
23
|
+
&& sha256sum -c "$tarball.sha256" \
|
|
24
|
+
&& tar -xzf "$tarball" -C /usr/local/bin --strip-components=1 "${tarball%.tar.gz}/grim" ) \
|
|
25
|
+
&& npm run enrich \
|
|
26
|
+
|| echo "warning: enrich failed - building without READMEs, logos and version lists"
|