wachiman 0.0.0-stage → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +105 -0
- package/README.md +178 -2
- package/bin/launcher-args.js +179 -0
- package/bin/wachiman.js +10 -0
- package/bunfig.toml +6 -0
- package/package.json +40 -4
- package/skills/wachiman-audit/SKILL.md +80 -0
- package/skills/wachiman-review/SKILL.md +105 -0
- package/src/accounts/commands.ts +680 -0
- package/src/accounts/config.ts +322 -0
- package/src/accounts/confirm.ts +231 -0
- package/src/accounts/profile-list.ts +70 -0
- package/src/accounts/resolve.ts +152 -0
- package/src/call-message.ts +54 -0
- package/src/cli.ts +299 -0
- package/src/config.ts +211 -0
- package/src/connectors/local-git.ts +473 -0
- package/src/contracts.ts +246 -0
- package/src/cost.ts +180 -0
- package/src/doctor/checks.ts +434 -0
- package/src/doctor/render.ts +76 -0
- package/src/doctor/run.ts +157 -0
- package/src/engine.ts +3653 -0
- package/src/evidence.ts +739 -0
- package/src/flags.ts +44 -0
- package/src/formats.ts +319 -0
- package/src/history/command.ts +175 -0
- package/src/history/fingerprint.ts +58 -0
- package/src/history/index.ts +103 -0
- package/src/history/match.ts +181 -0
- package/src/history/store.ts +325 -0
- package/src/history/types.ts +69 -0
- package/src/init/args.ts +27 -0
- package/src/init/options.ts +124 -0
- package/src/init/prompt.ts +51 -0
- package/src/init/render.ts +31 -0
- package/src/init/run.ts +152 -0
- package/src/init/write.ts +114 -0
- package/src/install-claude-code.sh +85 -0
- package/src/install-kind.ts +50 -0
- package/src/json-schema.ts +95 -0
- package/src/local.ts +403 -0
- package/src/precision.ts +175 -0
- package/src/profiles.ts +157 -0
- package/src/providers/claude-code.settings.json +5 -0
- package/src/providers/claude-code.ts +406 -0
- package/src/providers/codex.ts +373 -0
- package/src/providers/errors.ts +64 -0
- package/src/providers/index.ts +54 -0
- package/src/providers/locate.ts +15 -0
- package/src/providers/types.ts +116 -0
- package/src/questions/bounded-wait.ts +68 -0
- package/src/questions/destructive-step.ts +74 -0
- package/src/questions/index.ts +55 -0
- package/src/questions/overlapping-calls.ts +63 -0
- package/src/questions/read-error-as-empty.ts +138 -0
- package/src/questions/stale-write-back.ts +91 -0
- package/src/questions/types.ts +14 -0
- package/src/setup/args.ts +16 -0
- package/src/setup/options.ts +32 -0
- package/src/setup/prompt.ts +170 -0
- package/src/setup/run.ts +219 -0
- package/src/sinks/agent-prompt.ts +18 -0
- package/src/sinks/common.ts +238 -0
- package/src/sinks/json.ts +9 -0
- package/src/sinks/markdown.ts +87 -0
- package/src/sinks/terminal.ts +55 -0
- package/src/skills/install.ts +379 -0
- package/src/verify.ts +413 -0
- package/src/wizard/interactive.ts +16 -0
- package/src/wizard/prompt.ts +176 -0
- package/templates/caller-workflow.yml +308 -0
- package/templates/wachiman-review-caller.yml +34 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Functional Source License, Version 1.1, ALv2 Future License
|
|
2
|
+
|
|
3
|
+
## Abbreviation
|
|
4
|
+
|
|
5
|
+
FSL-1.1-ALv2
|
|
6
|
+
|
|
7
|
+
## Notice
|
|
8
|
+
|
|
9
|
+
Copyright 2026 Niway S.A.C.
|
|
10
|
+
|
|
11
|
+
## Terms and Conditions
|
|
12
|
+
|
|
13
|
+
### Licensor ("We")
|
|
14
|
+
|
|
15
|
+
The party offering the Software under these Terms and Conditions.
|
|
16
|
+
|
|
17
|
+
### The Software
|
|
18
|
+
|
|
19
|
+
The "Software" is each version of the software that we make available under
|
|
20
|
+
these Terms and Conditions, as indicated by our inclusion of these Terms and
|
|
21
|
+
Conditions with the Software.
|
|
22
|
+
|
|
23
|
+
### License Grant
|
|
24
|
+
|
|
25
|
+
Subject to your compliance with this License Grant and the Patents,
|
|
26
|
+
Redistribution and Trademark clauses below, we hereby grant you the right to
|
|
27
|
+
use, copy, modify, create derivative works, publicly perform, publicly display
|
|
28
|
+
and redistribute the Software for any Permitted Purpose identified below.
|
|
29
|
+
|
|
30
|
+
### Permitted Purpose
|
|
31
|
+
|
|
32
|
+
A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
|
|
33
|
+
means making the Software available to others in a commercial product or
|
|
34
|
+
service that:
|
|
35
|
+
|
|
36
|
+
1. substitutes for the Software;
|
|
37
|
+
|
|
38
|
+
2. substitutes for any other product or service we offer using the Software
|
|
39
|
+
that exists as of the date we make the Software available; or
|
|
40
|
+
|
|
41
|
+
3. offers the same or substantially similar functionality as the Software.
|
|
42
|
+
|
|
43
|
+
Permitted Purposes specifically include using the Software:
|
|
44
|
+
|
|
45
|
+
1. for your internal use and access;
|
|
46
|
+
|
|
47
|
+
2. for non-commercial education;
|
|
48
|
+
|
|
49
|
+
3. for non-commercial research; and
|
|
50
|
+
|
|
51
|
+
4. in connection with professional services that you provide to a licensee
|
|
52
|
+
using the Software in accordance with these Terms and Conditions.
|
|
53
|
+
|
|
54
|
+
### Patents
|
|
55
|
+
|
|
56
|
+
To the extent your use for a Permitted Purpose would necessarily infringe our
|
|
57
|
+
patents, the license grant above includes a license under our patents. If you
|
|
58
|
+
make a claim against any party that the Software infringes or contributes to
|
|
59
|
+
the infringement of any patent, then your patent license to the Software ends
|
|
60
|
+
immediately.
|
|
61
|
+
|
|
62
|
+
### Redistribution
|
|
63
|
+
|
|
64
|
+
The Terms and Conditions apply to all copies, modifications and derivatives of
|
|
65
|
+
the Software.
|
|
66
|
+
|
|
67
|
+
If you redistribute any copies, modifications or derivatives of the Software,
|
|
68
|
+
you must include a copy of or a link to these Terms and Conditions and not
|
|
69
|
+
remove any copyright notices provided in or with the Software.
|
|
70
|
+
|
|
71
|
+
### Disclaimer
|
|
72
|
+
|
|
73
|
+
THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
|
|
74
|
+
IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
|
|
75
|
+
PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
|
|
76
|
+
|
|
77
|
+
IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
|
|
78
|
+
SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
|
|
79
|
+
EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
|
|
80
|
+
|
|
81
|
+
### Trademarks
|
|
82
|
+
|
|
83
|
+
Except for displaying the License Details and identifying us as the origin of
|
|
84
|
+
the Software, you have no right under these Terms and Conditions to use our
|
|
85
|
+
trademarks, trade names, service marks or product names.
|
|
86
|
+
|
|
87
|
+
## Grant of Future License
|
|
88
|
+
|
|
89
|
+
We hereby irrevocably grant you an additional license to use the Software under
|
|
90
|
+
the Apache License, Version 2.0 that is effective on the second anniversary of
|
|
91
|
+
the date we make the Software available. On or after that date, you may use the
|
|
92
|
+
Software under the Apache License, Version 2.0, in which case the following
|
|
93
|
+
will apply:
|
|
94
|
+
|
|
95
|
+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
|
|
96
|
+
this file except in compliance with the License.
|
|
97
|
+
|
|
98
|
+
You may obtain a copy of the License at
|
|
99
|
+
|
|
100
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
101
|
+
|
|
102
|
+
Unless required by applicable law or agreed to in writing, software distributed
|
|
103
|
+
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
|
|
104
|
+
CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
105
|
+
specific language governing permissions and limitations under the License.
|
package/README.md
CHANGED
|
@@ -1,3 +1,179 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Wachiman
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
<img src="docs/assets/wachi.png" alt="Wachiman, a cartoon watchman with a cap and a whistle" width="96" align="right">
|
|
4
|
+
|
|
5
|
+
Advisory pull request reviews by an agent CLI, run inside your own GitHub Actions with your own
|
|
6
|
+
API key. Wachiman (pronounced *watchman*) is named after the Peruvian *wachimán*, the street
|
|
7
|
+
watchman who warns you and lets you decide; the Spanish word comes from the English one.
|
|
8
|
+
|
|
9
|
+
## What it is
|
|
10
|
+
|
|
11
|
+
A GitHub Actions workflow that reads a pull request and posts one GitHub review on it. Each
|
|
12
|
+
finding is an inline comment on its line, labelled with a severity (when to act: Blocker, Fix,
|
|
13
|
+
Consider) and a topic (what it is about: bug, docs, security…), for example `🟠📝 Fix · docs`.
|
|
14
|
+
Information that asks for no reply, such as a notice that the pull request edits a decision
|
|
15
|
+
record, goes in the review body. Every comment opens with "👮 **Wachiman**".
|
|
16
|
+
|
|
17
|
+
- **It runs in your repository.** The workflow runs on your runners and spends your own provider
|
|
18
|
+
key. This project operates no service that receives your code or your key.
|
|
19
|
+
- **It is advisory.** The review is posted as a plain comment review. It never approves,
|
|
20
|
+
requests changes or blocks a merge, and "Blocker" is a label on a comment, not a status
|
|
21
|
+
GitHub enforces.
|
|
22
|
+
- **It starts on a pull request and on request.** It reviews when a pull request opens or leaves
|
|
23
|
+
draft, and, if your caller lists `synchronize`, on each push (only the new commits). The operator
|
|
24
|
+
you name can comment `/wachiman review` to ask for another review of what changed, or any other
|
|
25
|
+
`/wachiman …` text to ask a question, and a reply the operator writes in a review thread gets an
|
|
26
|
+
answer without a push. All automatic runs on one pull request together stop at $6 by default
|
|
27
|
+
(`auto_budget_usd`); `/wachiman review` is not limited. Details: [docs/reviews.md](docs/reviews.md).
|
|
28
|
+
- **Large pull requests are reviewed in batches**, riskiest files first (the risk is a score
|
|
29
|
+
computed from file paths, not a reading of the code). Each batch is one agent call with a
|
|
30
|
+
spend limit. What that limit does and does not guarantee, and what "all files reviewed"
|
|
31
|
+
establishes, is in [docs/costs.md](docs/costs.md).
|
|
32
|
+
|
|
33
|
+
## What it will not do
|
|
34
|
+
|
|
35
|
+
- It does not approve, request changes, gate a merge or replace a human review.
|
|
36
|
+
- It does not run your code or tests. In the Claude Code workflow the agent is started with
|
|
37
|
+
read-only tools and no shell; that is a tested configuration, not a sandbox
|
|
38
|
+
([SECURITY.md](SECURITY.md)).
|
|
39
|
+
- It does not check every changed file in every case: lockfiles, build output and generated files
|
|
40
|
+
are dropped first, and coverage is what the model reports per file. If some files are not
|
|
41
|
+
reviewed, the first line of the review says so.
|
|
42
|
+
- It is not a sandbox, and it is not a tool for reviewing code from people you do not trust (see
|
|
43
|
+
below).
|
|
44
|
+
|
|
45
|
+
## The trust decision, in brief
|
|
46
|
+
|
|
47
|
+
Installing it gives a workflow your model API key and read access to your pull requests. Before
|
|
48
|
+
you do:
|
|
49
|
+
|
|
50
|
+
- **Everyone who can push a branch to the repository must be someone you trust with the key.** On
|
|
51
|
+
a pull request GitHub runs the workflow file from the pull request's own branch, so a person
|
|
52
|
+
who can push can change the workflow and read the key. Nothing in this tool defends against
|
|
53
|
+
that. Fork pull requests are skipped.
|
|
54
|
+
- **Your code goes to the model provider.** The review sends the pull request's diff, and the
|
|
55
|
+
files the agent chooses to read, to the provider you configured (Anthropic, for Claude Code)
|
|
56
|
+
from your runner.
|
|
57
|
+
- **The version you pin decides what runs with your key.** See
|
|
58
|
+
[docs/install.md](docs/install.md#what-pinning-a-version-does-and-does-not-guarantee).
|
|
59
|
+
|
|
60
|
+
This is not a sandbox. What is enforced, what the tests show and what is still unverified:
|
|
61
|
+
[SECURITY.md](SECURITY.md).
|
|
62
|
+
|
|
63
|
+
## Status and providers
|
|
64
|
+
|
|
65
|
+
The current release is **v1.5.0** ([CHANGELOG.md](CHANGELOG.md)). The command-line contract
|
|
66
|
+
([docs/cli.md](docs/cli.md)) and the provider adapter boundary are in place. The older
|
|
67
|
+
`CLAUDE_REVIEW_*` variables, `.github/agent-review.md`, `.github/claude-review.md`, `/wachi` and the `@claude` mention are still read
|
|
68
|
+
with a notice; see [docs/upgrading.md](docs/upgrading.md).
|
|
69
|
+
|
|
70
|
+
- **Claude Code** (`claude -p`) is the provider the reusable workflow installs and runs, with
|
|
71
|
+
`ANTHROPIC_API_KEY` ([docs/providers/claude-code.md](docs/providers/claude-code.md)).
|
|
72
|
+
- **Codex CLI** (`codex exec`, for OpenAI models) is a second adapter, available only from the
|
|
73
|
+
command line. The reusable workflow does not install it or pass an OpenAI key
|
|
74
|
+
([docs/providers/codex.md](docs/providers/codex.md)).
|
|
75
|
+
- **npm:** the package `wachiman` is built and tested for npm (a Node launcher; the package installs
|
|
76
|
+
Bun through its `bun` dependency;
|
|
77
|
+
[docs/local-review.md](docs/local-review.md#3-install)) and is published by the release workflow
|
|
78
|
+
from the first release that includes this change. Until the first publish succeeds, install from
|
|
79
|
+
source (`scripts/install-local.sh`). macOS and Linux are covered by CI; **Windows: pending CI**.
|
|
80
|
+
|
|
81
|
+
## Install
|
|
82
|
+
|
|
83
|
+
The recommended way is the reusable workflow: a short caller file in your repository pins a
|
|
84
|
+
release, passes your key and names the operator.
|
|
85
|
+
|
|
86
|
+
**Start here: [docs/install.md](docs/install.md)**, the first advisory review from an empty
|
|
87
|
+
repository to a posted review. The caller file itself is
|
|
88
|
+
[`templates/wachiman-review-caller.yml`](templates/wachiman-review-caller.yml).
|
|
89
|
+
|
|
90
|
+
Already copied the engine into a repository? That setup still runs, and
|
|
91
|
+
[docs/setup-copy.md](docs/setup-copy.md) explains how to move to the maintained workflow.
|
|
92
|
+
|
|
93
|
+
### Review locally on your own subscription
|
|
94
|
+
|
|
95
|
+
`wachiman local` reviews the branch you have checked out, on your own signed-in `claude`, and
|
|
96
|
+
prints the review in the terminal or writes it as Markdown or JSON. No API key, no GitHub token,
|
|
97
|
+
nothing posted: Wachiman sends no telemetry and makes no GitHub requests in local mode; your code
|
|
98
|
+
is sent to your configured Claude account for the review. Read
|
|
99
|
+
[docs/local-review.md](docs/local-review.md) before running it on an employer's code.
|
|
100
|
+
|
|
101
|
+
Install it from source: run `scripts/install-local.sh` once from your clone. The npm package
|
|
102
|
+
(`npm i -g wachiman` or `bun add -g wachiman`; it is written to work on Windows (Git Bash, PowerShell) and is tested on `windows-latest` in CI, and the Windows tier stays pending until that job has passed on a published commit: it ships the
|
|
103
|
+
TypeScript source and a small Node launcher, and it installs Bun for you by depending on the
|
|
104
|
+
official npm package `bun`, which means installing it downloads Bun, 23 to 42 MB compressed; with
|
|
105
|
+
`--ignore-scripts` it uses a Bun already on your `PATH`; details and the supply-chain note are in
|
|
106
|
+
[docs/local-review.md](docs/local-review.md#from-npm)) is published from the first release that
|
|
107
|
+
includes this change; until the first publish succeeds, use the script. Both give a global `wachiman local …`
|
|
108
|
+
command; the script also links the `wachiman-review` agent skill, usable from any project, and the
|
|
109
|
+
`wachiman-audit` skill, which compares Wachiman's findings with another reviewer's on the same
|
|
110
|
+
commits (data and procedure in
|
|
111
|
+
[wachiman-benchmarks](https://github.com/niway-dev/wachiman-benchmarks)). Details and the Windows
|
|
112
|
+
status are in [docs/local-review.md](docs/local-review.md#3-install).
|
|
113
|
+
|
|
114
|
+
## Documentation map
|
|
115
|
+
|
|
116
|
+
| I want to… | Read |
|
|
117
|
+
| --- | --- |
|
|
118
|
+
| Get a first review on a repository | [docs/install.md](docs/install.md) |
|
|
119
|
+
| Know what I am trusting, and what is unverified | [SECURITY.md](SECURITY.md) |
|
|
120
|
+
| Look up an input, a default or an environment setting | [docs/configuration.md](docs/configuration.md) |
|
|
121
|
+
| Find out why there was no review, or an error | [docs/troubleshooting.md](docs/troubleshooting.md) |
|
|
122
|
+
| Ask for reviews, scope them, read the results | [docs/reviews.md](docs/reviews.md) |
|
|
123
|
+
| Understand what a run can cost and what coverage means | [docs/costs.md](docs/costs.md) |
|
|
124
|
+
| Tell the reviewer what matters in my repository | [docs/review-policy.md](docs/review-policy.md) |
|
|
125
|
+
| Adopt a new release, or roll one back | [docs/upgrading.md](docs/upgrading.md) |
|
|
126
|
+
| Turn it off and remove it | [docs/uninstall.md](docs/uninstall.md) |
|
|
127
|
+
| Quick answers and limitations | [docs/faq.md](docs/faq.md) |
|
|
128
|
+
| Run the engine from the command line | [docs/cli.md](docs/cli.md) |
|
|
129
|
+
| Review a branch on my laptop, on my own Claude subscription | [docs/local-review.md](docs/local-review.md) |
|
|
130
|
+
| See what a provider adapter runs and inherits | [docs/providers/claude-code.md](docs/providers/claude-code.md), [docs/providers/codex.md](docs/providers/codex.md) |
|
|
131
|
+
| Move from a copied installation | [docs/setup-copy.md](docs/setup-copy.md) |
|
|
132
|
+
| Understand how a review is built | [docs/architecture.md](docs/architecture.md) |
|
|
133
|
+
| Measure false positives | [docs/precision.md](docs/precision.md) |
|
|
134
|
+
| See how it compares with Pullfrog and CodeRabbit | [wachiman-benchmarks](https://github.com/niway-dev/wachiman-benchmarks) |
|
|
135
|
+
| See what changed in a release | [CHANGELOG.md](CHANGELOG.md) |
|
|
136
|
+
| Cut a release (maintainers) | [docs/releasing.md](docs/releasing.md) |
|
|
137
|
+
|
|
138
|
+
Design decisions are recorded as ADRs in [docs/adr/](docs/adr/), starting with
|
|
139
|
+
[ADR 0001: distribution and trust](docs/adr/0001-distribution-and-trust.md) and
|
|
140
|
+
[ADR 0002: TypeScript on Bun, no runtime dependencies besides Bun itself](docs/adr/0002-typescript-on-bun-zero-dependencies.md).
|
|
141
|
+
|
|
142
|
+
## License
|
|
143
|
+
|
|
144
|
+
Wachiman is **source available**, not open source. It is licensed under the
|
|
145
|
+
[Functional Source License, Version 1.1, ALv2 Future License](LICENSE) (`FSL-1.1-ALv2`), a
|
|
146
|
+
[Fair Source](https://fair.io/) license. Copyright 2026 Niway S.A.C.
|
|
147
|
+
|
|
148
|
+
- **You may** use it for any purpose except a Competing Use: run it on your own repositories and
|
|
149
|
+
in your company, read and modify it, fork it for internal use, use it for non-commercial
|
|
150
|
+
education and research, and use it to give professional services to a client who may also
|
|
151
|
+
use it.
|
|
152
|
+
- **You may not**, without a commercial license, make it available to others as a commercial
|
|
153
|
+
product or service that does what Wachiman does, hosted or packaged.
|
|
154
|
+
- **Each version published under FSL-1.1-ALv2 also becomes available under Apache 2.0 two years
|
|
155
|
+
after that version is made available.** The license says so under "Grant of Future License".
|
|
156
|
+
- **Commercial license:** write to contact@niway.com. There is no price list.
|
|
157
|
+
- **Name and logo:** the license grants no rights to them. See [TRADEMARKS.md](TRADEMARKS.md).
|
|
158
|
+
|
|
159
|
+
### Licensing history
|
|
160
|
+
|
|
161
|
+
Everything published before the commit that replaced LICENSE, including releases through v1.3.0,
|
|
162
|
+
remains available under GNU AGPL version 3 or any later version, as stated in those versions'
|
|
163
|
+
LICENSE files. The
|
|
164
|
+
`LICENSE` file inside each release tag says which license that release carries. The reason for the
|
|
165
|
+
change is recorded in [ADR 0013](docs/adr/0013-license-fsl-1-1-alv2.md).
|
|
166
|
+
|
|
167
|
+
Benchmark reports and the review dataset live in
|
|
168
|
+
[niway-dev/wachiman-benchmarks](https://github.com/niway-dev/wachiman-benchmarks); they moved there
|
|
169
|
+
from `docs/benchmark/` on 2026-11-02. Quoted third-party text in them, and in this repository's
|
|
170
|
+
history, belongs to its authors and is reproduced for research and comparison; it is not covered by
|
|
171
|
+
the license above.
|
|
172
|
+
|
|
173
|
+
## Commit messages
|
|
174
|
+
|
|
175
|
+
Releases and the changelog are generated from commit messages, so pull request titles (squash
|
|
176
|
+
merges use them) follow [Conventional Commits](https://www.conventionalcommits.org/):
|
|
177
|
+
`feat:` is a minor release, `fix:` and `perf:` a patch, `feat!:` or a `BREAKING CHANGE:` footer a
|
|
178
|
+
major one. `docs:`, `test:`, `ci:`, `build:`, `chore:` and `style:` appear in no release notes.
|
|
179
|
+
Release pull requests are titled `chore: release X.Y.Z`.
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
// The launcher's logic, in one dependency-free module (Node built-ins only). `bin/wachiman.js`
|
|
2
|
+
// calls `launch`; the Bun tests import this same file, so no second copy of the argv exists.
|
|
3
|
+
//
|
|
4
|
+
// The launcher's one job is to make the npm install portable. It never changes the environment,
|
|
5
|
+
// never turns a variable into a flag, and never downloads or installs anything: the Bun it runs is
|
|
6
|
+
// the one the package's own `bun` dependency installed, or one already on PATH.
|
|
7
|
+
import { spawn as nodeSpawn, spawnSync as nodeSpawnSync } from "node:child_process";
|
|
8
|
+
import { existsSync } from "node:fs";
|
|
9
|
+
import { createRequire } from "node:module";
|
|
10
|
+
import { constants } from "node:os";
|
|
11
|
+
import { dirname, join, posix, win32 } from "node:path";
|
|
12
|
+
|
|
13
|
+
export const BUN_INSTALL_LINE = "wachiman needs Bun 1.3 or later: https://bun.sh";
|
|
14
|
+
export const MIN_BUN = [1, 3, 0];
|
|
15
|
+
|
|
16
|
+
/**
|
|
17
|
+
* Pure. The argv the launcher gives to `bun`: the isolation flags first (Bun reads `.env` and a
|
|
18
|
+
* `bunfig.toml`, which can run code, from its working directory, and that directory is the
|
|
19
|
+
* reviewed repository), then the package's own entry point, then the user's arguments untouched.
|
|
20
|
+
*/
|
|
21
|
+
export function launcherArgs(pkgDir, argv) {
|
|
22
|
+
return ["--no-env-file", `--config=${join(pkgDir, "bunfig.toml")}`, join(pkgDir, "src", "cli.ts"), ...argv];
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
/**
|
|
26
|
+
* Where the official npm package `bun` (a regular dependency of this package) keeps its binary:
|
|
27
|
+
* `<package>/bin/bun.exe`, on every OS. Resolved from this file's own location through Node's
|
|
28
|
+
* module lookup, never from the working directory, so a `node_modules/bun` inside the reviewed
|
|
29
|
+
* repository is never reached. Null when the package is not installed. The file may still be the
|
|
30
|
+
* placeholder the package ships when its postinstall did not run (`--ignore-scripts`): see `probeBun`.
|
|
31
|
+
*/
|
|
32
|
+
export function bundledBunPath(resolve = createRequire(import.meta.url).resolve) {
|
|
33
|
+
try {
|
|
34
|
+
return join(dirname(resolve("bun/package.json")), "bin", "bun.exe");
|
|
35
|
+
} catch {
|
|
36
|
+
return null;
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
/** Pure. `[major, minor, patch]` from the first line of `bun --version` output, or null. */
|
|
41
|
+
export function parseBunVersion(text) {
|
|
42
|
+
const match = /^v?(\d+)\.(\d+)\.(\d+)/.exec(String(text ?? "").trim());
|
|
43
|
+
return match ? [Number(match[1]), Number(match[2]), Number(match[3])] : null;
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/** Pure. True when version `a` is the same as or later than version `b` (both `[major, minor, patch]`). */
|
|
47
|
+
export function versionAtLeast(a, b = MIN_BUN) {
|
|
48
|
+
for (let i = 0; i < 3; i++) if (a[i] !== b[i]) return a[i] > b[i];
|
|
49
|
+
return true;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
/**
|
|
53
|
+
* Runs `<binary> --version` and judges the answer. A file that exists is not enough: with
|
|
54
|
+
* `--ignore-scripts` the `bun` package leaves a placeholder in `bin/bun.exe` that cannot run. Returns `{ ok: true, version }`, `{ ok: false, reason: "old", version }`
|
|
55
|
+
* `{ ok: false, reason: "timeout" }` or `{ ok: false, reason: "unusable" }`. No shell, a timeout generous
|
|
56
|
+
* enough for the first start of a freshly installed binary, the caller's environment.
|
|
57
|
+
*/
|
|
58
|
+
export function probeBun(binary, { env, cwd, spawnSync = nodeSpawnSync, timeout = 15000 }) {
|
|
59
|
+
let result;
|
|
60
|
+
try {
|
|
61
|
+
result = spawnSync(binary, ["--version"], { shell: false, timeout, env, cwd, encoding: "utf8", stdio: ["ignore", "pipe", "ignore"], windowsHide: true });
|
|
62
|
+
} catch {
|
|
63
|
+
return { ok: false, reason: "unusable" };
|
|
64
|
+
}
|
|
65
|
+
if (result?.error?.code === "ETIMEDOUT" || result?.signal === "SIGTERM") return { ok: false, reason: "timeout" };
|
|
66
|
+
if (!result || result.error || result.status !== 0) return { ok: false, reason: "unusable" };
|
|
67
|
+
const version = parseBunVersion(String(result.stdout ?? "").split("\n")[0]);
|
|
68
|
+
if (version === null) return { ok: false, reason: "unusable" };
|
|
69
|
+
const text = version.join(".");
|
|
70
|
+
return versionAtLeast(version) ? { ok: true, version: text } : { ok: false, reason: "old", version: text };
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
/**
|
|
74
|
+
* Pure apart from `exists`. The first `bun` found on PATH, or null. Only absolute entries count,
|
|
75
|
+
* on every OS: an empty or relative entry (or ".") means the working directory, which is the
|
|
76
|
+
* reviewed repository, and a binary there must never run. On Windows a bare name would also be
|
|
77
|
+
* looked up in the working directory first, so the PATH is always scanned here. Each entry may
|
|
78
|
+
* hold `bun` (`bun.exe` on Windows) or, on Windows, the npm-global layout
|
|
79
|
+
* `node_modules\bun\bin\bun.exe` that `npm install -g bun` creates next to its `bun.cmd` shim.
|
|
80
|
+
*/
|
|
81
|
+
export function resolveBun(platform, env, exists = existsSync) {
|
|
82
|
+
const windows = platform === "win32";
|
|
83
|
+
const path = windows ? win32 : posix;
|
|
84
|
+
const key = Object.keys(env).find((name) => (windows ? name.toLowerCase() === "path" : name === "PATH"));
|
|
85
|
+
const value = key === undefined ? "" : env[key] ?? "";
|
|
86
|
+
for (const raw of value.split(windows ? ";" : ":")) {
|
|
87
|
+
const entry = windows ? raw.trim().replace(/^"(.*)"$/, "$1") : raw;
|
|
88
|
+
if (entry === "" || !path.isAbsolute(entry)) continue;
|
|
89
|
+
const candidates = windows ? [path.join(entry, "bun.exe"), path.join(entry, "node_modules", "bun", "bin", "bun.exe")] : [path.join(entry, "bun")];
|
|
90
|
+
for (const candidate of candidates) if (exists(candidate)) return candidate;
|
|
91
|
+
}
|
|
92
|
+
return null;
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
/**
|
|
96
|
+
* The Bun to run: the bundled one when it works, else a Bun on PATH when it works, else the
|
|
97
|
+
* reason there is none. Returns `{ binary, version }`, or `{ binary: null, old, failed }` where `old`
|
|
98
|
+
* is the `{ version, path }` of the first Bun that runs but is older than 1.3 (null when none) and
|
|
99
|
+
* `failed` is the path of the first candidate that did not run at all (a placeholder, an error or
|
|
100
|
+
* a timeout), undefined when there was none.
|
|
101
|
+
*/
|
|
102
|
+
export function findBun({ platform, env, cwd, exists = existsSync, spawnSync = nodeSpawnSync, bundled = bundledBunPath }) {
|
|
103
|
+
let old = null;
|
|
104
|
+
let failed;
|
|
105
|
+
const candidates = [bundled(), resolveBun(platform, env, exists)].filter((path) => path !== null && path !== undefined);
|
|
106
|
+
for (const binary of [...new Set(candidates)]) {
|
|
107
|
+
const probe = probeBun(binary, { env, cwd, spawnSync });
|
|
108
|
+
if (probe.ok) return { binary, version: probe.version };
|
|
109
|
+
if (probe.reason === "old" && old === null) old = { version: probe.version, path: binary };
|
|
110
|
+
else if (probe.reason !== "old" && failed === undefined) failed = binary;
|
|
111
|
+
}
|
|
112
|
+
return { binary: null, old, failed };
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
/** Pure. The stderr lines for "no usable Bun": which case it is, then the commands. */
|
|
116
|
+
export function missingBunLines(found, platform) {
|
|
117
|
+
const first = found.old
|
|
118
|
+
? `wachiman needs Bun 1.3 or later, but found Bun ${found.old.version} (${found.old.path}): https://bun.sh`
|
|
119
|
+
: BUN_INSTALL_LINE;
|
|
120
|
+
const lines = [
|
|
121
|
+
first,
|
|
122
|
+
found.old
|
|
123
|
+
? "Upgrade it, or install a newer one:"
|
|
124
|
+
: found.failed
|
|
125
|
+
? `A Bun was found (${found.failed}) but did not run (an install with --ignore-scripts leaves a placeholder that cannot run; a first start can also be slow, try again). Install or repair one:`
|
|
126
|
+
: "No Bun was found: the `bun` package that npm installs with wachiman is missing and no Bun is on PATH. Install one:",
|
|
127
|
+
" npm install -g bun",
|
|
128
|
+
];
|
|
129
|
+
if (platform === "win32") lines.push(' powershell -c "irm bun.sh/install.ps1|iex" (then open a new terminal)');
|
|
130
|
+
lines.push(" https://bun.sh");
|
|
131
|
+
return lines;
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
/**
|
|
135
|
+
* Runs the engine under Bun and resolves to the exit code. Everything it touches is a parameter,
|
|
136
|
+
* so a test can replace `spawn` and `stderr`; `env` is handed to the child as the same object it
|
|
137
|
+
* was given and is never written to. While the child runs the launcher ignores SIGINT (the
|
|
138
|
+
* terminal already delivers it to the child's process group, and the child cleans up on its own
|
|
139
|
+
* schedule) and forwards SIGTERM and SIGHUP; it exits with the child's code, or 128+n when the
|
|
140
|
+
* child died from signal n. A SIGINT sent to the launcher's pid alone is dropped; use SIGTERM,
|
|
141
|
+
* which is forwarded. The Bun comes from the package's own `bun` dependency or from the absolute
|
|
142
|
+
* PATH entries only (see `findBun`), so a `bun` in the reviewed repository (the cwd) is never run.
|
|
143
|
+
*/
|
|
144
|
+
export function launch({ pkgDir, argv, platform = process.platform, env = process.env, spawn = nodeSpawn, stderr = (line) => console.error(line), exists = existsSync, spawnSync = nodeSpawnSync, bundled = bundledBunPath }) {
|
|
145
|
+
return new Promise((resolve) => {
|
|
146
|
+
const { binary, old, failed } = findBun({ platform, env, cwd: pkgDir, exists, spawnSync, bundled });
|
|
147
|
+
if (binary === null) {
|
|
148
|
+
for (const line of missingBunLines({ old, failed }, platform)) stderr(line);
|
|
149
|
+
resolve(2);
|
|
150
|
+
return;
|
|
151
|
+
}
|
|
152
|
+
const child = spawn(binary, launcherArgs(pkgDir, argv), { stdio: "inherit", env });
|
|
153
|
+
const ignore = () => {};
|
|
154
|
+
const forward = { SIGTERM: () => child.kill("SIGTERM"), SIGHUP: () => child.kill("SIGHUP") };
|
|
155
|
+
process.on("SIGINT", ignore);
|
|
156
|
+
for (const [name, handler] of Object.entries(forward)) process.on(name, handler);
|
|
157
|
+
let done = false;
|
|
158
|
+
const finish = (code) => {
|
|
159
|
+
if (done) return;
|
|
160
|
+
done = true;
|
|
161
|
+
process.off("SIGINT", ignore);
|
|
162
|
+
for (const [name, handler] of Object.entries(forward)) process.off(name, handler);
|
|
163
|
+
resolve(code);
|
|
164
|
+
};
|
|
165
|
+
child.on("error", (error) => {
|
|
166
|
+
if (error.code === "ENOENT") {
|
|
167
|
+
stderr(BUN_INSTALL_LINE);
|
|
168
|
+
finish(2);
|
|
169
|
+
} else {
|
|
170
|
+
stderr(`wachiman: could not start bun: ${error.message}`);
|
|
171
|
+
finish(1);
|
|
172
|
+
}
|
|
173
|
+
});
|
|
174
|
+
child.on("close", (code, signal) => {
|
|
175
|
+
if (code !== null && code !== undefined) finish(code);
|
|
176
|
+
else finish(signal && constants.signals[signal] ? 128 + constants.signals[signal] : 1);
|
|
177
|
+
});
|
|
178
|
+
});
|
|
179
|
+
}
|
package/bin/wachiman.js
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// The npm `bin` for `wachiman` and `agent-review`: finds Bun on PATH and runs this package's
|
|
3
|
+
// TypeScript engine with it. All logic is in ./launcher-args.js.
|
|
4
|
+
import { dirname } from "node:path";
|
|
5
|
+
import { fileURLToPath } from "node:url";
|
|
6
|
+
import { launch } from "./launcher-args.js";
|
|
7
|
+
|
|
8
|
+
// Node resolves the symlink npm puts in `bin/`, so this is the real package directory.
|
|
9
|
+
const pkgDir = dirname(dirname(fileURLToPath(import.meta.url)));
|
|
10
|
+
process.exitCode = await launch({ pkgDir, argv: process.argv.slice(2) });
|
package/bunfig.toml
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
# Engine-owned Bun configuration. The workflow runs the engine with
|
|
2
|
+
# `bun --no-env-file --config=<this file>`, so Bun never reads a bunfig.toml (and its
|
|
3
|
+
# `preload`, which runs code) or a .env file from the reviewed pull request's checkout.
|
|
4
|
+
# Keep this file minimal: no `preload`, no `env`.
|
|
5
|
+
[install]
|
|
6
|
+
auto = "disable"
|
package/package.json
CHANGED
|
@@ -1,6 +1,42 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "wachiman",
|
|
3
|
-
"version": "
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "1.10.0",
|
|
4
|
+
"description": "Advisory pull request reviews by an agent CLI, run in your own GitHub Actions with your own key, or locally on your own Claude subscription",
|
|
5
|
+
"private": false,
|
|
6
|
+
"type": "module",
|
|
7
|
+
"license": "FSL-1.1-ALv2",
|
|
8
|
+
"repository": {
|
|
9
|
+
"type": "git",
|
|
10
|
+
"url": "git+https://github.com/niway-dev/wachiman.git"
|
|
11
|
+
},
|
|
12
|
+
"bin": {
|
|
13
|
+
"wachiman": "bin/wachiman.js",
|
|
14
|
+
"agent-review": "bin/wachiman.js"
|
|
15
|
+
},
|
|
16
|
+
"files": [
|
|
17
|
+
"bin/wachiman.js",
|
|
18
|
+
"bin/launcher-args.js",
|
|
19
|
+
"src/",
|
|
20
|
+
"skills/",
|
|
21
|
+
"templates/",
|
|
22
|
+
"bunfig.toml",
|
|
23
|
+
"LICENSE",
|
|
24
|
+
"README.md",
|
|
25
|
+
"!**/*.test.ts",
|
|
26
|
+
"!**/fixtures/"
|
|
27
|
+
],
|
|
28
|
+
"dependencies": {
|
|
29
|
+
"bun": "1.3.4"
|
|
30
|
+
},
|
|
31
|
+
"engines": {
|
|
32
|
+
"bun": ">=1.3",
|
|
33
|
+
"node": ">=20"
|
|
34
|
+
},
|
|
35
|
+
"publishConfig": {
|
|
36
|
+
"access": "public",
|
|
37
|
+
"provenance": true
|
|
38
|
+
},
|
|
39
|
+
"scripts": {
|
|
40
|
+
"test": "bun test"
|
|
41
|
+
}
|
|
42
|
+
}
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: wachiman-audit
|
|
3
|
+
description: Use when the developer asks to compare Wachiman's findings with CodeRabbit's or Pullfrog's on real pull requests, what one found that the other missed, or whether a day or a release of reviews was worth its cost. Not for reviewing code; that is wachiman-review.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Audit Wachiman against another reviewer
|
|
7
|
+
|
|
8
|
+
## Why
|
|
9
|
+
|
|
10
|
+
Comparing two reviewers by hand mixes things that must stay apart: a finding posted on one commit
|
|
11
|
+
with a finding posted on the next, a live CI run with a rerun on a laptop, a tool that found nothing
|
|
12
|
+
with a tool that never ran. The pairing is mechanical and a script does it; the judgement is not,
|
|
13
|
+
and that is the only part you do.
|
|
14
|
+
|
|
15
|
+
## The one command
|
|
16
|
+
|
|
17
|
+
From a worktree of `niway-dev/wachiman-benchmarks`:
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
bun scripts/pair.ts --repo <owner/name> --prs <a-b[,c]> --other coderabbit|pullfrog --out <YYYY-MM-DD-<topic>>/pairs
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
`--other` accepts only those two tools. Asked about another bot, stop and say that `pair.ts` needs a
|
|
24
|
+
parser for it first; do not compare by hand.
|
|
25
|
+
|
|
26
|
+
It writes `pairs.csv`, `special-cases.csv`, `reviews.csv`, `findings.csv` and a generated `pairs.md`.
|
|
27
|
+
Never edit those files, and never build the comparison from the PR page instead of them.
|
|
28
|
+
|
|
29
|
+
## What a pair is
|
|
30
|
+
|
|
31
|
+
A **pair** is one commit that both tools finished reviewing. Only findings with
|
|
32
|
+
`comparison: paired` in `findings.csv` are compared. A commit one tool reviewed and the other did
|
|
33
|
+
not, a refusal (no credit), a skip (path filters) or a partial review is a **special case**: it is
|
|
34
|
+
reported, with its reason and what would make it comparable, and it is never counted in the
|
|
35
|
+
comparison. A rerun on a laptop at the same commit makes a new row with `basis: rerun`; it goes in
|
|
36
|
+
the special-cases section, not in the pairs.
|
|
37
|
+
|
|
38
|
+
## Judge, then write
|
|
39
|
+
|
|
40
|
+
For every paired finding, add one row to a hand-written `verdicts.csv` next to the pairs:
|
|
41
|
+
|
|
42
|
+
| column | values |
|
|
43
|
+
| --- | --- |
|
|
44
|
+
| `id` | the `findings.csv` id; a comment that bundles several points gets `<id>-1`, `<id>-2` |
|
|
45
|
+
| `verdict` | `verified` (you read the code at that commit), `owner-accepted` (fixed after the comment), `answered` (a question with a checkable answer), `wrong`, `unverified` |
|
|
46
|
+
| `defect_severity` | `high`, `medium`, `low`, `none`: yours, not the tool's label |
|
|
47
|
+
| `difficulty` | 1 to 5: how hard for a human reviewer of the diff to find |
|
|
48
|
+
| `matched_by` | the other tool's finding id on the same defect, or empty |
|
|
49
|
+
| `judge` | the model that decided, e.g. `claude-fable-5-1` |
|
|
50
|
+
| `evidence` | `path:line` at the commit, or the fixing sha |
|
|
51
|
+
|
|
52
|
+
Then write `README.md` in the same folder, in this order:
|
|
53
|
+
|
|
54
|
+
1. **One-minute block**: PRs, pairs, special cases, verified defects, and the one-line answer to
|
|
55
|
+
"what did the other tool find that Wachiman did not", with `basis` and the Wachiman versions
|
|
56
|
+
seen (from `pairs.csv`, not from memory: a footer never shows the version).
|
|
57
|
+
2. **Pairs**: the `pairs.md` table, plus per pair what each tool found and the verdicts.
|
|
58
|
+
3. **Gaps**: for each finding only the other tool made and you verified: what happens, why that
|
|
59
|
+
tool found it (its posted commands or cited files), severity, difficulty.
|
|
60
|
+
4. **Special cases**: the `special-cases.csv` table ordered by what was lost: `refused` first (a
|
|
61
|
+
change merged with no Wachiman review), then `skipped`, `partial`, `unpaired`.
|
|
62
|
+
5. **Cost**: Wachiman's sum from `pairs.csv`. The other tool's cost only when it published one;
|
|
63
|
+
otherwise "not published", never an estimate.
|
|
64
|
+
6. **Limits**: days, PRs, repositories, who judged, what is still `unverified`.
|
|
65
|
+
|
|
66
|
+
Before pushing, check the audited repository's visibility (`gh repo view <owner/name> --json visibility`).
|
|
67
|
+
Public: open the result as a draft PR on `wachiman-benchmarks`. Private: the folder holds counts
|
|
68
|
+
only, no finding text, and the developer confirms before the push. Nothing about a comparison goes
|
|
69
|
+
under `wachiman/docs`.
|
|
70
|
+
|
|
71
|
+
## Never
|
|
72
|
+
|
|
73
|
+
| Tempting | Instead |
|
|
74
|
+
| --- | --- |
|
|
75
|
+
| "Both tools reviewed the PR" | Both reviewed the **commit**: check `pairs.csv` |
|
|
76
|
+
| Counting a laptop rerun with the live runs | `basis: rerun`, special-cases section |
|
|
77
|
+
| Writing the Wachiman version from the release of the day | `wachiman_version` column: #535 ran v1.7.0 on the night v1.8.0 shipped |
|
|
78
|
+
| Guessing the other tool's cost | "not published" |
|
|
79
|
+
| Scoring a finding only one tool made and nobody verified | `unverified`, listed, not scored |
|
|
80
|
+
| Committing a private repository's findings to the benchmarks repo | Counts only: the posted findings quote the code, and wachiman-benchmarks is public |
|