@hrg/gitignore 0.0.0-stage → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +287 -2
- package/js/gitignore.mjs +332 -0
- package/js/glob.mjs +87 -0
- package/package.json +30 -4
package/README.md
CHANGED
|
@@ -1,3 +1,288 @@
|
|
|
1
|
-
#
|
|
1
|
+
# gitignore — one matcher, one set of vectors, many ports
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
The `.gitignore` matcher and the directory-chain lookup that `fs-meta` used, lifted out so that **ports can share
|
|
4
|
+
the format, the vectors and the measurements** instead of each re-deriving them.
|
|
5
|
+
|
|
6
|
+
**These basis projects are vibe coded.** That is said plainly because it is true: the Zig implementation here was
|
|
7
|
+
written with an AI agent, in one long session, against a real tree (Apache Fory) whose `.gitignore` defeated three
|
|
8
|
+
earlier designs. Every hand-crafted improvement is welcome — a faster matcher, a clearer API, a different data
|
|
9
|
+
structure — and so is using this as the **basis for another port or a reimplementation**. The vectors and the
|
|
10
|
+
benchmark exist so that a rewrite can prove it is both faster and still correct; they are not here to protect this
|
|
11
|
+
implementation.
|
|
12
|
+
|
|
13
|
+
## What moves here
|
|
14
|
+
|
|
15
|
+
| from `fs-meta/src` | what it is |
|
|
16
|
+
| ------------------ | -------------------------------------------------------------------------------------- |
|
|
17
|
+
| `gitignore.zig` | pattern parsing, the `Rules` matcher, the segment-DP `globMatch` |
|
|
18
|
+
| `ignore.zig` | the directory chain: which `.gitignore` files govern a directory, flattened and cached |
|
|
19
|
+
|
|
20
|
+
`fs-meta` will depend on this project rather than carry the code. The batch diff, the trio, the engine and the
|
|
21
|
+
watcher stay in `fs-meta`; nothing about a trio belongs here.
|
|
22
|
+
|
|
23
|
+
## Why it is its own project
|
|
24
|
+
|
|
25
|
+
Three problems are the same for every port, and each one is cheaper to solve once:
|
|
26
|
+
|
|
27
|
+
1. **Correctness.** Git's precedence is subtle — last match wins, a deeper file wins, an excluded directory is not
|
|
28
|
+
entered, `**` has three forms, `!` re-includes, a trailing `/` means directories only. A port that gets one wrong
|
|
29
|
+
behaves plausibly and wrongly.
|
|
30
|
+
2. **Performance.** The failures were not mysterious once measured, and every one of them was a shape a synthetic tree
|
|
31
|
+
can reproduce:
|
|
32
|
+
- a **naive recursive** matcher, exponential on `**` with a deep path: ~4,700 ms for a 3,900-entry pass;
|
|
33
|
+
- a **superlinear `decide`** — 20 rules already cost 1,466 ms, because every entry scanned every rule;
|
|
34
|
+
- **anchored `**` rules** left in the general-glob class, each walking ancestor prefixes: 2.97× the no-matcher
|
|
35
|
+
pass;
|
|
36
|
+
- the same rules **with their literal first segment pruned**: 1.6–1.8×, inside budget.
|
|
37
|
+
3. **Comparability.** A Java or JS port cannot know whether its 4 ms is good or 40× worse than it should be. One
|
|
38
|
+
workload, one output format, one number.
|
|
39
|
+
|
|
40
|
+
## Shape
|
|
41
|
+
|
|
42
|
+
```text
|
|
43
|
+
gitignore/
|
|
44
|
+
src/gitignore.zig the matcher (moved from fs-meta)
|
|
45
|
+
src/ignore.zig the chain (moved from fs-meta)
|
|
46
|
+
src/conformance_test.zig the vectors as a test
|
|
47
|
+
src/decide_main.zig the bench contract as a CLI (`ignore-decide`)
|
|
48
|
+
js/ the JavaScript port, exported by package.json as `@hrg/gitignore`
|
|
49
|
+
java/src/ the Java port, packaged by pom.xml as `hr.hrg:gitignore`
|
|
50
|
+
conformance/ vectors every port must pass, the runner's file, and the corpus generator
|
|
51
|
+
bench/ the workloads and the runner
|
|
52
|
+
doc/ the rules as implemented, the porting guide, and where the corpus comes from
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
The **vectors** are JSON: a case is `{ name, ruleFiles, path, isDir, expect }` where each rule file is a `{ dir,
|
|
56
|
+
rules }` pair and `expect` is `include`, `exclude` or `no-opinion` (the chain's three-valued answer). They were
|
|
57
|
+
hand-authored from [`doc/rules.md`](doc/rules.md) and are run by
|
|
58
|
+
[`src/conformance_test.zig`](src/conformance_test.zig), which materialises each case as a real temporary tree and
|
|
59
|
+
answers it through the chain. The file is **not** generated from the Zig implementation: it is written from the
|
|
60
|
+
rules, so a disagreement is a port bug or a finding to report back — the unterminated-class case was the latter, and
|
|
61
|
+
[`conformance/README.md`](conformance/README.md) records the correction.
|
|
62
|
+
|
|
63
|
+
The **bench contract** is one CLI:
|
|
64
|
+
|
|
65
|
+
```text
|
|
66
|
+
<port> --rules <file.gitignore> --paths <file.txt> [--dirs <file>] → one line per path: include|exclude|no-opinion
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
so a port is measured on identical input, and the runner does not need to know the language. `--dirs` takes one
|
|
70
|
+
`0`/`1` per path, so directory-only rules are measured without a second format. The reference is
|
|
71
|
+
[`src/decide_main.zig`](src/decide_main.zig); the workloads are generated, never checked into a repository, and the
|
|
72
|
+
runner verifies the **verdict count equals the path count** before it reports a time.
|
|
73
|
+
|
|
74
|
+
### Building and running it
|
|
75
|
+
|
|
76
|
+
```text
|
|
77
|
+
cd gitignore
|
|
78
|
+
D:\wrk\zig\17\zig.exe build -Doptimize=ReleaseFast # zig-out/bin/ignore-decide
|
|
79
|
+
D:\wrk\zig\17\zig.exe build -Dnaive=true -Doptimize=ReleaseFast # zig-out/bin/ignore-decide-naive
|
|
80
|
+
bun bench/perf.mjs --emit
|
|
81
|
+
bun bench/perf.mjs --port "zig-out/bin/ignore-decide.exe --rules {rules} --paths {paths}" --repeat 5 --label indexed
|
|
82
|
+
bun bench/perf.mjs --port "zig-out/bin/ignore-decide-naive.exe --rules {rules} --paths {paths}" --repeat 5 --label naive
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
`--naive` is the historical design kept as a benchmark mode: every rule of the chain is scanned instead of using the
|
|
86
|
+
per-kind index, and `**` is matched by a recursion that retries the rest of the pattern at every position instead of
|
|
87
|
+
the segment dynamic program. The naive code is **not in the default executable** — nothing it runs references either
|
|
88
|
+
function, so Zig never analyses them there; the tests do call both, which is how they are kept honest — and the naive
|
|
89
|
+
build installs under a distinct name so both can sit in `zig-out/bin`.
|
|
90
|
+
|
|
91
|
+
### Measured
|
|
92
|
+
|
|
93
|
+
ReleaseFast, best of 5, Windows 11, Zig 0.17.0 (2026-10-21). One process per workload, so the fixed cost of process
|
|
94
|
+
start-up, reading the inputs and creating/removing the temporary root (about 12 ms) is included in every row; compare
|
|
95
|
+
within a row, not across rows.
|
|
96
|
+
|
|
97
|
+
| workload | verdicts | indexed ms | naive ms |
|
|
98
|
+
| ---------- | -------: | ---------: | -------: |
|
|
99
|
+
| flat | 1000 | 13.0 | 13.4 |
|
|
100
|
+
| deep | 2000 | 34.8 | 37.3 |
|
|
101
|
+
| doublestar | 2000 | 37.9 | 38.1 |
|
|
102
|
+
| anchored | 2000 | 22.0 | 24.1 |
|
|
103
|
+
| dirs | 1000 | 13.8 | 15.9 |
|
|
104
|
+
| backtrack | 2000 | 89.2 | 558.1 |
|
|
105
|
+
|
|
106
|
+
`backtrack` is the row that makes the recursive matcher's shape visible: a rule with three `**` segments and a path
|
|
107
|
+
that repeats their literal neighbours, so every split is tried and the final `*.g4` never matches. It is 6.3× slower
|
|
108
|
+
in the naive build, and its size is chosen so the row fits in a normal bench run. The other rows are close because
|
|
109
|
+
`**/x` normalises to an indexed class (`**/*.cpp` becomes a suffix rule) and the process start-up dominates 1–2k
|
|
110
|
+
paths at these times.
|
|
111
|
+
|
|
112
|
+
The historical figures in this README — the ~4,700 ms recursive pass, the 1,466 ms superlinear `decide`, the 2.97×
|
|
113
|
+
anchored-`**` pass — were measured on a real 3,900-entry tree against the designs that preceded this one. The table
|
|
114
|
+
above reproduces the **shapes** at a size a gate can afford; it does not reproduce those magnitudes, and a port should
|
|
115
|
+
quote its own numbers with the tree, build and date rather than these.
|
|
116
|
+
|
|
117
|
+
## Status
|
|
118
|
+
|
|
119
|
+
Steps 1–6 of [`plans/next-steps.md`](plans/next-steps.md) are implemented:
|
|
120
|
+
|
|
121
|
+
- the vectors are a gate: [`src/conformance_test.zig`](src/conformance_test.zig) materialises all 34 cases, and a
|
|
122
|
+
generated corpus can be run with `-Dvectors=<file>`;
|
|
123
|
+
- the rules table is [`doc/rules.md`](doc/rules.md), with every vector name in a row and every row linked to a test;
|
|
124
|
+
- the bench contract is a CLI ([`src/decide_main.zig`](src/decide_main.zig)), with `--dirs` for directory-only rules;
|
|
125
|
+
- the historical designs are `-Dnaive=true`, measured against the current one in the table above;
|
|
126
|
+
- the porting guide is [`doc/porting.md`](doc/porting.md);
|
|
127
|
+
- the corpus generator is [`conformance/from-git-check-ignore.mjs`](conformance/from-git-check-ignore.mjs), a
|
|
128
|
+
separate step because it needs a checkout CI will not have.
|
|
129
|
+
|
|
130
|
+
What remains is **publishing** (step 7), scheduled for **0.3.0**: [Publishing](#publishing) is the checklist, and once
|
|
131
|
+
the version is on the registry, `fs-meta`'s path dependency (`../gitignore` in its `build.zig.zon`, `file:` in its
|
|
132
|
+
`package.json`) becomes a version range, the way `hr.hrg:wyhash` is consumed. The Java and JS
|
|
133
|
+
ports still consume the vectors and the bench rather than re-deriving them, and a `bench/compare.mjs` that runs
|
|
134
|
+
several ports in one table is a convenience, not a contract.
|
|
135
|
+
|
|
136
|
+
## Research: where correctness cases come from
|
|
137
|
+
|
|
138
|
+
The matcher's rules are documented, but *cases* are what a port is judged on, and a hand-written dozen is not a
|
|
139
|
+
catalog. The sources are described in [`conformance/README.md`](conformance/README.md), and the one that is automated
|
|
140
|
+
is the checkout:
|
|
141
|
+
|
|
142
|
+
- **git's own tests** — `t/t0008-ignore.sh` and `t/t7063-status-untracked-cache.sh` in git's tree are the
|
|
143
|
+
authoritative behaviour, written as shell with `.gitignore` fixtures; they can be read as cases, not executed.
|
|
144
|
+
- **`.gitignore` files in the wild, with their trees** — a rule set alone does not say whether a path is ignored, but
|
|
145
|
+
a real repository plus `git check-ignore` does. Any checkout with `git` available is a generator: enumerate paths
|
|
146
|
+
with `git ls-files --others --ignored`, pair them with the rules, and keep only cases where git and this matcher
|
|
147
|
+
agree *or* where the disagreement is understood. This is the highest-value source and the one to automate first.
|
|
148
|
+
- **specification-shaped cases** — for every row of the rules table, one case that exercises it and one that
|
|
149
|
+
exercises its boundary (`**/foo`, `foo/**`, `a/**/b`, `!` after a match, a dir-only rule on a file, an escaped `#`,
|
|
150
|
+
a trailing space, a character class with `]` first).
|
|
151
|
+
- **issue trackers** — git's and other tools' ignore bugs are cases already distilled by someone else.
|
|
152
|
+
- **licence and provenance**: cases extracted from git's tests carry git's licence and must be attributed as such;
|
|
153
|
+
generated cases from a checkout must not carry the repository's file contents, only names and rules.
|
|
154
|
+
|
|
155
|
+
The catalog is a *plan*, not a promise: the first version of this project ships the rules table's cases and the
|
|
156
|
+
[`git check-ignore` generator](conformance/from-git-check-ignore.mjs), and a port that finds a disagreement is
|
|
157
|
+
expected to send the case back here.
|
|
158
|
+
|
|
159
|
+
## Using it from Zig
|
|
160
|
+
|
|
161
|
+
```zig
|
|
162
|
+
// The import name is the consumer's choice; `fs-meta` mounts the two source files as `gitignore.zig` and
|
|
163
|
+
// `ignore.zig`, which is why these look like file names.
|
|
164
|
+
const ignore = @import("ignore.zig");
|
|
165
|
+
|
|
166
|
+
var stack = ignore.Stack.init(arena, io, root, true); // no error union; `arena` must outlive the stack
|
|
167
|
+
defer stack.deinit();
|
|
168
|
+
|
|
169
|
+
const verdict = try stack.decide("src/main/java/Foo.java", false);
|
|
170
|
+
// verdict: ?bool — true = excluded, false = re-included, null = no-opinion
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
The library does no file walking of its own: it reads the `.gitignore` files under the root you give it, answers about
|
|
174
|
+
a path, and caches the chains it built. That is what lets a port's own walker use it without adopting its I/O. The
|
|
175
|
+
walker-shaped entry point is `ignored(dir_path, name, is_dir)`; `decide(rel_path, is_dir)` is the three-valued form
|
|
176
|
+
the vectors and the bench CLI need, and it also honours an excluded ancestor (which is why it exists next to the
|
|
177
|
+
per-entry one).
|
|
178
|
+
|
|
179
|
+
## Using it from JavaScript and Java
|
|
180
|
+
|
|
181
|
+
The matcher is packaged for both languages, so a consumer depends on it instead of copying the files.
|
|
182
|
+
|
|
183
|
+
```js
|
|
184
|
+
// JavaScript: package.json exports `.` and `./glob`.
|
|
185
|
+
import { GitIgnoreRules, parseGitignore, readGitignore } from '@hrg/gitignore';
|
|
186
|
+
import { compileSegments, segmentsMatch } from '@hrg/gitignore/glob';
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
The package is `package.json` here (`"type": "module"`, `"sideEffects": false`, `.` → `js/gitignore.mjs`,
|
|
190
|
+
`./glob` → `js/glob.mjs`, `./package.json` → `package.json`); `bun pm pack` ships exactly those two modules plus
|
|
191
|
+
`package.json` and the README, nothing else.
|
|
192
|
+
|
|
193
|
+
```xml
|
|
194
|
+
<!-- Java: hr.hrg:gitignore, resolved the way hr.hrg:wyhash is. -->
|
|
195
|
+
<dependency>
|
|
196
|
+
<groupId>hr.hrg</groupId>
|
|
197
|
+
<artifactId>gitignore</artifactId>
|
|
198
|
+
<version>0.3.0</version>
|
|
199
|
+
</dependency>
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
`pom.xml` builds the three matcher classes — `GitIgnoreFile`, `GitIgnoreRules` and the package-private `Glob` —
|
|
203
|
+
with JDK 25. `bun java/run.mjs` remains the zero-build gate for the port. Until `0.3.0` is published, a machine
|
|
204
|
+
needs the checkout installed once into the local repository:
|
|
205
|
+
|
|
206
|
+
```text
|
|
207
|
+
cd gitignore
|
|
208
|
+
mvn -q install
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
## Publishing
|
|
212
|
+
|
|
213
|
+
`@hrg/gitignore` is published under the `hrg` npm scope; `publishConfig` sets `access` to `public`, so a plain
|
|
214
|
+
`npm publish` publishes it publicly. This release is **0.3.0** — the same version as `build.zig.zon` and
|
|
215
|
+
`pom.xml`. The JavaScript package is plain ESM with no Bun API and no dependencies, so it installs and runs under
|
|
216
|
+
Node as well as Bun (`engines.node` is `>=18`).
|
|
217
|
+
|
|
218
|
+
```bash
|
|
219
|
+
cd gitignore
|
|
220
|
+
bun js/conformance.mjs # 34/34
|
|
221
|
+
bun pm pack --dry-run # package.json, README.md, js/gitignore.mjs, js/glob.mjs — nothing else
|
|
222
|
+
npm publish # from the repository root; publishConfig supplies --access public
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
After it is on the registry, [`fs-meta`](https://github.com/hrgdavor/fs-meta)'s `package.json` should switch its
|
|
226
|
+
`@hrg/gitignore` dependency from the local `file:` link to the published version range (`^0.3.0`), exactly as the
|
|
227
|
+
Java port moves from a local `mvn install` to the released `hr.hrg:gitignore:0.3.0`.
|
|
228
|
+
|
|
229
|
+
## Licence
|
|
230
|
+
|
|
231
|
+
Same as `fs-meta`: `LicenseRef-GPL-3.0-with-Commons-Clause`, `Copyright (c) 2026 Davor Hrg`.
|
|
232
|
+
|
|
233
|
+
## Ports: measured, and what the numbers say
|
|
234
|
+
|
|
235
|
+
Three implementations, the same 34 vectors, the same bytes from `bench/perf.mjs`. Start-up is measured separately (a run
|
|
236
|
+
with an empty path list) and **subtracted**, so the table is the matcher, not the loader:
|
|
237
|
+
|
|
238
|
+
| workload | Zig | JS (hybrid) | JS (regex) | JS (full DP) |
|
|
239
|
+
| -------- | --: | ----------: | ---------: | -----------: |
|
|
240
|
+
| flat | 0.7 ms | 5.2 ms | 0.8 ms | 8.1 ms |
|
|
241
|
+
| deep | 17.6 ms | 115.8 ms | 111.6 ms | 241.9 ms |
|
|
242
|
+
| doublestar | 23.5 ms | 120.5 ms | 75.3 ms | 494.8 ms |
|
|
243
|
+
| anchored | 9.0 ms | 35.3 ms | 32.0 ms | 126.9 ms |
|
|
244
|
+
| dirs | 1.9 ms | 13.7 ms | 6.1 ms | 26.4 ms |
|
|
245
|
+
| **backtrack** | **68.3 ms** | **1370.6 ms** | **2902.4 ms** | **1462.6 ms** |
|
|
246
|
+
| total | **121 ms** | **1661.2 ms (13.9x)** | **3128.2 ms (26.1x)** | **2360.7 ms (19.5x)** |
|
|
247
|
+
|
|
248
|
+
Start-up, for completeness: Zig ~11 ms, JS ~48 ms, Java ~110 ms.
|
|
249
|
+
|
|
250
|
+
The three JS columns are **the same vectors, the same port, three matching strategies**, measured on this machine with
|
|
251
|
+
`--repeat 3` and processing only:
|
|
252
|
+
|
|
253
|
+
- **regex** — one compiled regex per rule. Fast on five workloads, and catastrophic on `backtrack` (2902 ms), where the
|
|
254
|
+
engine explores exponentially many `**` splits.
|
|
255
|
+
- **full DP** — the segment dynamic program for every rule. The total looks good only because `backtrack` dominates: it is
|
|
256
|
+
**1.6x-6x slower on the other five rows**, which the first version of this table hid by reporting only the total.
|
|
257
|
+
- **hybrid** — the DP only for patterns containing a `**` segment, the compiled regex for everything else. Best total
|
|
258
|
+
(1661 ms), `backtrack` down 2.1x, and every other row within ~1.2x of the regex — except `doublestar` (120 vs 75 ms),
|
|
259
|
+
which is a pattern with `**` where the regex happened to be fast. That row is the open tuning question, not a settled
|
|
260
|
+
result.
|
|
261
|
+
|
|
262
|
+
The Java port still runs the full DP (4397 ms total, `backtrack` 3205 ms, `anchored` and `doublestar` regressed against its
|
|
263
|
+
regex version). The same hybrid applies to it and is the next step; the numbers above are why it should be measured per
|
|
264
|
+
workload rather than by the total.
|
|
265
|
+
|
|
266
|
+
### How these numbers are produced
|
|
267
|
+
|
|
268
|
+
Cold-process runs, with start-up measured separately and subtracted. That is the honest number for a CLI and it is
|
|
269
|
+
also noisy: the same Java build has reported `backtrack` at 3205 ms and 5129 ms. [+doc/measurement.md](doc/measurement.md)](doc/measurement.md) records the two measurement modes, the steady-state table (12 rounds inside one process) and every
|
|
270
|
+
strategy tried on the Java port with what it cost. The Java column above predates the literal-segment fast path.
|
|
271
|
+
|
|
272
|
+
### Two caveats
|
|
273
|
+
|
|
274
|
+
- The runner measures a process; the start-up column is a subtraction, and on a loaded machine it is a noisy one. The
|
|
275
|
+
large rows are far outside that noise; the small ones are not.
|
|
276
|
+
- The ports are young. Their remaining distance to Zig is a design difference, not tuning: Java has not adopted the DP, and
|
|
277
|
+
neither port has the Zig implementation's per-chain indexing.
|
|
278
|
+
|
|
279
|
+
## Running the three
|
|
280
|
+
|
|
281
|
+
```text
|
|
282
|
+
bun js/conformance.mjs # JS port: 34/34
|
|
283
|
+
$env:JAVA_HOME='C:\Program Files\Java\jdk-25'; bun java/run.mjs # Java port: 34/34, compiles and runs it
|
|
284
|
+
bun bench/perf.mjs --repeat 3 \
|
|
285
|
+
--port "zig=zig-out/bin/ignore-decide.exe --rules {rules} --paths {paths}" \
|
|
286
|
+
--port "js=bun js/decide.mjs --rules {rules} --paths {paths}" \
|
|
287
|
+
--port "java=\"…\bin\java.exe\" -cp java/out hr.hrg.gitignore.Decide --rules {rules} --paths {paths}"
|
|
288
|
+
```
|
package/js/gitignore.mjs
ADDED
|
@@ -0,0 +1,332 @@
|
|
|
1
|
+
// SPDX-License-Identifier: LicenseRef-GPL-3.0-with-Commons-Clause
|
|
2
|
+
// Copyright (c) 2026 Davor Hrg
|
|
3
|
+
//
|
|
4
|
+
// `.gitignore` semantics, for the walk `fsmeta scan` performs. The Java port (`GitIgnoreFile`,
|
|
5
|
+
// `GitIgnoreRules`) implements the same rules, and conformance/fsmeta-vectors.json's `gitignoreCases`
|
|
6
|
+
// materialise the same trees in both ports and compare the file lists — so this file may be read as the
|
|
7
|
+
// statement of the rules for either language.
|
|
8
|
+
//
|
|
9
|
+
// What a trio describes is "the files a project considers ours", and the definition of "not ours" is a
|
|
10
|
+
// .gitignore file read per directory. That is this repository's own answer (DEC-044: a hand-written skip
|
|
11
|
+
// list is a promise somebody has to remember to keep, and its failure is a silently incomplete index). So
|
|
12
|
+
// the rules below are git's:
|
|
13
|
+
//
|
|
14
|
+
// * a file's patterns apply only to paths BELOW the directory holding it (which is also why a directory's
|
|
15
|
+
// own .gitignore cannot exclude the directory itself);
|
|
16
|
+
// * a pattern with a separator at the start or in the middle is anchored to that directory; otherwise it
|
|
17
|
+
// matches at any depth;
|
|
18
|
+
// * the LAST matching pattern wins, so `!keep.log` after `*.log` re-includes;
|
|
19
|
+
// * a deeper file beats a shallower one;
|
|
20
|
+
// * a trailing slash means directories only, and a file inside an excluded directory counts as excluded —
|
|
21
|
+
// which is what makes pruning a directory equivalent to ignoring everything below it;
|
|
22
|
+
// * matching is case-SENSITIVE on every platform, deliberately: a trio is written on one machine and read
|
|
23
|
+
// on another, so the ports must agree with each other before they agree with a file system.
|
|
24
|
+
|
|
25
|
+
import { readFileSync, statSync } from 'node:fs';
|
|
26
|
+
import { join } from 'node:path';
|
|
27
|
+
|
|
28
|
+
/** The file name a walk reads per directory. */
|
|
29
|
+
import { compileSegments, segmentsMatch } from './glob.mjs';
|
|
30
|
+
|
|
31
|
+
export const GITIGNORE_FILE_NAME = '.gitignore';
|
|
32
|
+
|
|
33
|
+
/**
|
|
34
|
+
* Translates one glob into a regular expression source, git's `**` rules included.
|
|
35
|
+
*
|
|
36
|
+
* Exported because it is the part worth testing on its own: everything else is precedence.
|
|
37
|
+
*/
|
|
38
|
+
export function translateGlob(glob) {
|
|
39
|
+
let regex = '';
|
|
40
|
+
for (let index = 0; index < glob.length; index++) {
|
|
41
|
+
const c = glob[index];
|
|
42
|
+
if (c === '\\') {
|
|
43
|
+
if (index + 1 < glob.length) {
|
|
44
|
+
regex += escapeRegExp(glob[++index]);
|
|
45
|
+
} else {
|
|
46
|
+
regex += '\\\\';
|
|
47
|
+
}
|
|
48
|
+
continue;
|
|
49
|
+
}
|
|
50
|
+
if (c === '*') {
|
|
51
|
+
const doubleStar = index + 1 < glob.length && glob[index + 1] === '*';
|
|
52
|
+
if (!doubleStar) {
|
|
53
|
+
regex += '[^/]*';
|
|
54
|
+
continue;
|
|
55
|
+
}
|
|
56
|
+
const first = index;
|
|
57
|
+
const second = index + 1;
|
|
58
|
+
const atSegmentStart = first === 0 || glob[first - 1] === '/';
|
|
59
|
+
const followedBySlash = second + 1 < glob.length && glob[second + 1] === '/';
|
|
60
|
+
const atEnd = second + 1 === glob.length;
|
|
61
|
+
if (atSegmentStart && followedBySlash) {
|
|
62
|
+
regex += '(?:.*/)?'; // "**/" — zero or more directories
|
|
63
|
+
index = second + 1; // consume the '/'
|
|
64
|
+
} else if (atSegmentStart && atEnd) {
|
|
65
|
+
regex += '.*'; // a trailing "/**" matches everything inside
|
|
66
|
+
index = second;
|
|
67
|
+
} else {
|
|
68
|
+
regex += '[^/]*'; // any other consecutive asterisks are just asterisks
|
|
69
|
+
index = second;
|
|
70
|
+
}
|
|
71
|
+
continue;
|
|
72
|
+
}
|
|
73
|
+
if (c === '?') {
|
|
74
|
+
regex += '[^/]';
|
|
75
|
+
continue;
|
|
76
|
+
}
|
|
77
|
+
if (c === '[') {
|
|
78
|
+
const translated = translateCharacterClass(glob, index);
|
|
79
|
+
regex += translated.regex;
|
|
80
|
+
index = translated.close;
|
|
81
|
+
continue;
|
|
82
|
+
}
|
|
83
|
+
regex += escapeRegExp(c);
|
|
84
|
+
}
|
|
85
|
+
return regex;
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
function escapeRegExp(character) {
|
|
89
|
+
return /[.*+?^${}()|[\]\\/]/.test(character) ? `\\${character}` : character;
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
function translateCharacterClass(glob, open) {
|
|
93
|
+
// A `]` immediately after the opener is a member, not the terminator: `[]]x` holds `]` then `x`.
|
|
94
|
+
const from = glob[open + 1] === ']' ? open + 2 : open + 1;
|
|
95
|
+
const close = glob.indexOf(']', from);
|
|
96
|
+
if (close < 0) {
|
|
97
|
+
// git's wildmatch aborts the whole pattern, so the class matches nothing and the rest is not read.
|
|
98
|
+
return { regex: '(?!)', close: glob.length - 1 };
|
|
99
|
+
}
|
|
100
|
+
const body = glob.slice(open + 1, close);
|
|
101
|
+
let translated = '[';
|
|
102
|
+
let start = 0;
|
|
103
|
+
if (body.length > 0 && (body[0] === '!' || body[0] === '^')) {
|
|
104
|
+
translated += '^';
|
|
105
|
+
start = 1;
|
|
106
|
+
}
|
|
107
|
+
for (let i = start; i < body.length; i++) {
|
|
108
|
+
const c = body[i];
|
|
109
|
+
if (c === '\\' && i + 1 < body.length) {
|
|
110
|
+
translated += `\\${body[++i]}`;
|
|
111
|
+
} else if (c === '[' || c === ']' || c === '^') {
|
|
112
|
+
translated += `\\${c}`;
|
|
113
|
+
} else {
|
|
114
|
+
translated += c;
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
if (translated === '[' || translated === '[^') {
|
|
118
|
+
return { regex: '(?!)', close }; // an empty class matches nothing, as in git
|
|
119
|
+
}
|
|
120
|
+
return { regex: `${translated}]`, close };
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
/** One parsed pattern. */
|
|
124
|
+
function parseRule(rawLine) {
|
|
125
|
+
let line = stripTrailingUnescapedSpaces(rawLine);
|
|
126
|
+
if (line.length === 0 || line[0] === '#') {
|
|
127
|
+
return null;
|
|
128
|
+
}
|
|
129
|
+
let negated = false;
|
|
130
|
+
if (line[0] === '!') {
|
|
131
|
+
negated = true;
|
|
132
|
+
line = line.slice(1);
|
|
133
|
+
if (line.length === 0) {
|
|
134
|
+
return null;
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
const directoryOnly = line.endsWith('/') && !line.endsWith('\\/');
|
|
138
|
+
if (directoryOnly) {
|
|
139
|
+
line = line.slice(0, -1);
|
|
140
|
+
}
|
|
141
|
+
const anchored = line.startsWith('/') || line.includes('/');
|
|
142
|
+
if (line.startsWith('/')) {
|
|
143
|
+
line = line.slice(1);
|
|
144
|
+
}
|
|
145
|
+
if (line.length === 0) {
|
|
146
|
+
return null;
|
|
147
|
+
}
|
|
148
|
+
return {
|
|
149
|
+
negated,
|
|
150
|
+
directoryOnly,
|
|
151
|
+
// Both anchors are required: Java's `Matcher.matches()` anchors the whole pattern, while JavaScript's
|
|
152
|
+
// `RegExp.test` searches — and a missing `^` lets an anchored pattern match a path *suffix*. That one
|
|
153
|
+
// difference is what the ignore vector `anchored-pattern` caught between the two ports.
|
|
154
|
+
//
|
|
155
|
+
// Only a pattern with a `**` segment needs the DP: that is the construct that makes a regex back-track
|
|
156
|
+
// exponentially. Everything else is faster as one compiled regex, so it keeps one.
|
|
157
|
+
pattern: line.split('/').includes('**')
|
|
158
|
+
? null
|
|
159
|
+
: new RegExp(`^${anchored ? '' : '(?:.*/)?'}${translateGlob(line)}$`),
|
|
160
|
+
segments: line.split('/').includes('**') ? compileSegments(line, anchored, translateGlob) : null,
|
|
161
|
+
};
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
/** Trailing spaces are ignored unless they are escaped with a backslash. */
|
|
165
|
+
function stripTrailingUnescapedSpaces(line) {
|
|
166
|
+
let end = line.length;
|
|
167
|
+
const isEscaped = (index) => {
|
|
168
|
+
let backslashes = 0;
|
|
169
|
+
for (let i = index - 1; i >= 0 && line[i] === '\\'; i--) {
|
|
170
|
+
backslashes++;
|
|
171
|
+
}
|
|
172
|
+
return (backslashes & 1) === 1;
|
|
173
|
+
};
|
|
174
|
+
while (end > 0 && line[end - 1] === ' ' && !isEscaped(end - 1)) {
|
|
175
|
+
end--;
|
|
176
|
+
}
|
|
177
|
+
return line.slice(0, end);
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
/**
|
|
181
|
+
* One `.gitignore` file: the directory it governs (relative to the walk root, `''` at the root) and its
|
|
182
|
+
* patterns.
|
|
183
|
+
*
|
|
184
|
+
* @param {string} relativeDirectory the directory holding the file, `/`-separated
|
|
185
|
+
* @param {string} text the file's text
|
|
186
|
+
*/
|
|
187
|
+
export function parseGitignore(relativeDirectory, text) {
|
|
188
|
+
let directory = relativeDirectory ?? '';
|
|
189
|
+
while (directory.endsWith('/')) {
|
|
190
|
+
directory = directory.slice(0, -1);
|
|
191
|
+
}
|
|
192
|
+
const rules = [];
|
|
193
|
+
// A CRLF checkout must parse like an LF one: the carriage return is not part of the pattern.
|
|
194
|
+
for (const line of text.replace(/\r\n/g, '\n').split('\n')) {
|
|
195
|
+
const rule = parseRule(line);
|
|
196
|
+
if (rule !== null) {
|
|
197
|
+
rules.push(rule);
|
|
198
|
+
}
|
|
199
|
+
}
|
|
200
|
+
return {
|
|
201
|
+
relativeDirectory: directory,
|
|
202
|
+
ruleCount: rules.length,
|
|
203
|
+
|
|
204
|
+
/**
|
|
205
|
+
* What this file decides about a path relative to the walk root.
|
|
206
|
+
*
|
|
207
|
+
* @returns {boolean|undefined} true to exclude, false to re-include, and `undefined` when no pattern
|
|
208
|
+
* matches — which is not "included", because a shallower file may still have an opinion
|
|
209
|
+
*/
|
|
210
|
+
decide(relativePath, isDirectory) {
|
|
211
|
+
let scoped = relativePath;
|
|
212
|
+
if (directory !== '') {
|
|
213
|
+
const prefix = `${directory}/`;
|
|
214
|
+
if (!relativePath.startsWith(prefix)) {
|
|
215
|
+
return undefined;
|
|
216
|
+
}
|
|
217
|
+
scoped = relativePath.slice(prefix.length);
|
|
218
|
+
}
|
|
219
|
+
for (let index = rules.length - 1; index >= 0; index--) {
|
|
220
|
+
if (matches(rules[index], scoped, isDirectory)) {
|
|
221
|
+
return !rules[index].negated;
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
return undefined;
|
|
225
|
+
},
|
|
226
|
+
};
|
|
227
|
+
}
|
|
228
|
+
|
|
229
|
+
function matches(rule, path, isDirectory, text = null) {
|
|
230
|
+
if (rule.segments === null) {
|
|
231
|
+
// No `**`: the compiled regex is both correct and faster (see the A/B in conformance/findings-js.md).
|
|
232
|
+
if (!rule.directoryOnly) {
|
|
233
|
+
return rule.pattern.test(path);
|
|
234
|
+
}
|
|
235
|
+
if (isDirectory && rule.pattern.test(path)) {
|
|
236
|
+
return true;
|
|
237
|
+
}
|
|
238
|
+
for (let slash = path.indexOf('/'); slash >= 0; slash = path.indexOf('/', slash + 1)) {
|
|
239
|
+
if (rule.pattern.test(path.slice(0, slash))) {
|
|
240
|
+
return true;
|
|
241
|
+
}
|
|
242
|
+
}
|
|
243
|
+
return false;
|
|
244
|
+
}
|
|
245
|
+
|
|
246
|
+
// Split once per decision, not once per rule: `text` arrives from the caller that already has it.
|
|
247
|
+
const segments = text ?? path.split('/');
|
|
248
|
+
if (!rule.directoryOnly) {
|
|
249
|
+
return segmentsMatch(rule.segments, segments);
|
|
250
|
+
}
|
|
251
|
+
if (isDirectory && segmentsMatch(rule.segments, segments)) {
|
|
252
|
+
return true;
|
|
253
|
+
}
|
|
254
|
+
// A file inside an excluded directory counts as excluded, so a directory-only pattern matches any ancestor too.
|
|
255
|
+
// The walk prunes before it gets here; a caller testing a path on its own gets the same answer either way.
|
|
256
|
+
for (let count = segments.length - 1; count >= 1; count -= 1) {
|
|
257
|
+
if (segmentsMatch(rule.segments, segments, count)) {
|
|
258
|
+
return true;
|
|
259
|
+
}
|
|
260
|
+
}
|
|
261
|
+
return false;
|
|
262
|
+
}
|
|
263
|
+
|
|
264
|
+
/**
|
|
265
|
+
* The rules of one directory, read from disk: an empty file object when the directory has none, so a walk's
|
|
266
|
+
* push/pop pairs stay balanced.
|
|
267
|
+
*/
|
|
268
|
+
export function readGitignore(directoryPath, relativeDirectory) {
|
|
269
|
+
const file = join(directoryPath, GITIGNORE_FILE_NAME);
|
|
270
|
+
try {
|
|
271
|
+
if (!statSync(file).isFile()) {
|
|
272
|
+
return parseGitignore(relativeDirectory, '');
|
|
273
|
+
}
|
|
274
|
+
} catch {
|
|
275
|
+
return parseGitignore(relativeDirectory, '');
|
|
276
|
+
}
|
|
277
|
+
return parseGitignore(relativeDirectory, readFileSync(file, 'utf8'));
|
|
278
|
+
}
|
|
279
|
+
|
|
280
|
+
/**
|
|
281
|
+
* The `.gitignore` files that apply at one point of a walk — git's precedence in a stack.
|
|
282
|
+
*
|
|
283
|
+
* A walk pushes a directory's file when it enters and pops it when it leaves, so the stack is exactly the
|
|
284
|
+
* set of files governing the path being visited. The deepest file with an opinion decides; within a file the
|
|
285
|
+
* last matching pattern does.
|
|
286
|
+
*/
|
|
287
|
+
export class GitIgnoreRules {
|
|
288
|
+
#stack = [];
|
|
289
|
+
|
|
290
|
+
push(file) {
|
|
291
|
+
this.#stack.push(file);
|
|
292
|
+
}
|
|
293
|
+
|
|
294
|
+
pop() {
|
|
295
|
+
this.#stack.pop();
|
|
296
|
+
}
|
|
297
|
+
|
|
298
|
+
get depth() {
|
|
299
|
+
return this.#stack.length;
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
decide(relativePath, isDirectory) {
|
|
303
|
+
const direct = this.decideDirect(relativePath, isDirectory);
|
|
304
|
+
if (direct === true) {
|
|
305
|
+
return true;
|
|
306
|
+
}
|
|
307
|
+
// An excluded directory is never entered, so a rule for a file inside it cannot re-include that file: ask the
|
|
308
|
+
// ancestors, deepest first. A parent that a later rule re-includes answers false and is skipped.
|
|
309
|
+
const segments = relativePath.split('/');
|
|
310
|
+
for (let count = segments.length - 1; count >= 1; count -= 1) {
|
|
311
|
+
if (this.decideDirect(segments.slice(0, count).join('/'), true) === true) {
|
|
312
|
+
return true;
|
|
313
|
+
}
|
|
314
|
+
}
|
|
315
|
+
return direct;
|
|
316
|
+
}
|
|
317
|
+
|
|
318
|
+
/** What the rules say about this exact path, ignoring its ancestors. */
|
|
319
|
+
decideDirect(relativePath, isDirectory) {
|
|
320
|
+
for (let index = this.#stack.length - 1; index >= 0; index--) {
|
|
321
|
+
const decision = this.#stack[index].decide(relativePath, isDirectory);
|
|
322
|
+
if (decision !== undefined) {
|
|
323
|
+
return decision;
|
|
324
|
+
}
|
|
325
|
+
}
|
|
326
|
+
return undefined;
|
|
327
|
+
}
|
|
328
|
+
|
|
329
|
+
isIgnored(relativePath, isDirectory) {
|
|
330
|
+
return this.decide(relativePath, isDirectory) === true;
|
|
331
|
+
}
|
|
332
|
+
}
|
package/js/glob.mjs
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
// The segment dynamic program, ported from the Zig reference — with the allocations removed.
|
|
2
|
+
//
|
|
3
|
+
// The first version was correct and *slower* than the regex it replaced: it split the path inside every rule and
|
|
4
|
+
// allocated two arrays per pattern segment, and in a 2,000-path run that allocation is the cost. This version splits the
|
|
5
|
+
// path once per decision (the caller does it), reuses two scratch arrays, and lets a caller ask about the first `count`
|
|
6
|
+
// segments so the directory-only ancestor check needs no second split.
|
|
7
|
+
|
|
8
|
+
// A segment with no wildcard metacharacter and no escape compares by string equality. The regex it would have
|
|
9
|
+
// compiled to is an anchored, fully escaped literal, so `===` is the same decision without the engine's work —
|
|
10
|
+
// the same fast path the Java port gained 3.03x from (doc/measurement.md).
|
|
11
|
+
const hasWildcard = /[*?[\\]/;
|
|
12
|
+
|
|
13
|
+
/**
|
|
14
|
+
* Compile one pattern into the form the DP wants: an array of `{ doubleStar }`, `{ literal }` or `{ regex }`
|
|
15
|
+
* entries, in order.
|
|
16
|
+
*
|
|
17
|
+
* @param {string} glob the pattern, with any leading `/` already removed
|
|
18
|
+
* @param {boolean} anchored whether the pattern is anchored to the walk root
|
|
19
|
+
* @param {(segment: string) => string} translate one segment to a regex body (the port's own translator)
|
|
20
|
+
*/
|
|
21
|
+
export function compileSegments(glob, anchored, translate) {
|
|
22
|
+
const segments = glob.split('/').map((segment) => {
|
|
23
|
+
if (segment === '**') return { doubleStar: true };
|
|
24
|
+
// A backslash may escape a wildcard, so it stays on the regex path rather than being guessed at here.
|
|
25
|
+
if (!hasWildcard.test(segment)) return { literal: segment };
|
|
26
|
+
return { regex: new RegExp(`^${translate(segment)}$`) };
|
|
27
|
+
});
|
|
28
|
+
// An unanchored pattern applies at any depth, which is a leading `**` and nothing else.
|
|
29
|
+
return anchored ? segments : [{ doubleStar: true }, ...segments];
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
// Two scratch rows, reused across calls: the DP needs a "previous" and a "next" row and nothing else. They grow to the
|
|
33
|
+
// longest path seen and are never freed, which is the point — allocation was the cost, not the comparison.
|
|
34
|
+
let scratchA = [];
|
|
35
|
+
let scratchB = [];
|
|
36
|
+
|
|
37
|
+
function rows(size) {
|
|
38
|
+
if (scratchA.length < size) {
|
|
39
|
+
scratchA = new Array(size).fill(false);
|
|
40
|
+
scratchB = new Array(size).fill(false);
|
|
41
|
+
}
|
|
42
|
+
return [scratchA, scratchB];
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
/**
|
|
46
|
+
* Does the compiled pattern match the first `count` segments of `text`?
|
|
47
|
+
*
|
|
48
|
+
* @param {Array<{doubleStar?: boolean, literal?: string, regex?: RegExp}>} segments
|
|
49
|
+
* @param {string[]} text a path already split on `/`
|
|
50
|
+
* @param {number} [count] how many segments to consider (default: all of them)
|
|
51
|
+
*/
|
|
52
|
+
export function segmentsMatch(segments, text, count = text.length) {
|
|
53
|
+
const [first, second] = rows(count + 1);
|
|
54
|
+
let reach = first;
|
|
55
|
+
let next = second;
|
|
56
|
+
for (let index = 0; index <= count; index += 1) reach[index] = index === 0;
|
|
57
|
+
|
|
58
|
+
for (let index = 0; index < segments.length; index += 1) {
|
|
59
|
+
const segment = segments[index];
|
|
60
|
+
for (let position = 0; position <= count; position += 1) next[position] = false;
|
|
61
|
+
|
|
62
|
+
if (segment.doubleStar === true) {
|
|
63
|
+
const last = index === segments.length - 1;
|
|
64
|
+
let any = false;
|
|
65
|
+
for (let position = 0; position <= count; position += 1) {
|
|
66
|
+
// A trailing `**` is one-or-more (`build/**` is inside build), a middle one is zero-or-more.
|
|
67
|
+
if (last) next[position] = any;
|
|
68
|
+
if (reach[position]) any = true;
|
|
69
|
+
if (!last) next[position] = any;
|
|
70
|
+
}
|
|
71
|
+
} else if (segment.literal !== undefined) {
|
|
72
|
+
for (let position = 0; position < count; position += 1) {
|
|
73
|
+
if (reach[position] && text[position] === segment.literal) next[position + 1] = true;
|
|
74
|
+
}
|
|
75
|
+
} else {
|
|
76
|
+
for (let position = 0; position < count; position += 1) {
|
|
77
|
+
if (reach[position] && segment.regex.test(text[position])) next[position + 1] = true;
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
const swap = reach;
|
|
82
|
+
reach = next;
|
|
83
|
+
next = swap;
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
return reach[count];
|
|
87
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,32 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@hrg/gitignore",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.3.0",
|
|
4
|
+
"description": "git's .gitignore (wildmatch) semantics: one matcher, shared by the Zig, JavaScript and Java ports.",
|
|
5
|
+
"license": "LicenseRef-GPL-3.0-with-Commons-Clause",
|
|
6
|
+
"author": "Davor Hrg",
|
|
7
|
+
"repository": {
|
|
8
|
+
"type": "git",
|
|
9
|
+
"url": "git+https://github.com/hrgdavor/gitignore.git"
|
|
10
|
+
},
|
|
11
|
+
"type": "module",
|
|
12
|
+
"sideEffects": false,
|
|
13
|
+
"exports": {
|
|
14
|
+
".": "./js/gitignore.mjs",
|
|
15
|
+
"./glob": "./js/glob.mjs",
|
|
16
|
+
"./package.json": "./package.json"
|
|
17
|
+
},
|
|
18
|
+
"files": [
|
|
19
|
+
"js/gitignore.mjs",
|
|
20
|
+
"js/glob.mjs",
|
|
21
|
+
"README.md"
|
|
22
|
+
],
|
|
23
|
+
"engines": {
|
|
24
|
+
"node": ">=18"
|
|
25
|
+
},
|
|
26
|
+
"publishConfig": {
|
|
27
|
+
"access": "public"
|
|
28
|
+
},
|
|
29
|
+
"scripts": {
|
|
30
|
+
"conformance": "bun js/conformance.mjs"
|
|
31
|
+
}
|
|
32
|
+
}
|