jobcompat 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +9 -0
- data/LICENSE +21 -0
- data/README.md +161 -0
- data/SECURITY.md +5 -0
- data/docs/architecture.md +587 -0
- data/docs/competitive-analysis.md +268 -0
- data/docs/implementation-plan.md +806 -0
- data/docs/release-checklist-v0.1.0.md +49 -0
- data/docs/release-notes-v0.1.0.md +24 -0
- data/docs/spec-v0.1.md +1157 -0
- data/exe/jobcompat +3 -0
- data/lib/jobcompat/analysis.rb +340 -0
- data/lib/jobcompat/cli.rb +107 -0
- data/lib/jobcompat/config.rb +89 -0
- data/lib/jobcompat/engine.rb +247 -0
- data/lib/jobcompat/errors.rb +12 -0
- data/lib/jobcompat/formatter.rb +64 -0
- data/lib/jobcompat/git_repository.rb +78 -0
- data/lib/jobcompat/version.rb +3 -0
- data/lib/jobcompat.rb +8 -0
- metadata +84 -0
|
@@ -0,0 +1,587 @@
|
|
|
1
|
+
# jobcompat v0.1 architecture
|
|
2
|
+
|
|
3
|
+
Status: implementation design for `0.1.0`
|
|
4
|
+
Date: 2026-09-23
|
|
5
|
+
|
|
6
|
+
## 1. Architecture principles
|
|
7
|
+
|
|
8
|
+
1. **Immutable inputs:** analyze Git commits without changing the checkout.
|
|
9
|
+
2. **No application execution:** parse source; never require, boot, or evaluate it.
|
|
10
|
+
3. **Extraction before semantics:** AST visitors produce facts; a pure engine compares facts.
|
|
11
|
+
4. **Conservative unknowns:** unsupported evidence is explicit and cannot become a pass.
|
|
12
|
+
5. **Small v0.1 surface:** one framework adapter, one parser, one command, two formatters.
|
|
13
|
+
6. **Determinism:** stable sorting and fully resolved commit SHAs are part of correctness.
|
|
14
|
+
7. **No speculative framework abstraction:** namespaces organize current responsibilities; no generic plugin API ships in v0.1.
|
|
15
|
+
|
|
16
|
+
## 2. Data flow
|
|
17
|
+
|
|
18
|
+
```text
|
|
19
|
+
Git base ref Git head ref
|
|
20
|
+
| |
|
|
21
|
+
v v
|
|
22
|
+
resolve SHA + ls-tree metadata resolve SHA + ls-tree metadata
|
|
23
|
+
| |
|
|
24
|
+
v v
|
|
25
|
+
selected blobs -> Prism AST selected blobs -> Prism AST
|
|
26
|
+
| |
|
|
27
|
+
v v
|
|
28
|
+
worker fragments + producer facts + selected DefinedConstantIndex facts
|
|
29
|
+
|
|
|
30
|
+
v
|
|
31
|
+
pure engine presence-request preflight
|
|
32
|
+
|
|
|
33
|
+
v
|
|
34
|
+
lazy presence-only pass on requested snapshot/name pairs
|
|
35
|
+
|
|
|
36
|
+
v
|
|
37
|
+
pure Compatibility::Engine (contracts, matrices, rules,
|
|
38
|
+
JC007 semantic pairing and finding aggregation)
|
|
39
|
+
|
|
|
40
|
+
v
|
|
41
|
+
suppression -> text / JSON
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
The same selected source blob is parsed once per distinct blob OID. When base and head contain the same OID, the parse/extraction result MAY be memoized and rebound to the appropriate revision label. The presence-only pass is lazy for potential JC004/JC005 names and checks tracked `.rb` blobs outside normal scan scope without using their producer facts. Caching is in-process only.
|
|
45
|
+
|
|
46
|
+
## 3. Proposed project layout
|
|
47
|
+
|
|
48
|
+
```text
|
|
49
|
+
jobcompat.gemspec
|
|
50
|
+
Gemfile
|
|
51
|
+
Rakefile
|
|
52
|
+
LICENSE
|
|
53
|
+
README.md
|
|
54
|
+
|
|
55
|
+
exe/
|
|
56
|
+
jobcompat
|
|
57
|
+
|
|
58
|
+
lib/
|
|
59
|
+
jobcompat.rb
|
|
60
|
+
jobcompat/
|
|
61
|
+
version.rb
|
|
62
|
+
cli.rb
|
|
63
|
+
config.rb
|
|
64
|
+
errors.rb
|
|
65
|
+
|
|
66
|
+
git_repository.rb
|
|
67
|
+
revision_snapshot.rb
|
|
68
|
+
|
|
69
|
+
analysis/
|
|
70
|
+
sidekiq_analyzer.rb
|
|
71
|
+
worker_discovery_visitor.rb
|
|
72
|
+
producer_discovery_visitor.rb
|
|
73
|
+
defined_constant_index.rb
|
|
74
|
+
semantic_evidence.rb
|
|
75
|
+
constant_name.rb
|
|
76
|
+
|
|
77
|
+
model/
|
|
78
|
+
source_location.rb
|
|
79
|
+
worker_contract.rb
|
|
80
|
+
enqueue_call.rb
|
|
81
|
+
compatibility_result.rb
|
|
82
|
+
finding.rb
|
|
83
|
+
|
|
84
|
+
compatibility/
|
|
85
|
+
engine.rb
|
|
86
|
+
rules.rb
|
|
87
|
+
|
|
88
|
+
formatter/
|
|
89
|
+
text.rb
|
|
90
|
+
json.rb
|
|
91
|
+
|
|
92
|
+
test/
|
|
93
|
+
test_helper.rb
|
|
94
|
+
support/
|
|
95
|
+
temporary_repository.rb
|
|
96
|
+
unit/
|
|
97
|
+
config_test.rb
|
|
98
|
+
constant_name_test.rb
|
|
99
|
+
worker_contract_test.rb
|
|
100
|
+
compatibility_engine_test.rb
|
|
101
|
+
rules_test.rb
|
|
102
|
+
text_formatter_test.rb
|
|
103
|
+
json_formatter_test.rb
|
|
104
|
+
analysis/
|
|
105
|
+
worker_discovery_visitor_test.rb
|
|
106
|
+
producer_discovery_visitor_test.rb
|
|
107
|
+
sidekiq_analyzer_test.rb
|
|
108
|
+
integration/
|
|
109
|
+
git_repository_test.rb
|
|
110
|
+
check_command_test.rb
|
|
111
|
+
determinism_test.rb
|
|
112
|
+
|
|
113
|
+
docs/
|
|
114
|
+
competitive-analysis.md
|
|
115
|
+
spec-v0.1.md
|
|
116
|
+
architecture.md
|
|
117
|
+
implementation-plan.md
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
This structure is intentionally shallower than a multi-framework architecture. If a second framework is implemented later, extraction can then gain an adapter interface based on observed commonality.
|
|
121
|
+
|
|
122
|
+
## 4. Component responsibilities
|
|
123
|
+
|
|
124
|
+
### 4.1 `Jobcompat::CLI`
|
|
125
|
+
|
|
126
|
+
- parse `ARGV` using `OptionParser`;
|
|
127
|
+
- implement `check`, top-level help, command help, and version;
|
|
128
|
+
- locate the Git root;
|
|
129
|
+
- load configuration;
|
|
130
|
+
- orchestrate snapshots, analysis, engine, suppression, formatting, and exit status;
|
|
131
|
+
- translate expected exceptions into user-facing tool errors;
|
|
132
|
+
- never contain rule logic or AST branching.
|
|
133
|
+
|
|
134
|
+
The executable calls one public entry point and exits with its integer result. It MUST NOT rescue `SystemExit` broadly or hide programming failures as compatibility findings.
|
|
135
|
+
|
|
136
|
+
### 4.2 `Jobcompat::Config`
|
|
137
|
+
|
|
138
|
+
- supply defaults;
|
|
139
|
+
- load YAML with `Psych.safe_load` and aliases disabled;
|
|
140
|
+
- reject unknown keys/types/duplicates;
|
|
141
|
+
- compile scan glob lists without reading source files;
|
|
142
|
+
- expose `scan?(path)` and `suppresses?(finding)`;
|
|
143
|
+
- retain display path for JSON output.
|
|
144
|
+
|
|
145
|
+
No rule severity customization or plugin configuration belongs here in v0.1.
|
|
146
|
+
|
|
147
|
+
### 4.3 `Jobcompat::GitRepository`
|
|
148
|
+
|
|
149
|
+
- find repository root;
|
|
150
|
+
- resolve a ref to a full commit SHA;
|
|
151
|
+
- enumerate regular blobs in a tree with NUL-delimited output;
|
|
152
|
+
- stream selected blob contents using one `git cat-file --batch` process per snapshot or per complete comparison;
|
|
153
|
+
- retain path/mode/OID metadata for every tracked `.rb` blob so a later bounded presence-only pass can examine excluded paths without rescanning the Git tree;
|
|
154
|
+
- build `RevisionSnapshot` metadata;
|
|
155
|
+
- raise typed Git errors with sanitized messages.
|
|
156
|
+
|
|
157
|
+
It MUST NOT expose a general shell runner.
|
|
158
|
+
|
|
159
|
+
### 4.4 `Jobcompat::RevisionSnapshot`
|
|
160
|
+
|
|
161
|
+
Immutable metadata and extracted analysis for one revision:
|
|
162
|
+
|
|
163
|
+
```text
|
|
164
|
+
label : base | head
|
|
165
|
+
requested_ref : String
|
|
166
|
+
sha : 40/64-character object ID as emitted by Git
|
|
167
|
+
workers : Hash<String, WorkerContract or UnknownContract>
|
|
168
|
+
enqueue_calls : Array<EnqueueCall or UnknownEnqueueCall>
|
|
169
|
+
constant_index : DefinedConstantIndex
|
|
170
|
+
tracked_ruby : lazy path/OID metadata for regular .rb blobs
|
|
171
|
+
files_scanned : Integer
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
The snapshot does not retain every source file after analysis. Selected ASTs contribute declaration facts to `DefinedConstantIndex`; unselected blobs are opened only if a potential JC004/JC005 needs a presence proof. `files_scanned` counts only normal selected files, not presence-only blobs.
|
|
175
|
+
|
|
176
|
+
### 4.5 `Jobcompat::Analysis::SidekiqAnalyzer`
|
|
177
|
+
|
|
178
|
+
- call `Prism.parse(source, filepath: path)`;
|
|
179
|
+
- turn any Prism error diagnostic into a typed parse failure;
|
|
180
|
+
- run worker and producer visitors over the same AST;
|
|
181
|
+
- return syntax facts without compatibility conclusions;
|
|
182
|
+
- preserve source locations and concise signature/call excerpts.
|
|
183
|
+
|
|
184
|
+
It receives source bytes and a revision/path context. It does not open files, invoke Git, or know base/head rule semantics.
|
|
185
|
+
|
|
186
|
+
### 4.6 AST visitors
|
|
187
|
+
|
|
188
|
+
`WorkerDiscoveryVisitor` owns class/module namespace tracking, direct Sidekiq includes, and direct `perform` definitions.
|
|
189
|
+
|
|
190
|
+
`ProducerDiscoveryVisitor` owns calls named `perform_async`, `perform_in`, and `perform_at`, supported `.set` chains, syntactic payload counts, lexical namespace capture, and unknown call facts.
|
|
191
|
+
|
|
192
|
+
`ConstantName` is a small utility that flattens only `ConstantReadNode` and `ConstantPathNode`. It returns a value object with `segments`, `root_qualified`, and `static`; it does not resolve Ruby constants.
|
|
193
|
+
|
|
194
|
+
`DefinedConstantIndex` records exact or plausible canonical class/module declarations and statically named constant bindings, source locations, and normal-scan membership. It is distinct from supported Sidekiq worker recognition. For a name missing from one revision's recognized workers, it returns `recognized_worker`, `defined_unrecognized`, `outside_scan_scope`, `absent`, or `unverified` using the formal proof in the spec. A present-but-unrecognized class or incomplete presence pass blocks JC004/JC005 ERROR.
|
|
195
|
+
|
|
196
|
+
`SemanticEvidence` builds the revision-free JC007 fingerprint from uncertainty kind/reason, canonical worker when known, sorted relative paths, syntactic enclosing scope, trivia-free relevant token stream and enclosing statement, plus deterministic group size and occurrence ordinal. It retains every revision-tagged root location. Same-fingerprint roots pair one-to-one across snapshots; distinct roots remain separate.
|
|
197
|
+
|
|
198
|
+
### 4.7 `Compatibility::Engine`
|
|
199
|
+
|
|
200
|
+
- accept fully extracted base/head snapshots and configuration-independent facts;
|
|
201
|
+
- resolve producer names against the combined worker-name index;
|
|
202
|
+
- expose a pure preflight that identifies the canonical names needing opposite-snapshot presence checks after producer resolution; consume the resulting immutable presence statuses during rule evaluation;
|
|
203
|
+
- calculate matrices;
|
|
204
|
+
- evaluate rule algorithms and precedence;
|
|
205
|
+
- aggregate/de-duplicate findings;
|
|
206
|
+
- return a `CompatibilityResult` without printing or exiting.
|
|
207
|
+
|
|
208
|
+
The engine MUST be pure with respect to filesystem, Git, environment, clock, and output streams. This is the main unit-test seam.
|
|
209
|
+
|
|
210
|
+
### 4.8 Formatters
|
|
211
|
+
|
|
212
|
+
Formatters receive a completed or failed result envelope. They MUST NOT recalculate compatibility or suppression. They render already matched suppression audit records without reconstructing omitted findings.
|
|
213
|
+
|
|
214
|
+
- `Formatter::Text` renders concise human remediation.
|
|
215
|
+
- `Formatter::Json` constructs schema-version-1 primitive Hash/Array data, including each finding's `revisions`, `directions`, and `unknown_reason` and each worker's base/head presence status, calls `JSON.pretty_generate`, and appends exactly one newline. The formal specification fixes two-space pretty JSON; exact-output tests lock it.
|
|
216
|
+
|
|
217
|
+
Both consume already sorted results.
|
|
218
|
+
|
|
219
|
+
## 5. Git revision loading
|
|
220
|
+
|
|
221
|
+
### 5.1 Why not checkout/worktree copies
|
|
222
|
+
|
|
223
|
+
| Approach | Advantages | Problems | Decision |
|
|
224
|
+
| --- | --- | --- | --- |
|
|
225
|
+
| `git checkout` base/head | simple filesystem reads | mutates user state, conflicts with uncommitted changes, slow | reject |
|
|
226
|
+
| temporary `git worktree` | isolates checkout | creates/deletes directories and Git metadata; checkout filters may run | reject for v0.1 |
|
|
227
|
+
| one `git show <sha>:<path>` per file | immutable and simple | one subprocess per file; poor large-repo performance | reject |
|
|
228
|
+
| `git archive` | one stream | honors archive behavior such as `export-ignore`; tar layer; less direct object control | reject |
|
|
229
|
+
| `ls-tree` + `cat-file --batch` | immutable, raw blobs, few processes, path-safe enumeration | framed-stream parser required | adopt |
|
|
230
|
+
|
|
231
|
+
### 5.2 Command protocol
|
|
232
|
+
|
|
233
|
+
All commands are invoked as argv arrays with `Open3`, never through a shell.
|
|
234
|
+
|
|
235
|
+
1. Locate root:
|
|
236
|
+
|
|
237
|
+
```text
|
|
238
|
+
git rev-parse --show-toplevel
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
2. Resolve ref:
|
|
242
|
+
|
|
243
|
+
```text
|
|
244
|
+
git rev-parse --verify --end-of-options <ref>^{commit}
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Every command after root discovery runs with `chdir` set to that exact Git root.
|
|
248
|
+
|
|
249
|
+
3. Enumerate tree using the resolved SHA:
|
|
250
|
+
|
|
251
|
+
```text
|
|
252
|
+
git ls-tree -r -z --full-tree <sha>
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
4. Split each NUL-terminated default-format record at its first tab into `<mode> <type> <oid>` and the path; then filter mode/type/path in Ruby. Keep metadata for all regular tracked `.rb` blobs, but parse only normal-scan paths initially.
|
|
256
|
+
5. Feed only hexadecimal object IDs to:
|
|
257
|
+
|
|
258
|
+
```text
|
|
259
|
+
git cat-file --batch
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
6. Parse each response as `<oid> <type> <size>\n`, exactly `size` bytes, then one delimiter newline.
|
|
263
|
+
|
|
264
|
+
Only OIDs, not user paths or refs, enter the batch input. NUL-delimited tree enumeration preserves unusual Git pathnames. v0.1 rejects paths containing NUL by construction (Git paths cannot contain NUL) and can preserve newlines because the path is never sent through line-based batch input.
|
|
265
|
+
|
|
266
|
+
Set `GIT_OPTIONAL_LOCKS=0` for read-only intent. Do not request textconv, filters, or symlink following. Capture stderr and convert non-zero status into `GitError` without leaking irrelevant environment data.
|
|
267
|
+
|
|
268
|
+
### 5.3 Missing or moving refs
|
|
269
|
+
|
|
270
|
+
The ref may move after resolution without affecting analysis because every later command uses its SHA. Missing refs, non-commit objects that cannot peel to commits, missing objects in a partial clone, corrupt batch framing, or Git command failure make the analysis incomplete and exit 2.
|
|
271
|
+
|
|
272
|
+
No implicit `git fetch` occurs. Network access is never required by jobcompat.
|
|
273
|
+
|
|
274
|
+
### 5.4 Lazy presence-only pass
|
|
275
|
+
|
|
276
|
+
The pure engine preflight returns HEAD names for all base-only workers and base names for head-only workers with an attributable head enqueue. The CLI then resolves those requests through each snapshot's index before final engine evaluation; the engine never performs I/O. A head-only worker with no head enqueue needs no base query and has `not_checked` base presence in JSON. Selected ASTs already supply declarations. For unselected regular tracked `.rb` blobs, stream bytes and prefilter on the candidate leaf constant token; only candidate-bearing blobs need Prism. Non-ASCII candidate leaves require parsing all unselected blobs within the same budget. This pass never extracts producers or worker contracts, and excluded-file syntax errors do not become global parse errors. An exact declaration/binding gives `outside_scan_scope`; an ambiguous declaration, candidate-bearing parse failure, or exhausted budget gives `unverified`. `absent` is returned only after every relevant blob is checked within the 64 MiB (67,108,864 byte) unselected-byte budget. The prefilter and budget bound expensive vendor trees; a budget hit sacrifices an ERROR proof and produces JC007. No checkout, application execution, or source retention is required.
|
|
277
|
+
|
|
278
|
+
## 6. Prism strategy
|
|
279
|
+
|
|
280
|
+
### 6.1 Parser decision
|
|
281
|
+
|
|
282
|
+
Prism is selected over `parser` because:
|
|
283
|
+
|
|
284
|
+
- Prism is official, production-ready, error-tolerant, and bundled from Ruby 3.3;
|
|
285
|
+
- the target runtime is Ruby 3.3+;
|
|
286
|
+
- native Prism AST exposes source locations and named node fields directly;
|
|
287
|
+
- the `parser` project itself recommends native Prism for Ruby 3.3+ when its legacy AST compatibility is not required;
|
|
288
|
+
- jobcompat is new code and has no existing `parser` AST investment.
|
|
289
|
+
|
|
290
|
+
The `parser` gem remains stronger when one tool must parse historical Ruby grammars with its long-stable AST. That is not a v0.1 goal. No parser abstraction is added.
|
|
291
|
+
|
|
292
|
+
### 6.2 Parse result handling
|
|
293
|
+
|
|
294
|
+
For every selected blob:
|
|
295
|
+
|
|
296
|
+
1. `result = Prism.parse(source, filepath: path)`;
|
|
297
|
+
2. if `result.errors.any?`, record every located error; continue over sorted selected files only to aggregate parse errors, then abort compatibility analysis with exit 2;
|
|
298
|
+
3. parse warnings MAY be logged or asserted during development but do not appear in v0.1 public findings or completed diagnostics;
|
|
299
|
+
4. visit `result.value` once, dispatching both fact collectors or one combined traversal;
|
|
300
|
+
5. use node `location.start_line` and `start_column` and convert columns to documented 1-based values.
|
|
301
|
+
|
|
302
|
+
Prism columns are byte-based in source encoding; JSON/text reports 1-based byte columns in v0.1. This MUST be stated in developer documentation to avoid Unicode ambiguity.
|
|
303
|
+
|
|
304
|
+
Presence-only blobs reuse this parser with a narrower visitor. A candidate-bearing parse/encoding error makes the candidate's presence `unverified`; it does not fail the entire analysis because the file was excluded from the normal scan. Selected-file errors retain exit 2.
|
|
305
|
+
|
|
306
|
+
### 6.3 Node mapping
|
|
307
|
+
|
|
308
|
+
| Concern | Prism nodes/fields | Extracted information |
|
|
309
|
+
| --- | --- | --- |
|
|
310
|
+
| class | `ClassNode#constant_path`, `#body` | declared static name, body namespace |
|
|
311
|
+
| module | `ModuleNode#constant_path`, `#body` | lexical namespace |
|
|
312
|
+
| constant path | `ConstantReadNode#name`, `ConstantPathNode#parent/#name` | static segments and root qualification |
|
|
313
|
+
| class/module presence | `ClassNode`/`ModuleNode` and statically named constant writes | exact or plausible canonical name, normal-scan status, source location |
|
|
314
|
+
| include | `CallNode#name == :include`, `#receiver == nil`, `ArgumentsNode#arguments` | exact Sidekiq module argument and location |
|
|
315
|
+
| perform | `DefNode#name == :perform`, `#receiver == nil`, `#parameters` | direct instance signature only |
|
|
316
|
+
| positional parameters | `ParametersNode#requireds/#optionals/#rest/#posts` | min/max arity |
|
|
317
|
+
| keywords | `ParametersNode#keywords/#keyword_rest` | unsupported status, except forwarding node |
|
|
318
|
+
| forwarding | `ForwardingParameterNode` | unbounded positional acceptance |
|
|
319
|
+
| enqueue call | `CallNode#name/#receiver/#arguments/#block` | method, receiver fact, argument nodes, location |
|
|
320
|
+
| safe navigation | `CallNode#call_operator_loc` whose source slice is `&.` | unknown receiver evidence |
|
|
321
|
+
| call arguments | `ArgumentsNode#arguments` | syntactic count |
|
|
322
|
+
| splat | `SplatNode` | unknown producer arity |
|
|
323
|
+
| forwarded call args | `ForwardingArgumentsNode` | unknown producer arity |
|
|
324
|
+
| `.set` chain | outer `CallNode(:perform_async)` receiver inner `CallNode(:set)` | underlying worker receiver; outer payload args |
|
|
325
|
+
| source location | every node's `#location` | revision, path, line, column, concise source excerpt |
|
|
326
|
+
| unknown evidence | relevant node source tokens and enclosing statement | revision-free semantic fingerprint parts and occurrence ordering |
|
|
327
|
+
|
|
328
|
+
### 6.4 Visitor state
|
|
329
|
+
|
|
330
|
+
Namespace tracking uses a stack of static segment arrays. Entering `module Admin; class ExportJob` pushes `Admin`, then `ExportJob`. At top level, entering `class Admin::ExportJob` pushes the full declared path as one lexical scope value. A root-qualified path is exact. An unrooted multi-segment class/module path inside a non-empty lexical namespace is unsupported rather than guessed. The visitor MUST restore state with `ensure` so exceptions do not corrupt sibling traversal.
|
|
331
|
+
|
|
332
|
+
Each class fragment has its own context:
|
|
333
|
+
|
|
334
|
+
```text
|
|
335
|
+
canonical_name
|
|
336
|
+
declaration_location
|
|
337
|
+
recognized_include_locations[]
|
|
338
|
+
perform_nodes[]
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
The discovery visitor must avoid treating `include` inside a nested class as belonging to its parent.
|
|
342
|
+
|
|
343
|
+
Group all selected class fragments by canonical name within one revision before deciding whether the worker has one direct `perform`. An include fragment and a `perform` fragment can be in different files; a file move alone cannot delete the canonical worker. The merge derives only the supported Sidekiq positional contract: zero or multiple direct `perform` definitions use the existing `missing_perform` or `multiple_perform_definitions` unknown reasons. Unsupported canonical paths remain unknown, and the separate presence index blocks a false JC004. This aggregation does not reconcile general Ruby class/module kind, superclass, constant, autoload, or runtime load-order conflicts.
|
|
344
|
+
|
|
345
|
+
### 6.5 Producer facts before name resolution
|
|
346
|
+
|
|
347
|
+
The producer visitor emits an unresolved fact:
|
|
348
|
+
|
|
349
|
+
```text
|
|
350
|
+
UnresolvedEnqueueCall
|
|
351
|
+
receiver_kind static_constant | dynamic | unsupported
|
|
352
|
+
receiver_segments Array<String> | nil
|
|
353
|
+
root_qualified true | false
|
|
354
|
+
lexical_namespace Array<String>
|
|
355
|
+
payload_arity Integer | nil
|
|
356
|
+
arity_known true | false
|
|
357
|
+
method perform_async | perform_in | perform_at
|
|
358
|
+
location
|
|
359
|
+
unknown_reason Symbol | nil
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
The symbol values map one-to-one to the schema-v1 strings enumerated in `docs/spec-v0.1.md`; visitors do not invent free-form reason text.
|
|
363
|
+
|
|
364
|
+
Only after both revisions' worker names are known does the engine resolve these facts. This avoids ordering dependence between files and enables head calls to removed base workers.
|
|
365
|
+
|
|
366
|
+
## 7. Internal models
|
|
367
|
+
|
|
368
|
+
Models SHOULD be immutable `Data.define` values on Ruby 3.3. Constructors validate invariants at the boundary; rule methods do not repeatedly revalidate them.
|
|
369
|
+
|
|
370
|
+
### 7.1 `SourceLocation`
|
|
371
|
+
|
|
372
|
+
```text
|
|
373
|
+
revision : :base | :head
|
|
374
|
+
path : String
|
|
375
|
+
line : Integer >= 1
|
|
376
|
+
column : Integer >= 1
|
|
377
|
+
role : :consumer | :producer | :worker_declaration | :unknown_call
|
|
378
|
+
excerpt : String | nil
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
### 7.2 `WorkerContract`
|
|
382
|
+
|
|
383
|
+
```text
|
|
384
|
+
name : String
|
|
385
|
+
min_arity : Integer >= 0
|
|
386
|
+
max_arity : Integer >= min_arity | nil
|
|
387
|
+
variadic : Boolean derived from max_arity.nil?
|
|
388
|
+
signature_kind : :positional | :forwarding
|
|
389
|
+
declaration_locations : Array<SourceLocation>
|
|
390
|
+
include_locations : Array<SourceLocation>
|
|
391
|
+
perform_location : SourceLocation
|
|
392
|
+
signature : String
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
Methods:
|
|
396
|
+
|
|
397
|
+
```text
|
|
398
|
+
accepts?(arity) -> Boolean
|
|
399
|
+
superset_of?(other_contract) -> Boolean
|
|
400
|
+
display_range -> String
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
All location arrays are sorted and deduplicated after canonical fragment merge. Unknown contracts are a separate `UnknownWorkerContract` carrying name, all relevant fragment locations, and a reason enum. Do not encode unknown as fake `0..∞`.
|
|
404
|
+
|
|
405
|
+
The JSON formatter serializes both classes as a contract object with `status: known|unknown`. A null contract means no recognized worker contract in that revision; the separate `base_presence`/`head_presence` enum distinguishes proven class absence from an unrecognized, out-of-scope, or unverified class. Null alone MUST NOT drive JC004 or JC005.
|
|
406
|
+
|
|
407
|
+
### 7.3 `EnqueueCall`
|
|
408
|
+
|
|
409
|
+
```text
|
|
410
|
+
worker_name : String | nil
|
|
411
|
+
payload_arity : Integer | nil
|
|
412
|
+
arity_known : Boolean
|
|
413
|
+
method : :perform_async | :perform_in | :perform_at
|
|
414
|
+
location : SourceLocation
|
|
415
|
+
unknown_reason : Symbol | nil
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
Invariant: known arity implies non-null worker and payload arity. A known worker with unknown arity is valid. A null worker is always unknown evidence.
|
|
419
|
+
|
|
420
|
+
### 7.4 `CompatibilityResult`
|
|
421
|
+
|
|
422
|
+
```text
|
|
423
|
+
base_snapshot
|
|
424
|
+
head_snapshot
|
|
425
|
+
workers[]
|
|
426
|
+
findings[]
|
|
427
|
+
matched_suppressions[]
|
|
428
|
+
diagnostics[]
|
|
429
|
+
summary
|
|
430
|
+
status
|
|
431
|
+
```
|
|
432
|
+
|
|
433
|
+
Per-worker results contain base/head contracts, unique producer arity lists, unknown call counts, and four matrix cell statuses.
|
|
434
|
+
|
|
435
|
+
### 7.5 `Finding`
|
|
436
|
+
|
|
437
|
+
```text
|
|
438
|
+
rule_id : JC001..JC007
|
|
439
|
+
title : String
|
|
440
|
+
severity : :error | :warning
|
|
441
|
+
worker : String | nil
|
|
442
|
+
revisions : non-empty Array<Symbol> in base/head order
|
|
443
|
+
directions : non-empty Array<Symbol>
|
|
444
|
+
unknown_reason : Symbol | nil, set for JC007 only
|
|
445
|
+
message : String
|
|
446
|
+
risk : String
|
|
447
|
+
remediation : Array<String>
|
|
448
|
+
payload_arity : Integer | nil
|
|
449
|
+
locations : Array<SourceLocation>
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
Rule titles and default remediation text live in `compatibility/rules.rb`, not formatters.
|
|
453
|
+
|
|
454
|
+
## 8. Compatibility engine
|
|
455
|
+
|
|
456
|
+
### 8.1 Pure calculation order
|
|
457
|
+
|
|
458
|
+
1. index known and unknown workers by canonical name;
|
|
459
|
+
2. build union name index;
|
|
460
|
+
3. resolve producer facts for each revision;
|
|
461
|
+
4. group known calls by worker and arity;
|
|
462
|
+
5. build base/base, base/head, head/base, head/head cells;
|
|
463
|
+
6. produce pure presence requests for potential JC004/JC005; after the CLI supplies immutable statuses, emit ERROR only on `absent`, otherwise queue a presence JC007;
|
|
464
|
+
7. evaluate JC001, then JC003 and JC002 membership relations; merge the same worker/arity's head-to-head failure into JC001;
|
|
465
|
+
8. evaluate JC006 interval inclusion;
|
|
466
|
+
9. translate unknown facts/contracts/presence transitions to JC007;
|
|
467
|
+
10. pair exact semantic fingerprints across base/head, union revisions/directions/locations, then aggregate and sort;
|
|
468
|
+
11. apply config suppressions outside the engine or in a dedicated result filter, retaining matched rule/worker/reason/count audit records;
|
|
469
|
+
12. compute summary and exit policy.
|
|
470
|
+
|
|
471
|
+
### 8.2 Rule isolation
|
|
472
|
+
|
|
473
|
+
Rule implementation SHOULD use small functions returning zero or more findings:
|
|
474
|
+
|
|
475
|
+
```text
|
|
476
|
+
Rules.worker_removed(context)
|
|
477
|
+
Rules.new_worker_activated(context)
|
|
478
|
+
Rules.current_mismatch(context)
|
|
479
|
+
Rules.old_payload_rejected(context)
|
|
480
|
+
Rules.new_payload_rejected_by_old(context)
|
|
481
|
+
Rules.contract_narrowed(context)
|
|
482
|
+
Rules.unproven(context)
|
|
483
|
+
```
|
|
484
|
+
|
|
485
|
+
The engine owns precedence by passing an evidence-ownership set keyed by worker and payload arity. JC001 claims any matching head-to-head failure before JC003; JC003 claims remaining head failures before JC002. Individual rules do not inspect already formatted findings.
|
|
486
|
+
|
|
487
|
+
### 8.3 Aggregation key
|
|
488
|
+
|
|
489
|
+
Default key:
|
|
490
|
+
|
|
491
|
+
```text
|
|
492
|
+
[rule_id, worker, payload_arity]
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
JC004, JC005, and JC006 use a worker-level key with null payload arity. `directions` and `revisions` are unioned values, never key components. JC007 uses the full semantic evidence fingerprint from the spec, excluding revision and line/column, rather than primary location. Multiple identical roots within one revision remain separate by occurrence identity. A changed count or ambiguous pairing does not merge across revisions.
|
|
496
|
+
|
|
497
|
+
Before returning a finding, the engine attaches the complete proof locations defined in the rule catalog: relevant producer callsites and every base/head consumer contract used by the membership decision. Formatters never infer or add evidence.
|
|
498
|
+
|
|
499
|
+
## 9. Error architecture
|
|
500
|
+
|
|
501
|
+
Expected failures use typed exceptions under `Jobcompat::Error`:
|
|
502
|
+
|
|
503
|
+
```text
|
|
504
|
+
UsageError
|
|
505
|
+
ConfigError
|
|
506
|
+
GitError
|
|
507
|
+
ParseError
|
|
508
|
+
InternalError (conversion boundary only)
|
|
509
|
+
```
|
|
510
|
+
|
|
511
|
+
`CLI` converts these to exit 2 and a text or JSON diagnostic. The original Git/Prism message can be included after path/ref sanitization. No application source body beyond a single relevant line should be echoed in errors.
|
|
512
|
+
|
|
513
|
+
Unexpected exceptions are caught only at the executable boundary to preserve CLI exit 2. During tests, a configurable runner MAY re-raise so failures retain backtraces. v0.1 has no public debug flag.
|
|
514
|
+
|
|
515
|
+
## 10. Output architecture
|
|
516
|
+
|
|
517
|
+
The engine/result layer supplies all semantic strings or structured data needed by both formats. Text-specific wrapping and labels stay in Text; JSON field construction stays in JSON.
|
|
518
|
+
|
|
519
|
+
Do not derive the JSON document by parsing text output. Do not embed ANSI codes. Use explicit singular/plural helpers only in Text.
|
|
520
|
+
|
|
521
|
+
JSON output is treated as a public API. A fixture/golden test locks a representative success, warning, error, suppression, and tool-failure document.
|
|
522
|
+
|
|
523
|
+
## 11. Dependency decisions
|
|
524
|
+
|
|
525
|
+
### Runtime
|
|
526
|
+
|
|
527
|
+
| Dependency | Decision | Reason |
|
|
528
|
+
| --- | --- | --- |
|
|
529
|
+
| `prism >= 1.9, < 2` | include | official native AST and diagnostics; only non-stdlib runtime dependency |
|
|
530
|
+
| `sidekiq` | exclude | analysis must not boot or load target framework |
|
|
531
|
+
| Thor | exclude | one subcommand; `OptionParser` is sufficient |
|
|
532
|
+
| `parser`/`ast` | exclude | Prism native AST selected; extra abstraction/dependencies add no v0.1 value |
|
|
533
|
+
| JSON/YAML/Open3/OptionParser | standard library/default gems | sufficient for serialization, safe config, subprocess, CLI |
|
|
534
|
+
|
|
535
|
+
### Development/test
|
|
536
|
+
|
|
537
|
+
Choose Minitest over RSpec. Minitest provides assertions, spec-style syntax if desired, low boot overhead, and fewer dependencies. The project needs data-driven tables and integration helpers more than an extensive DSL. Use `minitest`, `rake`, and optionally `simplecov` only if coverage enforcement is explicitly adopted during implementation; do not add SimpleCov by default merely for a number.
|
|
538
|
+
|
|
539
|
+
## 12. Performance and resource behavior
|
|
540
|
+
|
|
541
|
+
- enumerate each commit once;
|
|
542
|
+
- parse selected `.rb` blobs for full analysis; for only potential JC004/JC005 names, stream excluded tracked `.rb` blobs through a literal leaf-token prefilter and parse candidate-bearing blobs for presence only;
|
|
543
|
+
- cap the unselected presence pass at 64 MiB per snapshot; if exceeded, mark unresolved candidates `unverified` and emit JC007 rather than an ERROR;
|
|
544
|
+
- parse each unique blob OID once per process;
|
|
545
|
+
- stream blobs and release source/AST references after extracting facts;
|
|
546
|
+
- avoid worker threads in v0.1 until profiling proves parsing is a bottleneck;
|
|
547
|
+
- do not use `git diff` to scan only changed files because unchanged producers/consumers are required for cross-revision semantics;
|
|
548
|
+
- avoid repository-wide source snippets in results.
|
|
549
|
+
|
|
550
|
+
No hard source-file-size limit is configured in v0.1. The implementation SHOULD handle I/O incrementally and MAY add a documented safety limit only with a clear diagnostic, not silent skipping.
|
|
551
|
+
|
|
552
|
+
## 13. Security and privacy
|
|
553
|
+
|
|
554
|
+
The v0.1 security posture is a product characteristic:
|
|
555
|
+
|
|
556
|
+
- local source remains local;
|
|
557
|
+
- no source upload or telemetry;
|
|
558
|
+
- no external APIs;
|
|
559
|
+
- no Redis credentials;
|
|
560
|
+
- no application execution;
|
|
561
|
+
- no YAML object construction;
|
|
562
|
+
- no shell interpolation;
|
|
563
|
+
- no Git checkout filters or hooks;
|
|
564
|
+
- findings contain only necessary source lines/locations.
|
|
565
|
+
|
|
566
|
+
Threat boundary: the repository and config may be untrusted input. Prism and Git process those bytes, but jobcompat never evaluates Ruby. JSON/text escaping MUST prevent control characters in paths/excerpts from corrupting output framing. Text formatter should escape non-printable path characters; JSON handles them through the JSON encoder.
|
|
567
|
+
|
|
568
|
+
## 14. Future extension points without v0.1 over-engineering
|
|
569
|
+
|
|
570
|
+
The following seams are sufficient:
|
|
571
|
+
|
|
572
|
+
- `SidekiqAnalyzer` can later be joined by another analyzer behind a newly designed interface;
|
|
573
|
+
- `EnqueueCall` and `WorkerContract` represent framework-neutral concepts at the arity level, but fields should not be generalized prematurely;
|
|
574
|
+
- rule IDs remain product-level even if future adapters add their own namespaces;
|
|
575
|
+
- JSON `schema_version` and additive fields support integrations;
|
|
576
|
+
- a later SARIF formatter can consume `Finding` without changing the engine;
|
|
577
|
+
- a later live-queue mode must be a separate explicit command/input source, not silently mixed into static `check`.
|
|
578
|
+
|
|
579
|
+
Potential roadmap order:
|
|
580
|
+
|
|
581
|
+
1. more native Sidekiq producer forms (`Client.push`, bulk) after precision research;
|
|
582
|
+
2. SARIF/GitHub integration;
|
|
583
|
+
3. optional changed-path performance hints without weakening whole-snapshot facts;
|
|
584
|
+
4. ActiveJob as a separately specified adapter;
|
|
585
|
+
5. other ecosystems only after defining their serialization/deployment contracts.
|
|
586
|
+
|
|
587
|
+
Do not add an adapter registry, dependency injection container, generic AST facade, or rule plugin system in v0.1.
|