metanorma-release 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.rspec +3 -0
- data/.rubocop.yml +1 -0
- data/.rubocop_todo.yml +504 -0
- data/CHANGELOG.md +15 -0
- data/PROMPT.md +282 -0
- data/README.adoc +430 -0
- data/Rakefile +8 -0
- data/exe/mn-release +6 -0
- data/lib/metanorma/release/aggregation_interfaces.rb +33 -0
- data/lib/metanorma/release/aggregation_pipeline.rb +155 -0
- data/lib/metanorma/release/asset_processor.rb +58 -0
- data/lib/metanorma/release/cache_store.rb +86 -0
- data/lib/metanorma/release/change_detector.rb +20 -0
- data/lib/metanorma/release/channel.rb +64 -0
- data/lib/metanorma/release/channel_audience.rb +24 -0
- data/lib/metanorma/release/channel_config.rb +55 -0
- data/lib/metanorma/release/channel_filter.rb +26 -0
- data/lib/metanorma/release/channel_manifest.rb +192 -0
- data/lib/metanorma/release/channel_registry.rb +60 -0
- data/lib/metanorma/release/cli.rb +129 -0
- data/lib/metanorma/release/commands/aggregate.rb +126 -0
- data/lib/metanorma/release/commands/package.rb +46 -0
- data/lib/metanorma/release/commands/publish.rb +51 -0
- data/lib/metanorma/release/config_fetcher.rb +11 -0
- data/lib/metanorma/release/config_locator.rb +37 -0
- data/lib/metanorma/release/config_resolver.rb +37 -0
- data/lib/metanorma/release/content_hash.rb +51 -0
- data/lib/metanorma/release/delta_state.rb +108 -0
- data/lib/metanorma/release/document_id.rb +45 -0
- data/lib/metanorma/release/document_index.rb +183 -0
- data/lib/metanorma/release/document_metadata.rb +39 -0
- data/lib/metanorma/release/document_stage.rb +86 -0
- data/lib/metanorma/release/document_type.rb +55 -0
- data/lib/metanorma/release/document_version.rb +50 -0
- data/lib/metanorma/release/file_routing.rb +51 -0
- data/lib/metanorma/release/interfaces.rb +47 -0
- data/lib/metanorma/release/naming_strategy.rb +158 -0
- data/lib/metanorma/release/platform/github/config_fetcher.rb +40 -0
- data/lib/metanorma/release/platform/github/manifest_reader.rb +32 -0
- data/lib/metanorma/release/platform/github/publisher.rb +73 -0
- data/lib/metanorma/release/platform/github/release_fetcher.rb +52 -0
- data/lib/metanorma/release/platform/github/topic_discoverer.rb +29 -0
- data/lib/metanorma/release/platform/github.rb +25 -0
- data/lib/metanorma/release/platform/local/config_fetcher.rb +20 -0
- data/lib/metanorma/release/platform/local/directory_discoverer.rb +26 -0
- data/lib/metanorma/release/platform/local/fetcher.rb +76 -0
- data/lib/metanorma/release/platform/local/publisher.rb +44 -0
- data/lib/metanorma/release/platform/local.rb +14 -0
- data/lib/metanorma/release/platform/null/publisher.rb +17 -0
- data/lib/metanorma/release/platform/null.rb +11 -0
- data/lib/metanorma/release/platform.rb +11 -0
- data/lib/metanorma/release/platform_factory.rb +78 -0
- data/lib/metanorma/release/rake_tasks.rb +71 -0
- data/lib/metanorma/release/relaton_enricher.rb +138 -0
- data/lib/metanorma/release/release_metadata.rb +79 -0
- data/lib/metanorma/release/release_pipeline.rb +115 -0
- data/lib/metanorma/release/release_tag.rb +49 -0
- data/lib/metanorma/release/repo_ref.rb +34 -0
- data/lib/metanorma/release/rxl_extractor.rb +115 -0
- data/lib/metanorma/release/stage_filter.rb +18 -0
- data/lib/metanorma/release/version.rb +7 -0
- data/lib/metanorma/release/zip_packager.rb +37 -0
- data/lib/metanorma/release.rb +116 -0
- metadata +156 -0
data/PROMPT.md
ADDED
|
@@ -0,0 +1,282 @@
|
|
|
1
|
+
# PROMPT.md — metanorma-release Implementation Guide
|
|
2
|
+
|
|
3
|
+
This file provides guidance to AI agents and developers implementing the `metanorma-release` gem. Read it before starting any task. Re-read it when in doubt.
|
|
4
|
+
|
|
5
|
+
## What This Gem Does
|
|
6
|
+
|
|
7
|
+
`metanorma-release` manages the full release lifecycle of Metanorma documents:
|
|
8
|
+
|
|
9
|
+
1. **Release** (producer side): Discover compiled documents → extract metadata from RXL → detect changes → package as zip → publish to a platform (GitHub Releases, GitLab Releases, local filesystem)
|
|
10
|
+
2. **Aggregate** (consumer side): Discover repos → fetch published releases → filter by channel/stage → extract zip assets → generate `index.json` + file tree
|
|
11
|
+
|
|
12
|
+
It works locally (offline, no CI) and in CI (GitHub Actions, GitLab CI, Bitbucket). The output is platform-agnostic: a directory containing `index.json` and a tree of document files. Any site generator (Jekyll, Hugo, Vite, handcrafted) consumes that output independently.
|
|
13
|
+
|
|
14
|
+
## Implementation Tasks
|
|
15
|
+
|
|
16
|
+
Detailed task specifications are in `TODO.init/01-*.md` through `TODO.init/16-*.md`. Execute them in numerical order. Each task file specifies: files to create, specs to write, design decisions, and acceptance criteria. Do not skip ahead — earlier tasks are dependencies for later ones.
|
|
17
|
+
|
|
18
|
+
Before starting a task, read the task file completely. After completing a task, verify all acceptance criteria pass.
|
|
19
|
+
|
|
20
|
+
## Architecture
|
|
21
|
+
|
|
22
|
+
### Dependency Flow (unidirectional, no cycles)
|
|
23
|
+
|
|
24
|
+
```
|
|
25
|
+
domain/ → release/ → platform/
|
|
26
|
+
→ aggregation/ → platform/
|
|
27
|
+
→ cli/
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
- `domain/` has zero knowledge of pipelines, platforms, or CLI
|
|
31
|
+
- Pipelines depend on domain + interfaces, not on platform implementations
|
|
32
|
+
- Platform adapters depend on interfaces + domain, not on pipelines
|
|
33
|
+
- CLI depends on pipelines + platform adapters, wiring them together
|
|
34
|
+
|
|
35
|
+
### Key Patterns
|
|
36
|
+
|
|
37
|
+
**Value Objects**: Immutable, frozen, value-based equality. Created via `self.from_*` factory methods, never mutated. Examples: `DocumentId`, `Channel`, `ReleaseTag`, `ContentHash`.
|
|
38
|
+
|
|
39
|
+
**Strategy Pattern**: Pluggable algorithms behind a common interface. Resolved via registry (Open/Closed). Examples: `NamingStrategy` (5 implementations), `FileRouting` (3 implementations).
|
|
40
|
+
|
|
41
|
+
**Pipeline with DI**: Orchestrators receive all dependencies through constructors. No global state, no service locators, no `send`, no `respond_to?`. Pipelines compose domain objects and delegate to injected adapters.
|
|
42
|
+
|
|
43
|
+
**Null Object**: When a feature is disabled, inject a null implementation instead of adding conditional checks. Examples: `NullDeltaState`, `NullPublisher`, `NullCacheStore`.
|
|
44
|
+
|
|
45
|
+
**Result Types**: Pipelines return frozen Structs (`ReleaseResult`, `AggregationResult`). Errors are collected, not raised. Callers inspect `result.failed` to decide whether to abort.
|
|
46
|
+
|
|
47
|
+
## Hard Rules
|
|
48
|
+
|
|
49
|
+
### Never use `send` or `__send__`
|
|
50
|
+
|
|
51
|
+
It breaks encapsulation by calling private methods from outside. If you need a behavior, make it public or restructure the design. If you need polymorphic dispatch, use the Strategy pattern with a common interface.
|
|
52
|
+
|
|
53
|
+
### Never use `respond_to?`
|
|
54
|
+
|
|
55
|
+
It probes an object's internals instead of trusting the type contract. If an object includes `Metanorma::Release::Publisher`, it implements `publish`. Period. If it doesn't, you get a `NoMethodError` — that's the correct failure mode.
|
|
56
|
+
|
|
57
|
+
### Never use `method_missing`
|
|
58
|
+
|
|
59
|
+
If you need dynamic dispatch, use a registry or strategy pattern. `method_missing` hides bugs and makes debugging painful.
|
|
60
|
+
|
|
61
|
+
### No runtime dependencies in the gemspec
|
|
62
|
+
|
|
63
|
+
The gem core must be `require`-able with zero gem dependencies beyond stdlib. Platform-specific libraries (`octokit`, `rubyzip`, `nokogiri`, `concurrent-ruby`) are optional. Adapters that need them check at load time:
|
|
64
|
+
|
|
65
|
+
```ruby
|
|
66
|
+
begin
|
|
67
|
+
require "octokit"
|
|
68
|
+
rescue LoadError
|
|
69
|
+
raise LoadError, "The octokit gem is required for GitHub adapters. Add `gem 'octokit'` to your Gemfile."
|
|
70
|
+
end
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### All value objects are frozen
|
|
74
|
+
|
|
75
|
+
After construction, call `freeze`. No setters, no mutation, no `@field =` after `initialize`. Equality is value-based: implement `eql?` and `hash`.
|
|
76
|
+
|
|
77
|
+
### No conditionals on type
|
|
78
|
+
|
|
79
|
+
Don't write `if x.is_a?(GitHubPublisher)` or `case platform when :github`. Use the Strategy or Adapter pattern — the caller shouldn't know which concrete type it holds.
|
|
80
|
+
|
|
81
|
+
## Design Principles
|
|
82
|
+
|
|
83
|
+
### OOP
|
|
84
|
+
|
|
85
|
+
Objects own their data and behavior. No anaemic data structures with separate service classes. A `DocumentStage` knows whether it's published. A `NamingStrategy` knows how to compute a tag. A `ChannelManifest` knows how to resolve a policy.
|
|
86
|
+
|
|
87
|
+
### MECE
|
|
88
|
+
|
|
89
|
+
Every concern is handled by exactly one class. No two classes do the same thing. Together, all classes cover the entire domain. If you're unsure where a piece of logic belongs, check the responsibility boundary:
|
|
90
|
+
|
|
91
|
+
| Concern | Owner |
|
|
92
|
+
|---------|-------|
|
|
93
|
+
| Identifier normalization | `DocumentId` |
|
|
94
|
+
| Stage classification | `DocumentStage` |
|
|
95
|
+
| Tag format | `ReleaseTag` + `NamingStrategy` |
|
|
96
|
+
| Channel matching | `ChannelFilter` |
|
|
97
|
+
| Change detection | `ChangeDetector` implementations |
|
|
98
|
+
| Zip creation | `ZipPackager` |
|
|
99
|
+
| Release publishing | Platform `Publisher` adapters |
|
|
100
|
+
| Repo discovery | Platform `Discoverer` adapters |
|
|
101
|
+
| Delta state | `DeltaState` |
|
|
102
|
+
| Index schema | `DocumentIndex` |
|
|
103
|
+
| File organization | `FileRouting` strategies |
|
|
104
|
+
| Pipeline orchestration | `ReleasePipeline` / `AggregationPipeline` |
|
|
105
|
+
| Argument parsing | `CLI` |
|
|
106
|
+
| Task registration | `RakeTasks` |
|
|
107
|
+
|
|
108
|
+
### Open/Closed
|
|
109
|
+
|
|
110
|
+
New document types: register a new `NamingStrategy`. New platforms: create a new directory under `platform/`. New file routing modes: create a new `FileRouting` class. New release filters: create a new `Filter` class. **Never modify existing code to add a new variant.**
|
|
111
|
+
|
|
112
|
+
The registry pattern is the primary mechanism:
|
|
113
|
+
|
|
114
|
+
```ruby
|
|
115
|
+
# Open/Closed: adding a new platform requires zero changes to existing code
|
|
116
|
+
class NamingRegistry
|
|
117
|
+
def register(document_type, strategy) # Add new types here
|
|
118
|
+
def resolve(document_type) # No case/when — strategy lookup
|
|
119
|
+
end
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
### DRY
|
|
123
|
+
|
|
124
|
+
Don't duplicate the channel model between release and aggregation — both use `Channel`. Don't duplicate content hashing — both pipelines use `ContentHash`. Don't duplicate naming logic — all strategies go through `NamingStrategy`. The `ReleaseMetadata` format is the single data contract between release and aggregate.
|
|
125
|
+
|
|
126
|
+
### Performance
|
|
127
|
+
|
|
128
|
+
- Parallel processing where safe: repos in aggregation, documents in release. Use `Thread` with bounded concurrency (no external deps).
|
|
129
|
+
- ETag-based skip: don't re-fetch unchanged repo data.
|
|
130
|
+
- Content-hash dedup: don't re-extract unchanged release zips.
|
|
131
|
+
- Lazy loading: don't `require` platform adapters until needed.
|
|
132
|
+
- Frozen strings: add `# frozen_string_literal: true` to every Ruby file.
|
|
133
|
+
|
|
134
|
+
## Interface Contracts
|
|
135
|
+
|
|
136
|
+
Ruby doesn't have native interfaces. We use modules with stub methods that raise `NotImplementedError`:
|
|
137
|
+
|
|
138
|
+
```ruby
|
|
139
|
+
module Metanorma::Release::Publisher
|
|
140
|
+
def publish(tag, artifact, metadata, channels:, force_replace: false)
|
|
141
|
+
raise NotImplementedError, "#{self.class} must implement #publish"
|
|
142
|
+
end
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
class GitHubPublisher
|
|
146
|
+
include Metanorma::Release::Publisher
|
|
147
|
+
|
|
148
|
+
def publish(tag, artifact, metadata, channels:, force_replace: false)
|
|
149
|
+
# concrete implementation
|
|
150
|
+
end
|
|
151
|
+
end
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
This gives us:
|
|
155
|
+
- Documentation: the module lists the contract
|
|
156
|
+
- Runtime enforcement: missing method → clear error with class name
|
|
157
|
+
- Type checking: `publisher.is_a?(Metanorma::Release::Publisher)` if needed
|
|
158
|
+
|
|
159
|
+
**Never check the interface with `respond_to?`**. If an object includes the module, trust the contract.
|
|
160
|
+
|
|
161
|
+
## Spec Requirements
|
|
162
|
+
|
|
163
|
+
### Every value object has exhaustive specs
|
|
164
|
+
|
|
165
|
+
Test normalization edge cases, equality, hash consistency, rejection of invalid input. Don't just test the happy path — test that `"---"` is rejected by `DocumentId`, that ISO stage 37 falls back to "working-draft", that `Channel.parse("standards")` defaults to public audience.
|
|
166
|
+
|
|
167
|
+
### Every interface has a shared example
|
|
168
|
+
|
|
169
|
+
```ruby
|
|
170
|
+
RSpec.shared_examples "a naming strategy" do |id:, version:, expected_tag:, expected_asset:, expected_canonical:|
|
|
171
|
+
let(:strategy) { described_class.new }
|
|
172
|
+
|
|
173
|
+
it "computes the correct tag" do
|
|
174
|
+
expect(strategy.compute_tag(id, version).to_s).to eq(expected_tag)
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
it "computes the correct asset name" do
|
|
178
|
+
expect(strategy.compute_asset_name(id, version)).to eq(expected_asset)
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
it "computes the correct canonical base" do
|
|
182
|
+
expect(strategy.compute_canonical_base(id, version)).to eq(expected_canonical)
|
|
183
|
+
end
|
|
184
|
+
end
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
### Every pipeline has mock-based specs
|
|
188
|
+
|
|
189
|
+
Pipelines are tested with mock adapters that implement the interfaces. No real file I/O, no HTTP, no platform APIs in pipeline specs. Use `Struct.new` for quick mocks:
|
|
190
|
+
|
|
191
|
+
```ruby
|
|
192
|
+
let(:mock_publisher) do
|
|
193
|
+
Struct.new(:published) do
|
|
194
|
+
include Metanorma::Release::Publisher
|
|
195
|
+
|
|
196
|
+
def published = @published ||= []
|
|
197
|
+
|
|
198
|
+
def publish(tag, artifact, metadata, channels:, force_replace: false)
|
|
199
|
+
published << { tag: tag, channels: channels }
|
|
200
|
+
Metanorma::Release::PublishResult.new(tag: tag, url: "mock://#{tag}", created?: true)
|
|
201
|
+
end
|
|
202
|
+
end.new([])
|
|
203
|
+
end
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
### Integration specs use local adapters
|
|
207
|
+
|
|
208
|
+
End-to-end specs exercise the full pipeline with `Local::Publisher`, `Local::DirectoryDiscoverer`, `Local::Fetcher`. No network. Fixtures are committed in `spec/fixtures/`.
|
|
209
|
+
|
|
210
|
+
### Spec file organization mirrors source
|
|
211
|
+
|
|
212
|
+
```
|
|
213
|
+
lib/metanorma/release/channel.rb → spec/domain/channel_spec.rb
|
|
214
|
+
lib/metanorma/release/release_pipeline.rb → spec/release/release_pipeline_spec.rb
|
|
215
|
+
lib/metanorma/release/delta_state.rb → spec/aggregation/delta_state_spec.rb
|
|
216
|
+
lib/metanorma/release/platform/github/publisher.rb → spec/platform/github/publisher_spec.rb
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
## Code Style
|
|
220
|
+
|
|
221
|
+
- `# frozen_string_literal: true` at the top of every file
|
|
222
|
+
- No comments unless the WHY is non-obvious (hidden constraint, workaround, surprising invariant)
|
|
223
|
+
- No multi-line docstrings
|
|
224
|
+
- `Struct.new(..., keyword_init: true)` for result types and data objects
|
|
225
|
+
- Factory methods (`self.from_raw`, `self.from_json`, `self.parse`) for construction with validation
|
|
226
|
+
- `include` module interfaces in concrete implementations
|
|
227
|
+
- Constructor keyword arguments for DI
|
|
228
|
+
- `raise` for programmer errors, return error results for expected failures
|
|
229
|
+
- `abort` with message for CLI-level fatal errors
|
|
230
|
+
|
|
231
|
+
## Key Design Decisions
|
|
232
|
+
|
|
233
|
+
### `by-document` is the default file routing
|
|
234
|
+
|
|
235
|
+
Each document gets its own subdirectory. This avoids 500+ files in a flat directory and gives clean URL structure. `flat` mode exists for backward compatibility only.
|
|
236
|
+
|
|
237
|
+
### `formats` in `index.json` includes all extensions
|
|
238
|
+
|
|
239
|
+
Including `"rxl"`. The TS aggregate action omits it from `formats` (only in `files`), which forces consumers to merge both arrays. We fix this.
|
|
240
|
+
|
|
241
|
+
### `doctype` may be empty in `index.json`
|
|
242
|
+
|
|
243
|
+
It comes from the release metadata, which may not specify it. Consumers derive it from channels if needed. This is a consumer concern, not a gem concern.
|
|
244
|
+
|
|
245
|
+
### Channel manifest defaults to private
|
|
246
|
+
|
|
247
|
+
When a `metanorma.release.yml` is loaded but a document is not listed in it, the document is treated as **private** (not released). This is the safe default — you must explicitly declare what gets released.
|
|
248
|
+
|
|
249
|
+
### Two-phase release pipeline
|
|
250
|
+
|
|
251
|
+
Phase 1 (change detection) is read-only. Phase 2 (package + publish) is write. This enables dry-run mode and prevents partial state on failure.
|
|
252
|
+
|
|
253
|
+
### Pipeline errors are collected, not raised
|
|
254
|
+
|
|
255
|
+
`ReleaseResult.failed` and `AggregationResult.failed_repos` contain errors. The pipeline continues processing. The caller (CLI, Rake) decides whether to abort.
|
|
256
|
+
|
|
257
|
+
## Porting Notes
|
|
258
|
+
|
|
259
|
+
The TypeScript implementations (`actions-mn/release` and `actions-mn/aggregate`) are the reference. Port the domain logic faithfully, but apply Ruby idioms:
|
|
260
|
+
|
|
261
|
+
| TypeScript | Ruby |
|
|
262
|
+
|-----------|------|
|
|
263
|
+
| `class DocumentId { private constructor(...) }` | `class DocumentId; def self.from_raw(...); end` |
|
|
264
|
+
| `interface IChangeDetector { ... }` | `module ChangeDetector; def detect(...) = raise NotImplementedError; end` |
|
|
265
|
+
| `readonly` properties | `attr_reader` + `freeze` |
|
|
266
|
+
| `enum ChannelAudience` | Module constants + `from_string` method |
|
|
267
|
+
| `Record<string, RepoState>` | `Hash` with string keys |
|
|
268
|
+
| `readonly Array<T>` | frozen `Array` |
|
|
269
|
+
| `Promise<T>` | synchronous Ruby (use `Thread` for parallelism) |
|
|
270
|
+
| `minimatch(pattern)` | `File.fnmatch(pattern)` |
|
|
271
|
+
|
|
272
|
+
Don't port TypeScript type guards (`typeof x === "string"`). Don't port `as` type assertions. Trust the Ruby interface contracts.
|
|
273
|
+
|
|
274
|
+
## What To Improve Beyond The Plan
|
|
275
|
+
|
|
276
|
+
As you implement, watch for:
|
|
277
|
+
|
|
278
|
+
- **Missing value object behavior**: If you're writing the same normalization/comparison logic in multiple places, it belongs in a value object.
|
|
279
|
+
- **Leaky abstractions**: If a pipeline starts knowing about GitHub API response formats, the adapter boundary is wrong.
|
|
280
|
+
- **Premature concurrency**: Don't add `Thread` until the sequential version is correct and tested. Concurrency is an optimization, not a feature.
|
|
281
|
+
- **Over-specification**: Don't test private methods. Test the public interface. Don't test implementation details (method call order, intermediate variables).
|
|
282
|
+
- **Schema drift**: If you're adding fields to `AggregatedDocument` or `DocumentIndex`, increment the schema version and add validation.
|