asgard 0.3.1 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.envrc +5 -0
- data/.loki +8 -1
- data/.reek.yml +112 -0
- data/.rubocop.yml +6 -0
- data/Archspec.rb +24 -0
- data/CHANGELOG.md +57 -1
- data/CLAUDE.md +4 -7
- data/README.md +10 -1
- data/bin/asgard +1 -1
- data/docs/api.md +122 -5
- data/docs/changelog.md +21 -1
- data/docs/dependencies.md +83 -2
- data/docs/getting-started.md +4 -2
- data/docs/index.md +3 -3
- data/docs/options.md +16 -6
- data/examples/bad.loki +63 -0
- data/gem_tasks.loki +17 -1
- data/git.loki +13 -0
- data/lib/asgard/base/dependency_graph.rb +124 -0
- data/lib/asgard/base/dispatch.rb +160 -0
- data/lib/asgard/base/registry.rb +38 -0
- data/lib/asgard/base/task_dsl.rb +65 -0
- data/lib/asgard/base.rb +36 -250
- data/lib/asgard/doctor/report.rb +73 -0
- data/lib/asgard/doctor/task_sections.rb +67 -0
- data/lib/asgard/doctor.rb +150 -0
- data/lib/asgard/kernel_methods.rb +6 -2
- data/lib/asgard/tasks.rb +6 -0
- data/lib/asgard/version.rb +1 -1
- data/lib/asgard.rb +7 -1
- data/quality.loki +218 -32
- data/quality_rails.loki +46 -0
- data/xyzzy.loki +12 -0
- metadata +15 -16
data/docs/dependencies.md
CHANGED
|
@@ -145,9 +145,54 @@ When `asgard ci` runs, `setup` executes once even though both `test` and `lint`
|
|
|
145
145
|
|
|
146
146
|
---
|
|
147
147
|
|
|
148
|
+
## Transitive Dependencies
|
|
149
|
+
|
|
150
|
+
When a dependency has its own dependencies, Asgard resolves them recursively before running the dependent task. The deduplication set ensures each task runs at most once regardless of how many paths lead to it.
|
|
151
|
+
|
|
152
|
+
Consider this graph:
|
|
153
|
+
|
|
154
|
+
```ruby
|
|
155
|
+
class Tasks
|
|
156
|
+
desc "Fetch gems"
|
|
157
|
+
def setup = sh "bundle install"
|
|
158
|
+
|
|
159
|
+
depends_on :setup
|
|
160
|
+
desc "Compile assets"
|
|
161
|
+
def build = sh "rake assets:precompile"
|
|
162
|
+
|
|
163
|
+
desc "Check code style"
|
|
164
|
+
def lint = sh "bundle exec rubocop"
|
|
165
|
+
|
|
166
|
+
depends_on :build, :lint, :setup
|
|
167
|
+
desc "Run the full pipeline"
|
|
168
|
+
def ci = puts "Done."
|
|
169
|
+
end
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
`ci` declares three sequential dependencies: `build`, `lint`, `setup`. But `build` itself depends on `setup`. The effective execution order is:
|
|
173
|
+
|
|
174
|
+
```
|
|
175
|
+
setup ← run as build's prerequisite
|
|
176
|
+
↓
|
|
177
|
+
build
|
|
178
|
+
↓
|
|
179
|
+
lint
|
|
180
|
+
↓
|
|
181
|
+
(setup skipped — already done)
|
|
182
|
+
↓
|
|
183
|
+
ci
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
`setup` runs once — on its first encounter as `build`'s prerequisite. When `ci`'s own stage for `setup` is reached, the deduplication set skips it.
|
|
187
|
+
|
|
188
|
+
!!! tip
|
|
189
|
+
When a task is both a transitive dependency and a direct dependency, declare it only where it logically belongs — as a prerequisite of the task that needs it. Declaring it redundantly at the top level is harmless (deduplication handles it) but adds noise.
|
|
190
|
+
|
|
191
|
+
---
|
|
192
|
+
|
|
148
193
|
## Circular Dependency Detection
|
|
149
194
|
|
|
150
|
-
Asgard validates the full dependency graph using [
|
|
195
|
+
Asgard validates the full dependency graph using stdlib [TSort](https://docs.ruby-lang.org/en/master/TSort.html) before any task runs. A circular dependency produces a clean error and exits:
|
|
151
196
|
|
|
152
197
|
```ruby
|
|
153
198
|
class Tasks
|
|
@@ -187,7 +232,43 @@ class Tasks
|
|
|
187
232
|
end
|
|
188
233
|
```
|
|
189
234
|
|
|
190
|
-
|
|
235
|
+
Because `*.loki` files are loaded alphabetically when `import "*.loki"` is used, `build.loki` loads before `test.loki`. If you need to control load order precisely, use explicit `import` calls with full filenames rather than a glob.
|
|
236
|
+
|
|
237
|
+
---
|
|
238
|
+
|
|
239
|
+
## Dynamic Dependencies (Proc Form)
|
|
240
|
+
|
|
241
|
+
`depends_on` normally takes a fixed list, recorded the moment the `def` right after it is encountered — which is why load order matters, as above. Pass a `Proc` or lambda instead, and that list is computed *later*, after every `.loki` file has finished loading, rather than at the point `depends_on` itself is evaluated:
|
|
242
|
+
|
|
243
|
+
```ruby
|
|
244
|
+
depends_on -> { [all_commands.keys.grep(/_check\z/).sort.map(&:to_sym)] }
|
|
245
|
+
desc "Run every *_check quality gate task in parallel"
|
|
246
|
+
def quality
|
|
247
|
+
# ...
|
|
248
|
+
end
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
This solves exactly the "load order matters" problem from the previous section: a plain array can only name tasks that already exist in `.loki` files loaded *before* this one. A Proc is resolved once every file has loaded, so it can safely reference a task defined in a file that hasn't been imported yet at the point `depends_on` is written — including one that only exists conditionally, e.g. a Rails-specific task file imported with `import "quality_rails.loki" if defined?(Rails)`.
|
|
252
|
+
|
|
253
|
+
**Shape:** the Proc must return exactly what the plain-array form would receive as its splat arguments — an array of stages, each a `Symbol` (sequential) or `Array` (parallel group). The example above returns `[[:a_check, :b_check, :c_check]]`: one stage, containing every matching task, all running in parallel — the same shape as `depends_on [:a_check, :b_check, :c_check]`.
|
|
254
|
+
|
|
255
|
+
**When it runs:** once, when `validate_deps!` runs (right after the `.loki` chain finishes loading, before any task dispatches). The resolved result replaces the Proc in the dependency table, so cycle detection, undefined-task checks, and arity checks all run against the *resolved* list — a Proc that references an undefined task, or that itself introduces a cycle, is caught at startup exactly like a plain array would be:
|
|
256
|
+
|
|
257
|
+
```bash
|
|
258
|
+
asgard quality
|
|
259
|
+
# asgard: undefined task(s) in depends_on: ghost_check
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
**`self` inside the Proc:** since the Proc is written directly in a `class Tasks` body, it lexically captures that class as `self` — so it can call `all_commands`, `_deps`, or any other class-level method bare, without a `self.class.` prefix, even though it's actually invoked later from inside `validate_deps!`.
|
|
263
|
+
|
|
264
|
+
**If the Proc raises**, the error is caught and re-raised as `Asgard::Error` naming the task it was declared for:
|
|
265
|
+
|
|
266
|
+
```
|
|
267
|
+
asgard: depends_on proc for 'quality' raised RuntimeError: boom
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
!!! tip
|
|
271
|
+
Reach for this only when the dependency list genuinely can't be known until every file has loaded — like "every task whose name ends in `_check`," discovered across several `.loki` files. For a fixed, known-upfront list, the plain array form is simpler and reads just as clearly.
|
|
191
272
|
|
|
192
273
|
---
|
|
193
274
|
|
data/docs/getting-started.md
CHANGED
|
@@ -138,24 +138,26 @@ myproject/
|
|
|
138
138
|
qa.loki ← test and lint tasks
|
|
139
139
|
```
|
|
140
140
|
|
|
141
|
-
Each `*.loki` file reopens `class Tasks`. To load them,
|
|
141
|
+
Each `*.loki` file reopens `class Tasks`. To load them, call `import "*.loki"` at the top of `.loki` — files are loaded in alphabetical order. See [Task Files](task-files.md) for full details.
|
|
142
142
|
|
|
143
143
|
---
|
|
144
144
|
|
|
145
145
|
## Built-in Flags
|
|
146
146
|
|
|
147
|
-
Every task automatically has
|
|
147
|
+
Every task automatically has four flags available, defined as `class_option` on `Tasks`:
|
|
148
148
|
|
|
149
149
|
| Flag | Description |
|
|
150
150
|
|---|---|
|
|
151
151
|
| `--version` | Print the Asgard version and exit |
|
|
152
152
|
| `--debug` | Set `$DEBUG = true` before the task runs |
|
|
153
153
|
| `--verbose` | Set `$VERBOSE = true` before the task runs |
|
|
154
|
+
| `--doctor` | Diagnose `.loki` resolution, imports, and task definitions for the CWD, then exit |
|
|
154
155
|
|
|
155
156
|
```bash
|
|
156
157
|
asgard --version
|
|
157
158
|
asgard hello --debug
|
|
158
159
|
asgard hello --verbose
|
|
160
|
+
asgard --doctor
|
|
159
161
|
```
|
|
160
162
|
|
|
161
163
|
Inside a task body, use the `debug?` and `verbose?` predicates:
|
data/docs/index.md
CHANGED
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
<li><strong>Dotenv Support</strong> — load <code>.env</code> files into the environment with <code>dotenv</code></li>
|
|
19
19
|
<li><strong>Auto-Discovery</strong> — <code>.loki</code> root marker searched from CWD upward through parent directories</li>
|
|
20
20
|
<li><strong>Multi-File Tasks</strong> — split tasks across <code>*.loki</code> files loaded via <code>import</code></li>
|
|
21
|
-
<li><strong>Built-in Flags</strong> — <code>--version</code>, <code>--debug</code>, and <code>--
|
|
21
|
+
<li><strong>Built-in Flags</strong> — <code>--version</code>, <code>--debug</code>, <code>--verbose</code>, and <code>--doctor</code> available on every task</li>
|
|
22
22
|
</ul>
|
|
23
23
|
</td>
|
|
24
24
|
</tr>
|
|
@@ -72,7 +72,7 @@ The full Thor DSL is available: `desc`, `method_option`, `class_option`, `long_d
|
|
|
72
72
|
| [Subcommands](subcommands.md) | Grouping tasks under a namespace |
|
|
73
73
|
| [Shell Helpers](shell.md) | `sh`, `shebang`, and supported interpreters |
|
|
74
74
|
| [Environment](environment.md) | Loading `.env` files with `dotenv` |
|
|
75
|
-
| [Task Files](task-files.md) | `.loki` root marker,
|
|
75
|
+
| [Task Files](task-files.md) | `.loki` root marker, `import`, multi-file layout |
|
|
76
76
|
| [API Reference](api.md) | Module methods, DSL methods, error classes |
|
|
77
77
|
| [Examples](examples.md) | Working `.loki` files for every feature |
|
|
78
78
|
| [Changelog](changelog.md) | Release history |
|
|
@@ -82,4 +82,4 @@ The full Thor DSL is available: `desc`, `method_option`, `class_option`, `long_d
|
|
|
82
82
|
## Requirements
|
|
83
83
|
|
|
84
84
|
- Ruby >= 3.2.0
|
|
85
|
-
- Dependencies: [thor](https://github.com/rails/thor) `~> 1.0`, [
|
|
85
|
+
- Dependencies: [thor](https://github.com/rails/thor) `~> 1.0`, [dotenv](https://github.com/bkeepers/dotenv) `~> 3.0`
|
data/docs/options.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Options & Flags
|
|
2
2
|
|
|
3
|
-
Asgard tasks use the full Thor option system. Options declared with `method_option` (alias: `option`) apply to a single task. Options declared with `class_option` apply to every task in the class. Asgard ships with
|
|
3
|
+
Asgard tasks use the full Thor option system. Options declared with `method_option` (alias: `option`) apply to a single task. Options declared with `class_option` apply to every task in the class. Asgard ships with four built-in `class_option` declarations on `Tasks`: `--debug`, `--verbose`, `--version`, and `--doctor`.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -80,7 +80,7 @@ class Tasks
|
|
|
80
80
|
class_option :color,
|
|
81
81
|
type: :boolean,
|
|
82
82
|
default: true,
|
|
83
|
-
desc: "
|
|
83
|
+
desc: "Colorize output"
|
|
84
84
|
no_negate :color
|
|
85
85
|
end
|
|
86
86
|
```
|
|
@@ -88,13 +88,13 @@ end
|
|
|
88
88
|
Help output before `no_negate`:
|
|
89
89
|
|
|
90
90
|
```
|
|
91
|
-
[--color], [--no-color], [--skip-color] #
|
|
91
|
+
[--color], [--no-color], [--skip-color] # Colorize output
|
|
92
92
|
```
|
|
93
93
|
|
|
94
94
|
Help output after `no_negate`:
|
|
95
95
|
|
|
96
96
|
```
|
|
97
|
-
[--color] #
|
|
97
|
+
[--color] # Colorize output
|
|
98
98
|
```
|
|
99
99
|
|
|
100
100
|
`no_negate` accepts multiple option names in a single call:
|
|
@@ -109,7 +109,7 @@ It has no effect on runtime behaviour — `--no-color` still works on the CLI; o
|
|
|
109
109
|
|
|
110
110
|
## Built-in Flags
|
|
111
111
|
|
|
112
|
-
`Tasks` ships with
|
|
112
|
+
`Tasks` ships with four built-in `class_option` declarations — `--debug`, `--verbose`, `--version`, and `--doctor` — all visible in the Options section of `asgard help`.
|
|
113
113
|
|
|
114
114
|
### `--version`
|
|
115
115
|
|
|
@@ -120,6 +120,16 @@ asgard --version
|
|
|
120
120
|
# 0.3.0
|
|
121
121
|
```
|
|
122
122
|
|
|
123
|
+
### `--doctor`
|
|
124
|
+
|
|
125
|
+
A `class_option :doctor` of type `:boolean`. Diagnoses `.loki` resolution, import chains, and task definitions for the current directory, then exits. Handled by `Asgard.run!` before the `.loki` file is loaded (same pattern as `--version`) — deliberately so, since it needs to keep working in the exact situations that would otherwise abort `run!`: a broken `.loki` file, a circular or undefined dependency, or a task silently redefined by a later `def`. The report includes a "Tasks by file" listing — every command grouped by the file it's defined in, as `file:line` — with any silently-overridden task called out inline, right where it's defined. `no_negate :doctor` suppresses the `[--no-doctor]` / `[--skip-doctor]` variants:
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
asgard --doctor
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
See [`Asgard::Doctor` in the API Reference](api.md#asgarddoctor) for the full breakdown of what it checks.
|
|
132
|
+
|
|
123
133
|
### `--debug`
|
|
124
134
|
|
|
125
135
|
A `class_option :debug` of type `:boolean`. When passed, sets `$DEBUG = true` before the task body runs (via the `invoke_command` hook in `Asgard::Base`):
|
|
@@ -211,4 +221,4 @@ asgard _something
|
|
|
211
221
|
# asgard: unknown command '_something'
|
|
212
222
|
```
|
|
213
223
|
|
|
214
|
-
If you define your own methods on `Tasks`, avoid the `_` prefix to prevent them from being blocked. Built-in `class_option` declarations (like `--version`, `--debug`, `--verbose`) do not use the `_` prefix because they are options, not commands.
|
|
224
|
+
If you define your own methods on `Tasks`, avoid the `_` prefix to prevent them from being blocked. Built-in `class_option` declarations (like `--version`, `--debug`, `--verbose`, `--doctor`) do not use the `_` prefix because they are options, not commands.
|
data/examples/bad.loki
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
# Demonstrates review.md Critical Issue #1: parallel depends_on tasks that
|
|
3
|
+
# write to shared instance state on `self` race each other.
|
|
4
|
+
#
|
|
5
|
+
# quality.loki's *_check tasks each write a *different* named ivar
|
|
6
|
+
# (@test_result, @rubocop_result, ...), so MRI's GVL happens to hide the
|
|
7
|
+
# race in practice — every write lands on a distinct slot. This file
|
|
8
|
+
# isolates the actual hazard by having 4 parallel workers read-modify-write
|
|
9
|
+
# the SAME @hits counter, the way any real *_check task would if it
|
|
10
|
+
# accumulated into one shared results structure instead of one ivar each.
|
|
11
|
+
#
|
|
12
|
+
# `@hits = @hits + 1` is not one atomic operation — it's a read, an add,
|
|
13
|
+
# then a write — and MRI can switch threads between those steps. Thread.pass
|
|
14
|
+
# inside the loop forces exactly those switches, so the lost-update race
|
|
15
|
+
# shows up reliably instead of "most of the time."
|
|
16
|
+
#
|
|
17
|
+
# Run with:
|
|
18
|
+
# asgard bad_race
|
|
19
|
+
#
|
|
20
|
+
# Expected @hits: 4000 (4 workers x 1000 increments each)
|
|
21
|
+
# Actual: consistently lower — proof of the lost-update race described in
|
|
22
|
+
# review.md item #1.
|
|
23
|
+
|
|
24
|
+
BAD_RACE_WORKERS = 4
|
|
25
|
+
BAD_RACE_REPS = 1000
|
|
26
|
+
|
|
27
|
+
class Tasks
|
|
28
|
+
desc "Increment the shared @hits counter, unsynchronized (worker)"
|
|
29
|
+
def racer_1 = bump_shared_counter
|
|
30
|
+
|
|
31
|
+
desc "Increment the shared @hits counter, unsynchronized (worker)"
|
|
32
|
+
def racer_2 = bump_shared_counter
|
|
33
|
+
|
|
34
|
+
desc "Increment the shared @hits counter, unsynchronized (worker)"
|
|
35
|
+
def racer_3 = bump_shared_counter
|
|
36
|
+
|
|
37
|
+
desc "Increment the shared @hits counter, unsynchronized (worker)"
|
|
38
|
+
def racer_4 = bump_shared_counter
|
|
39
|
+
|
|
40
|
+
depends_on [:racer_1, :racer_2, :racer_3, :racer_4]
|
|
41
|
+
desc "Show the lost-update race in @hits"
|
|
42
|
+
def bad_race
|
|
43
|
+
expected = BAD_RACE_WORKERS * BAD_RACE_REPS
|
|
44
|
+
puts "expected @hits == #{expected}, got #{@hits}"
|
|
45
|
+
|
|
46
|
+
if @hits == expected
|
|
47
|
+
puts "no corruption this run — the race is timing-dependent, rerun a few times"
|
|
48
|
+
else
|
|
49
|
+
puts "DATA RACE CONFIRMED: #{expected - @hits} update(s) lost"
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
no_commands do
|
|
54
|
+
def bump_shared_counter
|
|
55
|
+
@hits ||= 0
|
|
56
|
+
BAD_RACE_REPS.times do
|
|
57
|
+
current = @hits # read
|
|
58
|
+
Thread.pass # force a context switch before the write lands
|
|
59
|
+
@hits = current + 1 # write — another thread's read may already be stale
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
end
|
data/gem_tasks.loki
CHANGED
|
@@ -2,6 +2,15 @@
|
|
|
2
2
|
# Gem lifecycle tasks — imported by .loki
|
|
3
3
|
|
|
4
4
|
class Tasks
|
|
5
|
+
desc "Open IRB console with the gem loaded"
|
|
6
|
+
def console
|
|
7
|
+
if File.exist?("bin/console")
|
|
8
|
+
sh "bin/console"
|
|
9
|
+
else
|
|
10
|
+
sh "bundle exec irb -Ilib -r #{@@project}"
|
|
11
|
+
end
|
|
12
|
+
end
|
|
13
|
+
|
|
5
14
|
desc "Build the gem package"
|
|
6
15
|
depends_on :quality
|
|
7
16
|
def build
|
|
@@ -16,12 +25,19 @@ class Tasks
|
|
|
16
25
|
sh "gem install pkg/asgard-#{project_version}.gem"
|
|
17
26
|
end
|
|
18
27
|
|
|
19
|
-
desc "Release to RubyGems"
|
|
28
|
+
desc "release", "Release gem to RubyGems (runs quality gate first)"
|
|
29
|
+
option :yes, aliases: "-y", type: :boolean, default: false, desc: "Skip confirmation prompt"
|
|
20
30
|
depends_on :quality
|
|
21
31
|
def release
|
|
22
32
|
tag = "v#{project_version}"
|
|
23
33
|
gem_file = "pkg/asgard-#{project_version}.gem"
|
|
24
34
|
|
|
35
|
+
unless options[:yes]
|
|
36
|
+
print "Release #{@@project} v#{project_version} to RubyGems? [y/N] "
|
|
37
|
+
$stdout.flush
|
|
38
|
+
return puts "Aborted." unless $stdin.gets.strip.downcase == "y"
|
|
39
|
+
end
|
|
40
|
+
|
|
25
41
|
abort "Working directory is not clean — commit or stash changes first." unless `git status --porcelain`.strip.empty?
|
|
26
42
|
abort "Tag #{tag} already exists." unless `git tag -l #{tag}`.strip.empty?
|
|
27
43
|
|
data/git.loki
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
# Per-repo git tasks — imported by .loki
|
|
3
|
+
|
|
4
|
+
class Tasks
|
|
5
|
+
desc "Push the current branch to its remote"
|
|
6
|
+
def push = sh "git push"
|
|
7
|
+
|
|
8
|
+
desc "Pull (fast-forward only) from the remote"
|
|
9
|
+
def pull = sh "git pull --ff-only"
|
|
10
|
+
|
|
11
|
+
desc "Fetch from the remote"
|
|
12
|
+
def fetch = sh "git fetch"
|
|
13
|
+
end
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Asgard
|
|
4
|
+
class Base < Thor
|
|
5
|
+
# Dependency declaration (`depends_on`) and full-graph validation
|
|
6
|
+
# (`validate_deps!`), backed by stdlib TSort for cycle detection.
|
|
7
|
+
module DependencyGraph
|
|
8
|
+
def _deps
|
|
9
|
+
@_deps ||= {}
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
# Declare dependencies for the next task.
|
|
13
|
+
# Bare symbols run sequentially; arrays within the splat run in parallel.
|
|
14
|
+
#
|
|
15
|
+
# depends_on :build # sequential
|
|
16
|
+
# depends_on :build, :lint # both sequential
|
|
17
|
+
# depends_on [:build, :lint] # build and lint in parallel
|
|
18
|
+
# depends_on :setup, [:build, :lint], :test # setup, then build+lint, then test
|
|
19
|
+
#
|
|
20
|
+
# A sole Proc/lambda defers resolution to validate_deps! (after every
|
|
21
|
+
# .loki file has loaded), instead of now. It must return the same shape
|
|
22
|
+
# the splat form above would: an array of stages, each a Symbol
|
|
23
|
+
# (sequential) or Array (parallel group).
|
|
24
|
+
#
|
|
25
|
+
# depends_on -> { [all_commands.keys.grep(/_check\z/).map(&:to_sym)] }
|
|
26
|
+
def depends_on(*tasks)
|
|
27
|
+
@_pending_deps = tasks
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# Validate the full dep graph for cycles using stdlib TSort.
|
|
31
|
+
def validate_deps!
|
|
32
|
+
_check_orphaned_deps!
|
|
33
|
+
return if _deps.empty?
|
|
34
|
+
|
|
35
|
+
_resolve_lazy_deps!
|
|
36
|
+
all_task_names = all_commands.keys.map(&:to_sym)
|
|
37
|
+
_check_undefined_deps!(all_task_names)
|
|
38
|
+
_check_dep_arities!
|
|
39
|
+
_build_and_sort_graph(all_task_names)
|
|
40
|
+
rescue TSort::Cyclic => e
|
|
41
|
+
raise Asgard::CircularDependencyError, e.message
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
private
|
|
45
|
+
|
|
46
|
+
# Normalizes depends_on's raw splat args into a _deps-ready value: the
|
|
47
|
+
# sole Proc/lambda unresolved (see #depends_on), or a concrete stage
|
|
48
|
+
# array.
|
|
49
|
+
def _normalize_pending_deps(pending)
|
|
50
|
+
sole = pending.first
|
|
51
|
+
return sole if pending.size == 1 && sole.respond_to?(:call)
|
|
52
|
+
|
|
53
|
+
_stages_from(pending)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
# Each element is a Symbol (sequential) or Array (parallel group).
|
|
57
|
+
def _stages_from(list)
|
|
58
|
+
list.map { |d| Array(d).map(&:to_sym) }
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# Replaces any Proc-valued _deps entry (see #depends_on) with its
|
|
62
|
+
# resolved stage array. Runs once, after the full .loki chain has
|
|
63
|
+
# loaded, so the Proc can safely reference tasks defined in any file.
|
|
64
|
+
def _resolve_lazy_deps!
|
|
65
|
+
_deps.each do |task, stages_or_proc|
|
|
66
|
+
next unless stages_or_proc.respond_to?(:call)
|
|
67
|
+
|
|
68
|
+
_deps[task] = _stages_from(Array(_call_dep_proc(task, stages_or_proc)))
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def _call_dep_proc(task, dep_proc)
|
|
73
|
+
dep_proc.call
|
|
74
|
+
rescue StandardError => e
|
|
75
|
+
raise Asgard::Error, "depends_on proc for '#{task}' raised #{e.class}: #{e.message}"
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
def _check_orphaned_deps!
|
|
79
|
+
pending = Array(@_pending_deps)
|
|
80
|
+
return unless pending.any?
|
|
81
|
+
|
|
82
|
+
raise Asgard::Error,
|
|
83
|
+
"depends_on(#{pending.join(', ')}) declared without a following task definition"
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
def _check_undefined_deps!(all_task_names)
|
|
87
|
+
undefined = _deps.values.flatten.uniq - all_task_names
|
|
88
|
+
return unless undefined.any?
|
|
89
|
+
|
|
90
|
+
raise Asgard::Error, "undefined task(s) in depends_on: #{undefined.sort.join(', ')}"
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def _check_dep_arities!
|
|
94
|
+
_deps.each_value do |stages|
|
|
95
|
+
stages.flatten.each do |dep|
|
|
96
|
+
meth = instance_method(dep.to_s)
|
|
97
|
+
required = meth.parameters.count { |type, _| type == :req }
|
|
98
|
+
next unless required.positive?
|
|
99
|
+
|
|
100
|
+
raise Asgard::Error,
|
|
101
|
+
"task '#{dep}' has #{required} required argument(s) and cannot be used as a dependency"
|
|
102
|
+
end
|
|
103
|
+
end
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
# Runs a full topological sort purely to raise TSort::Cyclic on a cycle;
|
|
107
|
+
# the order itself isn't otherwise used (execution order comes from the
|
|
108
|
+
# stage groups each task's own depends_on declared).
|
|
109
|
+
def _build_and_sort_graph(all_task_names)
|
|
110
|
+
full_graph = all_task_names.to_h { |task| [task, _deps.fetch(task, []).flatten] }
|
|
111
|
+
Graph.new(full_graph).tsort
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
# Minimal TSort-able wrapper around a task => dependency-list Hash.
|
|
115
|
+
Graph = Struct.new(:edges) do
|
|
116
|
+
include TSort
|
|
117
|
+
|
|
118
|
+
def tsort_each_node(&) = edges.each_key(&)
|
|
119
|
+
def tsort_each_child(node, &) = edges.fetch(node).each(&)
|
|
120
|
+
end
|
|
121
|
+
private_constant :Graph
|
|
122
|
+
end
|
|
123
|
+
end
|
|
124
|
+
end
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Asgard
|
|
4
|
+
class Base < Thor
|
|
5
|
+
# Task execution engine: resolves and runs declared dependencies (in
|
|
6
|
+
# parallel where declared) before the target command runs.
|
|
7
|
+
#
|
|
8
|
+
# Completion-based deduplication: a task is only marked done after its
|
|
9
|
+
# body finishes. Threads that arrive at an already-running shared dep
|
|
10
|
+
# wait on its ConditionVariable rather than proceeding immediately,
|
|
11
|
+
# preventing the race where parallel tasks start before a shared dep
|
|
12
|
+
# has actually completed.
|
|
13
|
+
module Dispatch
|
|
14
|
+
def self.included(base)
|
|
15
|
+
base.extend(ClassMethods)
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
module ClassMethods
|
|
19
|
+
def _running
|
|
20
|
+
@_running ||= Set.new
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def _done
|
|
24
|
+
@_done ||= Set.new
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def _cond
|
|
28
|
+
@_cond ||= Hash.new { |h, k| h[k] = ConditionVariable.new }
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
# Completed tasks' return values, keyed by task name — lets a task
|
|
32
|
+
# already run by one thread hand its result to another thread that
|
|
33
|
+
# shares the dependency, without either touching `self`.
|
|
34
|
+
def _results
|
|
35
|
+
@_results ||= {}
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def _ran_mutex
|
|
39
|
+
@_ran_mutex ||= Mutex.new
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
# Reset execution tracking for a fresh asgard invocation.
|
|
43
|
+
def _reset_ran!
|
|
44
|
+
_ran_mutex.synchronize do
|
|
45
|
+
@_running = Set.new
|
|
46
|
+
@_done = Set.new
|
|
47
|
+
@_cond = Hash.new { |h, k| h[k] = ConditionVariable.new }
|
|
48
|
+
@_results = {}
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
# Dispatch hook: resolves and runs all deps (in parallel where declared)
|
|
54
|
+
# before executing the target command.
|
|
55
|
+
def invoke_command(command, *)
|
|
56
|
+
$DEBUG = true if options[:debug]
|
|
57
|
+
$VERBOSE = true if options[:verbose]
|
|
58
|
+
target = command.name.to_sym
|
|
59
|
+
return cached_result(target) unless acquire_run_token(target)
|
|
60
|
+
|
|
61
|
+
result = nil
|
|
62
|
+
begin
|
|
63
|
+
resolved_deps = run_deps_for(target)
|
|
64
|
+
result = with_dep_results(resolved_deps) { command.run(self, *) }
|
|
65
|
+
ensure
|
|
66
|
+
signal_done(target, result)
|
|
67
|
+
end
|
|
68
|
+
result
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
# The results of +target+'s direct dependencies, keyed by task name —
|
|
72
|
+
# available to a task body while it runs, e.g. `dep_result(:test_check)`.
|
|
73
|
+
def dep_results
|
|
74
|
+
Thread.current[:asgard_dep_results] || {}
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def dep_result(task)
|
|
78
|
+
dep_results[task.to_sym]
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
private
|
|
82
|
+
|
|
83
|
+
def with_dep_results(results)
|
|
84
|
+
thread = Thread.current
|
|
85
|
+
previous = thread[:asgard_dep_results]
|
|
86
|
+
thread[:asgard_dep_results] = results
|
|
87
|
+
yield
|
|
88
|
+
ensure
|
|
89
|
+
thread[:asgard_dep_results] = previous
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
def cached_result(target)
|
|
93
|
+
klass = self.class
|
|
94
|
+
klass._ran_mutex.synchronize { klass._results[target] }
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
def acquire_run_token(target)
|
|
98
|
+
klass = self.class
|
|
99
|
+
mutex = klass._ran_mutex
|
|
100
|
+
done = klass._done
|
|
101
|
+
running = klass._running
|
|
102
|
+
|
|
103
|
+
mutex.synchronize do
|
|
104
|
+
if done.include?(target)
|
|
105
|
+
false
|
|
106
|
+
elsif running.include?(target)
|
|
107
|
+
klass._cond[target].wait(mutex) until done.include?(target)
|
|
108
|
+
false
|
|
109
|
+
else
|
|
110
|
+
running.add(target)
|
|
111
|
+
true
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def run_deps_for(target)
|
|
117
|
+
stages = self.class._deps[target]
|
|
118
|
+
return {} unless stages&.any?
|
|
119
|
+
|
|
120
|
+
stages.each_with_object({}) { |group, acc| acc.merge!(run_dep_group(group)) }
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
def run_dep_group(group)
|
|
124
|
+
return { group.first => run_dep(group.first) } unless group.size > 1
|
|
125
|
+
|
|
126
|
+
threads = group.map { |task| [task, Thread.new { run_dep(task) }] }
|
|
127
|
+
errors = []
|
|
128
|
+
results = {}
|
|
129
|
+
threads.each do |task, t|
|
|
130
|
+
results[task] = t.value
|
|
131
|
+
rescue => e
|
|
132
|
+
errors << e
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
if errors.size == 1
|
|
136
|
+
raise errors.first
|
|
137
|
+
elsif errors.any?
|
|
138
|
+
errors.each { |e| warn "asgard: #{e.message}" }
|
|
139
|
+
raise Asgard::Error, "#{errors.size} parallel dependencies failed"
|
|
140
|
+
end
|
|
141
|
+
|
|
142
|
+
results
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
def signal_done(target, result)
|
|
146
|
+
klass = self.class
|
|
147
|
+
klass._ran_mutex.synchronize do
|
|
148
|
+
klass._done.add(target)
|
|
149
|
+
klass._results[target] = result
|
|
150
|
+
klass._cond[target].broadcast
|
|
151
|
+
end
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
def run_dep(task)
|
|
155
|
+
command = self.class.all_commands[task.to_s]
|
|
156
|
+
invoke_command(command) if command
|
|
157
|
+
end
|
|
158
|
+
end
|
|
159
|
+
end
|
|
160
|
+
end
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Asgard
|
|
4
|
+
class Base < Thor
|
|
5
|
+
# Tracks every Asgard::Base subclass ever defined, and every method
|
|
6
|
+
# name's full source_location history — the data Doctor (`asgard
|
|
7
|
+
# --doctor`) uses to detect a silently-overridden `def`, same file or
|
|
8
|
+
# across files.
|
|
9
|
+
module Registry
|
|
10
|
+
def subclasses
|
|
11
|
+
@subclasses ||= []
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def inherited(subclass)
|
|
15
|
+
super
|
|
16
|
+
Asgard::Base.subclasses << subclass
|
|
17
|
+
subclass.instance_variable_set(:@_deps, {})
|
|
18
|
+
subclass.instance_variable_set(:@_method_log, Hash.new { |h, k| h[k] = [] })
|
|
19
|
+
subclass.instance_variable_set(:@_pending_deps, [])
|
|
20
|
+
subclass.instance_variable_set(:@_pending_single_desc, nil)
|
|
21
|
+
subclass.instance_variable_set(:@_pending_single_desc_opts, nil)
|
|
22
|
+
subclass.instance_variable_set(:@_running, Set.new)
|
|
23
|
+
subclass.instance_variable_set(:@_done, Set.new)
|
|
24
|
+
subclass.instance_variable_set(:@_cond, Hash.new { |h, k| h[k] = ConditionVariable.new })
|
|
25
|
+
subclass.instance_variable_set(:@_ran_mutex, Mutex.new)
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
# Every source_location a method name has ever been defined at, in
|
|
29
|
+
# definition order — so a later `def` silently overriding an earlier
|
|
30
|
+
# one (same name, different file) is still visible after the fact.
|
|
31
|
+
# Doctor is the consumer; asgard itself doesn't otherwise care once
|
|
32
|
+
# the last definition wins.
|
|
33
|
+
def _method_log
|
|
34
|
+
@_method_log ||= Hash.new { |h, k| h[k] = [] }
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|