scout-essentials 1.9.1 → 1.9.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.vimproject +18 -1
- data/Rakefile +9 -0
- data/VERSION +1 -1
- data/doc/Improvements.md +20 -0
- data/doc/StartHere.md +7 -4
- data/doc/developer/AnnotationSystem.md +10 -15
- data/doc/developer/Configuration.md +20 -30
- data/doc/developer/CoreUtilities.md +52 -22
- data/doc/developer/DesignPrinciples.md +7 -9
- data/doc/developer/DocumentationPractices.md +476 -0
- data/doc/developer/ErrorHandling.md +11 -20
- data/doc/developer/LockingAndConcurrency.md +20 -13
- data/doc/developer/StreamingModel.md +13 -14
- data/doc/developer/Testing.md +389 -0
- data/doc/user/AnnotatingData.md +9 -13
- data/doc/user/CommandLineOptions.md +9 -12
- data/doc/user/Cookbook.md +7 -10
- data/doc/user/HandlingStreams.md +22 -29
- data/doc/user/LoggingAndProgress.md +9 -11
- data/doc/user/ProducingResources.md +0 -1
- data/doc/user/RemoteData.md +10 -13
- data/doc/user/RunningCommands.md +12 -17
- data/doc/user/WorkingWithFiles.md +10 -2
- data/lib/scout/config.rb +1 -0
- data/lib/scout/indiferent_hash.rb +8 -2
- data/lib/scout/log/fingerprint.rb +1 -1
- data/lib/scout/named_array.rb +4 -0
- data/lib/scout/open/final.rb +1 -1
- data/lib/scout/open/stream.rb +14 -13
- data/lib/scout/path/util.rb +1 -1
- data/lib/scout/testing/availability.rb +264 -0
- data/lib/scout/testing/scout_state.rb +193 -0
- data/lib/scout/testing.rb +82 -0
- data/research/doc-audit-findings.md +7 -5
- data/research/doc-structure.md +89 -0
- data/research/lock-namespace-blindness.md +73 -0
- data/research/probing-guide.md +311 -0
- data/research/testing-availability-scout-ai-handoff.md +235 -0
- data/research/testing-availability-scout-rig-handoff.md +355 -0
- data/scout-essentials.gemspec +24 -3
- data/test/scout/annotation/test_dup_clone.rb +31 -0
- data/test/scout/indiferent_hash/test_case_insensitive_staleness.rb +27 -0
- data/test/scout/indiferent_hash/test_string2hash_false.rb +13 -0
- data/test/scout/misc/test_timespan.rb +30 -0
- data/test/scout/open/test_compression_detection.rb +32 -0
- data/test/scout/open/test_forced_gzip_broken.rb +25 -0
- data/test/scout/test_named_array_resolution.rb +49 -0
- data/test/scout/test_testing.rb +195 -0
- data/test/scout/test_tmpfile_cleanup.rb +23 -0
- data/test/scout/testing/test_availability.rb +222 -0
- data/test/scout/testing/test_state_bindings.rb +163 -0
- data/test/test_helper.rb +26 -12
- metadata +23 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: aa2a5b23a4e6627229255301bd5e8e9263e20333314e3334ed0359b6d387496b
|
|
4
|
+
data.tar.gz: 1c2fd66f3b370a1325ab60d877cda1ba2d19f1c41930bc4e45c879cbf1aafec0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a6c8151e1d4323e586c85eeb28bf4a1ada40f829d7b7c2e35f52f442e7f03ceed5a0ca64d5c93ef38033849fc9d3d7ce95e325d76bfe953068dd51594a6f6a2c
|
|
7
|
+
data.tar.gz: 3db4ca0f364ecfb5a4949a030f9ed35952e7d338e75663d46b185bf939f10821066efd80c8b80c852bd382da4fc8c4114293743977b5faabcf44d4f3dd3a16b3
|
data/.vimproject
CHANGED
|
@@ -11,6 +11,11 @@ scout-essentials=/$PWD filter="*.rb *.txt *.md *.conf *.yaml" {
|
|
|
11
11
|
cmd
|
|
12
12
|
|
|
13
13
|
timeout
|
|
14
|
+
|
|
15
|
+
cortex_coder
|
|
16
|
+
cortex_coder.analyze
|
|
17
|
+
|
|
18
|
+
testing_basics
|
|
14
19
|
}
|
|
15
20
|
doc=doc{
|
|
16
21
|
Improvements.md
|
|
@@ -23,21 +28,33 @@ scout-essentials=/$PWD filter="*.rb *.txt *.md *.conf *.yaml" {
|
|
|
23
28
|
HandlingStreams.md
|
|
24
29
|
LoggingAndProgress.md
|
|
25
30
|
ProducingResources.md
|
|
31
|
+
RemoteData.md
|
|
26
32
|
RunningCommands.md
|
|
27
33
|
WorkingWithFiles.md
|
|
28
34
|
}
|
|
29
35
|
developer=developer{
|
|
30
36
|
AnnotationSystem.md
|
|
31
37
|
Architecture.md
|
|
38
|
+
Configuration.md
|
|
39
|
+
CoreUtilities.md
|
|
32
40
|
DesignPrinciples.md
|
|
41
|
+
DocumentationPractices.md
|
|
42
|
+
ErrorHandling.md
|
|
43
|
+
LockingAndConcurrency.md
|
|
33
44
|
PathResolution.md
|
|
34
45
|
PersistenceAndResources.md
|
|
35
46
|
StreamingModel.md
|
|
47
|
+
Testing.md
|
|
36
48
|
}
|
|
37
49
|
}
|
|
38
50
|
lib=lib {
|
|
39
51
|
scout-essentials.rb
|
|
40
52
|
scout=scout{
|
|
53
|
+
testing.rb
|
|
54
|
+
testing=testing{
|
|
55
|
+
availability.rb
|
|
56
|
+
scout_state.rb
|
|
57
|
+
}
|
|
41
58
|
annotation.rb
|
|
42
59
|
annotation=annotation{
|
|
43
60
|
annotated_object.rb
|
|
@@ -141,7 +158,7 @@ scout-essentials=/$PWD filter="*.rb *.txt *.md *.conf *.yaml" {
|
|
|
141
158
|
test_tmpfile.rb
|
|
142
159
|
annotation=annotation{
|
|
143
160
|
test_array.rb
|
|
144
|
-
|
|
161
|
+
}
|
|
145
162
|
misc=misc{
|
|
146
163
|
test_digest.rb
|
|
147
164
|
test_filesystem.rb
|
data/Rakefile
CHANGED
|
@@ -21,6 +21,15 @@ Juwelier::Tasks.new do |gem|
|
|
|
21
21
|
gem.add_runtime_dependency 'yaml'
|
|
22
22
|
gem.add_runtime_dependency 'rake'
|
|
23
23
|
gem.add_runtime_dependency 'listen'
|
|
24
|
+
|
|
25
|
+
# Juwelier's default file list comes from Git's tracked files. Include these
|
|
26
|
+
# current additions explicitly so an uncommitted working-tree build packages them.
|
|
27
|
+
gem.files += %w[
|
|
28
|
+
doc/developer/Testing.md
|
|
29
|
+
lib/scout/testing.rb
|
|
30
|
+
lib/scout/testing/scout_state.rb
|
|
31
|
+
test/scout/test_testing.rb
|
|
32
|
+
]
|
|
24
33
|
end
|
|
25
34
|
Juwelier::RubygemsDotOrgTasks.new
|
|
26
35
|
require 'rake/testtask'
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
1.9.
|
|
1
|
+
1.9.3
|
data/doc/Improvements.md
CHANGED
|
@@ -224,3 +224,23 @@ to a temporary cache dir or raising a clearer error message.
|
|
|
224
224
|
| Bugs (🔴) | 0 |
|
|
225
225
|
| Improvements (🟡) | 15 |
|
|
226
226
|
| Done (🟢) | 1 |
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
230
|
+
## Promotion ledger
|
|
231
|
+
|
|
232
|
+
- **2026-09-12** — `research/doc-practices-guide.md` (491 lines, Critic-
|
|
233
|
+
reviewed 93/100) promoted to
|
|
234
|
+
`doc/developer/DocumentationPractices.md`. Evidence labels (`[V]` /
|
|
235
|
+
`[R1]` / `[R2]` / `[REPT]`) and all Critic-mandated caveats preserved
|
|
236
|
+
verbatim in the page. Provenance retained in the Cortex artifacts
|
|
237
|
+
`doc-guide/*` and `search/doc-guide-source-map.md`, and in the three
|
|
238
|
+
sessions `~/chats/scout-essentials/cortex_coder`,
|
|
239
|
+
`~/chats/scout-gear/cortex_coder`, `~/chats/scout-ai/cortex_coder`.
|
|
240
|
+
The source file was retired in a follow-up commit after independent
|
|
241
|
+
verification (Critic PASS 97/100); its unique §17 associated-elements
|
|
242
|
+
inventory is preserved in the Cortex closure artifact
|
|
243
|
+
`doc-guide/promotion-record.md`. Gemspec file list reconciled (4 additions:
|
|
244
|
+
`doc/developer/DocumentationPractices.md`,
|
|
245
|
+
`research/doc-structure.md`, `research/lock-namespace-blindness.md`,
|
|
246
|
+
`research/probing-guide.md`).
|
data/doc/StartHere.md
CHANGED
|
@@ -40,7 +40,7 @@ Log.info "wrote 2 lines"
|
|
|
40
40
|
Transitive pulls: `scout/open` (hence `Open`, `CMD`, `NamedStream`),
|
|
41
41
|
`scout/annotation` (hence `Annotation`), `scout/misc` (via
|
|
42
42
|
`scout/tmpfile`/`scout/open`). Notably **not** loaded by the umbrella
|
|
43
|
-
require (
|
|
43
|
+
require (against the current `lib/`):
|
|
44
44
|
|
|
45
45
|
- `NamedArray` — `require 'scout/named_array'`
|
|
46
46
|
- `Hook` (a **top-level** `module Hook`, *not* `Misc::Hook`) —
|
|
@@ -105,6 +105,7 @@ ruby -Ilib -Itest test/scout/test_misc.rb
|
|
|
105
105
|
|
|
106
106
|
**Developer guides** — how the library *works*:
|
|
107
107
|
|
|
108
|
+
- [Testing](developer/Testing.md) — test layout, generic and Scout-specific support, and isolation conventions
|
|
108
109
|
- [Architecture](developer/Architecture.md)
|
|
109
110
|
- [Design Principles](developer/DesignPrinciples.md)
|
|
110
111
|
- [Annotation System](developer/AnnotationSystem.md)
|
|
@@ -115,8 +116,10 @@ ruby -Ilib -Itest test/scout/test_misc.rb
|
|
|
115
116
|
- [Error Handling](developer/ErrorHandling.md)
|
|
116
117
|
- [Locking and Concurrency](developer/LockingAndConcurrency.md)
|
|
117
118
|
- [Core Utilities](developer/CoreUtilities.md)
|
|
119
|
+
- [Documentation Practices](developer/DocumentationPractices.md)
|
|
118
120
|
|
|
119
121
|
**Research artifacts** (`../research/`) are non-normative supporting
|
|
120
|
-
material:
|
|
121
|
-
|
|
122
|
-
`
|
|
122
|
+
material: design and behaviour analyses written while this
|
|
123
|
+
documentation was produced. The behaviour they describe is captured in
|
|
124
|
+
`doc/` and in the `test/` suite, which now carry the same facts in
|
|
125
|
+
executable form.
|
|
@@ -6,16 +6,12 @@ makes `Path`, `Resource` and `NamedArray` work, so understanding it is a
|
|
|
6
6
|
prerequisite for [Path Resolution](PathResolution.md) and
|
|
7
7
|
[Persistence and Resources](PersistenceAndResources.md).
|
|
8
8
|
|
|
9
|
-
Everything on this page is backed by `tmp/rewrite_C/probe_01..04.rb` and
|
|
10
|
-
probes P36–P42 in `research/behavior-probes.md`, run against the current
|
|
11
|
-
`lib/scout/annotation*.rb`.
|
|
12
|
-
|
|
13
9
|
## Load path
|
|
14
10
|
|
|
15
11
|
`lib/scout-essentials.rb` does **not** require `scout/annotation` directly.
|
|
16
12
|
`Annotation` becomes available because `scout/path` requires it
|
|
17
13
|
(`lib/scout/path.rb:1`), so `require 'scout-essentials'` gives you the
|
|
18
|
-
constant (`Object.const_defined?(:Annotation) => true
|
|
14
|
+
constant (`Object.const_defined?(:Annotation) => true`) — but a
|
|
19
15
|
library that only wants annotations can `require 'scout/annotation'` on its
|
|
20
16
|
own.
|
|
21
17
|
|
|
@@ -49,7 +45,7 @@ SampleInfo.setup('S004', %w[Human Liver]) # positional, same order
|
|
|
49
45
|
SampleInfo.setup { 'S005' } # block-as-object
|
|
50
46
|
```
|
|
51
47
|
|
|
52
|
-
Rules
|
|
48
|
+
Rules:
|
|
53
49
|
|
|
54
50
|
- **In-place extension.** `setup` calls `obj.extend SampleInfo` on the object
|
|
55
51
|
you pass and returns **that same object** (`setup(s).equal?(s) => true`) —
|
|
@@ -60,7 +56,7 @@ Rules verified in probe_01/probe_02:
|
|
|
60
56
|
`setup` rescues it and returns the *plain* object with no metadata.
|
|
61
57
|
- **`:annotation_types` is reserved.** Each annotated object gets an
|
|
62
58
|
`@annotation_types` array; declaring `annotation :annotation_types` collides
|
|
63
|
-
with it
|
|
59
|
+
with it; writing to it raised `NoMethodError` against `nil`.
|
|
64
60
|
|
|
65
61
|
## Introspection and serialisation
|
|
66
62
|
|
|
@@ -108,7 +104,7 @@ s.annotation_id # => "860c73f490edb8115064663b7f579d73"
|
|
|
108
104
|
- `Annotation.setup(obj, "A|B", hash)` — the generic deserialiser. The type
|
|
109
105
|
string is split on `|`, each name resolved with `Object.const_get`. **Unknown
|
|
110
106
|
names only warn and are skipped** (`Log.warn "Annotation NoSuchAnnotation not
|
|
111
|
-
defined"
|
|
107
|
+
defined"`) — no exception.
|
|
112
108
|
|
|
113
109
|
### Instance-side helpers
|
|
114
110
|
|
|
@@ -123,10 +119,10 @@ s.annotation_id # => "860c73f490edb8115064663b7f579d73"
|
|
|
123
119
|
|
|
124
120
|
## Round-trips and copies
|
|
125
121
|
|
|
126
|
-
- **Marshal round-trips** annotations
|
|
122
|
+
- **Marshal round-trips** annotations: the singleton modules
|
|
127
123
|
survive dump/load.
|
|
128
|
-
- **Only `dup` loses them; `clone` keeps them**
|
|
129
|
-
`dup is_annotated? => false`, `clone is_annotated? => true, .a => 1` —
|
|
124
|
+
- **Only `dup` loses them; `clone` keeps them**
|
|
125
|
+
(`dup is_annotated? => false`, `clone is_annotated? => true, .a => 1` —
|
|
130
126
|
`clone` copies the singleton class, `dup` does not): to re-annotate a `dup`
|
|
131
127
|
copy the metadata explicitly with `annotation_hash` →
|
|
132
128
|
`Annotation.setup` / `MyModule.setup`, or call `#annotate` on the copy.
|
|
@@ -147,7 +143,7 @@ arr.first.organism # => "Human"
|
|
|
147
143
|
Overrides provided in `lib/scout/annotation/array.rb`:
|
|
148
144
|
|
|
149
145
|
- `[]` `(pos, clean = false)` — the element is re-annotated unless the second
|
|
150
|
-
argument is truthy, in which case it is returned clean (
|
|
146
|
+
argument is truthy, in which case it is returned clean (a fresh
|
|
151
147
|
array's `fresh[0, true]` is an un-annotated `String` while `fresh[1]` is
|
|
152
148
|
annotated);
|
|
153
149
|
- `first`, `last`, `each_with_index`, `each`, `inject`, `collect`, `select` —
|
|
@@ -156,8 +152,7 @@ Overrides provided in `lib/scout/annotation/array.rb`:
|
|
|
156
152
|
- `subset(list)`, `remove(list)` — set operations (`&`, `-`) with
|
|
157
153
|
re-annotation.
|
|
158
154
|
|
|
159
|
-
**Limits**
|
|
160
|
-
owners plus actual results): `map`, `zip`, `filter_map`, `flat_map`,
|
|
155
|
+
**Limits** (method owners audited against `lib/`): `map`, `zip`, `filter_map`, `flat_map`,
|
|
161
156
|
`each_slice`, `values_at` and `count` are **not overridden** (their owner is
|
|
162
157
|
`Array`/`Enumerable`) and return plain results with no annotations — `ary.map
|
|
163
158
|
{ |x| x }` yields `[false, false, false]` under `Annotation.is_annotated?`,
|
|
@@ -165,7 +160,7 @@ while `ary.each` yields `[true, true, true]`, and `zip` keeps annotations only
|
|
|
165
160
|
on the container-side elements.
|
|
166
161
|
|
|
167
162
|
**Requirement:** elements must be extendable. `AnnotatedArray` over an Array
|
|
168
|
-
of `Integer`s raises `TypeError: can't define singleton`
|
|
163
|
+
of `Integer`s raises `TypeError: can't define singleton` — use
|
|
169
164
|
Strings or other extendable objects.
|
|
170
165
|
|
|
171
166
|
## NamedArray is a separate thing
|
|
@@ -24,8 +24,6 @@ token on `::` and returns `[token, priority]`:
|
|
|
24
24
|
| `key:` / `key:<name>` | 20 | the key itself; LOWEST priority |
|
|
25
25
|
| `token::N` | N | explicit numeric suffix always wins over the defaults |
|
|
26
26
|
|
|
27
|
-
Verified by `tmp/rewrite_D/probe_11_token_probe.rb`:
|
|
28
|
-
|
|
29
27
|
```text
|
|
30
28
|
workflow:x => ["workflow:x", 4]
|
|
31
29
|
task:x => ["task:x", 3]
|
|
@@ -44,8 +42,8 @@ plain value.
|
|
|
44
42
|
## A worked example
|
|
45
43
|
|
|
46
44
|
A common misreading is "workflow overrides task". The numbers say the
|
|
47
|
-
opposite — `line` (1) outranks `workflow` (4).
|
|
48
|
-
|
|
45
|
+
opposite — `line` (1) outranks `workflow` (4). Stacking all four tokens on
|
|
46
|
+
one key shows the chain end-to-end:
|
|
49
47
|
|
|
50
48
|
```ruby
|
|
51
49
|
def fresh(k); Scout::Config::CACHE.delete(k); end
|
|
@@ -73,11 +71,11 @@ Scout::Config.get(k,'workflow:W','task:T')
|
|
|
73
71
|
```
|
|
74
72
|
|
|
75
73
|
Within one priority bucket the LAST `set` wins (`priorities[prio].unshift`,
|
|
76
|
-
`lib/scout/config.rb:83`;
|
|
74
|
+
`lib/scout/config.rb:83`; two entries set with `task:` leave the *second*
|
|
75
|
+
as the winner).
|
|
77
76
|
|
|
78
77
|
Explicit suffixes punch through the table, both ways — `::1` promotes an
|
|
79
|
-
entry above every default token, `::30` buries it below `key:` (
|
|
80
|
-
cases `q6`/`q7`). Note the suffix is per-entry, not per-key: it re-prices one
|
|
78
|
+
entry above every default token, `::30` buries it below `key:` (cases `q6`/`q7`). Note the suffix is per-entry, not per-key: it re-prices one
|
|
81
79
|
`set` call, it does not change the token for later calls.
|
|
82
80
|
|
|
83
81
|
### Implicit caller tokens
|
|
@@ -85,8 +83,7 @@ cases `q6`/`q7`). Note the suffix is per-entry, not per-key: it re-prices one
|
|
|
85
83
|
Every `Config.get` appends `file:<caller>` and `line:<caller>:<n>` derived
|
|
86
84
|
from `caller` (`lib/scout/config.rb:119-127`), after filtering out frames
|
|
87
85
|
matching the legacy rbbt regexes (see *Legacy internals* below). Entries set
|
|
88
|
-
with those tokens match without being named in the call
|
|
89
|
-
`probe_12`, `matched by implicit caller file token => "v"`).
|
|
86
|
+
with those tokens match without being named in the call.
|
|
90
87
|
|
|
91
88
|
## `etc/config` files
|
|
92
89
|
|
|
@@ -103,9 +100,7 @@ key value token
|
|
|
103
100
|
(`lib/scout/config.rb:27-31`). `find_all` returns paths in `map_order`
|
|
104
101
|
(`current, user, home, local, global, usr, lib, fast, cache, bulk, ...`), so
|
|
105
102
|
the **reverse** order loads the *most specific* file (e.g. `$PWD/etc/config`)
|
|
106
|
-
**first** and `$HOME/.scout/etc/config` last.
|
|
107
|
-
with the same (implicit `key:`) token were tested by
|
|
108
|
-
`tmp/rewrite_D/probe_23_findall_order.rb`:
|
|
103
|
+
**first** and `$HOME/.scout/etc/config` last.
|
|
109
104
|
|
|
110
105
|
```text
|
|
111
106
|
map_order => [:current, :user, :home, ...]
|
|
@@ -130,33 +125,30 @@ one key it auto-adds the `key:<name>` token (`add_entry`,
|
|
|
130
125
|
`Config.get(key, *tokens)` (`lib/scout/config.rb:94-155`) does four things
|
|
131
126
|
after resolving the winner:
|
|
132
127
|
|
|
133
|
-
1. **`'false'` becomes `false`** — the ONLY string-to-object coercion
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
2. **`'nil'` becomes `nil`** (probe_10).
|
|
128
|
+
1. **`'false'` becomes `false`** — the ONLY string-to-object coercion
|
|
129
|
+
(`'TRUE'`, `'0'` and `'42'` all stay Strings).
|
|
130
|
+
2. **`'nil'` becomes `nil`**.
|
|
137
131
|
3. **`env:VAR[,VAR2]`** in the resolved value is replaced by the first set
|
|
138
|
-
environment variable, and `get` returns `nil` if none is set
|
|
139
|
-
`'from-env'`).
|
|
132
|
+
environment variable, and `get` returns `nil` if none is set.
|
|
140
133
|
4. Every `get` appends `[key, value, tokens]` to `GOT_KEYS` — an audit trail
|
|
141
|
-
of what was asked and answered
|
|
142
|
-
restores it.
|
|
134
|
+
of what was asked and answered. `with_config` snapshots and restores it.
|
|
143
135
|
|
|
144
136
|
Options hash as last argument:
|
|
145
137
|
|
|
146
138
|
- `:default => v` — used when no entry matches.
|
|
147
|
-
- `:env => 'VAR1,VAR2'` — first set env var becomes the default
|
|
139
|
+
- `:env => 'VAR1,VAR2'` — first set env var becomes the default.
|
|
148
140
|
|
|
149
141
|
A `Symbol` key raises `TypeError` — the cache is keyed by `key.to_s`, but
|
|
150
142
|
`CACHE[key.to_s]` is called after `match` has already used the raw key;
|
|
151
143
|
`Config.get(:symbol)` raises `TypeError: no implicit conversion of Symbol
|
|
152
|
-
into String
|
|
144
|
+
into String`. Use String keys.
|
|
153
145
|
|
|
154
146
|
## `with_config` and `process_config`
|
|
155
147
|
|
|
156
148
|
`with_config { ... }` (`lib/scout/config.rb:157-166`) dups every `CACHE`
|
|
157
149
|
bucket and `GOT_KEYS`, yields, and restores both in an `ensure`. Anything
|
|
158
150
|
`set` inside the block is rolled back — including sets made by code you
|
|
159
|
-
call (
|
|
151
|
+
call (`inside => "inside"`, `after => "outside"`). It is the
|
|
160
152
|
mechanism that keeps config changes from leaking out of a sub-operation.
|
|
161
153
|
|
|
162
154
|
`Config.process_config(config)` (`lib/scout/config.rb:168-180`) interprets
|
|
@@ -165,11 +157,9 @@ a CLI `--config` argument, in this order:
|
|
|
165
157
|
1. existing file path → `load_file`
|
|
166
158
|
2. name of an existing `Scout.etc.config_profile[config]` file → `load_file`
|
|
167
159
|
3. otherwise a `"key value token..."` string. The `::N` suffix is mandatory
|
|
168
|
-
|
|
169
|
-
|
|
160
|
+
in the stored token: any token without one gets `prio = "0"`, i.e. the
|
|
161
|
+
token is re-emitted as `token::0`.
|
|
170
162
|
|
|
171
|
-
Verified by `tmp/rewrite_D/probe_17b.rb` / `probe_17c.rb`. The subtle part
|
|
172
|
-
of case 3 is that the stored token keeps the suffix, so `process_config
|
|
173
163
|
"k3 cli workflow::0"` stores the token `workflow::0`, whose *name* is the
|
|
174
164
|
bare string `workflow` at priority 0 (stronger than `task`=3,
|
|
175
165
|
`file`=2, `line`=1, `key:`=20), and it does **not** match a call like
|
|
@@ -190,8 +180,8 @@ carrying an explicit smaller `::N` — but only for lookups that pass the bare
|
|
|
190
180
|
|
|
191
181
|
## Environment variables read by the gem
|
|
192
182
|
|
|
193
|
-
|
|
194
|
-
(
|
|
183
|
+
Established by grep over `lib/`
|
|
184
|
+
(covers the `env:`/`:env` machinery; the
|
|
195
185
|
rest are read directly from source):
|
|
196
186
|
|
|
197
187
|
| variable | read by | effect |
|
|
@@ -221,7 +211,7 @@ frames whose path matches those patterns are skipped when deriving
|
|
|
221
211
|
## Thread-safety
|
|
222
212
|
|
|
223
213
|
There is no mutex anywhere in `lib/scout/config.rb` (`Scout::Config` has no
|
|
224
|
-
`Mutex` ivar;
|
|
214
|
+
`Mutex` ivar;
|
|
225
215
|
and `GOT_KEYS` are plain shared structures and `add_entry` does
|
|
226
216
|
read-modify-write (`CACHE[key] ||= []; CACHE[key] << ...`). Concurrent
|
|
227
217
|
`set`/`get` from threads is therefore unprotected; the gem's own processes
|
|
@@ -14,8 +14,6 @@ The small modules everything else in the stack is built on:
|
|
|
14
14
|
type and every `Hash` operation still works. Reads and writes are
|
|
15
15
|
indifferent between String and Symbol keys.
|
|
16
16
|
|
|
17
|
-
Verified by `tmp/rewrite_D/probe_06_indiferent_misc.rb`:
|
|
18
|
-
|
|
19
17
|
```ruby
|
|
20
18
|
h = IndiferentHash.setup({ 'a' => 1 })
|
|
21
19
|
h[:b] = 2
|
|
@@ -38,7 +36,7 @@ h.keys # => ["a", "b"] (first-written form is kept)
|
|
|
38
36
|
### The destructive pair
|
|
39
37
|
|
|
40
38
|
`process_options(hash, *keys)` **destroys** the keys it extracts — it
|
|
41
|
-
`delete`s them (
|
|
39
|
+
`delete`s them (`process_options(opts, :a, :b)` leaves
|
|
42
40
|
`{:extra=>3}`). `pull_keys(hash, :prefix)` returns the sub-hash under that
|
|
43
41
|
prefix key AND removes it from the original
|
|
44
42
|
(`pull_keys({persist: true, x: 1}, :persist)` => `{:persist=>true}`, hash
|
|
@@ -56,8 +54,7 @@ in the coercion list — see the asymmetry below.
|
|
|
56
54
|
(`/\w+=(\"[^\"]*\"|[^\s\"]+)/`), strips surrounding quotes, and splits a
|
|
57
55
|
value containing a comma into an Array. Its coercion ladder is shorter than
|
|
58
56
|
`string2hash`'s: `true`/`false` are **not** special-cased at all, so both
|
|
59
|
-
survive as Strings (
|
|
60
|
-
`parse_options('a=false')` => `{"a"=>"false"}`,
|
|
57
|
+
survive as Strings (`parse_options('a=false')` => `{"a"=>"false"}`,
|
|
61
58
|
`parse_options('a=1,b=2')` => `{"a"=>["1", "b=2"]}` — the comma split eats
|
|
62
59
|
the second pair — and `parse_options('a="x y"')` => `{"a"=>"x y"}`).
|
|
63
60
|
`print_options(options)` is the inverse, re-quoting values with spaces.
|
|
@@ -67,11 +64,10 @@ branch (`options.rb:110`) is dead code for the plain case: `a=false` goes
|
|
|
67
64
|
through `options[key] = value` and the String survives. The only way the
|
|
68
65
|
branch is reached is when `false` is the *value* of a pair that the default
|
|
69
66
|
`#` separator has already split off — and the executed result is still
|
|
70
|
-
`"false"` in every
|
|
67
|
+
the executed result is still `"false"` in every form.
|
|
71
68
|
`parse_options` has no boolean branches at all. `Scout::Config.get`, by
|
|
72
69
|
contrast, turns a stored `'false'` into the boolean `false` (see
|
|
73
|
-
[Configuration.md](Configuration.md)).
|
|
74
|
-
`tmp/rewrite_D/probe_19_false_coercion.rb` and `probe_20_parse_options_false.rb`:
|
|
70
|
+
[Configuration.md](Configuration.md)).
|
|
75
71
|
|
|
76
72
|
```text
|
|
77
73
|
string2hash('a=true') => {"a"=>true}
|
|
@@ -94,7 +90,7 @@ single pair because the separator never appears — the value is the literal
|
|
|
94
90
|
`IndiferentHash.serializable(obj)` (`serialize.rb`) returns a deep copy of
|
|
95
91
|
Hash/Array structures. Arrays longer than 100 are truncated: first 70 +
|
|
96
92
|
`'...'` + last 30 + the marker `"TRUNCATED only 100 out of N shown"`
|
|
97
|
-
(
|
|
93
|
+
(250-element array => 102 elements, marker last). Use it for
|
|
98
94
|
logging fingerprints, not for round-tripping data.
|
|
99
95
|
|
|
100
96
|
### CaseInsensitiveHash
|
|
@@ -104,8 +100,7 @@ logging fingerprints, not for round-tripping data.
|
|
|
104
100
|
`values_at` are overridden; writes are plain `Hash` writes. Reads are
|
|
105
101
|
downcase-mapped against the *first* case seen for each key, so a Symbol
|
|
106
102
|
key can never be found by a String lookup (Symbol has no meaningful
|
|
107
|
-
downcase mapping in `downcase_keys`).
|
|
108
|
-
`tmp/rewrite_D/probe_24_cihash_writes.rb`:
|
|
103
|
+
downcase mapping in `downcase_keys`).
|
|
109
104
|
|
|
110
105
|
```text
|
|
111
106
|
ci = CaseInsensitiveHash.setup({"Key" => 1})
|
|
@@ -117,7 +112,18 @@ ci[:third] => 7
|
|
|
117
112
|
```
|
|
118
113
|
|
|
119
114
|
Treat it as a read-side convenience for String-keyed hashes; it is not an
|
|
120
|
-
indifferent-access container.
|
|
115
|
+
indifferent-access container. One more trap: the downcase map is
|
|
116
|
+
memoized on the first *missed* case-insensitive lookup and **never
|
|
117
|
+
invalidated by later writes** — if you look up a missing key before
|
|
118
|
+
storing it, the stored key stays invisible to case-insensitive reads:
|
|
119
|
+
|
|
120
|
+
```text
|
|
121
|
+
ci["fourth"] # => nil (builds @downcase_keys WITHOUT "Fourth")
|
|
122
|
+
ci["Fourth"] = 9
|
|
123
|
+
ci["fourth"] # => nil (stale map; only ci["Fourth"] works)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Build the hash completely (or call `ci.instance_variable_set(:@downcase_keys, nil)` to reset) before relying on case-insensitive reads.
|
|
121
127
|
|
|
122
128
|
## NamedArray
|
|
123
129
|
|
|
@@ -143,9 +149,34 @@ So `count`/`first`/`last`/`zip` are Array's, and any field that happens to
|
|
|
143
149
|
be named like an Array method is unreachable through the accessor. The
|
|
144
150
|
`key` is exposed via `all_fields` (`[key, fields].compact.flatten`).
|
|
145
151
|
|
|
152
|
+
### Name resolution is exact-first, then fuzzy — and fuzzy can bite
|
|
153
|
+
|
|
154
|
+
`identify_name` (`lib/scout/named_array.rb:22`) resolves a String
|
|
155
|
+
selector in this order: **exact match** on `to_s` equality → numeric
|
|
156
|
+
string as a position → **fuzzy match** via `field_match`
|
|
157
|
+
(`lib/scout/named_array.rb:10`) → `nil`. The fuzzy step accepts:
|
|
158
|
+
|
|
159
|
+
- a field containing `(selector)` — `'Human'` finds `Organism(Human)`
|
|
160
|
+
- a field that is `selector + ' ' + …` or the reverse — `'Sample'`
|
|
161
|
+
finds `'Sample other'`
|
|
162
|
+
|
|
163
|
+
Matching is **case-sensitive** (`'alpha'` does not find `'Alpha'`), and
|
|
164
|
+
a miss is silent:
|
|
165
|
+
|
|
166
|
+
```ruby
|
|
167
|
+
row["nope"] # => nil (no error)
|
|
168
|
+
row["nope"] = 3 # no-op (returns 3, stores nothing)
|
|
169
|
+
row.positions("nope") # => nil
|
|
170
|
+
row.values_at("nope") # raises TypeError (nil position reaches Array#values_at)
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
The practical risk: a slightly wrong column name can silently resolve
|
|
174
|
+
to a *different* column via the fuzzy prefix rule, so prefer exact
|
|
175
|
+
names — or `row.positions(name)` first when unsure.
|
|
176
|
+
|
|
146
177
|
## Misc.format family
|
|
147
178
|
|
|
148
|
-
All from `lib/scout/misc/format.rb
|
|
179
|
+
All from `lib/scout/misc/format.rb`:
|
|
149
180
|
|
|
150
181
|
| call | result |
|
|
151
182
|
|---|---|
|
|
@@ -164,8 +195,7 @@ leading `-` for negatives. The parser is `str.scan(/(\d+)(\w*)/)` and `\w*`
|
|
|
164
195
|
is greedy, so **each number may be followed by only one unit token**.
|
|
165
196
|
`"1h30m"` is scanned as the single pair `["1", "h30m"]`; `"h30m"` is not in
|
|
166
197
|
the token table, so the product becomes `1 * nil` and raises
|
|
167
|
-
`TypeError: nil can't be coerced into Integer`.
|
|
168
|
-
`tmp/rewrite_D/probe_16_timespan_exact.rb`:
|
|
198
|
+
`TypeError: nil can't be coerced into Integer`.
|
|
169
199
|
|
|
170
200
|
```text
|
|
171
201
|
Misc.timespan('1h') => 3600
|
|
@@ -176,6 +206,7 @@ Misc.timespan('1y') => 31536000
|
|
|
176
206
|
Misc.timespan('1:30') => 90 (HH:MM clock form)
|
|
177
207
|
Misc.timespan('-1h') => -3600
|
|
178
208
|
Misc.timespan('1h30m') => TypeError: nil can't be coerced into Integer
|
|
209
|
+
Misc.timespan('1h 30m') => 5400 (a space separates the pairs — works)
|
|
179
210
|
Misc.timespan('1x') => TypeError: nil can't be coerced into Integer
|
|
180
211
|
Misc.timespan('2') => 2 (bare number uses the default unit, seconds)
|
|
181
212
|
```
|
|
@@ -189,8 +220,7 @@ add the seconds yourself.
|
|
|
189
220
|
`"9dd4e461268c8034f5c8564e155c67a6"`). `Misc.file_md5(path)` hashes the
|
|
190
221
|
file **contents**; when the digest falls back to a plain `Misc.digest` on a
|
|
191
222
|
path string, it hashes the **path string**, not the content — a missing
|
|
192
|
-
`/nope/x` still yields a digest (`14470bb0...`) instead of raising
|
|
193
|
-
(probe_06). Check `File.exist?` yourself if that distinction matters.
|
|
223
|
+
`/nope/x` still yields a digest (`14470bb0...`) instead of raising. Check `File.exist?` yourself if that distinction matters.
|
|
194
224
|
|
|
195
225
|
## `Misc.insist`
|
|
196
226
|
|
|
@@ -203,7 +233,7 @@ See [ErrorHandling.md](ErrorHandling.md) for the retry protocol with
|
|
|
203
233
|
auto-required** (verify with `defined?(Hook)` => nil after
|
|
204
234
|
`require 'scout-essentials'`; it becomes a constant only after
|
|
205
235
|
`require 'scout/misc/hook'`). It exposes `Hook.extended`, `Hook.apply` and
|
|
206
|
-
`hook_method
|
|
236
|
+
`hook_method`. `Hook.apply(hook_class, base_class)` redefines the
|
|
207
237
|
methods the two classes share on `base_class`, aliasing the originals as
|
|
208
238
|
`orig_<name>` and dispatching first to the registered hooks, honouring an
|
|
209
239
|
optional `claim(*args)` predicate on each hook. It is the mechanism behind
|
|
@@ -214,7 +244,7 @@ tool registration in `CMD::TOOLS`-style setups.
|
|
|
214
244
|
Root: `TmpFile.tmpdir` => `$HOME/tmp/scout/tmpfiles` (here
|
|
215
245
|
`/home/mvazque2/tmp/scout/tmpfiles` — a machine-specific example value),
|
|
216
246
|
overridable with `TmpFile.tmpdir=`;
|
|
217
|
-
`TmpFile.user_tmp('sub')` => `$HOME/tmp/scout/sub
|
|
247
|
+
`TmpFile.user_tmp('sub')` => `$HOME/tmp/scout/sub`.
|
|
218
248
|
|
|
219
249
|
### Naming conventions (`tmp_for_file`, `lib/scout/tmpfile.rb:98-118`)
|
|
220
250
|
|
|
@@ -227,7 +257,7 @@ overridable with `TmpFile.tmpdir=`;
|
|
|
227
257
|
| `:md5` tail | digest of the remaining options |
|
|
228
258
|
| `MAX_FILE_LENGTH = 150` | names longer than this are truncated |
|
|
229
259
|
|
|
230
|
-
|
|
260
|
+
Supported shapes (`...` is
|
|
231
261
|
`TmpFile.tmpdir`, here `~/tmp/scout/tmpfiles`):
|
|
232
262
|
|
|
233
263
|
```text
|
|
@@ -245,14 +275,14 @@ the name and, because `other_options` is then non-empty, still gets the
|
|
|
245
275
|
**`nil` is not `#{}`**: `tmp_for_file('/a/b/c', nil, ...)` raises
|
|
246
276
|
`NoMethodError: super: no superclass method 'include?' for nil` from
|
|
247
277
|
`process_options` — pass an explicit empty Hash as the second argument when
|
|
248
|
-
you use `other_options
|
|
278
|
+
you use `other_options`.
|
|
249
279
|
|
|
250
280
|
### `with_file` leaks on raise
|
|
251
281
|
|
|
252
282
|
`TmpFile.with_file(content, erase, options)` writes the temp file, yields
|
|
253
283
|
it, then `Open.rm_rf tmpfile if Open.exist?(tmpfile) && erase` — there is
|
|
254
284
|
**no `ensure`**. If the block raises, the temp file stays on disk
|
|
255
|
-
(
|
|
285
|
+
(file still present after `raise "boom"`; removed normally when
|
|
256
286
|
the block succeeds). Add your own `begin/ensure` around `with_file` when the
|
|
257
287
|
block can fail. See [ErrorHandling.md](ErrorHandling.md).
|
|
258
288
|
|
|
@@ -3,8 +3,6 @@
|
|
|
3
3
|
The handful of conventions that hold across `lib/scout`. Everything below is
|
|
4
4
|
observable in the source of this repo; nothing here is aspirational.
|
|
5
5
|
|
|
6
|
-
Everything below is backed by `tmp/rewrite_C/probe_11_design.rb`
|
|
7
|
-
(probe_11), which reproduces each claim from a clean `require`.
|
|
8
6
|
|
|
9
7
|
## Composition by annotation
|
|
10
8
|
|
|
@@ -32,7 +30,7 @@ The public namespaces (`Path`, `Open`, `CMD`, `Persist`, `Misc`, `Log`,
|
|
|
32
30
|
`Resource`, `SOPT`, `IndiferentHash`, `TmpFile`, `Annotation`) are modules
|
|
33
31
|
whose methods are module-functions or class methods. Subclassing one of them
|
|
34
32
|
is not an option — `Persist` is a `Module`, so `class X < Persist` raises
|
|
35
|
-
`TypeError: superclass must be an instance of Class
|
|
33
|
+
`TypeError: superclass must be an instance of Class`. The reuse
|
|
36
34
|
pattern is `extend` (`resource/path.rb`, `annotation/annotated_object.rb`).
|
|
37
35
|
|
|
38
36
|
## Options hashes over positional flags
|
|
@@ -72,7 +70,7 @@ class KeepBar < Exception; end # log/progress/util.rb: keep the bar
|
|
|
72
70
|
bare `rescue` *does* catch.)
|
|
73
71
|
|
|
74
72
|
The point is deliberate: a bare `rescue => e` (which rescues
|
|
75
|
-
`StandardError`) **does not intercept them**
|
|
73
|
+
`StandardError`) **does not intercept them**
|
|
76
74
|
inside `begin ... rescue => e` and shows the signal escapes the bare
|
|
77
75
|
`rescue`. A producer that wants the caller to keep the stream it was handed
|
|
78
76
|
raises `DontClose` (carrying the result in `.payload`), and `Open.open`'s
|
|
@@ -90,8 +88,8 @@ Practical rules:
|
|
|
90
88
|
- aborting a `CMD` pipe surfaces as `Aborted` (a `StandardError`) or as the
|
|
91
89
|
`AbortedStream` **module** that is `extend`ed onto the stream object —
|
|
92
90
|
these are what streaming consumers should rescue
|
|
93
|
-
([Streaming Model](StreamingModel.md);
|
|
94
|
-
|
|
91
|
+
(see [Streaming Model](StreamingModel.md); `AbortedStream` is a
|
|
92
|
+
Module, not a Class).
|
|
95
93
|
|
|
96
94
|
## Atomic writes
|
|
97
95
|
|
|
@@ -116,9 +114,9 @@ Anything that produces a file other users may read concurrently goes through
|
|
|
116
114
|
|
|
117
115
|
Cleanup is explicit, not automatic. `TmpFile.with_file` removes the temporary
|
|
118
116
|
file only when the block ends normally — there is no `ensure` around the
|
|
119
|
-
`yield`, so an exception leaves the file behind
|
|
120
|
-
|
|
121
|
-
|
|
117
|
+
`yield`, so an exception leaves the file behind. Likewise stream closing
|
|
118
|
+
depends on the signal classes above, and progress bars rely on
|
|
119
|
+
`Log::ProgressBar.remove_bar` being called.
|
|
122
120
|
When correctness matters, put the cleanup in an `ensure` or use
|
|
123
121
|
`Open.consume_stream`.
|
|
124
122
|
|