scout-essentials 1.8.7 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.vimproject +26 -12
- data/README.md +83 -112
- data/VERSION +1 -1
- data/doc/Improvements.md +226 -0
- data/doc/StartHere.md +122 -0
- data/doc/developer/AnnotationSystem.md +184 -0
- data/doc/developer/Architecture.md +147 -0
- data/doc/developer/Configuration.md +238 -0
- data/doc/developer/CoreUtilities.md +265 -0
- data/doc/developer/DesignPrinciples.md +129 -0
- data/doc/developer/ErrorHandling.md +203 -0
- data/doc/developer/LockingAndConcurrency.md +157 -0
- data/doc/developer/PathResolution.md +200 -0
- data/doc/developer/PersistenceAndResources.md +119 -0
- data/doc/developer/StreamingModel.md +236 -0
- data/doc/user/AnnotatingData.md +202 -0
- data/doc/user/CachingResults.md +183 -0
- data/doc/user/CommandLineOptions.md +189 -0
- data/doc/user/Cookbook.md +211 -0
- data/doc/user/HandlingStreams.md +236 -0
- data/doc/user/LoggingAndProgress.md +158 -0
- data/doc/user/ProducingResources.md +177 -0
- data/doc/user/RemoteData.md +157 -0
- data/doc/user/RunningCommands.md +218 -0
- data/doc/user/WorkingWithFiles.md +217 -0
- data/lib/scout/cmd.rb +343 -40
- data/lib/scout/concurrent_stream.rb +14 -1
- data/lib/scout/indiferent_hash.rb +1 -1
- data/lib/scout/log/fingerprint.rb +13 -8
- data/lib/scout/log/progress/report.rb +1 -1
- data/lib/scout/log.rb +4 -1
- data/lib/scout/misc/digest.rb +6 -5
- data/lib/scout/misc/format.rb +24 -0
- data/lib/scout/named_array.rb +1 -1
- data/lib/scout/open/stream.rb +2 -2
- data/lib/scout/open/util.rb +8 -4
- data/lib/scout/open.rb +3 -3
- data/lib/scout/path/find.rb +3 -2
- data/lib/scout/persist.rb +14 -10
- data/lib/scout/resource/produce.rb +9 -1
- data/research/annotations-data-analysis.md +206 -0
- data/research/behavior-probes.md +1925 -0
- data/research/commands-streaming-analysis.md +272 -0
- data/research/design-philosophy-analysis.md +383 -0
- data/research/doc-audit-findings.md +294 -0
- data/research/ecosystem-attribution.md +118 -0
- data/research/implementation-inventory-core.md +1029 -0
- data/research/implementation-inventory-open.md +417 -0
- data/research/implementation-inventory-path-persist-resource.md +774 -0
- data/research/io-paths-analysis.md +228 -0
- data/research/persistence-resources-analysis.md +244 -0
- data/research/synthesis-report.md +80 -0
- data/scout-essentials.gemspec +37 -15
- data/test/scout/open/test_remote.rb +1 -2
- data/test/scout/test_cmd.rb +411 -0
- metadata +36 -14
- data/doc/Annotation.md +0 -352
- data/doc/CMD.md +0 -363
- data/doc/ConcurrentStream.md +0 -163
- data/doc/IndiferentHash.md +0 -240
- data/doc/Log.md +0 -235
- data/doc/NamedArray.md +0 -174
- data/doc/Open.md +0 -331
- data/doc/Path.md +0 -217
- data/doc/Persist.md +0 -214
- data/doc/Resource.md +0 -229
- data/doc/SimpleOPT.md +0 -236
- data/doc/TmpFile.md +0 -154
data/doc/IndiferentHash.md
DELETED
|
@@ -1,240 +0,0 @@
|
|
|
1
|
-
# IndiferentHash
|
|
2
|
-
|
|
3
|
-
IndiferentHash provides Hash utilities and a mixin that makes hash access indifferent to String vs Symbol keys. It also includes a set of helper utilities for parsing, merging and transforming option hashes used across the framework.
|
|
4
|
-
|
|
5
|
-
Two main pieces:
|
|
6
|
-
- IndiferentHash mixin (extend any Hash instance with IndiferentHash to get indifferent access behavior and extra hash helpers).
|
|
7
|
-
- CaseInsensitiveHash mixin (separate mixin to allow case-insensitive string keys).
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## Quick usage
|
|
12
|
-
|
|
13
|
-
Make a Hash indifferent (string/symbol interchangeable):
|
|
14
|
-
|
|
15
|
-
```ruby
|
|
16
|
-
h = { a: 1, "b" => 2 }
|
|
17
|
-
IndiferentHash.setup(h) # returns h extended with IndiferentHash
|
|
18
|
-
|
|
19
|
-
h[:a] # => 1
|
|
20
|
-
h["a"] # => 1
|
|
21
|
-
h[:b] # => 2
|
|
22
|
-
h["b"] # => 2
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
Make a Hash case-insensitive (string keys compared case-insensitively):
|
|
26
|
-
|
|
27
|
-
```ruby
|
|
28
|
-
h = { a: 1, "b" => 2 }
|
|
29
|
-
CaseInsensitiveHash.setup(h)
|
|
30
|
-
|
|
31
|
-
h[:a] # => 1
|
|
32
|
-
h["A"] # => 1
|
|
33
|
-
h[:A] # => 1
|
|
34
|
-
h["B"] # => 2
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
---
|
|
38
|
-
|
|
39
|
-
## IndiferentHash mixin (methods added to a Hash)
|
|
40
|
-
|
|
41
|
-
Call IndiferentHash.setup(hash) to extend a hash instance.
|
|
42
|
-
|
|
43
|
-
Behavior highlights:
|
|
44
|
-
- Access by Symbol or String: h[:k] and h["k"] resolve to the same entry when possible.
|
|
45
|
-
- Nested hashes returned from [] are automatically set up with IndiferentHash.
|
|
46
|
-
- Values are stored normally, but deletion and inclusion checks accept Symbol/String interchangeably.
|
|
47
|
-
|
|
48
|
-
Methods and behaviors:
|
|
49
|
-
|
|
50
|
-
- IndiferentHash.setup(hash)
|
|
51
|
-
- Extends the given hash with IndiferentHash and returns it.
|
|
52
|
-
|
|
53
|
-
- merge(other)
|
|
54
|
-
- Returns a new IndiferentHash with keys from self merged with other (other wins). The result is an IndiferentHash.
|
|
55
|
-
|
|
56
|
-
- deep_merge(other)
|
|
57
|
-
- Recursively merges nested hashes: if both have same key and values are hash-like, merges them deeply (preserving IndiferentHash behavior on nested hashes).
|
|
58
|
-
|
|
59
|
-
- [](key)
|
|
60
|
-
- Returns value for key. If not found directly, attempts the alternate form:
|
|
61
|
-
- If given Symbol/Module, will try the String key.
|
|
62
|
-
- If given String, will try the Symbol key.
|
|
63
|
-
- If the value is itself a Hash, it will be extended with IndiferentHash before returning.
|
|
64
|
-
- Note: behavior pays attention to hash default/default_proc. If a default exists and the key isn't explicitly present in keys, it returns the default without attempting alternative forms.
|
|
65
|
-
|
|
66
|
-
- []=(key, value)
|
|
67
|
-
- Deletes any existing matching key (either symbol or string) before setting the new one. This avoids duplicate representations of the same logical key.
|
|
68
|
-
|
|
69
|
-
- values_at(*keys)
|
|
70
|
-
- Returns an array of values for the provided keys (indifferent to symbol/string form).
|
|
71
|
-
|
|
72
|
-
- include?(key)
|
|
73
|
-
- Returns true if either symbol or string form exists.
|
|
74
|
-
|
|
75
|
-
- delete(key)
|
|
76
|
-
- Deletes by key; will try symbol and string form and return deleted value if found.
|
|
77
|
-
|
|
78
|
-
- clean_version
|
|
79
|
-
- Produces a plain Ruby hash where keys are strings (stringified keys), preferring the first occurrence.
|
|
80
|
-
|
|
81
|
-
- slice(*keys)
|
|
82
|
-
- Returns a new IndiferentHash containing only the requested keys. Accepts symbol or string forms; ensures both forms are considered.
|
|
83
|
-
|
|
84
|
-
- keys_to_sym! and keys_to_sym
|
|
85
|
-
- keys_to_sym! converts string keys in-place to symbols (best-effort; rescue on any failed conversion).
|
|
86
|
-
- keys_to_sym returns a new IndiferentHash with keys converted to symbols.
|
|
87
|
-
|
|
88
|
-
- prety_print
|
|
89
|
-
- A convenience wrapper that calls Misc.format_definition_list(self, sep: "\n") (keeps existing behavior/name `prety_print` as in code).
|
|
90
|
-
|
|
91
|
-
- except(*list)
|
|
92
|
-
- Returns a hash copy excluding provided keys. Accepts symbol/string forms; returns a result consistent with Hash#except but extended to be indifferent.
|
|
93
|
-
|
|
94
|
-
Notes:
|
|
95
|
-
- The implementation ensures nested hashes returned from [] or merges are set up with IndiferentHash automatically.
|
|
96
|
-
- Some method names are intentionally spelled as in the implementation (`prety_print`).
|
|
97
|
-
|
|
98
|
-
---
|
|
99
|
-
|
|
100
|
-
## CaseInsensitiveHash
|
|
101
|
-
|
|
102
|
-
A separate mixin to make key lookup case-insensitive (for string keys). Use CaseInsensitiveHash.setup(hash) to extend a hash.
|
|
103
|
-
|
|
104
|
-
Behavior:
|
|
105
|
-
- On lookup, it first tries the provided key directly. If no value, it converts the key to lowercase string and looks up a precomputed map (original_key_by_downcase) to find the actual stored key. This permits "A" and "a" to refer to the same entry.
|
|
106
|
-
- values_at returns values for provided keys using the case-insensitive lookup.
|
|
107
|
-
|
|
108
|
-
Example:
|
|
109
|
-
|
|
110
|
-
```ruby
|
|
111
|
-
h = { a: 1, "b" => 2 }
|
|
112
|
-
CaseInsensitiveHash.setup(h)
|
|
113
|
-
|
|
114
|
-
h["A"] # => 1
|
|
115
|
-
h[:A] # => 1
|
|
116
|
-
h["B"] # => 2
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
---
|
|
120
|
-
|
|
121
|
-
## Options helpers (IndiferentHash::Options functions)
|
|
122
|
-
|
|
123
|
-
These utilities are useful for parsing and handling option hashes and strings.
|
|
124
|
-
|
|
125
|
-
- add_defaults(options, defaults = {})
|
|
126
|
-
- Ensures options is an IndiferentHash, accepts defaults as Hash or string (string gets parsed). Adds defaults only for keys not present in options.
|
|
127
|
-
- Returns the options (modified/extended).
|
|
128
|
-
|
|
129
|
-
- process_options(hash, *keys)
|
|
130
|
-
- Sets up IndiferentHash on hash.
|
|
131
|
-
- If the last argument is a Hash, it is used as defaults (added first).
|
|
132
|
-
- If a single key passed, returns and removes that key from hash (prefers symbol then string).
|
|
133
|
-
- If multiple keys passed, returns array of removed values for each key.
|
|
134
|
-
- Example: IndiferentHash.process_options(h, :limit) or IndiferentHash.process_options(h, :a, :b, default: 1)
|
|
135
|
-
|
|
136
|
-
- pull_keys(hash, prefix)
|
|
137
|
-
- Pulls keys with prefix_... from hash and returns a new IndiferentHash with the suffixes as keys.
|
|
138
|
-
- Also consumes "#{prefix}_options" if present and merges into result.
|
|
139
|
-
- Example: given { foo_bar: 1, "foo_x" => 2 }, pull_keys(h, :foo) => { bar: 1, x: 2 } (keys matched as string/symbol appropriately).
|
|
140
|
-
|
|
141
|
-
- zip2hash(list1, list2)
|
|
142
|
-
- Zips two lists into a hash (keys from list1, values from list2) and sets it up as IndiferentHash.
|
|
143
|
-
|
|
144
|
-
- positional2hash(keys, *values)
|
|
145
|
-
- Converts positional values into a hash keyed by keys.
|
|
146
|
-
- Supports the common pattern where the last argument is a Hash of extra/defaults. In that case:
|
|
147
|
-
- Combines given values into a hash, removes nil/empty values, adds defaults from the extras, and prunes keys not in the original keys set.
|
|
148
|
-
- Example: IndiferentHash.positional2hash([:one,:two], 1, two: 2, extra: 4) => { one: 1, two: 2 }
|
|
149
|
-
|
|
150
|
-
- array2hash(array, default = nil)
|
|
151
|
-
- Accepts an array of [key, value] pairs and builds an IndiferentHash.
|
|
152
|
-
- If value is nil and default provided, uses a dup of default for that key.
|
|
153
|
-
|
|
154
|
-
- process_to_hash(list) { |list| ... }
|
|
155
|
-
- Yields list to block, expects a result list; zips original list with returned list into an IndiferentHash.
|
|
156
|
-
|
|
157
|
-
- hash2string(hash)
|
|
158
|
-
- Serializes a simple hash into a string representation (sorted by key). Only handles values of certain simple classes; others are omitted. Uses ":" prefix for symbol keys/values in output.
|
|
159
|
-
- Output format is key=value pairs joined with "#".
|
|
160
|
-
|
|
161
|
-
- string2hash(string, sep = "#")
|
|
162
|
-
- Parses the string produced by hash2string (or similar) and converts back into an IndiferentHash. Supports:
|
|
163
|
-
- :symbol keys/values (leading ":")
|
|
164
|
-
- quoted strings, integers, floats, booleans, regexps (/.../)
|
|
165
|
-
- empty values treated as true
|
|
166
|
-
- Example roundtrip: IndiferentHash.string2hash(IndiferentHash.hash2string(h)) == h (for supported types).
|
|
167
|
-
|
|
168
|
-
- parse_options(str)
|
|
169
|
-
- Parses a shell-like option string of key=value pairs, supporting quoted values and comma-separated lists (preserving quoted items with spaces).
|
|
170
|
-
- Returns an IndiferentHash.
|
|
171
|
-
- Example: IndiferentHash.parse_options('blueberries=true title="This is a title" list=one,two,"and three"')
|
|
172
|
-
|
|
173
|
-
- print_options(options)
|
|
174
|
-
- Serializes an options hash into a space-separated string of key=value pairs; array values become CSV (properly quoted if containing spaces).
|
|
175
|
-
|
|
176
|
-
---
|
|
177
|
-
|
|
178
|
-
## Examples (from tests and usage)
|
|
179
|
-
|
|
180
|
-
Indifferent access:
|
|
181
|
-
|
|
182
|
-
```ruby
|
|
183
|
-
h = { a: 1, "b" => 2 }
|
|
184
|
-
IndiferentHash.setup(h)
|
|
185
|
-
h[:a] # => 1
|
|
186
|
-
h["a"] # => 1
|
|
187
|
-
h["b"] # => 2
|
|
188
|
-
h[:b] # => 2
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
Deep merge:
|
|
192
|
-
|
|
193
|
-
```ruby
|
|
194
|
-
o = { h: { a: 1, b: 2 } }
|
|
195
|
-
n = { h: { c: 3 } }
|
|
196
|
-
IndiferentHash.setup(o)
|
|
197
|
-
o2 = o.deep_merge(n)
|
|
198
|
-
o2[:h]["a"] # => 1
|
|
199
|
-
o2[:h]["c"] # => 3
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
Options parsing:
|
|
203
|
-
|
|
204
|
-
```ruby
|
|
205
|
-
opts = IndiferentHash.parse_options('blueberries=true title="A title" list=one,two,"and three"')
|
|
206
|
-
opts["title"] # => "A title"
|
|
207
|
-
opts["list"] # => ["one", "two", "and three"]
|
|
208
|
-
```
|
|
209
|
-
|
|
210
|
-
String <-> hash roundtrip:
|
|
211
|
-
|
|
212
|
-
```ruby
|
|
213
|
-
h = { a: 1, b: :sym, c: true }
|
|
214
|
-
s = IndiferentHash.hash2string(h)
|
|
215
|
-
h2 = IndiferentHash.string2hash(s)
|
|
216
|
-
# h2 should equal h for the supported/simple types
|
|
217
|
-
```
|
|
218
|
-
|
|
219
|
-
pull_keys example:
|
|
220
|
-
|
|
221
|
-
```ruby
|
|
222
|
-
h = { "foo_bar" => 1, :foo_baz => 2, "other" => 3 }
|
|
223
|
-
IndiferentHash.setup(h)
|
|
224
|
-
prefixed = IndiferentHash.pull_keys(h, :foo)
|
|
225
|
-
# prefixed => { "bar" => 1, :baz => 2 } (returned as IndiferentHash)
|
|
226
|
-
# and h no longer contains those entries
|
|
227
|
-
```
|
|
228
|
-
|
|
229
|
-
---
|
|
230
|
-
|
|
231
|
-
## Implementation notes / caveats
|
|
232
|
-
|
|
233
|
-
- IndiferentHash.setup extends a single hash instance (not the Hash class). Use it on any hash instance you want to treat indifferently.
|
|
234
|
-
- Nested hashes returned from [] or created by zip/positional helpers get extended automatically with IndiferentHash.
|
|
235
|
-
- The [] lookup has special handling with Hash defaults: if the hash has a default or default_proc
|
|
236
|
-
and the requested key is not present in keys, it will return the default without trying the alternate symbol/string form.
|
|
237
|
-
- CaseInsensitiveHash is independent of IndiferentHash; you can mix both if needed, but their behaviors are separate.
|
|
238
|
-
- Some helper names are spelled as in the codebase (e.g., prety_print), used for compatibility.
|
|
239
|
-
|
|
240
|
-
This module covers common patterns required for flexible option handling in the framework: indifferent access, easy merging and defaulting, parsing/printing option strings, and extracting prefixed option subsets.
|
data/doc/Log.md
DELETED
|
@@ -1,235 +0,0 @@
|
|
|
1
|
-
# Log
|
|
2
|
-
|
|
3
|
-
The Log module is the framework-wide logging and progress utility. It provides:
|
|
4
|
-
- leveled logging (DEBUG, LOW, MEDIUM, HIGH, INFO, WARN, ERROR, NONE)
|
|
5
|
-
- colored output and utilities for color manipulation and gradients
|
|
6
|
-
- fingerprinting for compact object summaries
|
|
7
|
-
- a rich ProgressBar facility with multi-bar, ETA and history support
|
|
8
|
-
- helpers to trap and ignore stdout/stderr and to direct logs to a logfile
|
|
9
|
-
- convenience debug/inspect helpers
|
|
10
|
-
|
|
11
|
-
Files / components:
|
|
12
|
-
- log.rb — core logging API, level handling, formatting, logfile control
|
|
13
|
-
- log/color.rb — integration with Term::ANSIColor and concept color mapping
|
|
14
|
-
- log/color_class.rb — Color class for hex color parsing, blending, lightening/darkening
|
|
15
|
-
- log/fingerprint.rb — compact fingerprint representations for many object types
|
|
16
|
-
- log/progress{.rb, /util, /report} — ProgressBar implementation and helpers
|
|
17
|
-
- log/trap.rb — utilities to trap/ignore STDOUT/STDERR
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## Configuration & Environment
|
|
22
|
-
|
|
23
|
-
- Log.severity (module attribute): current logging level threshold. Messages below this level are ignored.
|
|
24
|
-
- SEVERITY constants: DEBUG, LOW, MEDIUM, HIGH, INFO, WARN, ERROR, NONE (assigned 0..7).
|
|
25
|
-
- Use Log.get_level(value) to convert numeric/string/symbol into numeric level.
|
|
26
|
-
- Default severity:
|
|
27
|
-
- Determined by environment variable `SCOUT_LOG` if set to one of the level names.
|
|
28
|
-
- Otherwise read from `~/.scout/etc/log_severity` if present, else INFO.
|
|
29
|
-
- Color disabled if `ENV["SCOUT_NOCOLOR"] == 'true'` or SOPT.nocolor set.
|
|
30
|
-
- Log.tty_size returns terminal rows (uses IO.console.winsize or `tput li`), falls back to ENV["TTY_SIZE"] or 80.
|
|
31
|
-
- Log.logfile(file_or_io) — set logfile target:
|
|
32
|
-
- Passing a String opens the file in append mode, sync=true.
|
|
33
|
-
- Passing an IO or File sets that as the logfile.
|
|
34
|
-
- If not set, logging writes to STDERR.
|
|
35
|
-
- Thread-safety: Log.log_write and Log.log_puts synchronize writes via MUTEX.
|
|
36
|
-
|
|
37
|
-
---
|
|
38
|
-
|
|
39
|
-
## Coloring & Color utilities
|
|
40
|
-
|
|
41
|
-
- Log extends Term::ANSIColor and exposes helpers:
|
|
42
|
-
- Log.color(color, str = nil, reset = false)
|
|
43
|
-
- color can be:
|
|
44
|
-
- Symbol naming an ansi color (e.g. :green)
|
|
45
|
-
- Integer index into SEVERITY_COLOR
|
|
46
|
-
- Concept color name (Log.CONCEPT_COLORS map keys like :title, :path, :value)
|
|
47
|
-
- A Color/hex handled by Color class (indirectly via Colorize)
|
|
48
|
-
- If str is nil, returns only the color control string (unless nocolor true).
|
|
49
|
-
- If nocolor is true, returns str unchanged.
|
|
50
|
-
- Log.highlight(str = nil): returns HIGHLIGHT sequence or wraps string.
|
|
51
|
-
- Log.uncolor(str) — strips ANSI color sequences.
|
|
52
|
-
|
|
53
|
-
- Color utilities:
|
|
54
|
-
- Color class (log/color_class.rb) handles hex parsing, lighten/darken/blend and returns hex strings.
|
|
55
|
-
- Colorize module (log/color.rb) provides:
|
|
56
|
-
- Color selection by name (Colorize.from_name),
|
|
57
|
-
- continuous gradient generation (Colorize.continuous),
|
|
58
|
-
- gradient/rank mapping and distinct color mapping for categorical values,
|
|
59
|
-
- TSV coloring helpers.
|
|
60
|
-
|
|
61
|
-
- Concept color map:
|
|
62
|
-
- Log.CONCEPT_COLORS (IndiferentHash) maps logical concepts to ANSI colors (e.g. :title => magenta).
|
|
63
|
-
|
|
64
|
-
---
|
|
65
|
-
|
|
66
|
-
## Basic Logging API
|
|
67
|
-
|
|
68
|
-
Main functions:
|
|
69
|
-
|
|
70
|
-
- Log.log(message = nil, severity = MEDIUM, &block)
|
|
71
|
-
- Adds newline (if missing) and delegates to Log.logn.
|
|
72
|
-
- Skips if severity < Log.severity.
|
|
73
|
-
|
|
74
|
-
- Log.logn(message = nil, severity = MEDIUM, &block)
|
|
75
|
-
- Emits formatted message without appending newline.
|
|
76
|
-
- Prefix includes timestamp and severity tag inside color.
|
|
77
|
-
- Uses Log.color to color the line. Messages with severity >= INFO are wrapped in Log.highlight.
|
|
78
|
-
- Writes via Log.log_write (synchronized).
|
|
79
|
-
|
|
80
|
-
- Convenience wrappers:
|
|
81
|
-
- Log.debug(msg), Log.low(msg), Log.medium(msg), Log.high(msg), Log.info(msg), Log.warn(msg), Log.error(msg)
|
|
82
|
-
- Each calls Log.log with the corresponding severity.
|
|
83
|
-
|
|
84
|
-
- Log.exception(e)
|
|
85
|
-
- Nicely logs an exception: formats message and backtrace using Log.fingerprint for very long messages and Log.color_stack for colored backtrace output. Honors environment `SCOUT_ORIGINAL_STACK` for ordering.
|
|
86
|
-
|
|
87
|
-
- Log.get_level(level)
|
|
88
|
-
- Accepts Numeric, String (case-insensitive), or Symbol and returns numeric level or 0/nil.
|
|
89
|
-
|
|
90
|
-
- Log.with_severity(level) { ... }
|
|
91
|
-
- Temporarily sets Log.severity for the block.
|
|
92
|
-
|
|
93
|
-
- Log.log_obj_inspect(obj, level, file = $stdout)
|
|
94
|
-
- Logs caller location and obj.inspect at given level.
|
|
95
|
-
|
|
96
|
-
- Log.log_obj_fingerprint(obj, level, file = $stdout)
|
|
97
|
-
- Logs caller location and Log.fingerprint(obj) for compact summary.
|
|
98
|
-
|
|
99
|
-
- Line/terminal helpers:
|
|
100
|
-
- Log.up_lines(n), Log.down_lines(n), Log.return_line, Log.clear_line(out = STDOUT)
|
|
101
|
-
|
|
102
|
-
---
|
|
103
|
-
|
|
104
|
-
## Fingerprinting
|
|
105
|
-
|
|
106
|
-
- Log.fingerprint(obj) produces a compact human-readable representation useful in logs:
|
|
107
|
-
- Strings are truncated with an MD5 snippet if longer than FP_MAX_STRING (150).
|
|
108
|
-
- Arrays and Hashes are truncated beyond FP_MAX_ARRAY / FP_MAX_HASH.
|
|
109
|
-
- Special handling for IO/File, Float formatting, Thread names, Symbol, nil/true/false etc.
|
|
110
|
-
- Used by other logging helpers to keep outputs concise.
|
|
111
|
-
|
|
112
|
-
---
|
|
113
|
-
|
|
114
|
-
## Progress bars
|
|
115
|
-
|
|
116
|
-
The ProgressBar is a full-featured facility for reporting progress from concurrent tasks.
|
|
117
|
-
|
|
118
|
-
Key pieces:
|
|
119
|
-
- Use Log::ProgressBar.with_bar(max_or_options, options = {}) {|bar| ... } to create a managed bar.
|
|
120
|
-
- new_bar(max, options) creates a bar; with_bar ensures removal on exit unless KeepBar is raised.
|
|
121
|
-
- ProgressBar instance attributes:
|
|
122
|
-
- max, ticks, frequency, depth, desc, file, bytes, process, callback, severity
|
|
123
|
-
- Behavior:
|
|
124
|
-
- bar.init — initialize and print first state.
|
|
125
|
-
- bar.tick(step = 1) — increment ticks and possibly report (depending on frequency and percent progress).
|
|
126
|
-
- bar.pos(position) — set bar to a specific position (pos - ticks).
|
|
127
|
-
- bar.process(elem) — calls `process` callback and interprets return to tick/pos based on type.
|
|
128
|
-
- bar.percent — computes percent (0..100).
|
|
129
|
-
- Bars are managed centrally in ProgressBar::BARS with concurrency via BAR_MUTEX.
|
|
130
|
-
- Bars can be nested (depth management), silenced, removed, persisted via `file` (save/load YAML state).
|
|
131
|
-
- ProgressBar.report and ProgressBar.report_msg produce formatted output lines including per-second rate, ETA, used time and ticks.
|
|
132
|
-
|
|
133
|
-
Helpers:
|
|
134
|
-
- ProgressBar.get_obj_bar(obj, bar) — helper to create a meaningful bar given an object (TSV, File, Array, Path, etc.) — guesses max records by inspecting file/TSV length if possible.
|
|
135
|
-
- ProgressBar.with_obj_bar(obj, bar = true) — convenience wrapper around with_bar using a guessed max.
|
|
136
|
-
|
|
137
|
-
Notes:
|
|
138
|
-
- Progress printing will skip if Log.no_bar is true (set via environment SCOUT_NO_PROGRESS or Log.no_bar=).
|
|
139
|
-
- ProgressBar persistence: if `file` option provided, the bar saves state to YAML.
|
|
140
|
-
|
|
141
|
-
---
|
|
142
|
-
|
|
143
|
-
## Trapping / ignoring STDOUT and STDERR
|
|
144
|
-
|
|
145
|
-
- Log.trap_std(msg = "STDOUT", msge = "STDERR", severity = 0, severity_err = nil) { ... }
|
|
146
|
-
- Redirects STDOUT/STDERR into pipes; background threads read and call Log.logn on captured lines with provided severity and prefix.
|
|
147
|
-
- Useful to capture external command output or to consolidate prints into the log.
|
|
148
|
-
|
|
149
|
-
- Log.trap_stderr(msg = "STDERR", severity = 0) { ... }
|
|
150
|
-
- Captures only STDERR and logs it.
|
|
151
|
-
|
|
152
|
-
- Log.ignore_stderr { ... } / Log.ignore_stdout { ... }
|
|
153
|
-
- Redirects respective stream to /dev/null for the block (silences output). Safe fallback if /dev/null missing.
|
|
154
|
-
|
|
155
|
-
These functions restore original streams when the block ends even if exceptions occur.
|
|
156
|
-
|
|
157
|
-
---
|
|
158
|
-
|
|
159
|
-
## Convenience debug/inspect helpers
|
|
160
|
-
|
|
161
|
-
Global helper methods (defined outside Log) for quick debugging:
|
|
162
|
-
|
|
163
|
-
- ppp(message) — pretty print (with color) and file/line location
|
|
164
|
-
- fff(object) — debug printing fingerprint (using Log.debug)
|
|
165
|
-
- ddd(obj, file = $stdout) — Log.log_obj_inspect(obj, :debug)
|
|
166
|
-
- lll(obj, file = $stdout) — low-level inspect wrapper
|
|
167
|
-
- mmm, iii, wwww, eee — wrappers for different severities (medium, info, warn, error)
|
|
168
|
-
- ddf/mmf/llf/iif/wwwf/eef — wrappers calling log_obj_fingerprint at different severities
|
|
169
|
-
- sss(level) { } — temporarily set severity or set it if no block
|
|
170
|
-
- ccc(obj=nil) — conditional debug printing based on $scout_debug_log (used as ad-hoc toggle)
|
|
171
|
-
|
|
172
|
-
These are small helpers used in tests (e.g., iif :foo writes INFO lines).
|
|
173
|
-
|
|
174
|
-
---
|
|
175
|
-
|
|
176
|
-
## Examples
|
|
177
|
-
|
|
178
|
-
Basic logging:
|
|
179
|
-
```ruby
|
|
180
|
-
Log.severity = Log::DEBUG
|
|
181
|
-
Log.info "Starting task"
|
|
182
|
-
Log.debug { "Expensive debug only evaluated when level allows" }
|
|
183
|
-
Log.error "Something failed"
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
Exception handling:
|
|
187
|
-
```ruby
|
|
188
|
-
begin
|
|
189
|
-
raise "boom"
|
|
190
|
-
rescue => e
|
|
191
|
-
Log.exception(e)
|
|
192
|
-
end
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
Progress bar:
|
|
196
|
-
```ruby
|
|
197
|
-
Log::ProgressBar.with_bar(100, desc: "Processing") do |bar|
|
|
198
|
-
100.times do
|
|
199
|
-
bar.tick
|
|
200
|
-
# work...
|
|
201
|
-
end
|
|
202
|
-
end
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
Trap STDOUT/STDERR block:
|
|
206
|
-
```ruby
|
|
207
|
-
Log.trap_std("OUT", "ERR", Log::INFO, Log::WARN) do
|
|
208
|
-
system("some_command")
|
|
209
|
-
end
|
|
210
|
-
```
|
|
211
|
-
|
|
212
|
-
Set logfile:
|
|
213
|
-
```ruby
|
|
214
|
-
Log.logfile("/tmp/mylog.txt")
|
|
215
|
-
Log.info "Wrote to logfile"
|
|
216
|
-
```
|
|
217
|
-
|
|
218
|
-
Fingerprint:
|
|
219
|
-
```ruby
|
|
220
|
-
s = "a very long string..."
|
|
221
|
-
Log.debug Log.fingerprint(s) # compact representation for logs
|
|
222
|
-
```
|
|
223
|
-
|
|
224
|
-
---
|
|
225
|
-
|
|
226
|
-
## Implementation notes & caveats
|
|
227
|
-
|
|
228
|
-
- Colors: if nocolor is enabled (env or Log.nocolor), color helpers return raw strings.
|
|
229
|
-
- Log.log_write and Log.log_puts are synchronized to avoid interleaved writes from threads.
|
|
230
|
-
- Log.logn uses caller and Log.last_caller to attempt to find meaningful source location for messages in stack traces.
|
|
231
|
-
- Fingerprint logic truncates long strings and large arrays/hashes to keep logs readable.
|
|
232
|
-
- ProgressBar uses a central registry (BARS) and a mutex for concurrency; nested bars are supported.
|
|
233
|
-
- The Log module depends on utility modules (Misc, IndiferentHash, TSV, Path, etc.) for some features — when used in isolation, those parts may not be available.
|
|
234
|
-
|
|
235
|
-
This document summarizes the Log module capabilities and usage. Use Log for all script-level diagnostic output, and use ProgressBar for long running operations where periodic status and ETA are useful.
|
data/doc/NamedArray.md
DELETED
|
@@ -1,174 +0,0 @@
|
|
|
1
|
-
# NamedArray
|
|
2
|
-
|
|
3
|
-
NamedArray is a small utility mixin built on top of the Annotation system that gives arrays named fields and name-based accessors. It lets you treat an Array like a record/tuple where elements can be accessed by name (symbol or string), supports fuzzy name matching, conversion to a hash (indifferent to string/symbol keys), and provides helpers for zipping/combining lists of named values.
|
|
4
|
-
|
|
5
|
-
NamedArray extends Annotation and declares two annotation attributes:
|
|
6
|
-
- fields — an ordered list of names for each position in the array
|
|
7
|
-
- key — an optional primary key field name
|
|
8
|
-
|
|
9
|
-
Since NamedArray extends Annotation, you can apply it to an array via NamedArray.setup(array, fields, key: ...), or by extending an instance.
|
|
10
|
-
|
|
11
|
-
Examples:
|
|
12
|
-
```ruby
|
|
13
|
-
a = NamedArray.setup([1,2], [:a, :b])
|
|
14
|
-
a[:a] # => 1
|
|
15
|
-
a["b"] # => 2
|
|
16
|
-
a.a # => 1 # method_missing lookup
|
|
17
|
-
a.to_hash # => IndiferentHash { a: 1, b: 2 }
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
## Core instance API
|
|
21
|
-
|
|
22
|
-
- fields, key
|
|
23
|
-
- Provided by Annotation (accessors). fields is an Array of field names associated with array positions.
|
|
24
|
-
|
|
25
|
-
- all_fields
|
|
26
|
-
- Returns [key, fields].compact.flatten — useful if you want the key included with other fields.
|
|
27
|
-
|
|
28
|
-
- [](name_or_index)
|
|
29
|
-
- Accepts a field name (symbol or string) or numeric index. Name is resolved to a numeric position via identify_name; if unresolved returns nil.
|
|
30
|
-
- Example: a[:a], a["a"], a[0]
|
|
31
|
-
|
|
32
|
-
- []=(name_or_index, value)
|
|
33
|
-
- Sets element by name (resolved to a position) or index; returns nil if name not found.
|
|
34
|
-
|
|
35
|
-
- positions(fields)
|
|
36
|
-
- Resolve one or many fields to their positions (delegates to NamedArray.identify_name).
|
|
37
|
-
|
|
38
|
-
- values_at(*positions)
|
|
39
|
-
- Accepts named fields or positions; it will translate names to indices before calling Array#values_at.
|
|
40
|
-
|
|
41
|
-
- concat(other)
|
|
42
|
-
- If other is a Hash: appends values of the hash in iteration order and adds the hash keys to this array's fields list.
|
|
43
|
-
Example:
|
|
44
|
-
a.concat({c: 3, d: 4}) # adds 3 and 4 to array and [:c, :d] to fields
|
|
45
|
-
- If other is another NamedArray: standard concat and fields from other are appended.
|
|
46
|
-
- Otherwise behaves like Array#concat.
|
|
47
|
-
|
|
48
|
-
- to_hash
|
|
49
|
-
- Returns a hash mapping fields => value for each field position. The returned hash is extended with IndiferentHash (so both string and symbol lookups work).
|
|
50
|
-
- Example: a.to_hash[:a] => 1
|
|
51
|
-
|
|
52
|
-
- prety_print
|
|
53
|
-
- Convenience pretty-print wrapper: uses Misc.format_definition_list(self.to_hash, sep: "\n").
|
|
54
|
-
|
|
55
|
-
- method_missing(name, *args)
|
|
56
|
-
- If name resolves to a field (via identify_name) returns self[name]; otherwise calls super. This gives quick accessors like a.foo
|
|
57
|
-
|
|
58
|
-
## Name resolution and matching
|
|
59
|
-
|
|
60
|
-
NamedArray provides flexible name resolution via:
|
|
61
|
-
|
|
62
|
-
- NamedArray.identify_name(names, selected, strict: false)
|
|
63
|
-
- names: array of field names (usually the NamedArray#fields)
|
|
64
|
-
- selected: value to resolve — may be nil, Range, Integer, Symbol, or String
|
|
65
|
-
- Returns:
|
|
66
|
-
- Integer index (position) for a single field selection
|
|
67
|
-
- Range (unchanged) if a Range is passed
|
|
68
|
-
- 0 for nil (treat nil as first field)
|
|
69
|
-
- :key for Symbol :key (special sentinel)
|
|
70
|
-
- nil if unresolved
|
|
71
|
-
|
|
72
|
-
Resolution rules:
|
|
73
|
-
- nil => 0
|
|
74
|
-
- Range => returned as-is
|
|
75
|
-
- Integer => returned as-is
|
|
76
|
-
- Symbol:
|
|
77
|
-
- if :key => returns :key
|
|
78
|
-
- otherwise finds first field whose to_s equals the symbol name
|
|
79
|
-
- String:
|
|
80
|
-
- exact string match first
|
|
81
|
-
- if string is numeric (^\d+$) it is treated as an index
|
|
82
|
-
- unless strict: fuzzy match using NamedArray.field_match
|
|
83
|
-
- field_match returns true if:
|
|
84
|
-
- exact equality
|
|
85
|
-
- one contains the other inside parentheses
|
|
86
|
-
- one starts with the other followed by a space
|
|
87
|
-
- returns the index found or nil if none
|
|
88
|
-
|
|
89
|
-
Instance helper identify_name(selected) delegates to the class method using this NamedArray's fields.
|
|
90
|
-
|
|
91
|
-
Note: identify_name accepts arrays for selected (returns an array of resolved positions), so values_at and other helpers can pass multiple names.
|
|
92
|
-
|
|
93
|
-
## Class-level helpers for lists
|
|
94
|
-
|
|
95
|
-
- NamedArray.field_match(field, name)
|
|
96
|
-
- Helper used by identify_name for fuzzy matching of two strings (parentheses and prefix matching).
|
|
97
|
-
|
|
98
|
-
- NamedArray._zip_fields(array, max = nil)
|
|
99
|
-
- Internal helper to zip together an array of lists, expanding single-element lists to match `max`.
|
|
100
|
-
|
|
101
|
-
- NamedArray.zip_fields(array)
|
|
102
|
-
- Zips a list-of-lists into per-position combined lists. Optimized to slice large inputs when array length is huge.
|
|
103
|
-
|
|
104
|
-
Example:
|
|
105
|
-
```ruby
|
|
106
|
-
NamedArray.zip_fields([ %w(a b), %w(1 1) ]) # => [["a","1"], ["b","1"]]
|
|
107
|
-
```
|
|
108
|
-
|
|
109
|
-
- NamedArray.add_zipped(source, new)
|
|
110
|
-
- Given two zipped-lists (source and new), concatenates each corresponding sub-array from `new` into `source` (skips nil entries).
|
|
111
|
-
- Useful to merge results incrementally.
|
|
112
|
-
|
|
113
|
-
## Concatenation with Hash
|
|
114
|
-
|
|
115
|
-
Calling concat with a Hash behaves like:
|
|
116
|
-
```ruby
|
|
117
|
-
a = NamedArray.setup([1,2], [:a, :b])
|
|
118
|
-
a.concat({c: 3, d: 4})
|
|
119
|
-
# resulting array becomes [1,2,3,4] and fields => [:a, :b, :c, :d]
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
This is handy when building named rows incrementally from keyed data.
|
|
123
|
-
|
|
124
|
-
## Integration with Annotation
|
|
125
|
-
|
|
126
|
-
Because NamedArray extends Annotation:
|
|
127
|
-
- You can call NamedArray.setup(array, fields) to set the `@fields` annotation on the array and extend it with NamedArray behavior.
|
|
128
|
-
- NamedArray.setup delegates to the Annotation::AnnotationModule.setup implementation for assigning @fields/@key values to the array.
|
|
129
|
-
|
|
130
|
-
Example:
|
|
131
|
-
```ruby
|
|
132
|
-
a = NamedArray.setup([1,2], [:a, :b])
|
|
133
|
-
a.fields # => [:a, :b]
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
## Examples (from tests)
|
|
137
|
-
|
|
138
|
-
Identify names:
|
|
139
|
-
```ruby
|
|
140
|
-
names = ["ValueA", "ValueB (Entity type)", "15"]
|
|
141
|
-
NamedArray.identify_name(names, "ValueA") # => 0
|
|
142
|
-
NamedArray.identify_name(names, :key) # => :key
|
|
143
|
-
NamedArray.identify_name(names, nil) # => 0
|
|
144
|
-
NamedArray.identify_name(names, "ValueB") # => 1 (fuzzy match)
|
|
145
|
-
NamedArray.identify_name(names, 1) # => 1
|
|
146
|
-
```
|
|
147
|
-
|
|
148
|
-
Basic named array usage:
|
|
149
|
-
```ruby
|
|
150
|
-
a = NamedArray.setup([1,2], [:a, :b])
|
|
151
|
-
a[:a] # => 1
|
|
152
|
-
a[:c] # => nil
|
|
153
|
-
a.a # => 1 (method_missing provides a getter)
|
|
154
|
-
a.to_hash # => IndiferentHash { a: 1, b: 2 }
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
Zipping and adding zipped:
|
|
158
|
-
```ruby
|
|
159
|
-
NamedArray.zip_fields([ %w(a b), %w(1 1) ]) # => [["a","1"], ["b","1"]]
|
|
160
|
-
|
|
161
|
-
a = [%w(a b), %w(1 1)]
|
|
162
|
-
NamedArray.add_zipped(a, [%w(c), %w(1)])
|
|
163
|
-
NamedArray.add_zipped(a, [%w(d), %w(1)])
|
|
164
|
-
# a => [%w(a b c d), %w(1 1 1 1)]
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
## Notes & caveats
|
|
168
|
-
|
|
169
|
-
- Name matching is intentionally forgiving (parentheses and space-prefix checks). Use `identify_name(..., strict: true)` to force exact matches only.
|
|
170
|
-
- The `fields` annotation must correspond to element positions in the array. If fields and array lengths differ, name resolution may return nil or indices outside current array bounds.
|
|
171
|
-
- to_hash returns an IndiferentHash (so consumers can use either string or symbol keys).
|
|
172
|
-
- method_missing exposes field getters only; it does not create setters (use []= to assign by name).
|
|
173
|
-
|
|
174
|
-
NamedArray is small but convenient when treating Arrays as records/rows with named columns and needing flexible lookup and composition tools.
|