scout-essentials 1.8.7 → 1.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (69) hide show
  1. checksums.yaml +4 -4
  2. data/.vimproject +26 -12
  3. data/README.md +83 -112
  4. data/VERSION +1 -1
  5. data/doc/Improvements.md +226 -0
  6. data/doc/StartHere.md +122 -0
  7. data/doc/developer/AnnotationSystem.md +184 -0
  8. data/doc/developer/Architecture.md +147 -0
  9. data/doc/developer/Configuration.md +238 -0
  10. data/doc/developer/CoreUtilities.md +265 -0
  11. data/doc/developer/DesignPrinciples.md +129 -0
  12. data/doc/developer/ErrorHandling.md +203 -0
  13. data/doc/developer/LockingAndConcurrency.md +157 -0
  14. data/doc/developer/PathResolution.md +200 -0
  15. data/doc/developer/PersistenceAndResources.md +119 -0
  16. data/doc/developer/StreamingModel.md +236 -0
  17. data/doc/user/AnnotatingData.md +202 -0
  18. data/doc/user/CachingResults.md +183 -0
  19. data/doc/user/CommandLineOptions.md +189 -0
  20. data/doc/user/Cookbook.md +211 -0
  21. data/doc/user/HandlingStreams.md +236 -0
  22. data/doc/user/LoggingAndProgress.md +158 -0
  23. data/doc/user/ProducingResources.md +177 -0
  24. data/doc/user/RemoteData.md +157 -0
  25. data/doc/user/RunningCommands.md +218 -0
  26. data/doc/user/WorkingWithFiles.md +217 -0
  27. data/lib/scout/cmd.rb +343 -40
  28. data/lib/scout/concurrent_stream.rb +14 -1
  29. data/lib/scout/indiferent_hash.rb +1 -1
  30. data/lib/scout/log/fingerprint.rb +13 -8
  31. data/lib/scout/log/progress/report.rb +1 -1
  32. data/lib/scout/log.rb +4 -1
  33. data/lib/scout/misc/digest.rb +6 -5
  34. data/lib/scout/misc/format.rb +24 -0
  35. data/lib/scout/named_array.rb +1 -1
  36. data/lib/scout/open/stream.rb +2 -2
  37. data/lib/scout/open/util.rb +8 -4
  38. data/lib/scout/open.rb +3 -3
  39. data/lib/scout/path/find.rb +3 -2
  40. data/lib/scout/persist.rb +14 -10
  41. data/lib/scout/resource/produce.rb +9 -1
  42. data/research/annotations-data-analysis.md +206 -0
  43. data/research/behavior-probes.md +1925 -0
  44. data/research/commands-streaming-analysis.md +272 -0
  45. data/research/design-philosophy-analysis.md +383 -0
  46. data/research/doc-audit-findings.md +294 -0
  47. data/research/ecosystem-attribution.md +118 -0
  48. data/research/implementation-inventory-core.md +1029 -0
  49. data/research/implementation-inventory-open.md +417 -0
  50. data/research/implementation-inventory-path-persist-resource.md +774 -0
  51. data/research/io-paths-analysis.md +228 -0
  52. data/research/persistence-resources-analysis.md +244 -0
  53. data/research/synthesis-report.md +80 -0
  54. data/scout-essentials.gemspec +37 -15
  55. data/test/scout/open/test_remote.rb +1 -2
  56. data/test/scout/test_cmd.rb +411 -0
  57. metadata +36 -14
  58. data/doc/Annotation.md +0 -352
  59. data/doc/CMD.md +0 -363
  60. data/doc/ConcurrentStream.md +0 -163
  61. data/doc/IndiferentHash.md +0 -240
  62. data/doc/Log.md +0 -235
  63. data/doc/NamedArray.md +0 -174
  64. data/doc/Open.md +0 -331
  65. data/doc/Path.md +0 -217
  66. data/doc/Persist.md +0 -214
  67. data/doc/Resource.md +0 -229
  68. data/doc/SimpleOPT.md +0 -236
  69. data/doc/TmpFile.md +0 -154
@@ -1,240 +0,0 @@
1
- # IndiferentHash
2
-
3
- IndiferentHash provides Hash utilities and a mixin that makes hash access indifferent to String vs Symbol keys. It also includes a set of helper utilities for parsing, merging and transforming option hashes used across the framework.
4
-
5
- Two main pieces:
6
- - IndiferentHash mixin (extend any Hash instance with IndiferentHash to get indifferent access behavior and extra hash helpers).
7
- - CaseInsensitiveHash mixin (separate mixin to allow case-insensitive string keys).
8
-
9
- ---
10
-
11
- ## Quick usage
12
-
13
- Make a Hash indifferent (string/symbol interchangeable):
14
-
15
- ```ruby
16
- h = { a: 1, "b" => 2 }
17
- IndiferentHash.setup(h) # returns h extended with IndiferentHash
18
-
19
- h[:a] # => 1
20
- h["a"] # => 1
21
- h[:b] # => 2
22
- h["b"] # => 2
23
- ```
24
-
25
- Make a Hash case-insensitive (string keys compared case-insensitively):
26
-
27
- ```ruby
28
- h = { a: 1, "b" => 2 }
29
- CaseInsensitiveHash.setup(h)
30
-
31
- h[:a] # => 1
32
- h["A"] # => 1
33
- h[:A] # => 1
34
- h["B"] # => 2
35
- ```
36
-
37
- ---
38
-
39
- ## IndiferentHash mixin (methods added to a Hash)
40
-
41
- Call IndiferentHash.setup(hash) to extend a hash instance.
42
-
43
- Behavior highlights:
44
- - Access by Symbol or String: h[:k] and h["k"] resolve to the same entry when possible.
45
- - Nested hashes returned from [] are automatically set up with IndiferentHash.
46
- - Values are stored normally, but deletion and inclusion checks accept Symbol/String interchangeably.
47
-
48
- Methods and behaviors:
49
-
50
- - IndiferentHash.setup(hash)
51
- - Extends the given hash with IndiferentHash and returns it.
52
-
53
- - merge(other)
54
- - Returns a new IndiferentHash with keys from self merged with other (other wins). The result is an IndiferentHash.
55
-
56
- - deep_merge(other)
57
- - Recursively merges nested hashes: if both have same key and values are hash-like, merges them deeply (preserving IndiferentHash behavior on nested hashes).
58
-
59
- - [](key)
60
- - Returns value for key. If not found directly, attempts the alternate form:
61
- - If given Symbol/Module, will try the String key.
62
- - If given String, will try the Symbol key.
63
- - If the value is itself a Hash, it will be extended with IndiferentHash before returning.
64
- - Note: behavior pays attention to hash default/default_proc. If a default exists and the key isn't explicitly present in keys, it returns the default without attempting alternative forms.
65
-
66
- - []=(key, value)
67
- - Deletes any existing matching key (either symbol or string) before setting the new one. This avoids duplicate representations of the same logical key.
68
-
69
- - values_at(*keys)
70
- - Returns an array of values for the provided keys (indifferent to symbol/string form).
71
-
72
- - include?(key)
73
- - Returns true if either symbol or string form exists.
74
-
75
- - delete(key)
76
- - Deletes by key; will try symbol and string form and return deleted value if found.
77
-
78
- - clean_version
79
- - Produces a plain Ruby hash where keys are strings (stringified keys), preferring the first occurrence.
80
-
81
- - slice(*keys)
82
- - Returns a new IndiferentHash containing only the requested keys. Accepts symbol or string forms; ensures both forms are considered.
83
-
84
- - keys_to_sym! and keys_to_sym
85
- - keys_to_sym! converts string keys in-place to symbols (best-effort; rescue on any failed conversion).
86
- - keys_to_sym returns a new IndiferentHash with keys converted to symbols.
87
-
88
- - prety_print
89
- - A convenience wrapper that calls Misc.format_definition_list(self, sep: "\n") (keeps existing behavior/name `prety_print` as in code).
90
-
91
- - except(*list)
92
- - Returns a hash copy excluding provided keys. Accepts symbol/string forms; returns a result consistent with Hash#except but extended to be indifferent.
93
-
94
- Notes:
95
- - The implementation ensures nested hashes returned from [] or merges are set up with IndiferentHash automatically.
96
- - Some method names are intentionally spelled as in the implementation (`prety_print`).
97
-
98
- ---
99
-
100
- ## CaseInsensitiveHash
101
-
102
- A separate mixin to make key lookup case-insensitive (for string keys). Use CaseInsensitiveHash.setup(hash) to extend a hash.
103
-
104
- Behavior:
105
- - On lookup, it first tries the provided key directly. If no value, it converts the key to lowercase string and looks up a precomputed map (original_key_by_downcase) to find the actual stored key. This permits "A" and "a" to refer to the same entry.
106
- - values_at returns values for provided keys using the case-insensitive lookup.
107
-
108
- Example:
109
-
110
- ```ruby
111
- h = { a: 1, "b" => 2 }
112
- CaseInsensitiveHash.setup(h)
113
-
114
- h["A"] # => 1
115
- h[:A] # => 1
116
- h["B"] # => 2
117
- ```
118
-
119
- ---
120
-
121
- ## Options helpers (IndiferentHash::Options functions)
122
-
123
- These utilities are useful for parsing and handling option hashes and strings.
124
-
125
- - add_defaults(options, defaults = {})
126
- - Ensures options is an IndiferentHash, accepts defaults as Hash or string (string gets parsed). Adds defaults only for keys not present in options.
127
- - Returns the options (modified/extended).
128
-
129
- - process_options(hash, *keys)
130
- - Sets up IndiferentHash on hash.
131
- - If the last argument is a Hash, it is used as defaults (added first).
132
- - If a single key passed, returns and removes that key from hash (prefers symbol then string).
133
- - If multiple keys passed, returns array of removed values for each key.
134
- - Example: IndiferentHash.process_options(h, :limit) or IndiferentHash.process_options(h, :a, :b, default: 1)
135
-
136
- - pull_keys(hash, prefix)
137
- - Pulls keys with prefix_... from hash and returns a new IndiferentHash with the suffixes as keys.
138
- - Also consumes "#{prefix}_options" if present and merges into result.
139
- - Example: given { foo_bar: 1, "foo_x" => 2 }, pull_keys(h, :foo) => { bar: 1, x: 2 } (keys matched as string/symbol appropriately).
140
-
141
- - zip2hash(list1, list2)
142
- - Zips two lists into a hash (keys from list1, values from list2) and sets it up as IndiferentHash.
143
-
144
- - positional2hash(keys, *values)
145
- - Converts positional values into a hash keyed by keys.
146
- - Supports the common pattern where the last argument is a Hash of extra/defaults. In that case:
147
- - Combines given values into a hash, removes nil/empty values, adds defaults from the extras, and prunes keys not in the original keys set.
148
- - Example: IndiferentHash.positional2hash([:one,:two], 1, two: 2, extra: 4) => { one: 1, two: 2 }
149
-
150
- - array2hash(array, default = nil)
151
- - Accepts an array of [key, value] pairs and builds an IndiferentHash.
152
- - If value is nil and default provided, uses a dup of default for that key.
153
-
154
- - process_to_hash(list) { |list| ... }
155
- - Yields list to block, expects a result list; zips original list with returned list into an IndiferentHash.
156
-
157
- - hash2string(hash)
158
- - Serializes a simple hash into a string representation (sorted by key). Only handles values of certain simple classes; others are omitted. Uses ":" prefix for symbol keys/values in output.
159
- - Output format is key=value pairs joined with "#".
160
-
161
- - string2hash(string, sep = "#")
162
- - Parses the string produced by hash2string (or similar) and converts back into an IndiferentHash. Supports:
163
- - :symbol keys/values (leading ":")
164
- - quoted strings, integers, floats, booleans, regexps (/.../)
165
- - empty values treated as true
166
- - Example roundtrip: IndiferentHash.string2hash(IndiferentHash.hash2string(h)) == h (for supported types).
167
-
168
- - parse_options(str)
169
- - Parses a shell-like option string of key=value pairs, supporting quoted values and comma-separated lists (preserving quoted items with spaces).
170
- - Returns an IndiferentHash.
171
- - Example: IndiferentHash.parse_options('blueberries=true title="This is a title" list=one,two,"and three"')
172
-
173
- - print_options(options)
174
- - Serializes an options hash into a space-separated string of key=value pairs; array values become CSV (properly quoted if containing spaces).
175
-
176
- ---
177
-
178
- ## Examples (from tests and usage)
179
-
180
- Indifferent access:
181
-
182
- ```ruby
183
- h = { a: 1, "b" => 2 }
184
- IndiferentHash.setup(h)
185
- h[:a] # => 1
186
- h["a"] # => 1
187
- h["b"] # => 2
188
- h[:b] # => 2
189
- ```
190
-
191
- Deep merge:
192
-
193
- ```ruby
194
- o = { h: { a: 1, b: 2 } }
195
- n = { h: { c: 3 } }
196
- IndiferentHash.setup(o)
197
- o2 = o.deep_merge(n)
198
- o2[:h]["a"] # => 1
199
- o2[:h]["c"] # => 3
200
- ```
201
-
202
- Options parsing:
203
-
204
- ```ruby
205
- opts = IndiferentHash.parse_options('blueberries=true title="A title" list=one,two,"and three"')
206
- opts["title"] # => "A title"
207
- opts["list"] # => ["one", "two", "and three"]
208
- ```
209
-
210
- String <-> hash roundtrip:
211
-
212
- ```ruby
213
- h = { a: 1, b: :sym, c: true }
214
- s = IndiferentHash.hash2string(h)
215
- h2 = IndiferentHash.string2hash(s)
216
- # h2 should equal h for the supported/simple types
217
- ```
218
-
219
- pull_keys example:
220
-
221
- ```ruby
222
- h = { "foo_bar" => 1, :foo_baz => 2, "other" => 3 }
223
- IndiferentHash.setup(h)
224
- prefixed = IndiferentHash.pull_keys(h, :foo)
225
- # prefixed => { "bar" => 1, :baz => 2 } (returned as IndiferentHash)
226
- # and h no longer contains those entries
227
- ```
228
-
229
- ---
230
-
231
- ## Implementation notes / caveats
232
-
233
- - IndiferentHash.setup extends a single hash instance (not the Hash class). Use it on any hash instance you want to treat indifferently.
234
- - Nested hashes returned from [] or created by zip/positional helpers get extended automatically with IndiferentHash.
235
- - The [] lookup has special handling with Hash defaults: if the hash has a default or default_proc
236
- and the requested key is not present in keys, it will return the default without trying the alternate symbol/string form.
237
- - CaseInsensitiveHash is independent of IndiferentHash; you can mix both if needed, but their behaviors are separate.
238
- - Some helper names are spelled as in the codebase (e.g., prety_print), used for compatibility.
239
-
240
- This module covers common patterns required for flexible option handling in the framework: indifferent access, easy merging and defaulting, parsing/printing option strings, and extracting prefixed option subsets.
data/doc/Log.md DELETED
@@ -1,235 +0,0 @@
1
- # Log
2
-
3
- The Log module is the framework-wide logging and progress utility. It provides:
4
- - leveled logging (DEBUG, LOW, MEDIUM, HIGH, INFO, WARN, ERROR, NONE)
5
- - colored output and utilities for color manipulation and gradients
6
- - fingerprinting for compact object summaries
7
- - a rich ProgressBar facility with multi-bar, ETA and history support
8
- - helpers to trap and ignore stdout/stderr and to direct logs to a logfile
9
- - convenience debug/inspect helpers
10
-
11
- Files / components:
12
- - log.rb — core logging API, level handling, formatting, logfile control
13
- - log/color.rb — integration with Term::ANSIColor and concept color mapping
14
- - log/color_class.rb — Color class for hex color parsing, blending, lightening/darkening
15
- - log/fingerprint.rb — compact fingerprint representations for many object types
16
- - log/progress{.rb, /util, /report} — ProgressBar implementation and helpers
17
- - log/trap.rb — utilities to trap/ignore STDOUT/STDERR
18
-
19
- ---
20
-
21
- ## Configuration & Environment
22
-
23
- - Log.severity (module attribute): current logging level threshold. Messages below this level are ignored.
24
- - SEVERITY constants: DEBUG, LOW, MEDIUM, HIGH, INFO, WARN, ERROR, NONE (assigned 0..7).
25
- - Use Log.get_level(value) to convert numeric/string/symbol into numeric level.
26
- - Default severity:
27
- - Determined by environment variable `SCOUT_LOG` if set to one of the level names.
28
- - Otherwise read from `~/.scout/etc/log_severity` if present, else INFO.
29
- - Color disabled if `ENV["SCOUT_NOCOLOR"] == 'true'` or SOPT.nocolor set.
30
- - Log.tty_size returns terminal rows (uses IO.console.winsize or `tput li`), falls back to ENV["TTY_SIZE"] or 80.
31
- - Log.logfile(file_or_io) — set logfile target:
32
- - Passing a String opens the file in append mode, sync=true.
33
- - Passing an IO or File sets that as the logfile.
34
- - If not set, logging writes to STDERR.
35
- - Thread-safety: Log.log_write and Log.log_puts synchronize writes via MUTEX.
36
-
37
- ---
38
-
39
- ## Coloring & Color utilities
40
-
41
- - Log extends Term::ANSIColor and exposes helpers:
42
- - Log.color(color, str = nil, reset = false)
43
- - color can be:
44
- - Symbol naming an ansi color (e.g. :green)
45
- - Integer index into SEVERITY_COLOR
46
- - Concept color name (Log.CONCEPT_COLORS map keys like :title, :path, :value)
47
- - A Color/hex handled by Color class (indirectly via Colorize)
48
- - If str is nil, returns only the color control string (unless nocolor true).
49
- - If nocolor is true, returns str unchanged.
50
- - Log.highlight(str = nil): returns HIGHLIGHT sequence or wraps string.
51
- - Log.uncolor(str) — strips ANSI color sequences.
52
-
53
- - Color utilities:
54
- - Color class (log/color_class.rb) handles hex parsing, lighten/darken/blend and returns hex strings.
55
- - Colorize module (log/color.rb) provides:
56
- - Color selection by name (Colorize.from_name),
57
- - continuous gradient generation (Colorize.continuous),
58
- - gradient/rank mapping and distinct color mapping for categorical values,
59
- - TSV coloring helpers.
60
-
61
- - Concept color map:
62
- - Log.CONCEPT_COLORS (IndiferentHash) maps logical concepts to ANSI colors (e.g. :title => magenta).
63
-
64
- ---
65
-
66
- ## Basic Logging API
67
-
68
- Main functions:
69
-
70
- - Log.log(message = nil, severity = MEDIUM, &block)
71
- - Adds newline (if missing) and delegates to Log.logn.
72
- - Skips if severity < Log.severity.
73
-
74
- - Log.logn(message = nil, severity = MEDIUM, &block)
75
- - Emits formatted message without appending newline.
76
- - Prefix includes timestamp and severity tag inside color.
77
- - Uses Log.color to color the line. Messages with severity >= INFO are wrapped in Log.highlight.
78
- - Writes via Log.log_write (synchronized).
79
-
80
- - Convenience wrappers:
81
- - Log.debug(msg), Log.low(msg), Log.medium(msg), Log.high(msg), Log.info(msg), Log.warn(msg), Log.error(msg)
82
- - Each calls Log.log with the corresponding severity.
83
-
84
- - Log.exception(e)
85
- - Nicely logs an exception: formats message and backtrace using Log.fingerprint for very long messages and Log.color_stack for colored backtrace output. Honors environment `SCOUT_ORIGINAL_STACK` for ordering.
86
-
87
- - Log.get_level(level)
88
- - Accepts Numeric, String (case-insensitive), or Symbol and returns numeric level or 0/nil.
89
-
90
- - Log.with_severity(level) { ... }
91
- - Temporarily sets Log.severity for the block.
92
-
93
- - Log.log_obj_inspect(obj, level, file = $stdout)
94
- - Logs caller location and obj.inspect at given level.
95
-
96
- - Log.log_obj_fingerprint(obj, level, file = $stdout)
97
- - Logs caller location and Log.fingerprint(obj) for compact summary.
98
-
99
- - Line/terminal helpers:
100
- - Log.up_lines(n), Log.down_lines(n), Log.return_line, Log.clear_line(out = STDOUT)
101
-
102
- ---
103
-
104
- ## Fingerprinting
105
-
106
- - Log.fingerprint(obj) produces a compact human-readable representation useful in logs:
107
- - Strings are truncated with an MD5 snippet if longer than FP_MAX_STRING (150).
108
- - Arrays and Hashes are truncated beyond FP_MAX_ARRAY / FP_MAX_HASH.
109
- - Special handling for IO/File, Float formatting, Thread names, Symbol, nil/true/false etc.
110
- - Used by other logging helpers to keep outputs concise.
111
-
112
- ---
113
-
114
- ## Progress bars
115
-
116
- The ProgressBar is a full-featured facility for reporting progress from concurrent tasks.
117
-
118
- Key pieces:
119
- - Use Log::ProgressBar.with_bar(max_or_options, options = {}) {|bar| ... } to create a managed bar.
120
- - new_bar(max, options) creates a bar; with_bar ensures removal on exit unless KeepBar is raised.
121
- - ProgressBar instance attributes:
122
- - max, ticks, frequency, depth, desc, file, bytes, process, callback, severity
123
- - Behavior:
124
- - bar.init — initialize and print first state.
125
- - bar.tick(step = 1) — increment ticks and possibly report (depending on frequency and percent progress).
126
- - bar.pos(position) — set bar to a specific position (pos - ticks).
127
- - bar.process(elem) — calls `process` callback and interprets return to tick/pos based on type.
128
- - bar.percent — computes percent (0..100).
129
- - Bars are managed centrally in ProgressBar::BARS with concurrency via BAR_MUTEX.
130
- - Bars can be nested (depth management), silenced, removed, persisted via `file` (save/load YAML state).
131
- - ProgressBar.report and ProgressBar.report_msg produce formatted output lines including per-second rate, ETA, used time and ticks.
132
-
133
- Helpers:
134
- - ProgressBar.get_obj_bar(obj, bar) — helper to create a meaningful bar given an object (TSV, File, Array, Path, etc.) — guesses max records by inspecting file/TSV length if possible.
135
- - ProgressBar.with_obj_bar(obj, bar = true) — convenience wrapper around with_bar using a guessed max.
136
-
137
- Notes:
138
- - Progress printing will skip if Log.no_bar is true (set via environment SCOUT_NO_PROGRESS or Log.no_bar=).
139
- - ProgressBar persistence: if `file` option provided, the bar saves state to YAML.
140
-
141
- ---
142
-
143
- ## Trapping / ignoring STDOUT and STDERR
144
-
145
- - Log.trap_std(msg = "STDOUT", msge = "STDERR", severity = 0, severity_err = nil) { ... }
146
- - Redirects STDOUT/STDERR into pipes; background threads read and call Log.logn on captured lines with provided severity and prefix.
147
- - Useful to capture external command output or to consolidate prints into the log.
148
-
149
- - Log.trap_stderr(msg = "STDERR", severity = 0) { ... }
150
- - Captures only STDERR and logs it.
151
-
152
- - Log.ignore_stderr { ... } / Log.ignore_stdout { ... }
153
- - Redirects respective stream to /dev/null for the block (silences output). Safe fallback if /dev/null missing.
154
-
155
- These functions restore original streams when the block ends even if exceptions occur.
156
-
157
- ---
158
-
159
- ## Convenience debug/inspect helpers
160
-
161
- Global helper methods (defined outside Log) for quick debugging:
162
-
163
- - ppp(message) — pretty print (with color) and file/line location
164
- - fff(object) — debug printing fingerprint (using Log.debug)
165
- - ddd(obj, file = $stdout) — Log.log_obj_inspect(obj, :debug)
166
- - lll(obj, file = $stdout) — low-level inspect wrapper
167
- - mmm, iii, wwww, eee — wrappers for different severities (medium, info, warn, error)
168
- - ddf/mmf/llf/iif/wwwf/eef — wrappers calling log_obj_fingerprint at different severities
169
- - sss(level) { } — temporarily set severity or set it if no block
170
- - ccc(obj=nil) — conditional debug printing based on $scout_debug_log (used as ad-hoc toggle)
171
-
172
- These are small helpers used in tests (e.g., iif :foo writes INFO lines).
173
-
174
- ---
175
-
176
- ## Examples
177
-
178
- Basic logging:
179
- ```ruby
180
- Log.severity = Log::DEBUG
181
- Log.info "Starting task"
182
- Log.debug { "Expensive debug only evaluated when level allows" }
183
- Log.error "Something failed"
184
- ```
185
-
186
- Exception handling:
187
- ```ruby
188
- begin
189
- raise "boom"
190
- rescue => e
191
- Log.exception(e)
192
- end
193
- ```
194
-
195
- Progress bar:
196
- ```ruby
197
- Log::ProgressBar.with_bar(100, desc: "Processing") do |bar|
198
- 100.times do
199
- bar.tick
200
- # work...
201
- end
202
- end
203
- ```
204
-
205
- Trap STDOUT/STDERR block:
206
- ```ruby
207
- Log.trap_std("OUT", "ERR", Log::INFO, Log::WARN) do
208
- system("some_command")
209
- end
210
- ```
211
-
212
- Set logfile:
213
- ```ruby
214
- Log.logfile("/tmp/mylog.txt")
215
- Log.info "Wrote to logfile"
216
- ```
217
-
218
- Fingerprint:
219
- ```ruby
220
- s = "a very long string..."
221
- Log.debug Log.fingerprint(s) # compact representation for logs
222
- ```
223
-
224
- ---
225
-
226
- ## Implementation notes & caveats
227
-
228
- - Colors: if nocolor is enabled (env or Log.nocolor), color helpers return raw strings.
229
- - Log.log_write and Log.log_puts are synchronized to avoid interleaved writes from threads.
230
- - Log.logn uses caller and Log.last_caller to attempt to find meaningful source location for messages in stack traces.
231
- - Fingerprint logic truncates long strings and large arrays/hashes to keep logs readable.
232
- - ProgressBar uses a central registry (BARS) and a mutex for concurrency; nested bars are supported.
233
- - The Log module depends on utility modules (Misc, IndiferentHash, TSV, Path, etc.) for some features — when used in isolation, those parts may not be available.
234
-
235
- This document summarizes the Log module capabilities and usage. Use Log for all script-level diagnostic output, and use ProgressBar for long running operations where periodic status and ETA are useful.
data/doc/NamedArray.md DELETED
@@ -1,174 +0,0 @@
1
- # NamedArray
2
-
3
- NamedArray is a small utility mixin built on top of the Annotation system that gives arrays named fields and name-based accessors. It lets you treat an Array like a record/tuple where elements can be accessed by name (symbol or string), supports fuzzy name matching, conversion to a hash (indifferent to string/symbol keys), and provides helpers for zipping/combining lists of named values.
4
-
5
- NamedArray extends Annotation and declares two annotation attributes:
6
- - fields — an ordered list of names for each position in the array
7
- - key — an optional primary key field name
8
-
9
- Since NamedArray extends Annotation, you can apply it to an array via NamedArray.setup(array, fields, key: ...), or by extending an instance.
10
-
11
- Examples:
12
- ```ruby
13
- a = NamedArray.setup([1,2], [:a, :b])
14
- a[:a] # => 1
15
- a["b"] # => 2
16
- a.a # => 1 # method_missing lookup
17
- a.to_hash # => IndiferentHash { a: 1, b: 2 }
18
- ```
19
-
20
- ## Core instance API
21
-
22
- - fields, key
23
- - Provided by Annotation (accessors). fields is an Array of field names associated with array positions.
24
-
25
- - all_fields
26
- - Returns [key, fields].compact.flatten — useful if you want the key included with other fields.
27
-
28
- - [](name_or_index)
29
- - Accepts a field name (symbol or string) or numeric index. Name is resolved to a numeric position via identify_name; if unresolved returns nil.
30
- - Example: a[:a], a["a"], a[0]
31
-
32
- - []=(name_or_index, value)
33
- - Sets element by name (resolved to a position) or index; returns nil if name not found.
34
-
35
- - positions(fields)
36
- - Resolve one or many fields to their positions (delegates to NamedArray.identify_name).
37
-
38
- - values_at(*positions)
39
- - Accepts named fields or positions; it will translate names to indices before calling Array#values_at.
40
-
41
- - concat(other)
42
- - If other is a Hash: appends values of the hash in iteration order and adds the hash keys to this array's fields list.
43
- Example:
44
- a.concat({c: 3, d: 4}) # adds 3 and 4 to array and [:c, :d] to fields
45
- - If other is another NamedArray: standard concat and fields from other are appended.
46
- - Otherwise behaves like Array#concat.
47
-
48
- - to_hash
49
- - Returns a hash mapping fields => value for each field position. The returned hash is extended with IndiferentHash (so both string and symbol lookups work).
50
- - Example: a.to_hash[:a] => 1
51
-
52
- - prety_print
53
- - Convenience pretty-print wrapper: uses Misc.format_definition_list(self.to_hash, sep: "\n").
54
-
55
- - method_missing(name, *args)
56
- - If name resolves to a field (via identify_name) returns self[name]; otherwise calls super. This gives quick accessors like a.foo
57
-
58
- ## Name resolution and matching
59
-
60
- NamedArray provides flexible name resolution via:
61
-
62
- - NamedArray.identify_name(names, selected, strict: false)
63
- - names: array of field names (usually the NamedArray#fields)
64
- - selected: value to resolve — may be nil, Range, Integer, Symbol, or String
65
- - Returns:
66
- - Integer index (position) for a single field selection
67
- - Range (unchanged) if a Range is passed
68
- - 0 for nil (treat nil as first field)
69
- - :key for Symbol :key (special sentinel)
70
- - nil if unresolved
71
-
72
- Resolution rules:
73
- - nil => 0
74
- - Range => returned as-is
75
- - Integer => returned as-is
76
- - Symbol:
77
- - if :key => returns :key
78
- - otherwise finds first field whose to_s equals the symbol name
79
- - String:
80
- - exact string match first
81
- - if string is numeric (^\d+$) it is treated as an index
82
- - unless strict: fuzzy match using NamedArray.field_match
83
- - field_match returns true if:
84
- - exact equality
85
- - one contains the other inside parentheses
86
- - one starts with the other followed by a space
87
- - returns the index found or nil if none
88
-
89
- Instance helper identify_name(selected) delegates to the class method using this NamedArray's fields.
90
-
91
- Note: identify_name accepts arrays for selected (returns an array of resolved positions), so values_at and other helpers can pass multiple names.
92
-
93
- ## Class-level helpers for lists
94
-
95
- - NamedArray.field_match(field, name)
96
- - Helper used by identify_name for fuzzy matching of two strings (parentheses and prefix matching).
97
-
98
- - NamedArray._zip_fields(array, max = nil)
99
- - Internal helper to zip together an array of lists, expanding single-element lists to match `max`.
100
-
101
- - NamedArray.zip_fields(array)
102
- - Zips a list-of-lists into per-position combined lists. Optimized to slice large inputs when array length is huge.
103
-
104
- Example:
105
- ```ruby
106
- NamedArray.zip_fields([ %w(a b), %w(1 1) ]) # => [["a","1"], ["b","1"]]
107
- ```
108
-
109
- - NamedArray.add_zipped(source, new)
110
- - Given two zipped-lists (source and new), concatenates each corresponding sub-array from `new` into `source` (skips nil entries).
111
- - Useful to merge results incrementally.
112
-
113
- ## Concatenation with Hash
114
-
115
- Calling concat with a Hash behaves like:
116
- ```ruby
117
- a = NamedArray.setup([1,2], [:a, :b])
118
- a.concat({c: 3, d: 4})
119
- # resulting array becomes [1,2,3,4] and fields => [:a, :b, :c, :d]
120
- ```
121
-
122
- This is handy when building named rows incrementally from keyed data.
123
-
124
- ## Integration with Annotation
125
-
126
- Because NamedArray extends Annotation:
127
- - You can call NamedArray.setup(array, fields) to set the `@fields` annotation on the array and extend it with NamedArray behavior.
128
- - NamedArray.setup delegates to the Annotation::AnnotationModule.setup implementation for assigning @fields/@key values to the array.
129
-
130
- Example:
131
- ```ruby
132
- a = NamedArray.setup([1,2], [:a, :b])
133
- a.fields # => [:a, :b]
134
- ```
135
-
136
- ## Examples (from tests)
137
-
138
- Identify names:
139
- ```ruby
140
- names = ["ValueA", "ValueB (Entity type)", "15"]
141
- NamedArray.identify_name(names, "ValueA") # => 0
142
- NamedArray.identify_name(names, :key) # => :key
143
- NamedArray.identify_name(names, nil) # => 0
144
- NamedArray.identify_name(names, "ValueB") # => 1 (fuzzy match)
145
- NamedArray.identify_name(names, 1) # => 1
146
- ```
147
-
148
- Basic named array usage:
149
- ```ruby
150
- a = NamedArray.setup([1,2], [:a, :b])
151
- a[:a] # => 1
152
- a[:c] # => nil
153
- a.a # => 1 (method_missing provides a getter)
154
- a.to_hash # => IndiferentHash { a: 1, b: 2 }
155
- ```
156
-
157
- Zipping and adding zipped:
158
- ```ruby
159
- NamedArray.zip_fields([ %w(a b), %w(1 1) ]) # => [["a","1"], ["b","1"]]
160
-
161
- a = [%w(a b), %w(1 1)]
162
- NamedArray.add_zipped(a, [%w(c), %w(1)])
163
- NamedArray.add_zipped(a, [%w(d), %w(1)])
164
- # a => [%w(a b c d), %w(1 1 1 1)]
165
- ```
166
-
167
- ## Notes & caveats
168
-
169
- - Name matching is intentionally forgiving (parentheses and space-prefix checks). Use `identify_name(..., strict: true)` to force exact matches only.
170
- - The `fields` annotation must correspond to element positions in the array. If fields and array lengths differ, name resolution may return nil or indices outside current array bounds.
171
- - to_hash returns an IndiferentHash (so consumers can use either string or symbol keys).
172
- - method_missing exposes field getters only; it does not create setters (use []= to assign by name).
173
-
174
- NamedArray is small but convenient when treating Arrays as records/rows with named columns and needing flexible lookup and composition tools.