scout-essentials 1.7.1 → 1.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.vimproject +200 -47
- data/README.md +136 -0
- data/Rakefile +1 -0
- data/VERSION +1 -1
- data/doc/Annotation.md +352 -0
- data/doc/CMD.md +203 -0
- data/doc/ConcurrentStream.md +163 -0
- data/doc/IndiferentHash.md +240 -0
- data/doc/Log.md +235 -0
- data/doc/NamedArray.md +174 -0
- data/doc/Open.md +331 -0
- data/doc/Path.md +217 -0
- data/doc/Persist.md +214 -0
- data/doc/Resource.md +229 -0
- data/doc/SimpleOPT.md +236 -0
- data/doc/TmpFile.md +154 -0
- data/lib/scout/annotation/annotated_object.rb +8 -0
- data/lib/scout/annotation/annotation_module.rb +1 -0
- data/lib/scout/cmd.rb +19 -12
- data/lib/scout/concurrent_stream.rb +3 -1
- data/lib/scout/config.rb +2 -2
- data/lib/scout/indiferent_hash/options.rb +2 -2
- data/lib/scout/indiferent_hash.rb +16 -0
- data/lib/scout/log/color.rb +5 -3
- data/lib/scout/log/fingerprint.rb +8 -8
- data/lib/scout/log/progress/report.rb +6 -6
- data/lib/scout/log.rb +7 -7
- data/lib/scout/misc/digest.rb +11 -13
- data/lib/scout/misc/format.rb +2 -2
- data/lib/scout/misc/system.rb +5 -0
- data/lib/scout/open/final.rb +16 -1
- data/lib/scout/open/remote.rb +0 -1
- data/lib/scout/open/stream.rb +30 -5
- data/lib/scout/open/util.rb +32 -0
- data/lib/scout/path/digest.rb +12 -2
- data/lib/scout/path/find.rb +19 -6
- data/lib/scout/path/util.rb +37 -1
- data/lib/scout/persist/open.rb +2 -0
- data/lib/scout/persist.rb +7 -1
- data/lib/scout/resource/path.rb +2 -2
- data/lib/scout/resource/util.rb +18 -4
- data/lib/scout/resource.rb +15 -1
- data/lib/scout/simple_opt/parse.rb +2 -0
- data/lib/scout/tmpfile.rb +1 -1
- data/scout-essentials.gemspec +19 -6
- data/test/scout/misc/test_hook.rb +2 -2
- data/test/scout/open/test_stream.rb +43 -15
- data/test/scout/path/test_find.rb +1 -1
- data/test/scout/path/test_util.rb +11 -0
- data/test/scout/test_path.rb +4 -4
- data/test/scout/test_persist.rb +10 -1
- metadata +31 -5
- data/README.rdoc +0 -18
data/doc/SimpleOPT.md
ADDED
|
@@ -0,0 +1,236 @@
|
|
|
1
|
+
# SimpleOPT (SOPT)
|
|
2
|
+
|
|
3
|
+
SimpleOPT (module SOPT) is a lightweight command-line option definition and parsing helper. It provides:
|
|
4
|
+
|
|
5
|
+
- a small DSL to declare command options (long names, optional single/multi-letter shortcuts),
|
|
6
|
+
- automatic generation of usage/documentation text,
|
|
7
|
+
- parsing of ARGV-style option lists (boolean flags and string-valued options),
|
|
8
|
+
- helpers to build option docs from compact strings or heredocs,
|
|
9
|
+
- storage of option metadata (types, descriptions, defaults, shortcuts),
|
|
10
|
+
- convenience functions to parse/require options in scripts.
|
|
11
|
+
|
|
12
|
+
SOPT is split across several files:
|
|
13
|
+
- accessor — stores option metadata and simple mutators
|
|
14
|
+
- parse — parsing option-definition strings and registering options
|
|
15
|
+
- doc — builds usage/documentation output
|
|
16
|
+
- get — consumes ARGV-like arrays to produce parsed options
|
|
17
|
+
- setup — parse a help string (heredoc) and auto-consume ARGV
|
|
18
|
+
|
|
19
|
+
Important storage/accessors (module-level):
|
|
20
|
+
- SOPT.inputs — ordered list of long option names
|
|
21
|
+
- SOPT.shortcuts — map from shortcut string -> long option name
|
|
22
|
+
- SOPT.input_shortcuts[long] — chosen shortcut for a long name
|
|
23
|
+
- SOPT.input_types[long] — :string or :boolean
|
|
24
|
+
- SOPT.input_descriptions[long] — description string
|
|
25
|
+
- SOPT.input_defaults[long] — default values (used in docs)
|
|
26
|
+
- SOPT.GOT_OPTIONS — IndiferentHash accumulating consumed options across consumes
|
|
27
|
+
- SOPT.all (internal map for future use)
|
|
28
|
+
|
|
29
|
+
Utility accessors: SOPT.reset, SOPT.delete_inputs.
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Declaring options
|
|
34
|
+
|
|
35
|
+
SOPT supports two main declaration styles:
|
|
36
|
+
|
|
37
|
+
1) Programmatic registration via SOPT.parse or SOPT.register
|
|
38
|
+
- parse accepts a compact definition string (see below) and registers options.
|
|
39
|
+
- register(short, long, asterisk, description) registers one option:
|
|
40
|
+
- `short` may be nil or a single/multi-letter string (SOPT will try to pick a unique short if you pass nil or true)
|
|
41
|
+
- `long` is the long option name (string)
|
|
42
|
+
- `asterisk` truthy value => treat the option as a string-valued option (type :string). falsy => boolean.
|
|
43
|
+
- `description` is the human-readable description.
|
|
44
|
+
|
|
45
|
+
2) Declaration via SOPT.setup with a help/heredoc string
|
|
46
|
+
- setup parses a help text in the common manpage-ish format: optional summary line, optional `$` synopsys line, description paragraphs, then option lines starting with a dash (one or more).
|
|
47
|
+
- Example format (see tests):
|
|
48
|
+
```
|
|
49
|
+
Test application
|
|
50
|
+
|
|
51
|
+
$ test cmd -arg 1
|
|
52
|
+
|
|
53
|
+
It does some imaginary stuff
|
|
54
|
+
|
|
55
|
+
-a--arg* Argument
|
|
56
|
+
-a2--arg2* Argument
|
|
57
|
+
```
|
|
58
|
+
- Lines with `-s--long* Description` register options; `*` after `--long` marks a string-valued option.
|
|
59
|
+
|
|
60
|
+
parse() supports two separators for entries:
|
|
61
|
+
- newline-separated entries, or colon-separated entries (when string has no newlines).
|
|
62
|
+
|
|
63
|
+
parse() expects each entry to contain short and long names in the pattern `-(short)--(long)(*)?` and optional trailing description.
|
|
64
|
+
|
|
65
|
+
Examples:
|
|
66
|
+
- SOPT.parse("-f--first* first arg:-f--fun") registers two options with long names "first" (string) and "fun".
|
|
67
|
+
- If you pass '*' in the name pattern, the option is treated as a string-valued input (input_types[long] = :string). Otherwise it is boolean.
|
|
68
|
+
|
|
69
|
+
SOPT.register handles choosing a unique shortcut:
|
|
70
|
+
- SOPT.fix_shortcut picks a non-conflicting short string (initially the first char, then adds additional characters if needed, skipping punctuation characters).
|
|
71
|
+
- If fix_shortcut cannot find a unique short, it may return nil — the option will be registered with no shortcut.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
## Parsing command-line arguments
|
|
76
|
+
|
|
77
|
+
Primary function: SOPT.consume(args = ARGV)
|
|
78
|
+
|
|
79
|
+
Behavior:
|
|
80
|
+
- Scans through an array of tokens (defaults to ARGV) and removes recognized option tokens from the args array.
|
|
81
|
+
- Recognizes:
|
|
82
|
+
- `--key=value` and `-k=value`
|
|
83
|
+
- `--key value` (if option type is :string) or `-k value`
|
|
84
|
+
- boolean flags: `--flag` sets true, `--flag=false` or `--flag=false` will be interpreted as false.
|
|
85
|
+
- When parsing, it finds which long option corresponds to the given token:
|
|
86
|
+
- token key string resolved either directly in SOPT.inputs or via SOPT.shortcuts lookup.
|
|
87
|
+
- if the token is not a registered option it is skipped (left in args).
|
|
88
|
+
- For :string-typed options, the parser will consume the next token as the value if no `=` was provided.
|
|
89
|
+
- For boolean options: if a token immediately following the flag is one of F/false/FALSE/no the parser will warn and treat that token as the value; otherwise presence sets true.
|
|
90
|
+
- After parsing, returned options are normalized:
|
|
91
|
+
- IndiferentHash.setup is run on the options hash and keys are converted to symbols via keys_to_sym!
|
|
92
|
+
- The parsed options are merged into SOPT.GOT_OPTIONS (cumulative across calls).
|
|
93
|
+
- SOPT.consume returns the parsed options hash (IndiferentHash).
|
|
94
|
+
|
|
95
|
+
Example:
|
|
96
|
+
```ruby
|
|
97
|
+
SOPT.parse("-f--first* first arg:-f--fun")
|
|
98
|
+
args = "-f myfile --fun".split(" ")
|
|
99
|
+
opts = SOPT.consume(args)
|
|
100
|
+
# opts[:first] == "myfile"
|
|
101
|
+
# opts[:fun] == true
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Utility:
|
|
105
|
+
- SOPT.get(opt_str) — convenience: parse(opt_str) then consume(ARGV) (useful inline to define and parse immediately).
|
|
106
|
+
- SOPT.require(options, *parameters) — raise ParameterException if one of the listed parameter names is not present in options (useful to assert required options).
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## Documentation / usage generation
|
|
111
|
+
|
|
112
|
+
SOPT can generate usage/help text:
|
|
113
|
+
|
|
114
|
+
- SOPT.input_format(name, type, default, shortcut)
|
|
115
|
+
- builds a colored option usage fragment, e.g. "-n,--name=<type> (default: ...)"
|
|
116
|
+
- type values used: :string, :boolean (boolean prints [=false] in usage), :tsv/:text/:array treated specially.
|
|
117
|
+
|
|
118
|
+
- SOPT.input_doc(inputs, input_types, input_descriptions, input_defaults, input_shortcuts)
|
|
119
|
+
- Builds a formatted options block for the help text for a list of inputs.
|
|
120
|
+
|
|
121
|
+
- SOPT.input_array_doc(input_array)
|
|
122
|
+
- Accepts an array of arrays: [name, type, description, default, options]
|
|
123
|
+
- options may be a shortcut or an options hash (with :shortcut or other info).
|
|
124
|
+
- Produces formatted help entries.
|
|
125
|
+
|
|
126
|
+
- SOPT.doc
|
|
127
|
+
- Produces a full manpage-style documentation string containing SYNOPSYS, DESCRIPTION and OPTIONS using the stored SOPT.* metadata (command, summary, synopsys, description, inputs, types, defaults).
|
|
128
|
+
- SOPT.usage prints doc text and exits.
|
|
129
|
+
|
|
130
|
+
- SOPT.setup(str)
|
|
131
|
+
- Conveniently builds the command doc from a help heredoc, registers options, and calls SOPT.consume to parse current ARGV.
|
|
132
|
+
|
|
133
|
+
Colors and layout use the framework's Log and Misc helpers (Log.color, Misc.format_definition_list, etc.). The doc generation includes default values in option usage.
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## Metadata & helpers
|
|
138
|
+
|
|
139
|
+
- SOPT.inputs — list of long option names (strings).
|
|
140
|
+
- SOPT.shortcuts — map of chosen shortcut => long option (strings).
|
|
141
|
+
- SOPT.input_shortcuts[long] — the chosen shortcut for the given long option.
|
|
142
|
+
- SOPT.input_types[long] — :string or :boolean
|
|
143
|
+
- SOPT.input_descriptions[long] — documentation string for the option
|
|
144
|
+
- SOPT.input_defaults[long] — documented default value (not used by parser automatically)
|
|
145
|
+
- SOPT.GOT_OPTIONS — IndiferentHash accumulating parsed options between calls
|
|
146
|
+
|
|
147
|
+
Mutators / maintenance helpers:
|
|
148
|
+
- SOPT.reset — clears internal maps (@shortcuts and @all).
|
|
149
|
+
- SOPT.delete_inputs(list_of_inputs) — remove listed inputs and related metadata from SOPT registry.
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## Edge cases and notes
|
|
154
|
+
|
|
155
|
+
- Shortcut selection:
|
|
156
|
+
- If you pass `nil` or request automatic shortcuts, SOPT.fix_shortcut attempts to choose a unique shortcut starting from the first char and adding letters if collisions exist. It may return multi-letter shortcuts.
|
|
157
|
+
- fix_shortcut skips punctuation chars (., -, _) when building multi-letter shortcuts.
|
|
158
|
+
- If a unique shortcut cannot be chosen (rare), no shortcut is registered for the option.
|
|
159
|
+
|
|
160
|
+
- Option value parsing:
|
|
161
|
+
- `*` in the option definition (parse/register) means the option is string-valued; otherwise it is boolean.
|
|
162
|
+
- For booleans, presence sets true; to specify false on the command line either use `--flag=false` or provide `false` (or `F`/`no`) token after the flag — the parser will accept these but warns that `--flag=[true|false]` is preferred.
|
|
163
|
+
- If a string option is given without an explicit `=` and the next token is missing, the value will become `nil` (the parser consumes next token if present).
|
|
164
|
+
|
|
165
|
+
- Default handling:
|
|
166
|
+
- input_defaults are only used for documentation output. The parser does not automatically apply defaults to parsed options; callers should fill in defaults after parsing if desired.
|
|
167
|
+
|
|
168
|
+
- Normalization:
|
|
169
|
+
- Parsed options are converted to an IndiferentHash and keys are converted to symbols (`keys_to_sym!`), so consumers can access with either symbol or string keys (but tests expect symbol keys after consume).
|
|
170
|
+
|
|
171
|
+
- GOT_OPTIONS:
|
|
172
|
+
- SOPT.GOT_OPTIONS accumulates parsed options across multiple consumes; useful for scripts that call consume more than once or want a global snapshot.
|
|
173
|
+
|
|
174
|
+
- Documentation parsing (setup):
|
|
175
|
+
- SOPT.setup expects a particular structure: optional summary line, optional `$` synopsys line, description paragraphs, followed by option lines beginning with `-`.
|
|
176
|
+
- It calls SOPT.parse on the options block and SOPT.consume to parse current ARGV.
|
|
177
|
+
|
|
178
|
+
- Error handling:
|
|
179
|
+
- SOPT.require raises ParameterException (from the surrounding framework) if a required option is missing.
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
## Examples
|
|
184
|
+
|
|
185
|
+
Define options and parse ARGV:
|
|
186
|
+
|
|
187
|
+
```ruby
|
|
188
|
+
# declare from compact string and parse ARGV
|
|
189
|
+
SOPT.parse("-f--first* first arg:-s--silent Silent flag")
|
|
190
|
+
opts = SOPT.consume(ARGV)
|
|
191
|
+
# opts[:first] => "value" (if provided)
|
|
192
|
+
# opts[:silent] => true/false
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Define options from a help heredoc and auto-consume:
|
|
196
|
+
|
|
197
|
+
```ruby
|
|
198
|
+
SOPT.setup <<-EOF
|
|
199
|
+
My command summary
|
|
200
|
+
|
|
201
|
+
$ mycmd [options] args
|
|
202
|
+
|
|
203
|
+
This does interesting work
|
|
204
|
+
|
|
205
|
+
-f--first* First input file
|
|
206
|
+
-s--silent Run quietly
|
|
207
|
+
EOF
|
|
208
|
+
|
|
209
|
+
# SOPT.setup registers options and consumes ARGV
|
|
210
|
+
# Use SOPT.GOT_OPTIONS or result of consume to access parsed options
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
Quick inline parse+consume:
|
|
214
|
+
|
|
215
|
+
```ruby
|
|
216
|
+
SOPT.get("-f--first* first arg:-s--silent")
|
|
217
|
+
# Equivalent to SOPT.parse(...) followed by SOPT.consume(ARGV)
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
Require an option:
|
|
221
|
+
|
|
222
|
+
```ruby
|
|
223
|
+
opts = SOPT.consume(ARGV)
|
|
224
|
+
SOPT.require(opts, :first) # raises ParameterException if :first missing
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Produce usage:
|
|
228
|
+
|
|
229
|
+
```ruby
|
|
230
|
+
puts SOPT.doc # assemble doc string from registered inputs and descriptions
|
|
231
|
+
SOPT.usage # prints doc and exits
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
This covers the common usage patterns and API surface of SimpleOPT. Use SOPT.parse/register to declare options programmatically for small utilities; use SOPT.setup to derive declarations from a help/heredoc and auto-parse ARGV for typical script workflows.
|
data/doc/TmpFile.md
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
# TmpFile
|
|
2
|
+
|
|
3
|
+
TmpFile provides small helpers to create and manage temporary files and directories used throughout the framework. It offers safe temporary-path generation, scoped helpers that create and remove temporary files/dirs automatically, and a persistence-path helper (tmp_for_file) used by caching/persistence code to build stable cache filenames.
|
|
4
|
+
|
|
5
|
+
Files: lib/scout/tmpfile.rb
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Key constants & configuration
|
|
10
|
+
|
|
11
|
+
- TmpFile.MAX_FILE_LENGTH = 150 — max length used by tmp_for_file before truncating and appending a digest.
|
|
12
|
+
- TmpFile.tmpdir — base temporary directory used by tmp utilities (defaults to user tmp under $HOME: `~/tmp/scout/tmpfiles`).
|
|
13
|
+
- You can set: `TmpFile.tmpdir = "/some/dir"`.
|
|
14
|
+
|
|
15
|
+
Helpers:
|
|
16
|
+
- TmpFile.user_tmp(subdir = nil) — returns user-scoped base tmp dir (under $HOME/tmp/scout). If `subdir` provided it is appended.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Filename helpers
|
|
21
|
+
|
|
22
|
+
- TmpFile.random_name(prefix = 'tmp-', max = 1_000_000_000)
|
|
23
|
+
- Return a random name with the given prefix and a random integer (0..max).
|
|
24
|
+
|
|
25
|
+
- TmpFile.tmp_file(prefix = 'tmp-', max = 1_000_000_000, dir = nil)
|
|
26
|
+
- Returns a path inside `dir` (defaults to `TmpFile.tmpdir`) composed of prefix + random number.
|
|
27
|
+
- If `dir` is a Path it will be `.find`ed.
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## Scoped helpers
|
|
32
|
+
|
|
33
|
+
These helpers create temporary files/directories, yield them to the caller, and delete them afterward (by default).
|
|
34
|
+
|
|
35
|
+
- TmpFile.with_file(content = nil, erase = true, options = {}) { |tmpfile| ... }
|
|
36
|
+
- Create a temporary file path and optionally pre-populate it with `content`.
|
|
37
|
+
- Parameters:
|
|
38
|
+
- `content`:
|
|
39
|
+
- If String: write content into file.
|
|
40
|
+
- If IO/StringIO: read its contents and write into tmp file.
|
|
41
|
+
- If nil: tmp file is created empty.
|
|
42
|
+
- If `content` is a Hash, it is treated as options (content=nil).
|
|
43
|
+
- `erase` (default true): remove the tmp file after the block completes.
|
|
44
|
+
- `options` (Hash):
|
|
45
|
+
- `:prefix` — filename prefix (default `'tmp-'`).
|
|
46
|
+
- `:max` — random suffix max integer.
|
|
47
|
+
- `:tmpdir` — directory to write the tmp file into.
|
|
48
|
+
- `:extension` — append `.extension` to tmp file name.
|
|
49
|
+
- Behavior:
|
|
50
|
+
- Ensures tmpdir exists (Open.mkdir).
|
|
51
|
+
- Handles IO content safely by reading readpartial until EOF.
|
|
52
|
+
- Yields the tmp file path (string) to the block.
|
|
53
|
+
- After the block returns, removes the tmp file when `erase` is true and file exists.
|
|
54
|
+
- Examples:
|
|
55
|
+
```ruby
|
|
56
|
+
TmpFile.with_file("Hello") do |file|
|
|
57
|
+
puts File.read(file) # => "Hello"
|
|
58
|
+
end
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
- TmpFile.with_dir(erase = true, options = {}) { |tmpdir| ... }
|
|
62
|
+
- Create a temporary directory (using tmp_file for a unique name), yield its path, and remove it after block if `erase` true.
|
|
63
|
+
- `options[:prefix]` may change directory name prefix.
|
|
64
|
+
- Example:
|
|
65
|
+
```ruby
|
|
66
|
+
TmpFile.with_dir do |dir|
|
|
67
|
+
# dir is a path to a temporary directory
|
|
68
|
+
end
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
- TmpFile.in_dir(*args) { |dir| ... }
|
|
72
|
+
- Convenience that creates a temporary directory and executes the block with the current working directory changed to that directory (uses `Misc.in_dir` internally).
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Persistence path helper
|
|
77
|
+
|
|
78
|
+
- TmpFile.tmp_for_file(file, tmp_options = {}, other_options = {})
|
|
79
|
+
- Generates a stable temporary/persistent filename for a logical file name plus options. Used by persistence/caching logic to build consistent cache files per logical input and options.
|
|
80
|
+
- Returns a Path (string extended with Path) under the chosen persistence directory.
|
|
81
|
+
- Parameters:
|
|
82
|
+
- `file` — logical filename or Path used to build the identifier.
|
|
83
|
+
- `tmp_options` may include:
|
|
84
|
+
- `:file` — return value override (internal)
|
|
85
|
+
- `:prefix` — prefix for the identifier (default: based on file)
|
|
86
|
+
- `:key` — optional key appended in identifier (`[...]`).
|
|
87
|
+
- `:dir` — base directory for persistence (defaults to `TmpFile.tmpdir`).
|
|
88
|
+
- `other_options` — additional options whose digest will be appended to the filename (used to make identifier unique for variations like filters).
|
|
89
|
+
- Special handling:
|
|
90
|
+
- Replaces path separators with `SLASH_REPLACE` (character `·`) to make a single filename.
|
|
91
|
+
- Truncates long filenames (over MAX_FILE_LENGTH) and appends a short digest to avoid filesystem limits.
|
|
92
|
+
- Appends a digest of `other_options` (unless empty) to ensure uniqueness when options differ.
|
|
93
|
+
- Use cases:
|
|
94
|
+
- Build cache file path for a content produced from input + parameters.
|
|
95
|
+
- Example (simplified):
|
|
96
|
+
```ruby
|
|
97
|
+
p = TmpFile.tmp_for_file("data.tsv", dir: Path.setup("var/cache"))
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## Behavior / edge cases
|
|
103
|
+
|
|
104
|
+
- `with_file` and `with_dir` remove created resources after the block only if `erase` true and the file/dir exists.
|
|
105
|
+
- `with_file` supports passing options as the first argument (if `content` is a Hash).
|
|
106
|
+
- `with_file` writes IO content using `readpartial` in chunks (handles large IOs without loading entire contents into memory).
|
|
107
|
+
- `tmp_for_file` uses a safe character `SLASH_REPLACE` (`'·'`) for replaced `/` characters — results in single-file names representing nested logical paths.
|
|
108
|
+
- `tmp_for_file` truncates overly long identifiers and appends a digest of the remainder to keep the filename length reasonable.
|
|
109
|
+
- The helper returns a Path-like value when `persistence_dir` is a Path.
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Examples (from tests)
|
|
114
|
+
|
|
115
|
+
- Create a temporary file with content:
|
|
116
|
+
```ruby
|
|
117
|
+
TmpFile.with_file("Hello World!") do |file|
|
|
118
|
+
assert_equal "Hello World!", File.read(file)
|
|
119
|
+
end
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
- Create a temporary file from an IO and consume into another temporary:
|
|
123
|
+
```ruby
|
|
124
|
+
TmpFile.with_file("Hello") do |file1|
|
|
125
|
+
Open.open(file1) do |io|
|
|
126
|
+
TmpFile.with_file(io) do |file2|
|
|
127
|
+
assert_equal "Hello", File.read(file2)
|
|
128
|
+
end
|
|
129
|
+
end
|
|
130
|
+
end
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
- Temporary directory and change into it:
|
|
134
|
+
```ruby
|
|
135
|
+
TmpFile.in_dir do |dir|
|
|
136
|
+
# current working directory is dir inside the block
|
|
137
|
+
end
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
- Build a persistent cache path for a logical filename + options:
|
|
141
|
+
```ruby
|
|
142
|
+
cache_path = TmpFile.tmp_for_file("input.tsv", dir: Path.setup("var/cache"))
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## Implementation notes
|
|
148
|
+
|
|
149
|
+
- TmpFile uses `Open` utilities to create directories and write files.
|
|
150
|
+
- `tmp_file` returns plain string paths; callers often wrap them in Path.setup when needed.
|
|
151
|
+
- Filenames are sanitized (spaces replaced with `_`) and slashes replaced with `·` for single-file representation of nested paths in `tmp_for_file`.
|
|
152
|
+
- The module is intentionally minimal but used pervasively by other framework components like Persist and Resource to generate stable temporary / cache paths.
|
|
153
|
+
|
|
154
|
+
Use TmpFile helpers when you need temporary files/dirs or stable cache filenames with predictable cleanup behavior.
|
|
@@ -4,6 +4,10 @@ module Annotation
|
|
|
4
4
|
@annotation_types ||= []
|
|
5
5
|
end
|
|
6
6
|
|
|
7
|
+
def base_type
|
|
8
|
+
annotation_types.last
|
|
9
|
+
end
|
|
10
|
+
|
|
7
11
|
def annotation_hash
|
|
8
12
|
attr_hash = {}
|
|
9
13
|
@annotations.each do |name|
|
|
@@ -56,6 +60,10 @@ module Annotation
|
|
|
56
60
|
new.remove_instance_variable(:@container)
|
|
57
61
|
end
|
|
58
62
|
|
|
63
|
+
if new.instance_variables.include?(:@container_index)
|
|
64
|
+
new.remove_instance_variable(:@container_index)
|
|
65
|
+
end
|
|
66
|
+
|
|
59
67
|
new
|
|
60
68
|
end
|
|
61
69
|
|
data/lib/scout/cmd.rb
CHANGED
|
@@ -35,7 +35,7 @@ module CMD
|
|
|
35
35
|
if test
|
|
36
36
|
CMD.cmd(test + " ")
|
|
37
37
|
else
|
|
38
|
-
CMD.cmd("#{cmd}
|
|
38
|
+
CMD.cmd("bash -c 'command -v #{cmd}'")
|
|
39
39
|
end
|
|
40
40
|
rescue
|
|
41
41
|
if claim
|
|
@@ -111,12 +111,12 @@ module CMD
|
|
|
111
111
|
when value.nil? || FalseClass === value
|
|
112
112
|
next
|
|
113
113
|
when TrueClass === value
|
|
114
|
-
string
|
|
114
|
+
string += "#{option} "
|
|
115
115
|
else
|
|
116
116
|
if option.to_s.chars.to_a.last == "="
|
|
117
|
-
string
|
|
117
|
+
string += "#{option}'#{value}' "
|
|
118
118
|
else
|
|
119
|
-
string
|
|
119
|
+
string += "#{option} '#{value}' "
|
|
120
120
|
end
|
|
121
121
|
end
|
|
122
122
|
end
|
|
@@ -130,6 +130,7 @@ module CMD
|
|
|
130
130
|
options = IndiferentHash.add_defaults options, :stderr => Log::DEBUG
|
|
131
131
|
in_content = options.delete(:in)
|
|
132
132
|
stderr = options.delete(:stderr)
|
|
133
|
+
sudo = options.delete(:sudo)
|
|
133
134
|
post = options.delete(:post)
|
|
134
135
|
pipe = options.delete(:pipe)
|
|
135
136
|
log = options.delete(:log)
|
|
@@ -172,9 +173,13 @@ module CMD
|
|
|
172
173
|
|
|
173
174
|
cmd_options = process_cmd_options options
|
|
174
175
|
if cmd =~ /'\{opt\}'/
|
|
175
|
-
cmd.sub
|
|
176
|
+
cmd = cmd.sub('\'{opt}\'', cmd_options)
|
|
176
177
|
else
|
|
177
|
-
cmd
|
|
178
|
+
cmd += " " + cmd_options
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
if sudo
|
|
182
|
+
cmd = "sudo " + cmd
|
|
178
183
|
end
|
|
179
184
|
|
|
180
185
|
in_content = StringIO.new in_content if String === in_content
|
|
@@ -247,9 +252,8 @@ module CMD
|
|
|
247
252
|
|
|
248
253
|
sout
|
|
249
254
|
else
|
|
250
|
-
|
|
255
|
+
err = ""
|
|
251
256
|
if bar
|
|
252
|
-
err = ""
|
|
253
257
|
err_thread = Thread.new do
|
|
254
258
|
while not serr.eof?
|
|
255
259
|
line = serr.gets
|
|
@@ -259,26 +263,28 @@ module CMD
|
|
|
259
263
|
serr.close
|
|
260
264
|
end
|
|
261
265
|
elsif log and Integer === stderr
|
|
262
|
-
err = ""
|
|
263
266
|
err_thread = Thread.new do
|
|
264
267
|
while not serr.eof?
|
|
265
|
-
err
|
|
268
|
+
err += serr.gets
|
|
266
269
|
end
|
|
267
270
|
serr.close
|
|
268
271
|
end
|
|
269
272
|
else
|
|
270
273
|
Open.consume_stream(serr, true)
|
|
271
274
|
err_thread = nil
|
|
272
|
-
err = ""
|
|
273
275
|
end
|
|
274
276
|
|
|
275
277
|
ConcurrentStream.setup sout, :pids => pids, :threads => [in_thread, err_thread].compact, :autojoin => autojoin, :no_fail => no_fail
|
|
276
278
|
|
|
277
279
|
begin
|
|
278
280
|
out = StringIO.new sout.read
|
|
281
|
+
status = wait_thr.value
|
|
282
|
+
|
|
283
|
+
sout.join
|
|
279
284
|
sout.close unless sout.closed?
|
|
285
|
+
sout.annotate(out)
|
|
280
286
|
|
|
281
|
-
|
|
287
|
+
out.exit_status = status.exitstatus
|
|
282
288
|
if status && ! status.success? && ! no_fail
|
|
283
289
|
if !err.empty?
|
|
284
290
|
raise ProcessFailed.new pid, "#{cmd} failed with error status #{status.exitstatus}.\n#{err}"
|
|
@@ -288,6 +294,7 @@ module CMD
|
|
|
288
294
|
else
|
|
289
295
|
Log.log err, stderr if Integer === stderr and log
|
|
290
296
|
end
|
|
297
|
+
out.std_err = err if save_stderr
|
|
291
298
|
out
|
|
292
299
|
ensure
|
|
293
300
|
post.call if post
|
|
@@ -9,7 +9,7 @@ module AbortedStream
|
|
|
9
9
|
end
|
|
10
10
|
|
|
11
11
|
module ConcurrentStream
|
|
12
|
-
attr_accessor :threads, :pids, :callback, :abort_callback, :filename, :joined, :aborted, :autojoin, :lock, :no_fail, :pair, :thread, :stream_exception, :log, :std_err, :next
|
|
12
|
+
attr_accessor :threads, :pids, :callback, :abort_callback, :filename, :joined, :aborted, :autojoin, :lock, :no_fail, :pair, :thread, :stream_exception, :log, :std_err, :next, :exit_status
|
|
13
13
|
|
|
14
14
|
def self.setup(stream, options = {}, &block)
|
|
15
15
|
threads, pids, callback, abort_callback, filename, autojoin, lock, no_fail, pair, next_stream = IndiferentHash.process_options options, :threads, :pids, :callback, :abort_callback, :filename, :autojoin, :lock, :no_fail, :pair, :next
|
|
@@ -117,6 +117,7 @@ module ConcurrentStream
|
|
|
117
117
|
@pids.each do |pid|
|
|
118
118
|
begin
|
|
119
119
|
Process.waitpid(pid, Process::WUNTRACED)
|
|
120
|
+
self.exit_status = $?.exitstatus
|
|
120
121
|
stream_raise_exception ConcurrentStreamProcessFailed.new(pid, "Error in waitpid", self) unless $?.success? or no_fail
|
|
121
122
|
rescue Errno::ECHILD
|
|
122
123
|
end
|
|
@@ -250,6 +251,7 @@ module ConcurrentStream
|
|
|
250
251
|
end
|
|
251
252
|
close if done
|
|
252
253
|
end
|
|
254
|
+
join if autojoin && (closed? || @stream_exception)
|
|
253
255
|
end
|
|
254
256
|
end
|
|
255
257
|
|
data/lib/scout/config.rb
CHANGED
|
@@ -116,8 +116,8 @@ module Scout::Config
|
|
|
116
116
|
|
|
117
117
|
File.expand_path(file)
|
|
118
118
|
|
|
119
|
-
tokens << ("file:"
|
|
120
|
-
tokens << ("line:"
|
|
119
|
+
tokens << ("file:" + file)
|
|
120
|
+
tokens << ("line:" + file << ":" << line.sub(/:in \`.*/,''))
|
|
121
121
|
|
|
122
122
|
entries = CACHE[key.to_s]
|
|
123
123
|
priorities = {}
|
|
@@ -88,8 +88,8 @@ module IndiferentHash
|
|
|
88
88
|
def self.hash2string(hash)
|
|
89
89
|
hash.sort_by{|k,v| k.to_s}.collect{|k,v|
|
|
90
90
|
next unless %w(Symbol String Float Fixnum Integer Numeric TrueClass FalseClass Module Class Object).include? v.class.to_s
|
|
91
|
-
[ Symbol === k ? ":"
|
|
92
|
-
Symbol === v ? ":"
|
|
91
|
+
[ Symbol === k ? ":" + k.to_s : k.to_s.chomp,
|
|
92
|
+
Symbol === v ? ":" + v.to_s : v.to_s.chomp] * "="
|
|
93
93
|
}.compact * "#"
|
|
94
94
|
end
|
|
95
95
|
|
|
@@ -157,5 +157,21 @@ module IndiferentHash
|
|
|
157
157
|
|
|
158
158
|
super(*full_list)
|
|
159
159
|
end
|
|
160
|
+
|
|
161
|
+
def dig(*keys)
|
|
162
|
+
current = self
|
|
163
|
+
while keys.any?
|
|
164
|
+
first = keys.shift
|
|
165
|
+
current = current[first]
|
|
166
|
+
break if current.nil?
|
|
167
|
+
IndiferentHash.setup(current) if Hash === current
|
|
168
|
+
end
|
|
169
|
+
current
|
|
170
|
+
end
|
|
171
|
+
|
|
172
|
+
def self.dig(obj, *keys)
|
|
173
|
+
IndiferentHash.setup obj
|
|
174
|
+
obj.dig(*keys)
|
|
175
|
+
end
|
|
160
176
|
end
|
|
161
177
|
|
data/lib/scout/log/color.rb
CHANGED
|
@@ -141,6 +141,8 @@ module Log
|
|
|
141
141
|
SEVERITY_COLOR = [reset, cyan, green, magenta, blue, yellow, red] #.collect{|e| "\033[#{e}"}
|
|
142
142
|
CONCEPT_COLORS = IndiferentHash.setup({
|
|
143
143
|
:title => magenta,
|
|
144
|
+
:subtitle => yellow,
|
|
145
|
+
:body => blue,
|
|
144
146
|
:path => blue,
|
|
145
147
|
:input => cyan,
|
|
146
148
|
:value => green,
|
|
@@ -161,7 +163,7 @@ module Log
|
|
|
161
163
|
HIGHLIGHT = "\033[1m"
|
|
162
164
|
|
|
163
165
|
def self.uncolor(str)
|
|
164
|
-
""
|
|
166
|
+
"" + Term::ANSIColor.uncolor(str)
|
|
165
167
|
end
|
|
166
168
|
|
|
167
169
|
def self.reset_color
|
|
@@ -169,7 +171,7 @@ module Log
|
|
|
169
171
|
end
|
|
170
172
|
|
|
171
173
|
def self.color(color, str = nil, reset = false)
|
|
172
|
-
return str.dup
|
|
174
|
+
return str.to_s.dup if nocolor
|
|
173
175
|
|
|
174
176
|
if (color == :integer || color == :float) && Numeric === str
|
|
175
177
|
color = if str < 0
|
|
@@ -205,7 +207,7 @@ module Log
|
|
|
205
207
|
str = str.to_s unless str.nil?
|
|
206
208
|
return str if Symbol === color
|
|
207
209
|
color_str = reset ? Term::ANSIColor.reset.dup : ""
|
|
208
|
-
color_str
|
|
210
|
+
color_str += color if color
|
|
209
211
|
if str.nil?
|
|
210
212
|
color_str
|
|
211
213
|
else
|
|
@@ -14,15 +14,15 @@ module Log
|
|
|
14
14
|
when FalseClass
|
|
15
15
|
"false"
|
|
16
16
|
when Symbol
|
|
17
|
-
":"
|
|
17
|
+
":" + obj.to_s
|
|
18
18
|
when String
|
|
19
19
|
if obj.length > FP_MAX_STRING
|
|
20
20
|
digest = Digest::MD5.hexdigest(obj)
|
|
21
21
|
middle = "<...#{obj.length} - #{digest[0..4]}...>"
|
|
22
22
|
s = (FP_MAX_STRING - middle.length) / 2
|
|
23
|
-
"'"
|
|
23
|
+
"'" + obj.slice(0,s-1) + middle + obj.slice(-s, obj.length ) + "'"
|
|
24
24
|
else
|
|
25
|
-
"'"
|
|
25
|
+
"'" + obj + "'"
|
|
26
26
|
end
|
|
27
27
|
when ConcurrentStream
|
|
28
28
|
name = obj.inspect + " " + obj.object_id.to_s
|
|
@@ -34,22 +34,22 @@ module Log
|
|
|
34
34
|
"<File:" + obj.path + ">"
|
|
35
35
|
when Array
|
|
36
36
|
if (length = obj.length) > FP_MAX_ARRAY
|
|
37
|
-
"[#{length}--"
|
|
37
|
+
"[#{length}--" + (obj.values_at(0,1, length / 2, -2, -1).collect{|e| fingerprint(e)} * ",") + "]"
|
|
38
38
|
else
|
|
39
|
-
"["
|
|
39
|
+
"[" + (obj.collect{|e| fingerprint(e) } * ", ") + "]"
|
|
40
40
|
end
|
|
41
41
|
when Hash
|
|
42
42
|
if obj.length > FP_MAX_HASH
|
|
43
|
-
"H:{"
|
|
43
|
+
"H:{" + fingerprint(obj.keys) + ";" + fingerprint(obj.values) + "}"
|
|
44
44
|
else
|
|
45
45
|
new = "{"
|
|
46
46
|
obj.each do |k,v|
|
|
47
|
-
new
|
|
47
|
+
new += fingerprint(k) + '=>' + fingerprint(v) + ' '
|
|
48
48
|
end
|
|
49
49
|
if new.length > 1
|
|
50
50
|
new[-1] = "}"
|
|
51
51
|
else
|
|
52
|
-
new
|
|
52
|
+
new += '}'
|
|
53
53
|
end
|
|
54
54
|
new
|
|
55
55
|
end
|