rexe 0.8.1 → 0.9.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (6) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +6 -0
  3. data/README.md +425 -168
  4. data/exe/rexe +9 -9
  5. data/rexe.gemspec +16 -1
  6. metadata +10 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e722054c78d64b4423f008c6c4e24e9d47b16fa4e07450f28d5a835246cd9810
4
- data.tar.gz: 4624fe6eacd8e6052b78a9410c2ac7ad9a527ffbd0977225cf0b91de41f90184
3
+ metadata.gz: 2aee272d801d5f73d5d742857baa103d108b366d2c4b82a64595d662dc47a438
4
+ data.tar.gz: ef605e5e1b9918215602abf481a20d1d24847c8fa36fc93ff9f91bcaf826547b
5
5
  SHA512:
6
- metadata.gz: 80f48114e7ff0e47fa6551e30b42aa0bc7326984bb68b5fec588d4cb4d2b1f0c246bc710aa709e9214dbcb52c18e0c9cae39bf89fec4f67c9a3e8fe44ea96621
7
- data.tar.gz: 64a01dcfa2daf21325f42203c4e74a090e6650aefb8bf6fec453f6c36b1a7f0e6ba38c05cd6a4f260cc4c4cb6ac7bac55ad3a6832ee00b927ea7a749d2ecc756
6
+ metadata.gz: 86ac1e3509c6ff78fc4037c25b441b62786f3cf473aa3409678ec1adcce0c1e23def881137a20972ea5bf2ab605364986e80c83b6d4bdab5c8845d203d3fce07
7
+ data.tar.gz: 1fd4c1d947117fa60ab722bf5a16aae4a378ee31d36d95503cfa15abe15e158d3350e7dbf010b86b22515eeb9492a0b9f72c0099cd8950fca0015438fae4c8ad
data/CHANGELOG.md CHANGED
@@ -1,6 +1,12 @@
1
1
  ## rexe -- Ruby Command Line Executor
2
2
 
3
3
 
4
+ ### v0.9.0
5
+
6
+ * Change -ms (single or separate string) mode to -ml (line) mode.
7
+ * Use article text for readme.
8
+
9
+
4
10
  ### v0.8.1
5
11
 
6
12
  * Fix and improve help text.
data/README.md CHANGED
@@ -1,33 +1,53 @@
1
- # Rexe
2
-
3
- A configurable Ruby command line executor and filter.
4
-
5
-
1
+ ---
2
+ title: The `rexe` Command Line Executor and Filter
3
+ date: 2019-02-15
4
+ ---
6
5
 
7
- ## Installation
6
+ I love the power of the command line, but not the awkwardness of shell scripting
7
+ languages. Sure, there's a lot that can be done with them, but it doesn't take
8
+ long before I get frustrated with their bluntness and verbosity.
8
9
 
9
- Installation can be performed by either installing the gem or copying the single executable file to your system and ensuring that it is executable:
10
+ Often, I solve this problem by writing a Ruby script instead. Ruby gives me fine
11
+ grained control in a "real" programming language with which I am comfortable.
12
+ However, when there are multiple OS commands to be called, then Ruby can be
13
+ awkward too.
10
14
 
11
- ```gem install rexe```
15
+ ### Using the Ruby Interpreter on the Command Line
12
16
 
13
- or
17
+ Sometimes a good solution is to combine Ruby and shell scripting on the same
18
+ command line. Here's an example, using an intermediate environment variable to
19
+ simplify the logic and save the data for use by future commands.
20
+ An excerpt of the output follows the code:
14
21
 
15
22
  ```
16
- curl https://raw.githubusercontent.com/keithrbennett/rexe/master/exe/rexe > rexe
17
- chmod +x rexe
23
+ ➜ ~  export EUR_RATES_JSON=`curl https://api.exchangeratesapi.io/latest`
24
+ ➜ ~  echo $EUR_RATES_JSON | ruby -r json -r awesome_print -e 'ap JSON.parse(STDIN.read)'
25
+ {
26
+ "rates" => {
27
+ "MXN" => 21.781,
28
+ ...
29
+ "DKK" => 7.462
30
+ },
31
+ "base" => "EUR",
32
+ "date" => "2019-02-22"
33
+ }
18
34
  ```
19
35
 
20
- ## Usage
36
+ However, the configuration setup (the `require`s) make the command long and tedious, discouraging this
37
+ approach.
21
38
 
22
- rexe is an _executor_, meaning it can execute Ruby without either standard input or output,
23
- but it is also a _filter_ in that it can implicitly consume standard input and emit standard output.
39
+ ### Rexe
24
40
 
25
- ### Help Text
41
+ Enter the `rexe` script. [^1]
42
+
43
+ `rexe` is at https://github.com/keithrbennett/rexe and can be installed with
44
+ `gem install rexe`. `rexe` provides several ways to simplify Ruby on the command
45
+ line, tipping the scale so that it is practical to do it more often.
26
46
 
27
- As a summary, here is the help text printed out by the application:
47
+ Here is `rexe`'s help text as of the time of this writing:
28
48
 
29
49
  ```
30
- rexe -- Ruby Command Line Filter/Executor -- v0.8.0 -- https://github.com/keithrbennett/rexe
50
+ rexe -- Ruby Command Line Executor/Filter -- v0.9.0 -- https://github.com/keithrbennett/rexe
31
51
 
32
52
  Executes Ruby code on the command line, optionally taking standard input and writing to standard output.
33
53
 
@@ -36,14 +56,14 @@ Options:
36
56
  -c --clear_options Clear all previous command line options specified up to now
37
57
  -h, --help Print help and exit
38
58
  -l, --load RUBY_FILE(S) Ruby file(s) to load, comma separated, or ! to clear
39
- -m, --mode MODE Mode with which to handle input (i.e. what `self` will be in the code):
40
- -ms for each line to be handled separately as a string
41
- -me for an enumerator of lines (least memory consumption for big data)
42
- -mb for 1 big string (all lines combined into single multiline string)
43
- -mn don't do special handling of input; self is not the input (default)
44
- -n', --[no-]noop Do not execute the code (useful with -v)
59
+ -m, --mode MODE Mode with which to handle input (i.e. what `self` will be in your code):
60
+ -ml line mode; each line is ingested as a separate string
61
+ -me enumerator mode
62
+ -mb big string mode; all lines combined into single multiline string
63
+ -mn (default) no input mode; no special handling of input; self is not input
64
+ -n, --[no-]noop Do not execute the code (useful with -v); see note (1) below
45
65
  -r, --require REQUIRES Gems and built-in libraries to require, comma separated, or ! to clear
46
- -v, --[no-]verbose verbose mode (logs to stderr); to disable, short options: -v n, -v false
66
+ -v, --[no-]verbose verbose mode (logs to stderr); see note (1) below
47
67
 
48
68
  If there is an .rexerc file in your home directory, it will be run as Ruby code
49
69
  before processing the input.
@@ -51,251 +71,488 @@ before processing the input.
51
71
  If there is a REXE_OPTIONS environment variable, its content will be prepended to the command line
52
72
  so that you can specify options implicitly (e.g. `export REXE_OPTIONS="-r awesome_print,yaml"`)
53
73
 
54
- For boolean verbose and noop options, the following are valid:
74
+ (1) For boolean 'verbose' and 'noop' options, the following are valid:
55
75
  -v no, -v yes, -v false, -v true, -v n, -v y, -v +, but not -v -
56
76
  ```
57
77
 
58
- ### Input Mode
78
+ For consistency with the `ruby` interpreter we called previously, `rexe` supports requires with the `-r` option, but as one tiny improvement it also allows grouping them together using commas:
59
79
 
60
- When it is used as a filter, the input is accessed differently in the source code
61
- depending on the mode that was specified (see example section below for examples).
62
- The mode letter is appended to `-m` on the command line;
63
- `n` (_no input_) mode is the default.
80
+ ```
81
+ vvvvvvvvvvvvvvvvvvvvv
82
+ ➜ ~  echo $EUR_RATES_JSON | rexe -r json,awesome_print 'ap JSON.parse(STDIN.read)'
83
+ ^^^^^^^^^^^^^^^^^^^^^
84
+ ```
64
85
 
65
- * `s` - _string_ mode - the source code is run once on each line of input, and `self` is each line of text
66
- * `e` - _enumerator_ mode - the code is run once on the enumerator of all lines; `self` is the enumerator, so you can call `map`, `to_a`, `select`, etc without explicitly specifying `self`.
67
- * `b` - _big string_ mode - all input is treated as one large (probably) multiline string with the newlines intact; `self` is this large string; this mode is required, for example, for parsing the text into a single object from JSON or YAML formatted data.
68
- * `n` - _no input_ mode - this instructs the program to proceed without looking for input
86
+ This command produces the same results as the previous `ruby` one.
69
87
 
88
+ ### Simplifying the Rexe Invocation with Configuration
70
89
 
71
- ### Requires
90
+ `rexe` provides two approaches to configuration:
72
91
 
73
- As with the Ruby interpreter itself, `require`s can be specified on the command line with the `-r` option. Multiple requires can be combined with commas between them.
92
+ * the `REXE_OPTIONS` environment variable
93
+ * loading Ruby files before executing the code using `-l`, or implicitly with `~/.rexerc`
74
94
 
95
+ These approaches enable removing configuration information from your `rexe` command,
96
+ making it shorter and simpler to read.
75
97
 
76
- ### Loading Ruby Files
77
98
 
78
- Other Ruby files can be loaded by `rexe`, to, for example, define classes and methods to use, set up resources, etc. They are specified with the `-l` option.
99
+ ### The REXE_OPTIONS Environment Variable
79
100
 
80
- If there is a file named `.rexerc` in the home directory, that will always be loaded without explicitly requesting it on the command line.
101
+ The `REXE_OPTIONS` environment variable can contain command line options that would otherwise
102
+ be specified on the `rexe` command line:
81
103
 
82
- ### Verbose Mode
104
+ ```
105
+ ➜ ~  export REXE_OPTIONS="-r json,awesome_print"
106
+ ➜ ~  echo $EUR_RATES_JSON | rexe 'ap JSON.parse(STDIN.read)'
107
+ ```
108
+
109
+ Like any environment variable, `REXE_OPTIONS` could also be set in your startup script, input on a command line using `export`, or in another script loaded with `source` or `.`.
110
+
111
+ ### Loading Files
83
112
 
84
- Verbose mode outputs information to standard error (stderr). This information can be redirected, for example to a file named `rexe.log`, by adding `2>& rexe.log` to the command line.
113
+ The environment variable approach works well for command line _options_, but what if we want to specify Ruby _code_ (e.g. methods) that can be used by multiple invocations of `rexe`?
85
114
 
86
- Here is an example of some text that might be output in verbose mode:
115
+ For this, `rexe` lets you _load_ Ruby files, using the `-l` option, or implicitly (without your specifying it) in the case of the `~/.rexerc` file. Here is an example of something you might include in such a file (this is an alternate approach to specifying `-r` in the `REXE_OPTIONS` environment variable):
87
116
 
88
117
  ```
89
- ➜ ~  rexe -r yaml -mn -v "%w(foo bar baz).to_yaml"
90
- rexe version 0.3.0 -- 2019-02-08 02:21:27 +0700
91
- Source Code: %w(foo bar baz).to_yaml
92
- Options: {:input_mode=>:no_input, :loads=>[], :requires=>["yaml"], :verbose=>true}
93
- Loading global config file /Users/kbennett/.rexerc
94
- ---
95
- - foo
96
- - bar
97
- - baz
98
- rexe time elapsed: 0.03874 seconds.
118
+ require 'json'
119
+ require 'yaml'
120
+ require 'awesome_print'
99
121
  ```
100
122
 
101
- To set verbose mode _off_ on the command line, the option parser accepts the long option `--no-verbose`, of course, but you could use instead `-v false` or `-v n`. If you should need to pass a value to `-v` to turn verbose mode _on_, then you can use `-v y` or `-v true`.
123
+ Requiring gems and modules for _all_ invocations of `rexe` will make your commands simpler and more concise, but will be a waste of execution time if they are not needed. You can inspect the execution times to see just how much time is being wasted. For example, we can find out that nokogiri takes about 0.7 seconds to load on my laptop by observing and comparing the execution times with and without the require (output has been abbreviated):
102
124
 
103
- ### The REXE_OPTIONS Environment Variable
125
+ ```
126
+ ➜ ~  rexe -v
127
+ rexe time elapsed: 0.094946 seconds.
104
128
 
105
- Very often you will want to call `rexe` several times with similar options. Instead of having to clutter the command line each time with these options, you can put them in an environment variable named `REXE_OPTIONS`, and they will be prepended automatically. Since they will be processed before the options on the command line, they are of lower precedence and can be overridden.
129
+ ➜ ~  rexe -v -r nokogiri
130
+ rexe time elapsed: 0.165996 seconds.
131
+ ```
106
132
 
133
+ ### Using Loaded Files in Your Commands
107
134
 
108
- ### The ~/.rexerc Configuation File
135
+ Here's something else you could include in such a load file:
109
136
 
110
- The `.rexerc` file in your home directory (if it exists) is loaded unconditionally. Here is a place to put things that you will _always_ want. You can include commonly needed requires, but be careful because the impact on startup time may be more than you want. You can test this using verbose mode, which outputs the execution time after completing.
137
+ ```
138
+ # Open YouTube to Wagner's "Ride of the Valkyries"
139
+ def valkyries
140
+ `open "http://www.youtube.com/watch?v=P73Z6291Pt8&t=0m28s"`
141
+ end
142
+ ```
111
143
 
144
+ Why would you want this? You might want to be able to go to another room until a long job completes, and be notified when it is done. The `valkyries` method will launch a browser window pointed to Richard Wagner's "Ride of the Valkyries" starting at a lively point in the music. (The `open` command is Mac specific and could be replaced with `start` on Windows, a browser command name, etc.) If you like this sort of thing, you could download public domain audio files and use a command like player like `afplay` on Mac OS, or `mpg123` or `ogg123` on Linux. This approach is lighter weight, requires no network access, and will not leave an open browser window for you to close.
112
145
 
113
- ### Directory-Specific Configuration Files
146
+ Here is an example of how you might use this, assuming the above configuration is loaded from your `~/.rexerc` file or
147
+ an explicitly loaded file:
114
148
 
115
- Although there is no built-in feature for directory-specific configuration files, this can be easily accomplished by inserting something like this in your global configuration file (~/.rexerc):
149
+ ```
150
+ ➜ ~  tar czf /tmp/my-whole-user-space.tar.gz ~ ; rexe valkyries
151
+ ```
152
+
153
+ You might be thinking that creating an alias or a minimal shell script for this open would be a simpler and more natural
154
+ approach, and I would agree with you. However, over time the number of these could become unmanageable, whereas using Ruby
155
+ you could build a pretty extensive and well organized library of functionality. Moreover, that functionality could be made available to _all_ your Ruby code (for example, by putting it in a gem), and not just command line one liners.
156
+
157
+ For example, you could have something like this in a configuration file:
116
158
 
117
159
  ```
118
- dir_specific_config_file = './rexe.rc'
119
- load dir_specific_config_file' if File.exist?(dir_specific_config_file)
160
+ def play(piece_code)
161
+ pieces = {
162
+ hallelujah: "https://www.youtube.com/watch?v=IUZEtVbJT5c&t=0m20s",
163
+ valkyries: "http://www.youtube.com/watch?v=P73Z6291Pt8&t=0m28s",
164
+ wm_tell: "https://www.youtube.com/watch?v=j3T8-aeOrbg"
165
+ # ... and many, many more
166
+ }
167
+ `open #{Shellwords.escape(pieces.fetch(piece_code))}`
168
+ end
120
169
  ```
121
-
122
- You should probably put this at the very end of this global configuration file so that the directory specific file will always be able to override global settings.
123
170
 
124
- Note that the directory specific filename used differs from the global config filename. This is a) so that there is no recursive loading of the file when in the home directory, and b) because it is more convenient for these files that they not be hidden.
171
+ ...which you could then call like this:
172
+
173
+ ```
174
+ ➜ ~  tar czf /tmp/my-whole-user-space.tar.gz ~ ; rexe 'play(:hallelujah)'
175
+ ```
176
+
177
+ (You need to quote the `play` call because otherwise the shell will process and remove the parentheses.
178
+ Alternatively you could escape the parentheses with backslashes.)
179
+
125
180
 
181
+ ### Clearing the Require and Load Lists
126
182
 
127
- ## Troubleshooting
183
+ There may be times when you have specified a load or require on the command line
184
+ or in the `REXE_OPTIONS` environment variable,
185
+ but you want to override it for a single invocation. Currently you cannot
186
+ unspecify a single resource, but you can unspecify _all_ the requires or loads
187
+ with the `-r!` and `-l!` command line options, respectively.
128
188
 
129
- One common problem relates to the shell's special handling of characters. Remember that the shell will process special characters, thereby changing your text before passing it on to the Ruby code. It is good to get in the habit of putting your source code in double quotes; and if the source code itself uses quotes, use `q{}` or `Q{}` instead. For example:
130
189
 
190
+ ### Clearing _All_ Options
191
+
192
+ You can also clear _all_ options specified up to a certain point in time with the _clear options_ option (`-c`).
193
+ This is especially useful if you have specified options in the `REXE_OPTIONS` environment variable,
194
+ and want to ignore all of them.
195
+
196
+
197
+ ### Verbose Mode
198
+
199
+ In addition to displaying the execution time, verbose mode will display the version, date/time of execution, source code
200
+ to be evaluated, options specified (by all approaches), and that the global file has been loaded (if it was found):
201
+
131
202
  ```
132
- ➜  rexe -mn "puts %Q{The time is now #{Time.now}}"
133
- The time is now 2019-02-04 18:49:31 +0700
203
+ ➜ ~ echo $EUR_RATES_JSON | rexe -v -rjson,awesome_print "ap JSON.parse(STDIN.read)"
204
+ rexe version 0.7.0 -- 2019-03-03 18:18:14 +0700
205
+ Source Code: ap JSON.parse(STDIN.read)
206
+ Options: {:input_mode=>:no_input, :loads=>[], :requires=>["json", "awesome_print"], :verbose=>true}
207
+ Loading global config file /Users/kbennett/.rexerc
208
+ ...
209
+ rexe time elapsed: 0.085913 seconds.
210
+ ```
211
+
212
+ This extra output is sent to standard error (_stderr_) instead of standard output
213
+ (_stdout_) so that it will not pollute the "real" data when stdout is piped to
214
+ another command.
215
+
216
+ If you would like to append this informational output to a file, you could do something like this:
217
+
134
218
  ```
219
+ ➜ ~  rexe ... -v 2>>rexe.log
220
+ ```
221
+
222
+ If verbose mode is enabled in configuration and you want to disable it, you can
223
+ do so by using any of the following: `--[no-]verbose`, `-v n`, or `-v false`.
135
224
 
136
- If you are troubleshooting the setup (i.e. the command line options, loaded files, and `REXE_OPTIONS` environment variable) using the verbose option, you may have the problem of the logging scrolling off the screen due to the length of your output. In this case you could easily fake or disable the output adding `; nil`, `; 'foo'`', etc. to the end of your expression. This way you don't have to mess with your code's logic.
225
+ ### Input Modes
137
226
 
138
- ## Examples
227
+ `rexe` tries to make it simple and convenient for you to handle standard input, and in different ways. Here is the help text relating to input modes:
139
228
 
140
229
  ```
141
- # Call reverse on listed file.
142
- # Need to specify the mode, since it defaults to "n" ("-mn"),
143
- # which treats every line separately.
144
- ➜ rexe git:(master) ✗  ls | head -2 | exe/rexe -ms "self + ' --> ' + reverse"
145
- CHANGELOG.md --> dm.GOLEGNAHC
146
- Gemfile --> elifmeG
230
+ -m, --mode MODE Mode with which to handle input (i.e. what `self` will be in your code):
231
+ -ml line mode; each line is ingested as a separate string
232
+ -me enumerator mode
233
+ -mb big string mode; all lines combined into single multiline string
234
+ -mn (default) no input mode; no special handling of input; self is not input
147
235
  ```
148
236
 
149
- ----
237
+ The first three are _filter_ modes; they make standard input available
238
+ to your code as `self`, and automatically output to standard output
239
+ the last value evaluated by your code.
240
+
241
+ The last (and default) is the _executor_ mode. It merely assists you in
242
+ executing the code you provide without any special implicit handling of standard input.
243
+
244
+
245
+ #### -ml "Line" Filter Mode
246
+
247
+ In this mode, your code would be called once per line of input,
248
+ and in each call, `self` would evaluate to the line of text:
150
249
 
151
250
  ```
152
- # Use input data to create a human friendly message:
153
- ➜ ~  uptime | rexe -ms "%Q{System has been up: #{split.first}.}"
154
- System has been up: 17:10.
251
+ ➜ ~  echo "hello\ngoodbye" | rexe -ms reverse
252
+ olleh
253
+ eybdoog
155
254
  ```
156
255
 
157
- ----
256
+ `reverse` is implicitly called on each line of standard input. `self`
257
+ is the input line in each call (we could also have used `self.reverse` but the `self` would have been redundant.).
258
+
259
+
260
+ #### -me "Enumerator" Filter Mode
261
+
262
+ In this mode, your code is called only once, and `self` is an enumerator
263
+ dispensing all lines of standard input. To be more precise, it is the enumerator returned by `STDIN.each_line`.
264
+
265
+ Dealing with input as an enumerator enables you to use the wealth of `Enumerable` methods such as `select`, `to_a`, `map`, etc.
266
+
267
+ Here is an example of using `-me` to add line numbers to the first 3
268
+ files in the directory listing:
158
269
 
159
270
  ```
160
- # Create a JSON array of a file listing.
161
- # Use the "-me" flag so that all input is treated as a single enumerator of lines.
162
- ➜ ~  ls | head -3 | rexe -me -r json "map(&:strip).to_a.to_json"
163
- ["AFP.conf","afpovertcp.cfg","afpovertcp.cfg~orig"]
271
+ ➜ ~  ls / | rexe -me "first(3).each_with_index { |ln,i| puts '%5d %s' % [i, ln] }; nil"
272
+
273
+ 0 AndroidStudioProjects
274
+ 1 Applications
275
+ 2 Desktop
164
276
  ```
165
277
 
166
- ----
278
+ Since `self` is an enumerable, we can call `first` and then `each_with_index`.
279
+
280
+
281
+ #### -mb "Big String" Filter Mode
282
+
283
+ In this mode, all standard input is combined into a single, (possibly)
284
+ large string, with newline characters joining the lines in the string.
285
+
286
+ A good example of when you would use this is when you parse JSON or YAML text; you need to pass the entire (probably) multiline string to the parse method.
287
+
288
+ An earlier example would be more simply specified using this mode, since `STDIN.read` could be replaced with `self`:
167
289
 
168
290
  ```
169
- # Create a "pretty" JSON array of a file listing:
170
- ➜ ~  ls | head -3 | rexe -me -r json "JSON.pretty_generate(map(&:strip).to_a)"
171
- [
172
- "AFP.conf",
173
- "afpovertcp.cfg",
174
- "afpovertcp.cfg~orig"
175
- ]
291
+ ➜ ~  echo $EUR_RATES_JSON | rexe -mb -r awesome_print,json 'ap JSON.parse(self)'
176
292
  ```
177
293
 
178
- ----
294
+ #### -mn "No Input" Executor Mode -- The Default
295
+
296
+ Examples up until this point have all used the default
297
+ `-mn` mode. This is the simplest use case, where `self`
298
+ does not evaluate to anything useful, and if you cared about standard
299
+ input, you would have to code it yourself (e.g. as we did earlier with `STDIN.read`).
300
+
301
+
302
+ #### Filter Input Mode Memory Considerations
303
+
304
+ If you may have more input than would fit in memory, you can do the following:
305
+
306
+ * use `-ml` (line) mode so you are fed only 1 line at a time
307
+ * use `-me` (enumerator) mode or use `-mn` (no input) mode with something like `STDIN.each_line`,
308
+ but make sure not to call any methods (e.g. `map`, `select`)
309
+ that will produce an array of all the input
310
+
311
+
312
+ ### Implementing Domain Specific Languages (DSL's)
313
+
314
+ Defining methods in your loaded files enables you to effectively define a [DSL](https://en.wikipedia.org/wiki/Domain-specific_language) for your command line use. You could use different load files for different projects, domains, or contexts, and define aliases or one line scripts to give them meaningful names. For example, if I wrote code to work with Ansible and put it in `~/projects/rexe-ansible.rb`, I could define an alias in my startup script:
179
315
 
180
316
  ```
181
- # Create a YAML array of a file listing:
182
- ➜ ~  ls | head -3 | rexe -me -r yaml "map(&:strip).to_a.to_yaml"
183
- ---
184
- - AFP.conf
185
- - afpovertcp.cfg
186
- - afpovertcp.cfg~orig
317
+ ➜ ~  alias rxans="rexe -l ~/projects/rexe-ansible.rb $*"
187
318
  ```
319
+ ...and then I would have an Ansible DSL available for me to use by calling `rxans`.
188
320
 
189
- ----
321
+ In addition, since you can also call `pry` on the context of any object, you
322
+ can provide a DSL in a [REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) (shell)
323
+ trivially easily. Just to illustrate, here's how you would open a REPL on the File class:
190
324
 
191
325
  ```
192
- # Use AwesomePrint to print a file listing.
193
- # (Rather than calling the `ap` method on the object to print,
194
- # call the `ai` method _on_ the object to print:
195
- ➜ ~  ls | head -3 | rexe -me -r awesome_print "map(&:chomp).ai"
196
- [
197
- [0] "AFP.conf",
198
- [1] "afpovertcp.cfg",
199
- [2] "afpovertcp.cfg~orig"
200
- ]
326
+ ➜ ~  ruby -r pry -e File.pry
327
+ # or
328
+ ➜ ~  rexe -r pry File.pry
201
329
  ```
202
330
 
203
- ----
331
+ `self` would evaluate to the `File` class, so you could call class methods implicitly using only their names:
204
332
 
205
333
  ```
206
- # Don't use input at all, so use "-mn" to tell rexe not to expect input.
207
- # -mn is the default, so it works with or without specifying that option:
208
- ➜ ~  rexe "puts %Q{The time is now #{Time.now}}"
209
- ➜ ~  rexe -mn "puts %Q{The time is now #{Time.now}}"
210
- The time is now 2019-02-04 17:20:03 +0700
334
+ ➜ rock_books git:(master) ✗  rexe -r pry File.pry
335
+
336
+ [6] pry(File)> size '/etc/passwd'
337
+ 6804
338
+ [7] pry(File)> directory? '.'
339
+ true
340
+ [8] pry(File)> file?('/etc/passwd')
341
+ true
211
342
  ```
212
343
 
213
- ----
344
+ This could be really handy if you call `pry` on a custom object that has methods especially suited to your task.
345
+
346
+ Ruby is supremely well suited for DSL's since it does not require parentheses for method calls,
347
+ so calls to your custom methods _look_ like built in language commands and keywords.
348
+
349
+
350
+ #### Suppressing Automatic Output in Filter Modes
351
+
352
+ The filter input modes will automatically call `puts` to output the last evaulated value of your code to stdout. There may be times you may want to do something _else_ with the input and send nothing to stdout. For example, you might want to write something to a file, send to a network, etc. The simplest way to suppress output is to make nil or the empty string the final value in the expression. This can be accomplished simply merely by appending `; nil` or `;''` to your code. For example, to only output directory entries containing the letter 'e' in `-ml` (line) mode:
214
353
 
215
354
  ```
216
- # Use REXE_OPTIONS environment variable to eliminate the need to specify
217
- # options on each invocation:
355
+ # Output only entries that contain the letter 'e':
356
+ ➜ /  ls | sort | rexe -ml "include?('e') ? self : nil"
357
+
358
+ Incompatible Software
218
359
 
219
- # First it will fail since these symbols have not been loaded via require
220
- (unless these requires have been specified elsewhere in the configuration):
221
- ➜ ~  rexe "[JSON, YAML, AwesomePrint]"
222
- Traceback (most recent call last):
360
+ Network
223
361
  ...
224
- (eval):1:in `block in call': uninitialized constant Rexe::JSON (NameError)
362
+ ```
363
+
364
+ However, as you can see, blank lines were displayed where `nil` was output. Why? Because in -ml mode,
365
+ puts will be called unconditionally on whatever value is the result of the expression. In the case of nil,
366
+ puts outputs an empty string and its usual newline. This is probably not what you want.
225
367
 
226
- # Now we specify the requires in the REXE_OPTIONS environment variable.
227
- # Contents of this variable will be prepended to the arguments
228
- # specified on the command line.
229
- ➜ ~  export REXE_OPTIONS="-r json,yaml,awesome_print"
368
+ If you want to see the `nil`s, you could replace `nil` with `nil.inspect`, which returns the string `'nil'`,
369
+ unlike `nil.to_s` which returns the empty string. Of course,
370
+ there may be some other custom string you would want,
371
+ such as `[no match]` or `-`, or you could just specify the string `'nil'`. [^2]
230
372
 
231
- # Now that command that previously failed will succeed:
232
- ➜ ~  rexe "[JSON, YAML, AwesomePrint].to_s"
233
- [JSON, Psych, AwesomePrint]
373
+ But you probably don't want any line at all to display for excluded objects. For this it is best to use
374
+ `-me` (enumerator) mode. If you won't have a huge amount of input data you could use `select`:
375
+
376
+ ```
377
+ # Output only entries that contain the letter 'e':
378
+ ➜ /  ls | sort | rexe -me "select { |s| s.include?('e') }"
379
+ Incompatible Software
380
+ Network
381
+ System
234
382
  ```
235
383
 
236
- ----
384
+ Here, `select` returns an array which is implicitly passed to `puts`. `puts` does _not_ call `to_s` when passed an array, but instead has special handling for arrays which prints each element on its own line. If instead we appended `.to_s`
385
+ or `.inspect` to the result array, we would get the more compact array notation:
386
+
387
+ ```
388
+ ➜ /  ls | sort | rexe -me "select { |s| s.include?('e') }.to_s"
389
+ ["Incompatible Software\n", "Network\n", ..., "private\n"]
390
+ ```
391
+
392
+ #### Quoting Strings in Your Ruby Code
393
+
394
+ One complication of using utilities like `rexe` where Ruby code is specified on the shell command line is that
395
+ you need to be careful about the shell's special treatment of certain characters. For this reason, it is often
396
+ necessary to quote the Ruby code. You can use single or double quotes.
397
+ An excellent reference for how they differ is [here](https://stackoverflow.com/questions/6697753/difference-between-single-and-double-quotes-in-bash).
398
+
399
+ Feel free to fall back on Ruby's super useful `%q{}` and `%Q{}`, equivalent to single and double quotes, respectively.
237
400
 
238
- Access public JSON data and print it with awesome_print, using the ruby interpreter directly:
239
401
 
402
+ #### Mimicking Method Arguments
403
+
404
+ You may want to support arguments in your code. One of the previous examples downloaded currency conversion rates. Let's find out the available currency codes:
405
+
406
+ ```
407
+ ➜ /  echo $JSON_TEXT | rexe -rjson -mb \
408
+ "JSON.parse(self)['rates'].keys.sort.join(' ')"
409
+ AUD BGN BRL CAD CHF CNY CZK DKK GBP HKD HRK HUF IDR ILS INR ISK JPY KRW MXN MYR NOK NZD PHP PLN RON RUB SEK SGD THB TRY USD ZAR
410
+ ```
411
+
412
+ Here would be a way to output a single rate:
413
+
240
414
  ```
241
- ➜ ~  export JSON_TEXT=`curl https://api.exchangeratesapi.io/latest`
242
- ➜ ~  echo $JSON_TEXT | ruby -r json -r awesome_print -e 'ap JSON.parse(STDIN.read)'
415
+ ➜ ~  echo PHP | rexe -ml -rjson \
416
+ "rate = JSON.parse(ENV['EUR_RATES_JSON'])['rates'][self];\
417
+ %Q{1 EUR = #{rate} #{self}}"
243
418
 
244
- {
245
- "base" => "EUR",
246
- "date" => "2019-02-20",
247
- "rates" => {
248
- "NZD" => 1.6513,
249
- "CAD" => 1.4956,
250
- "MXN" => 21.7301,
251
- ...
252
- }
419
+ 1 EUR = 58.986 PHP
253
420
  ```
254
421
 
422
+ In this code, `self` is the currency code `PHP` (Philippine Peso). We have accessed the JSON text to parse from the environment variable we previously populated.
423
+
255
424
 
256
- This `rexe` command will have the same effect:
425
+ ### More Examples
426
+
427
+ Here are some more examples to illustrate the use of `rexe`.
428
+
429
+ ----
430
+
431
+ Show disk space used/free on a Mac's main hard drive's main partition:
257
432
 
258
433
  ```
259
- ➜ ~  echo $JSON_TEXT | rexe -mb -r awesome_print,json "JSON.parse(self).ai"
434
+ ➜ ~  df -h | grep disk1s1 | rexe -ml \
435
+ "x = split; puts %Q{#{x[4]} Used: #{x[2]}, Avail #{x[3]}}"
436
+ 91% Used: 412Gi, Avail 44Gi
260
437
  ```
261
438
 
262
- The input modes that directly support standard input will send the last evaluated value to standard output.
263
- So instead of calling AwesomePrint's `ap`, we call `ai` (awesome inspect) on the object we want to display.
439
+ (Note that `split` is equivalent to `self.split`, and because the `-ml` option is used, `self` is the line of text.
440
+
441
+ ----
264
442
 
265
- In "no input" mode, there's nothing preventing us from handling the input ourselves (e.g. as `STDIN.read`,
266
- so we could have accomplished the same thing like this:
443
+ Print yellow (trust me!):
267
444
 
268
445
  ```
269
- echo $JSON_TEXT | rexe -r awesome_print,json "ap JSON.parse(STDIN.read)"
446
+ ➜ ~  cowsay hello | rexe -me "print %Q{\u001b[33m}; puts to_a"
447
+ ➜ ~  # or
448
+ ➜ ~  cowsay hello | rexe -mb "print %Q{\u001b[33m}; puts self"
449
+ ➜ ~  # or
450
+ ➜ ~  cowsay hello | rexe "print %Q{\u001b[33m}; puts STDIN.read"
451
+ _______
452
+ < hello >
453
+ -------
454
+ \ ^__^
455
+ \ (oo)\_______
456
+ (__)\ )\/\
457
+ ||----w |
458
+ || ||`
270
459
  ```
271
460
 
461
+
272
462
  ----
273
463
 
274
- Often we want to treat input as an array. Assuming there won't be too much to fit in memory,
275
- we can instruct `rexe` to treat input as an `Enumerable` (using the `-me` option), and then
276
- calling `to_a` on it. The following code prints the environment variables, sorted, with Awesome Print:
464
+
465
+ Show the 3 longest file names of the current directory, with their lengths, in descending order:
466
+
277
467
  ```
278
- ➜ ~  env | rexe -me -r awesome_print sort.to_a.ai
279
- [
280
- ...
281
- [ 4] "COLORFGBG=15;0\n",
282
- [ 5] "COLORTERM=truecolor\n",
283
- ...
468
+ ➜ ~  ls | rexe -ml "%Q{[%4d] %s} % [length, self]" | sort -r | head -3
469
+ [ 50] Agoda_Booking_ID_9999999 49_–_RECEIPT_enclosed.pdf
470
+ [ 40] 679a5c034994544aab4635ecbd50ab73-big.jpg
471
+ [ 28] 2018-abc-2019-01-16-2340.zip
284
472
  ```
285
473
 
474
+ Notice that when you right align numbers using printf formatting, sorting the lines
475
+ alphabetically will result in sorting them numerically as well.
476
+
286
477
  ----
287
478
 
288
- Here are two ways to print the number of entries in a directory.
289
- Notice that the two number differ by 1.
479
+ I was recently asked to provide a schema for the data in my `rock_books` accounting gem. `rock_books` data is intended to be very small in size, and no data base is used. Instead, the input data is parsed on every run, and reports generated on demand. However, there are data structures (actually class instances) in memory at runtime, and their classes inherit from `Struct`.
480
+ The definition lines look like this one:
481
+
482
+ ```
483
+ class JournalEntry < Struct.new(:date, :acct_amounts, :doc_short_name, :description, :receipts)
484
+ ```
485
+
486
+ The `grep` command line utility prepends each of these matches with a string like this:
290
487
 
291
488
  ```
292
- ➜ ~  rexe -mn "puts %Q{This directory has #{Dir['*'].size} entries.}"
293
- This directory has 210 entries.
294
- ➜ ~  echo `ls -l | wc -l` | rexe -ms "%Q{This directory has #{self} entries.}"
295
- This directory has 211 entries.
489
+ lib/rock_books/documents/journal_entry.rb:
296
490
  ```
297
491
 
492
+ So this is what worked well for me:
493
+
494
+ ```
495
+ ➜ ~  grep Struct **/*.rb | grep -v OpenStruct | rexe -ml \
496
+ "a = \
497
+ gsub('lib/rock_books/', '')\
498
+ .gsub('< Struct.new', '')\
499
+ .gsub('; end', '')\
500
+ .split('.rb:')\
501
+ .map(&:strip);\
502
+ \
503
+ %q{%-40s %-s} % [a[0] + %q{.rb}, a[1]]"
504
+ ```
505
+
506
+ ...which produced this output:
507
+
508
+ ```
509
+ cmd_line/command_line_interface.rb class Command (:min_string, :max_string, :action)
510
+ documents/book_set.rb class BookSet (:run_options, :chart_of_accounts, :journals)
511
+ documents/journal.rb class Entry (:date, :amount, :acct_amounts, :description)
512
+ documents/journal_entry.rb class JournalEntry (:date, :acct_amounts, :doc_short_name, :description, :receipts)
513
+ documents/journal_entry_builder.rb class JournalEntryBuilder (:journal_entry_context)
514
+ reports/report_context.rb class ReportContext (:chart_of_accounts, :journals, :page_width)
515
+ types/account.rb class Account (:code, :type, :name)
516
+ types/account_type.rb class AccountType (:symbol, :singular_name, :plural_name)
517
+ types/acct_amount.rb class AcctAmount (:date, :code, :amount, :journal_entry_context)
518
+ types/journal_entry_context.rb class JournalEntryContext (:journal, :linenum, :line)
519
+ ```
520
+
521
+ Although there's a lot going on here, the vertical and horizontal alignments and spacing make the code
522
+ straightforward to follow. Here's what it does:
523
+
524
+ * grep the code base for `"Struct"`
525
+ * exclude references to `"OpenStruct"` with `grep -v`
526
+ * remove unwanted text with `gsub`
527
+ * split the line into 1) a filespec relative to `lib/rockbooks`, and 2) the class definition
528
+ * strip unwanted space because that will mess up the horizontal alignment of the output.
529
+ * use C-style printf formatting to align the text into two columns
530
+
531
+
532
+
533
+
534
+
535
+ ### Conclusion
536
+
537
+ `rexe` is not revolutionary technology, it's just plumbing that removes low level
538
+ configuration from your command line so that you can focus on the high level
539
+ task at hand.
540
+
541
+ When we think of a new piece of software, we usually think "what would this be
542
+ helpful with now?". However, the power of `rexe` is not so much what can be done
543
+ with it in a single use case now, but rather what will it do for me as I get
544
+ used to the concept and my supporting code and its uses evolve.
545
+
546
+ I suggest starting to use `rexe` even for modest improvements in workflow, even
547
+ if it doesn't seem compelling. There's a good chance that as you use it over
548
+ time, new ideas will come to you and the workflow improvements will increase
549
+ exponentially.
550
+
551
+
552
+ #### Footnotes
298
553
 
299
- ## License
554
+ [^1]: `rexe` is an embellishment of the minimal but excellent `rb` script at
555
+ https://github.com/thisredone/rb. I started using `rb` and thought of lots of
556
+ other features I would like to have, so I started working on `rexe`.
300
557
 
301
- The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
558
+ [^2]: You might wonder why we don't just refrain from sending output to stdout on null or false. That is certainly easy to implement, but there are other ways to accomplish this (using _enumerable_ or _no input_ modes), and the lack of output might be surprising and disconcerting to the user. What do _you_ think, which approach makes more sense to you?
data/exe/rexe CHANGED
@@ -9,7 +9,7 @@ require 'shellwords'
9
9
 
10
10
  class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
11
11
 
12
- VERSION = '0.8.1'
12
+ VERSION = '0.9.0'
13
13
 
14
14
  def initialize
15
15
  clear_options
@@ -39,10 +39,10 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
39
39
  -h, --help Print help and exit
40
40
  -l, --load RUBY_FILE(S) Ruby file(s) to load, comma separated, or ! to clear
41
41
  -m, --mode MODE Mode with which to handle input (i.e. what `self` will be in your code):
42
- -ms for each line to be handled as a separate string
43
- -me for an enumerator of lines (least memory consumption for big data)
44
- -mb for 1 big string (all lines combined into single multiline string)
45
- -mn don't do special handling of input; self is not the input (default)
42
+ -ml line mode; each line is ingested as a separate string
43
+ -me enumerator mode
44
+ -mb big string mode; all lines combined into single multiline string
45
+ -mn (default) no input mode; no special handling of input; self is not input
46
46
  -n, --[no-]noop Do not execute the code (useful with -v); see note (1) below
47
47
  -r, --require REQUIRES Gems and built-in libraries to require, comma separated, or ! to clear
48
48
  -v, --[no-]verbose verbose mode (logs to stderr); see note (1) below
@@ -53,7 +53,7 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
53
53
  If there is a REXE_OPTIONS environment variable, its content will be prepended to the command line
54
54
  so that you can specify options implicitly (e.g. `export REXE_OPTIONS="-r awesome_print,yaml"`)
55
55
 
56
- (1) For boolean verbose and noop options, the following are valid:
56
+ (1) For boolean 'verbose' and 'noop' options, the following are valid:
57
57
  -v no, -v yes, -v false, -v true, -v n, -v y, -v +, but not -v -
58
58
 
59
59
  HEREDOC
@@ -94,10 +94,10 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
94
94
  end
95
95
 
96
96
  parser.on('-m', '--mode MODE',
97
- 'Mode with which to handle input (-ms (default), -me, -mb, mn)') do |v|
97
+ 'Mode with which to handle input (-ml, -me, -mb, -mn (default)') do |v|
98
98
 
99
99
  modes = {
100
- 's' => :string,
100
+ 'l' => :line,
101
101
  'e' => :enumerator,
102
102
  'b' => :one_big_string,
103
103
  'n' => :no_input
@@ -192,7 +192,7 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
192
192
  code = eval(source_code)
193
193
 
194
194
  actions = {
195
- string: -> { STDIN.each { |l| execute(l.chomp, code) } },
195
+ line: -> { STDIN.each { |l| execute(l.chomp, code) } },
196
196
  enumerator: -> { execute(STDIN.each_line, code) },
197
197
  one_big_string: -> { big_string = STDIN.read; execute(big_string, code) },
198
198
  no_input: -> { execute(nil, code) }
data/rexe.gemspec CHANGED
@@ -43,5 +43,20 @@ Gem::Specification.new do |spec|
43
43
  spec.add_development_dependency "rake", "~> 12.3"
44
44
  spec.add_development_dependency "rspec", "~> 3.0"
45
45
 
46
- spec.post_install_message = "\n\nWARNING! The default input mode was changed from -ms to -mn in version 0.6.0.\n\n"
46
+ spec.post_install_message =
47
+ <<~HEREDOC
48
+
49
+ !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
50
+
51
+ WARNING!
52
+
53
+ The default rexe input mode was changed from -ms to -mn in version 0.6.0
54
+ and
55
+ the -ms (separate string mode) mode name was changed
56
+ to -ml (line mode) in version 0.9.0.
57
+
58
+ !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
59
+
60
+ HEREDOC
61
+
47
62
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rexe
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.1
4
+ version: 0.9.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Keith Bennett
@@ -96,8 +96,16 @@ metadata:
96
96
  changelog_uri: https://github.com/keithrbennett/rexe/blob/master/README.md
97
97
  post_install_message: |2+
98
98
 
99
+ !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
99
100
 
100
- WARNING! The default input mode was changed from -ms to -mn in version 0.6.0.
101
+ WARNING!
102
+
103
+ The default rexe input mode was changed from -ms to -mn in version 0.6.0
104
+ and
105
+ the -ms (separate string mode) mode name was changed
106
+ to -ml (line mode) in version 0.9.0.
107
+
108
+ !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
101
109
 
102
110
  rdoc_options: []
103
111
  require_paths: