rexe 0.8.1 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e722054c78d64b4423f008c6c4e24e9d47b16fa4e07450f28d5a835246cd9810
-  data.tar.gz: 4624fe6eacd8e6052b78a9410c2ac7ad9a527ffbd0977225cf0b91de41f90184
+  metadata.gz: 2aee272d801d5f73d5d742857baa103d108b366d2c4b82a64595d662dc47a438
+  data.tar.gz: ef605e5e1b9918215602abf481a20d1d24847c8fa36fc93ff9f91bcaf826547b
 SHA512:
-  metadata.gz: 80f48114e7ff0e47fa6551e30b42aa0bc7326984bb68b5fec588d4cb4d2b1f0c246bc710aa709e9214dbcb52c18e0c9cae39bf89fec4f67c9a3e8fe44ea96621
-  data.tar.gz: 64a01dcfa2daf21325f42203c4e74a090e6650aefb8bf6fec453f6c36b1a7f0e6ba38c05cd6a4f260cc4c4cb6ac7bac55ad3a6832ee00b927ea7a749d2ecc756
+  metadata.gz: 86ac1e3509c6ff78fc4037c25b441b62786f3cf473aa3409678ec1adcce0c1e23def881137a20972ea5bf2ab605364986e80c83b6d4bdab5c8845d203d3fce07
+  data.tar.gz: 1fd4c1d947117fa60ab722bf5a16aae4a378ee31d36d95503cfa15abe15e158d3350e7dbf010b86b22515eeb9492a0b9f72c0099cd8950fca0015438fae4c8ad

data/CHANGELOG.md

@@ -1,6 +1,12 @@
 ## rexe -- Ruby Command Line Executor
 
 
+### v0.9.0
+
+* Change -ms (single or separate string) mode to -ml (line) mode.
+* Use article text for readme.
+
+
 ### v0.8.1
 
 * Fix and improve help text.

data/README.md

@@ -1,33 +1,53 @@
-# Rexe
-
-A configurable Ruby command line executor and filter.
-
-
+---
+title: The `rexe` Command Line Executor and Filter
+date: 2019-02-15
+---
 
-## Installation
+I love the power of the command line, but not the awkwardness of shell scripting
+languages. Sure, there's a lot that can be done with them, but it doesn't take
+long before I get frustrated with their bluntness and verbosity.
 
-Installation can be performed by either installing the gem or copying the single executable file to your system and ensuring that it is executable:
+Often, I solve this problem by writing a Ruby script instead. Ruby gives me fine
+grained control in a "real" programming language with which I am comfortable.
+However, when there are multiple OS commands to be called, then Ruby can be
+awkward too.
 
-```gem install rexe```
+### Using the Ruby Interpreter on the Command Line
 
-or
+Sometimes a good solution is to combine Ruby and shell scripting on the same
+command line. Here's an example, using an intermediate environment variable to
+simplify the logic and save the data for use by future commands.
+An excerpt of the output follows the code:
 
 ```
-curl https://raw.githubusercontent.com/keithrbennett/rexe/master/exe/rexe > rexe
-chmod +x rexe
+➜  ~   export EUR_RATES_JSON=`curl https://api.exchangeratesapi.io/latest`
+➜  ~   echo $EUR_RATES_JSON | ruby -r json -r awesome_print -e 'ap JSON.parse(STDIN.read)'
+{
+    "rates" => {
+        "MXN" => 21.781,
+        ...
+        "DKK" => 7.462
+    },
+     "base" => "EUR",
+     "date" => "2019-02-22"
+}
 ```
 
-## Usage
+However, the configuration setup (the `require`s) make the command long and tedious, discouraging this
+approach.
 
-rexe is an _executor_, meaning it can execute Ruby without either standard input or output,
-but it is also a _filter_ in that it can implicitly consume standard input and emit standard output.
+### Rexe
 
-### Help Text
+Enter the `rexe` script. [^1]
+ 
+`rexe` is at https://github.com/keithrbennett/rexe and can be installed with
+`gem install rexe`. `rexe` provides several ways to simplify Ruby on the command
+line, tipping the scale so that it is practical to do it more often.
 
-As a summary, here is the help text printed out by the application:
+Here is `rexe`'s help text as of the time of this writing:
 
 ```
-rexe -- Ruby Command Line Filter/Executor -- v0.8.0 -- https://github.com/keithrbennett/rexe
+rexe -- Ruby Command Line Executor/Filter -- v0.9.0 -- https://github.com/keithrbennett/rexe
 
 Executes Ruby code on the command line, optionally taking standard input and writing to standard output.
 
@@ -36,14 +56,14 @@ Options:
 -c  --clear_options        Clear all previous command line options specified up to now
 -h, --help                 Print help and exit
 -l, --load RUBY_FILE(S)    Ruby file(s) to load, comma separated, or ! to clear
--m, --mode MODE            Mode with which to handle input (i.e. what `self` will be in the code):
-                           -ms for each line to be handled separately as a string
-                           -me for an enumerator of lines (least memory consumption for big data)
-                           -mb for 1 big string (all lines combined into single multiline string)
-                           -mn don't do special handling of input; self is not the input (default) 
--n', --[no-]noop           Do not execute the code (useful with -v)
+-m, --mode MODE            Mode with which to handle input (i.e. what `self` will be in your code):
+                           -ml line mode; each line is ingested as a separate string
+                           -me enumerator mode
+                           -mb big string mode; all lines combined into single multiline string
+                           -mn (default) no input mode; no special handling of input; self is not input 
+-n, --[no-]noop            Do not execute the code (useful with -v); see note (1) below
 -r, --require REQUIRES     Gems and built-in libraries to require, comma separated, or ! to clear
--v, --[no-]verbose         verbose mode (logs to stderr); to disable, short options: -v n, -v false
+-v, --[no-]verbose         verbose mode (logs to stderr); see note (1) below
 
 If there is an .rexerc file in your home directory, it will be run as Ruby code 
 before processing the input.
@@ -51,251 +71,488 @@ before processing the input.
 If there is a REXE_OPTIONS environment variable, its content will be prepended to the command line
 so that you can specify options implicitly (e.g. `export REXE_OPTIONS="-r awesome_print,yaml"`)
 
-For boolean verbose and noop options, the following are valid:
+(1) For boolean 'verbose' and 'noop' options, the following are valid:
 -v no, -v yes, -v false, -v true, -v n, -v y, -v +, but not -v -
 ```
 
-### Input Mode
+For consistency with the `ruby` interpreter we called previously, `rexe` supports requires with the `-r` option, but as one tiny improvement it also allows grouping them together using commas:
 
-When it is used as a filter, the input is accessed differently in the source code 
-depending on the mode that was specified (see example section below for examples). 
-The mode letter is appended to `-m` on the command line;
-`n` (_no input_) mode is the default.
+```
+                                    vvvvvvvvvvvvvvvvvvvvv
+➜  ~   echo $EUR_RATES_JSON | rexe -r json,awesome_print 'ap JSON.parse(STDIN.read)'
+                                    ^^^^^^^^^^^^^^^^^^^^^
+```
 
-* `s` - _string_ mode - the source code is run once on each line of input, and `self` is each line of text
-* `e` - _enumerator_ mode - the code is run once on the enumerator of all lines; `self` is the enumerator, so you can call `map`, `to_a`, `select`, etc without explicitly specifying `self`.
-* `b` - _big string_ mode - all input is treated as one large (probably) multiline string with the newlines intact; `self` is this large string; this mode is required, for example, for parsing the text into a single object from JSON or YAML formatted data.
-* `n` - _no input_ mode - this instructs the program to proceed without looking for input
+This command produces the same results as the previous `ruby` one.
 
+### Simplifying the Rexe Invocation with Configuration
 
-### Requires
+`rexe` provides two approaches to configuration:
 
-As with the Ruby interpreter itself, `require`s can be specified on the command line with the `-r` option. Multiple requires can be combined with commas between them.
+* the `REXE_OPTIONS` environment variable
+* loading Ruby files before executing the code using `-l`, or implicitly with `~/.rexerc`
 
+These approaches enable removing configuration information from your `rexe` command,
+making it shorter and simpler to read.
 
-### Loading Ruby Files
 
-Other Ruby files can be loaded by `rexe`, to, for example, define classes and methods to use, set up resources, etc. They are specified with the `-l` option.
+### The REXE_OPTIONS Environment Variable
 
-If there is a file named `.rexerc` in the home directory, that will always be loaded without explicitly requesting it on the command line.
+The `REXE_OPTIONS` environment variable can contain command line options that would otherwise
+be specified on the `rexe` command line:
 
-### Verbose Mode
+```
+➜  ~   export REXE_OPTIONS="-r json,awesome_print"
+➜  ~   echo $EUR_RATES_JSON | rexe 'ap JSON.parse(STDIN.read)'
+```
+
+Like any environment variable, `REXE_OPTIONS` could also be set in your startup script, input on a command line using `export`, or in another script loaded with `source` or `.`.
+
+### Loading Files
 
-Verbose mode outputs information to standard error (stderr). This information can be redirected, for example to a file named `rexe.log`, by adding `2>& rexe.log` to the command line.
+The environment variable approach works well for command line _options_, but what if we want to specify Ruby _code_ (e.g. methods) that can be used by multiple invocations of `rexe`?
 
-Here is an example of some text that might be output in verbose mode:
+For this, `rexe` lets you _load_ Ruby files, using the `-l` option, or implicitly (without your specifying it) in the case of the `~/.rexerc` file. Here is an example of something you might include in such a file (this is an alternate approach to specifying `-r` in the `REXE_OPTIONS` environment variable):
 
 ```
-➜  ~   rexe -r yaml -mn -v "%w(foo bar baz).to_yaml"
-rexe version 0.3.0 -- 2019-02-08 02:21:27 +0700
-Source Code: %w(foo bar baz).to_yaml
-Options: {:input_mode=>:no_input, :loads=>[], :requires=>["yaml"], :verbose=>true}
-Loading global config file /Users/kbennett/.rexerc
----
-- foo
-- bar
-- baz
-rexe time elapsed: 0.03874 seconds.
+require 'json'
+require 'yaml'
+require 'awesome_print'
 ```
 
-To set verbose mode _off_ on the command line, the option parser accepts the long option `--no-verbose`, of course, but you could use instead `-v false` or `-v n`. If you should need to pass a value to `-v` to turn verbose mode _on_, then you can use `-v y` or `-v true`. 
+Requiring gems and modules for _all_ invocations of `rexe` will make your commands simpler and more concise, but will be a waste of execution time if they are not needed. You can inspect the execution times to see just how much time is being wasted. For example, we can find out that nokogiri takes about 0.7 seconds to load on my laptop by observing and comparing the execution times with and without the require (output has been abbreviated):
 
-### The REXE_OPTIONS Environment Variable
+```
+➜  ~   rexe -v
+rexe time elapsed: 0.094946 seconds.
 
-Very often you will want to call `rexe` several times with similar options. Instead of having to clutter the command line each time with these options, you can put them in an environment variable named `REXE_OPTIONS`, and they will be prepended automatically. Since they will be processed before the options on the command line, they are of lower precedence and can be overridden.
+➜  ~   rexe -v -r nokogiri
+rexe time elapsed: 0.165996 seconds.
+```
 
+### Using Loaded Files in Your Commands
 
-### The ~/.rexerc Configuation File
+Here's something else you could include in such a load file:
 
-The `.rexerc` file in your home directory (if it exists) is loaded unconditionally. Here is a place to put things that you will _always_ want. You can include commonly needed requires, but be careful because the impact on startup time may be more than you want. You can test this using verbose mode, which outputs the execution time after completing.
+```
+# Open YouTube to Wagner's "Ride of the Valkyries"
+def valkyries
+  `open "http://www.youtube.com/watch?v=P73Z6291Pt8&t=0m28s"`
+end
+```
 
+Why would you want this? You might want to be able to go to another room until a long job completes, and be notified when it is done. The `valkyries` method will launch a browser window pointed to Richard Wagner's "Ride of the Valkyries" starting at a lively point in the music. (The `open` command is Mac specific and could be replaced with `start` on Windows, a browser command name, etc.) If you like this sort of thing, you could download public domain audio files and use a command like player like `afplay` on Mac OS, or `mpg123` or `ogg123` on Linux. This approach is lighter weight, requires no network access, and will not leave an open browser window for you to close.
 
-### Directory-Specific Configuration Files
+Here is an example of how you might use this, assuming the above configuration is loaded from your `~/.rexerc` file or 
+an explicitly loaded file:
 
-Although there is no built-in feature for directory-specific configuration files, this can be easily accomplished by inserting something like this in your global configuration file (~/.rexerc):
+```
+➜  ~   tar czf /tmp/my-whole-user-space.tar.gz ~ ; rexe valkyries
+```
+
+You might be thinking that creating an alias or a minimal shell script for this open would be a simpler and more natural
+approach, and I would agree with you. However, over time the number of these could become unmanageable, whereas using Ruby
+you could build a pretty extensive and well organized library of functionality. Moreover, that functionality could be made available to _all_ your Ruby code (for example, by putting it in a gem), and not just command line one liners.
+
+For example, you could have something like this in a configuration file:
 
 ```
-dir_specific_config_file = './rexe.rc'
-load dir_specific_config_file' if File.exist?(dir_specific_config_file)
+def play(piece_code)
+  pieces = {
+    hallelujah: "https://www.youtube.com/watch?v=IUZEtVbJT5c&t=0m20s",
+    valkyries:  "http://www.youtube.com/watch?v=P73Z6291Pt8&t=0m28s",
+    wm_tell:    "https://www.youtube.com/watch?v=j3T8-aeOrbg"
+    # ... and many, many more
+  }
+  `open #{Shellwords.escape(pieces.fetch(piece_code))}`
+end
 ```
- 
-You should probably put this at the very end of this global configuration file so that the directory specific file will always be able to override global settings.
 
-Note that the directory specific filename used differs from the global config filename. This is a) so that there is no recursive loading of the file when in the home directory, and b) because it is more convenient for these files that they not be hidden.
+...which you could then call like this:
+
+```
+➜  ~   tar czf /tmp/my-whole-user-space.tar.gz ~ ; rexe 'play(:hallelujah)'
+```
+
+(You need to quote the `play` call because otherwise the shell will process and remove the parentheses.
+Alternatively you could escape the parentheses with backslashes.)
+
 
+### Clearing the Require and Load Lists
 
-## Troubleshooting
+There may be times when you have specified a load or require on the command line
+or in the `REXE_OPTIONS` environment variable,
+but you want to override it for a single invocation. Currently you cannot
+unspecify a single resource, but you can unspecify _all_ the requires or loads
+with the `-r!` and `-l!` command line options, respectively.
 
-One common problem relates to the shell's special handling of characters. Remember that the shell will process special characters, thereby changing your text before passing it on to the Ruby code. It is good to get in the habit of putting your source code in double quotes; and if the source code itself uses quotes, use `q{}` or `Q{}` instead. For example:
 
+### Clearing _All_ Options
+
+You can also clear _all_ options specified up to a certain point in time with the _clear options_ option (`-c`).
+This is especially useful if you have specified options in the `REXE_OPTIONS` environment variable, 
+and want to ignore all of them.
+
+
+### Verbose Mode
+
+In addition to displaying the execution time, verbose mode will display the version, date/time of execution, source code
+to be evaluated, options specified (by all approaches), and that the global file has been loaded (if it was found):
+ 
 ```
-➜   rexe -mn "puts %Q{The time is now #{Time.now}}"
-The time is now 2019-02-04 18:49:31 +0700
+➜  ~   echo $EUR_RATES_JSON | rexe -v -rjson,awesome_print "ap JSON.parse(STDIN.read)"
+rexe version 0.7.0 -- 2019-03-03 18:18:14 +0700
+Source Code: ap JSON.parse(STDIN.read)
+Options: {:input_mode=>:no_input, :loads=>[], :requires=>["json", "awesome_print"], :verbose=>true}
+Loading global config file /Users/kbennett/.rexerc
+...
+rexe time elapsed: 0.085913 seconds.
+``` 
+ 
+This extra output is sent to standard error (_stderr_) instead of standard output
+(_stdout_) so that it will not pollute the "real" data when stdout is piped to
+another command.
+
+If you would like to append this informational output to a file, you could do something like this:
+
 ```
+➜  ~   rexe ... -v 2>>rexe.log
+```
+
+If verbose mode is enabled in configuration and you want to disable it, you can
+do so by using any of the following: `--[no-]verbose`, `-v n`, or `-v false`.
 
-If you are troubleshooting the setup (i.e. the command line options, loaded files, and `REXE_OPTIONS` environment variable) using the verbose option, you may have the problem of the logging scrolling off the screen due to the length of your output. In this case you could easily fake or disable the output adding `; nil`, `; 'foo'`', etc. to the end of your expression. This way you don't have to mess with your code's logic.
+### Input Modes
 
-## Examples
+`rexe` tries to make it simple and convenient for you to handle standard input, and in different ways. Here is the help text relating to input modes:
 
 ```
-# Call reverse on listed file.
-# Need to specify the mode, since it defaults to "n" ("-mn"),
-# which treats every line separately.
-➜  rexe git:(master) ✗   ls | head -2 | exe/rexe -ms "self + ' --> ' + reverse"
-CHANGELOG.md --> dm.GOLEGNAHC
-Gemfile --> elifmeG
+-m, --mode MODE            Mode with which to handle input (i.e. what `self` will be in your code):
+                           -ml line mode; each line is ingested as a separate string
+                           -me enumerator mode
+                           -mb big string mode; all lines combined into single multiline string
+                           -mn (default) no input mode; no special handling of input; self is not input 
 ```
 
-----
+The first three are _filter_ modes; they make standard input available
+to your code as `self`, and automatically output to standard output
+the last value evaluated by your code.
+
+The last (and default) is the _executor_ mode. It merely assists you in
+executing the code you provide without any special implicit handling of standard input.
+
+
+#### -ml "Line" Filter Mode
+
+In this mode, your code would be called once per line of input,
+and in each call, `self` would evaluate to the line of text:
 
 ```
-# Use input data to create a human friendly message:
-➜  ~   uptime | rexe -ms "%Q{System has been up: #{split.first}.}"
-System has been up: 17:10.
+➜  ~   echo "hello\ngoodbye" | rexe -ms reverse
+olleh
+eybdoog
 ```
 
-----
+`reverse` is implicitly called on each line of standard input.  `self`
+ is the input line in each call (we could also have used `self.reverse` but the `self` would have been redundant.).
+  
+
+#### -me "Enumerator" Filter Mode
+
+In this mode, your code is called only once, and `self` is an enumerator
+dispensing all lines of standard input. To be more precise, it is the enumerator returned by `STDIN.each_line`.
+
+Dealing with input as an enumerator enables you to use the wealth of `Enumerable` methods such as `select`, `to_a`, `map`, etc.
+
+Here is an example of using `-me` to add line numbers to the first 3
+files in the directory listing:
 
 ```
-# Create a JSON array of a file listing.
-# Use the "-me" flag so that all input is treated as a single enumerator of lines.
-➜  ~   ls | head -3 | rexe -me -r json "map(&:strip).to_a.to_json"
-["AFP.conf","afpovertcp.cfg","afpovertcp.cfg~orig"]
+➜  ~   ls / | rexe -me "first(3).each_with_index { |ln,i| puts '%5d  %s' % [i, ln] }; nil"
+
+    0  AndroidStudioProjects
+    1  Applications
+    2  Desktop
 ```
 
-----
+Since `self` is an enumerable, we can call `first` and then `each_with_index`.
+
+
+#### -mb "Big String" Filter Mode
+
+In this mode, all standard input is combined into a single, (possibly)
+large string, with newline characters joining the lines in the string.
+
+A good example of when you would use this is when you parse JSON or YAML text; you need to pass the entire (probably) multiline string to the parse method.
+
+An earlier example would be more simply specified using this mode, since `STDIN.read` could be replaced with `self`:
 
 ```
-# Create a "pretty" JSON array of a file listing:
-➜  ~   ls | head -3 | rexe -me -r json "JSON.pretty_generate(map(&:strip).to_a)"
-[
-  "AFP.conf",
-  "afpovertcp.cfg",
-  "afpovertcp.cfg~orig"
-]
+➜  ~   echo $EUR_RATES_JSON | rexe -mb -r awesome_print,json 'ap JSON.parse(self)'
 ```
 
-----
+#### -mn "No Input" Executor Mode -- The Default
+
+Examples up until this point have all used the default
+`-mn` mode. This is the simplest use case, where `self`
+does not evaluate to anything useful, and if you cared about standard
+input, you would have to code it yourself (e.g. as we did earlier with `STDIN.read`).
+
+
+#### Filter Input Mode Memory Considerations
+
+If you may have more input than would fit in memory, you can do the following:
+
+* use `-ml` (line) mode so you are fed only 1 line at a time
+* use `-me` (enumerator) mode or use `-mn` (no input) mode with something like `STDIN.each_line`, 
+but make sure not to call any methods (e.g. `map`, `select`)
+ that will produce an array of all the input
+
+
+### Implementing Domain Specific Languages (DSL's)
+
+Defining methods in your loaded files enables you to effectively define a [DSL](https://en.wikipedia.org/wiki/Domain-specific_language) for your command line use. You could use different load files for different projects, domains, or contexts, and define aliases or one line scripts to give them meaningful names. For example, if I wrote code to work with Ansible and put it in `~/projects/rexe-ansible.rb`, I could define an alias in my startup script:
 
 ```
-# Create a YAML array of a file listing:
-➜  ~   ls | head -3 | rexe -me -r yaml "map(&:strip).to_a.to_yaml"
----
-- AFP.conf
-- afpovertcp.cfg
-- afpovertcp.cfg~orig
+➜  ~   alias rxans="rexe -l ~/projects/rexe-ansible.rb $*"
 ```
+...and then I would have an Ansible DSL available for me to use by calling `rxans`.
 
-----
+In addition, since you can also call `pry` on the context of any object, you
+can provide a DSL in a [REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) (shell)
+trivially easily. Just to illustrate, here's how you would open a REPL on the File class:
 
 ```
-# Use AwesomePrint to print a file listing.
-# (Rather than calling the `ap` method on the object to print, 
-# call the `ai` method _on_ the object to print:
-➜  ~   ls | head -3 | rexe -me -r awesome_print "map(&:chomp).ai"
-[
-    [0] "AFP.conf",
-    [1] "afpovertcp.cfg",
-    [2] "afpovertcp.cfg~orig"
-]
+➜  ~   ruby -r pry -e File.pry
+# or
+➜  ~   rexe -r pry File.pry
 ```
 
-----
+`self` would evaluate to the `File` class, so you could call class methods implicitly using only their names:
 
 ```
-# Don't use input at all, so use "-mn" to tell rexe not to expect input.
-# -mn is the default, so it works with or without specifying that option:
-➜  ~   rexe     "puts %Q{The time is now #{Time.now}}"
-➜  ~   rexe -mn "puts %Q{The time is now #{Time.now}}"
-The time is now 2019-02-04 17:20:03 +0700
+➜  rock_books git:(master) ✗   rexe  -r pry File.pry
+
+[6] pry(File)> size '/etc/passwd'
+6804
+[7] pry(File)> directory? '.'
+true
+[8] pry(File)> file?('/etc/passwd')
+true
 ```
 
-----
+This could be really handy if you call `pry` on a custom object that has methods especially suited to your task.
+
+Ruby is supremely well suited for DSL's since it does not require parentheses for method calls, 
+so calls to your custom methods _look_ like built in language commands and keywords. 
+
+
+#### Suppressing Automatic Output in Filter Modes
+
+The filter input modes will automatically call `puts` to output the last evaulated value of your code to stdout. There may be times you may want to do something _else_ with the input and send nothing to stdout. For example, you might want to write something to a file, send to a network, etc. The simplest way to suppress output is to make nil or the empty string the final value in the expression. This can be accomplished simply merely by appending `; nil` or `;''` to your code. For example, to only output directory entries containing the letter 'e' in `-ml` (line) mode:
 
 ```
-# Use REXE_OPTIONS environment variable to eliminate the need to specify
-# options on each invocation:
+# Output only entries that contain the letter 'e':
+➜  /   ls | sort | rexe -ml "include?('e') ? self : nil"
+
+Incompatible Software
 
-# First it will fail since these symbols have not been loaded via require
-(unless these requires have been specified elsewhere in the configuration):
-➜  ~   rexe "[JSON, YAML, AwesomePrint]"
-Traceback (most recent call last):
+Network
 ...
-(eval):1:in `block in call': uninitialized constant Rexe::JSON (NameError)
+```
+
+However, as you can see, blank lines were displayed where `nil` was output. Why? Because in -ml mode,
+ puts will be called unconditionally on whatever value is the result of the expression. In the case of nil,
+ puts outputs an empty string and its usual newline. This is probably not what you want.
 
-# Now we specify the requires in the REXE_OPTIONS environment variable.
-# Contents of this variable will be prepended to the arguments
-# specified on the command line.
-➜  ~   export REXE_OPTIONS="-r json,yaml,awesome_print"
+If you want to see the `nil`s, you could replace `nil` with `nil.inspect`, which returns the string `'nil'`,
+ unlike `nil.to_s` which returns the empty string. Of course, 
+ there may be some other custom string you would want, 
+ such as `[no match]` or `-`, or you could just specify the string `'nil'`. [^2]
 
-# Now that command that previously failed will succeed:
-➜  ~   rexe "[JSON, YAML, AwesomePrint].to_s"
-[JSON, Psych, AwesomePrint]
+But you probably don't want any line at all to display for excluded objects. For this it is best to use 
+`-me` (enumerator) mode. If you won't have a huge amount of input data you could use `select`:
+
+```
+# Output only entries that contain the letter 'e':
+➜  /   ls | sort | rexe -me "select { |s| s.include?('e') }"
+Incompatible Software
+Network
+System
 ```
 
-----
+Here, `select` returns an array which is implicitly passed to `puts`. `puts` does _not_ call `to_s` when passed an array, but instead has special handling for arrays which prints each element on its own line. If instead we appended `.to_s` 
+or `.inspect` to the result array, we would get the more compact array notation: 
+ 
+```
+➜  /   ls | sort | rexe -me "select { |s| s.include?('e') }.to_s"
+["Incompatible Software\n", "Network\n", ..., "private\n"]
+```
+
+#### Quoting Strings in Your Ruby Code
+
+One complication of using utilities like `rexe` where Ruby code is specified on the shell command line is that
+you need to be careful about the shell's special treatment of certain characters. For this reason, it is often
+necessary to quote the Ruby code. You can use single or double quotes. 
+An excellent reference for how they differ is [here](https://stackoverflow.com/questions/6697753/difference-between-single-and-double-quotes-in-bash).
+
+Feel free to fall back on Ruby's super useful `%q{}` and `%Q{}`, equivalent to single and double quotes, respectively.
 
-Access public JSON data and print it with awesome_print, using the ruby interpreter directly:
 
+#### Mimicking Method Arguments
+
+You may want to support arguments in your code. One of the previous examples downloaded currency conversion rates. Let's find out the available currency codes:
+
+```
+➜  /   echo $JSON_TEXT | rexe -rjson -mb \
+        "JSON.parse(self)['rates'].keys.sort.join(' ')"
+AUD BGN BRL CAD CHF CNY CZK DKK GBP HKD HRK HUF IDR ILS INR ISK JPY KRW MXN MYR NOK NZD PHP PLN RON RUB SEK SGD THB TRY USD ZAR
+```
+ 
+ Here would be a way to output a single rate:
+ 
 ```
-➜  ~   export JSON_TEXT=`curl https://api.exchangeratesapi.io/latest`
-➜  ~   echo $JSON_TEXT | ruby -r json -r awesome_print -e 'ap JSON.parse(STDIN.read)'
+➜  ~   echo PHP | rexe -ml -rjson \
+        "rate = JSON.parse(ENV['EUR_RATES_JSON'])['rates'][self];\
+        %Q{1 EUR = #{rate} #{self}}"
 
-{
-     "base" => "EUR",
-     "date" => "2019-02-20",
-    "rates" => {
-        "NZD" => 1.6513,
-        "CAD" => 1.4956,
-        "MXN" => 21.7301,
-    ...
-}
+1 EUR = 58.986 PHP
 ```
 
+In this code, `self` is the currency code `PHP` (Philippine Peso). We have accessed the JSON text to parse from the environment variable we previously populated.
+
 
-This `rexe` command will have the same effect:
+### More Examples
+
+Here are some more examples to illustrate the use of `rexe`.
+
+----
+
+Show disk space used/free on a Mac's main hard drive's main partition:
 
 ```
-➜  ~   echo $JSON_TEXT | rexe -mb -r awesome_print,json "JSON.parse(self).ai"
+➜  ~   df -h | grep disk1s1 | rexe -ml \
+"x = split; puts %Q{#{x[4]} Used: #{x[2]}, Avail #{x[3]}}"
+91% Used: 412Gi, Avail 44Gi
 ```
 
-The input modes that directly support standard input will send the last evaluated value to standard output.
-So instead of calling AwesomePrint's `ap`, we call `ai` (awesome inspect) on the object we want to display.
+(Note that `split` is equivalent to `self.split`, and because the `-ml` option is used, `self` is the line of text.
+
+----
 
-In "no input" mode, there's nothing preventing us from handling the input ourselves (e.g. as `STDIN.read`,
-so we could have accomplished the same thing like this:
+Print yellow (trust me!):
 
 ```
-echo $JSON_TEXT | rexe -r awesome_print,json "ap JSON.parse(STDIN.read)"
+➜  ~   cowsay hello | rexe -me "print %Q{\u001b[33m}; puts to_a"
+➜  ~     # or
+➜  ~   cowsay hello | rexe -mb "print %Q{\u001b[33m}; puts self"
+➜  ~     # or
+➜  ~   cowsay hello | rexe "print %Q{\u001b[33m}; puts STDIN.read"
+  _______
+ < hello >
+  -------
+         \   ^__^
+          \  (oo)\_______
+             (__)\       )\/\
+                 ||----w |
+                 ||     ||`
 ```
 
+
 ----
 
-Often we want to treat input as an array. Assuming there won't be too much to fit in memory,
-we can instruct `rexe` to treat input as an `Enumerable` (using the `-me` option), and then
-calling `to_a` on it. The following code prints the environment variables, sorted, with Awesome Print:
+    
+Show the 3 longest file names of the current directory, with their lengths, in descending order:
+
 ```
-➜  ~   env | rexe -me -r awesome_print sort.to_a.ai
-[
-...
-    [ 4] "COLORFGBG=15;0\n",
-    [ 5] "COLORTERM=truecolor\n",
-...    
+➜  ~   ls  | rexe -ml "%Q{[%4d] %s} % [length, self]" | sort -r | head -3
+[  50] Agoda_Booking_ID_9999999 49_–_RECEIPT_enclosed.pdf
+[  40] 679a5c034994544aab4635ecbd50ab73-big.jpg
+[  28] 2018-abc-2019-01-16-2340.zip
 ```
 
+Notice that when you right align numbers using printf formatting, sorting the lines
+alphabetically will result in sorting them numerically as well.
+
 ----
 
-Here are two ways to print the number of entries in a directory.
-Notice that the two number differ by 1.
+I was recently asked to provide a schema for the data in my `rock_books` accounting gem. `rock_books` data is intended to be very small in size, and no data base is used. Instead, the input data is parsed on every run, and reports generated on demand. However, there are data structures (actually class instances) in memory at runtime, and their classes inherit from `Struct`.
+ The definition lines look like this one:
+ 
+```
+class JournalEntry < Struct.new(:date, :acct_amounts, :doc_short_name, :description, :receipts)
+```
+
+The `grep` command line utility prepends each of these matches with a string like this:
 
 ```
-➜  ~   rexe -mn "puts %Q{This directory has #{Dir['*'].size} entries.}"
-This directory has 210 entries.
-➜  ~   echo `ls -l | wc -l` | rexe -ms "%Q{This directory has #{self} entries.}"
-This directory has 211 entries.
+lib/rock_books/documents/journal_entry.rb:
 ```
 
+So this is what worked well for me:
+
+```
+➜  ~   grep Struct **/*.rb | grep -v OpenStruct | rexe -ml \
+"a = \
+ gsub('lib/rock_books/', '')\
+.gsub('< Struct.new',    '')\
+.gsub('; end',           '')\
+.split('.rb:')\
+.map(&:strip);\
+\
+%q{%-40s %-s} % [a[0] + %q{.rb}, a[1]]"
+```
+
+...which produced this output:
+
+``` 
+cmd_line/command_line_interface.rb       class Command (:min_string, :max_string, :action)
+documents/book_set.rb                    class BookSet (:run_options, :chart_of_accounts, :journals)
+documents/journal.rb                     class Entry (:date, :amount, :acct_amounts, :description)
+documents/journal_entry.rb               class JournalEntry (:date, :acct_amounts, :doc_short_name, :description, :receipts)
+documents/journal_entry_builder.rb       class JournalEntryBuilder (:journal_entry_context)
+reports/report_context.rb                class ReportContext (:chart_of_accounts, :journals, :page_width)
+types/account.rb                         class Account (:code, :type, :name)
+types/account_type.rb                    class AccountType (:symbol, :singular_name, :plural_name)
+types/acct_amount.rb                     class AcctAmount (:date, :code, :amount, :journal_entry_context)
+types/journal_entry_context.rb           class JournalEntryContext (:journal, :linenum, :line)
+``` 
+
+Although there's a lot going on here, the vertical and horizontal alignments and spacing make the code
+straightforward to follow. Here's what it does:
+
+* grep the code base for `"Struct"`
+* exclude references to `"OpenStruct"` with `grep -v`
+* remove unwanted text with `gsub`
+* split the line into 1) a filespec relative to `lib/rockbooks`, and 2) the class definition
+* strip unwanted space because that will mess up the horizontal alignment of the output.
+* use C-style printf formatting to align the text into two columns
+
+ 
+
+
+
+### Conclusion
+
+`rexe` is not revolutionary technology, it's just plumbing that removes low level
+configuration from your command line so that you can focus on the high level
+task at hand.
+
+When we think of a new piece of software, we usually think "what would this be
+helpful with now?". However, the power of `rexe` is not so much what can be done
+with it in a single use case now, but rather what will it do for me as I get
+used to the concept and my supporting code and its uses evolve.
+
+I suggest starting to use `rexe` even for modest improvements in workflow, even
+if it doesn't seem compelling. There's a good chance that as you use it over
+time, new ideas will come to you and the workflow improvements will increase
+exponentially.
+
+
+#### Footnotes
 
-## License
+[^1]: `rexe` is an embellishment of the minimal but excellent `rb` script at
+https://github.com/thisredone/rb. I started using `rb` and thought of lots of
+other features I would like to have, so I started working on `rexe`.
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+[^2]: You might wonder why we don't just refrain from sending output to stdout on null or false. That is certainly easy to implement, but there are other ways to accomplish this (using _enumerable_ or _no input_ modes), and the lack of output might be surprising and disconcerting to the user. What do _you_ think, which approach makes more sense to you?

data/exe/rexe

@@ -9,7 +9,7 @@ require 'shellwords'
 
 class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
 
-  VERSION = '0.8.1'
+  VERSION = '0.9.0'
 
   def initialize
     clear_options
@@ -39,10 +39,10 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
     -h, --help                 Print help and exit
     -l, --load RUBY_FILE(S)    Ruby file(s) to load, comma separated, or ! to clear
     -m, --mode MODE            Mode with which to handle input (i.e. what `self` will be in your code):
-                               -ms for each line to be handled as a separate string
-                               -me for an enumerator of lines (least memory consumption for big data)
-                               -mb for 1 big string (all lines combined into single multiline string)
-                               -mn don't do special handling of input; self is not the input (default) 
+                               -ml line mode; each line is ingested as a separate string
+                               -me enumerator mode
+                               -mb big string mode; all lines combined into single multiline string
+                               -mn (default) no input mode; no special handling of input; self is not input 
     -n, --[no-]noop            Do not execute the code (useful with -v); see note (1) below
     -r, --require REQUIRES     Gems and built-in libraries to require, comma separated, or ! to clear
     -v, --[no-]verbose         verbose mode (logs to stderr); see note (1) below
@@ -53,7 +53,7 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
     If there is a REXE_OPTIONS environment variable, its content will be prepended to the command line
     so that you can specify options implicitly (e.g. `export REXE_OPTIONS="-r awesome_print,yaml"`)
 
-    (1) For boolean verbose and noop options, the following are valid:
+    (1) For boolean 'verbose' and 'noop' options, the following are valid:
     -v no, -v yes, -v false, -v true, -v n, -v y, -v +, but not -v -
 
     HEREDOC
@@ -94,10 +94,10 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
       end
 
       parser.on('-m', '--mode MODE',
-                'Mode with which to handle input (-ms (default), -me, -mb, mn)') do |v|
+                'Mode with which to handle input (-ml, -me, -mb, -mn (default)') do |v|
 
         modes = {
-            's' => :string,
+            'l' => :line,
             'e' => :enumerator,
             'b' => :one_big_string,
             'n' => :no_input
@@ -192,7 +192,7 @@ class Rexe < Struct.new(:input_mode, :loads, :requires, :verbose, :noop)
     code = eval(source_code)
 
     actions = {
-        string:         -> { STDIN.each { |l| execute(l.chomp, code) } },
+        line:           -> { STDIN.each { |l| execute(l.chomp, code) } },
         enumerator:     -> { execute(STDIN.each_line, code) },
         one_big_string: -> { big_string = STDIN.read; execute(big_string, code) },
         no_input:       -> { execute(nil, code) }

data/rexe.gemspec

@@ -43,5 +43,20 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "rake", "~> 12.3"
   spec.add_development_dependency "rspec", "~> 3.0"
 
-  spec.post_install_message = "\n\nWARNING! The default input mode was changed from -ms to -mn in version 0.6.0.\n\n"
+  spec.post_install_message =
+      <<~HEREDOC
+      
+      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+      
+      WARNING!
+
+      The default rexe input mode was changed from -ms to -mn in version 0.6.0
+      and
+      the -ms (separate string mode) mode name was changed
+      to -ml (line mode) in version 0.9.0.
+
+      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+
+      HEREDOC
+
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: rexe
 version: !ruby/object:Gem::Version
-  version: 0.8.1
+  version: 0.9.0
 platform: ruby
 authors:
 - Keith Bennett
@@ -96,8 +96,16 @@ metadata:
   changelog_uri: https://github.com/keithrbennett/rexe/blob/master/README.md
 post_install_message: |2+
 
+  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 
-  WARNING! The default input mode was changed from -ms to -mn in version 0.6.0.
+  WARNING!
+
+  The default rexe input mode was changed from -ms to -mn in version 0.6.0
+  and
+  the -ms (separate string mode) mode name was changed
+  to -ml (line mode) in version 0.9.0.
+
+  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 
 rdoc_options: []
 require_paths: