iostreams 2.1.0 → 2.2.0

data/docs/path.md

@@ -309,7 +309,8 @@ end
 
   A username and password supplied in the url remain part of it, so `#to_s` returns them,
   as does any log or error message that includes the path. To keep them out of logs, supply them
-  with the `username:` and `password:` arguments instead.
+  with the `username:` and `password:` arguments instead, and log `#display_name`, which never
+  includes them.
 
 #### Optional Arguments:
 
@@ -378,7 +379,8 @@ Notes:
 
   A username and password supplied in the url remain part of it, so `#to_s` returns them,
   as does any log or error message that includes the path. To keep them out of logs, supply them
-  with the `username:` and `password:` arguments instead.
+  with the `username:` and `password:` arguments instead, and log `#display_name`, which never
+  includes them.
 
 #### Optional Arguments:
 
@@ -487,6 +489,20 @@ IOStreams.path("sample/data").mkpath
 Inspect the components of a path's file name:
 
 ~~~ruby
+# The full name of the path, without any user name, password or query in an SFTP or HTTP url,
+# for logging. Unlike #to_s it cannot be used to create the path again.
+IOStreams.path("sftp://jack:secret@sftp.example.org/data/ruby.rb").display_name
+# => "sftp://sftp.example.org/data/ruby.rb"
+
+# The options for creating a path, such as a password, with each secret replaced, for an application
+# that stores a url and its options, and displays them. The path class of the url decides which are
+# secret. `redact_stream_options` does the same for the options of streams.
+IOStreams.redact_path_options("sftp://sftp.example.org/data/ruby.rb", username: "jack", password: "secret")
+# => {username: "jack", password: "[FILTERED]"}
+
+IOStreams.redact_stream_options(pgp: {passphrase: "secret", recipient: "a@b.org"})
+# => {pgp: {passphrase: "[FILTERED]", recipient: "a@b.org"}}
+
 # The last component of the path.
 IOStreams.path("/home/gumby/work/ruby.rb").basename
 # => "ruby.rb"

data/docs/streams.md

@@ -21,6 +21,12 @@ IOStreams.path("example.csv").reader do |io|
 end
 ~~~
 
+The stream that the block receives answers the methods of an IO that read, such as `#read` and `#eof?`. Rely on
+those, rather than on the class of the stream, which depends on how the file is stored, such as a `Zlib::GzipReader`
+for a `.gz` file. Where a library needs a `File`, copy the data into a local file first, such as with
+`IOStreams.temp_file` and `#copy_to`, and read that file. To read the data from the start again, open the reader
+again.
+
 Read one line at a time from the file:
 ~~~ruby
 IOStreams.path("example.csv").each do |line|
@@ -50,8 +56,13 @@ Notes:
   tabular format quotes its fields, such as CSV (detected from a `.csv` file name or set explicitly via
   `.format(:csv)`). `embedded_within` only needs to be supplied for quoted formats that are not
   auto-detected, or to override the quote character.
+* When reading rows or records, with `each(:array)` or `each(:hash)`, the format of the file decides where
+  its lines are split, since its parser must agree with them, so set the format with `format` instead.
+  Supplying `embedded_within` there is deprecated, and reports a deprecation warning, see
+  [Seeing deprecation warnings](upgrading#seeing-deprecation-warnings).
 * A file that is named `.csv` but is actually pipe-delimited can avoid quote parsing by declaring its
-  real format with `.format(:psv)`, or by passing `embedded_within: nil` to disable it explicitly:
+  real format with `.format(:psv)`, or when reading lines, by passing `embedded_within: nil` to disable it
+  explicitly:
 ~~~ruby
 IOStreams.path("pipe_delimited.csv").format(:psv).each(:line) do |line|
   puts line
@@ -72,6 +83,23 @@ IOStreams.path("example.csv").each(:hash) do |hash|
 end
 ~~~
 
+The tabular format of the rows and records, such as CSV or JSON, is detected from the file name, or set with
+`#format`, see [File Formats](formats). Set a schema on the path with `#schema` to read each value as the type of
+its field, such as an `Integer` or a `Date`, with the names of the fields as the keys of each record, see
+[Schema](formats#schema):
+~~~ruby
+IOStreams.path("example.csv").schema(
+  fieldsMatch: "subset",
+  fields:      [
+    {name: "name"},
+    {name: "zip_code", type: "integer"}
+  ]
+).each(:hash) do |hash|
+  p hash
+end
+# => {"name" => "Jack", "zip_code" => 1234}
+~~~
+
 Write data to the file.
 ~~~ruby
 IOStreams.path("abc.txt").writer do |io|
@@ -258,10 +286,12 @@ IOStreams.path("example.csv.pgp").
 
 An option that neither direction accepts, such as a misspelled one, raises an `ArgumentError`
 that lists the valid options. The exception is BZip2, which ignores options it does not accept and
-logs a warning instead. In v3.0 it will raise an `ArgumentError` too.
+logs a warning instead. One that neither reading nor writing takes also reports a deprecation warning,
+since in v3.0 it will raise an `ArgumentError` too, while v3.0 ignores an option of the other direction.
 
 Options for a stream that is not in the pipeline are still ignored, as described above,
-since they are not passed to any reader or writer.
+since they are not passed to any reader or writer. One that neither direction accepts reports a
+deprecation warning when it is set, for every stream, since v3.0 raises an `ArgumentError` for it then.
 
 #### Stream
 
@@ -305,7 +335,8 @@ In this example the file contains JSON data that was compressed with Zip, and si
 ~~~ruby 
 IOStreams.path("tempfile2527").
   stream(:zip).
-  each(:hash, format: :json) do |row|
+  format(:json).
+  each(:hash) do |row|
     p row
   end
 ~~~
@@ -329,9 +360,23 @@ path.setting(:pgp)
 # => {:passphrase=>"receiver_passphrase"}
 ~~~
 
-To ensure no streams are inferred or applied use stream `:none` 
-~~~ruby 
-path = IOStreams.path("file.zip")
-path.stream(:none)
-path.read
+#### Raw
+
+To read or write the data as it is stored, applying no streams, neither those that the file name implies nor any
+set with `option` or `stream`, and no encoding, use `raw`. For example, to checksum a compressed file as it is
+stored, or to write data that is already compressed:
+
+~~~ruby
+Digest::SHA256.hexdigest(IOStreams.path("data.csv.gz").raw.read)
+
+IOStreams.path("data.csv.gz").raw.write(gzipped_data)
 ~~~
+
+Setting a stream, option or encoding after `raw` raises `ArgumentError`, since the data is as stored.
+`raw?` returns whether `raw` was called. `raw` replaces `stream(:none)`, which is deprecated: it still works, and
+reports a deprecation warning, see [Upgrading](upgrading#deprecated-and-what-replaces-it).
+
+To copy a file as it is stored, such as downloading a zip file without unzipping it, use
+`copy_to(target, convert: false)` instead, since the target applies its own streams. To read a file that is not
+named for its format, such as a CSV file called `data.txt`, set the file name that names its format, for example
+`file_name("data.csv")`.

data/docs/tutorial.md

@@ -429,8 +429,20 @@ Using `writer(:hash)` makes it easier to develop the application without regard
 - Specialized escaping of values to handle row or column delimiters
 
 Note: The first row written determines the column names as well as the order of the elements to be written.
-To supply the header columns up front, to set the order or to filter out which columns should be written to the
-target file, pass them to the writer: `path.writer(:hash, columns: ["name", "login"])`.
+To set the columns up front, in your own order, describe them with a [schema](formats#schema) on the path: its
+fields are the columns, in order, and `fieldsMatch: "subset"` skips the other keys of each record:
+~~~ruby
+path = IOStreams.path("sample/logins.csv").schema(
+  fieldsMatch: "subset",
+  fields:      [{name: "login"}, {name: "name"}]
+)
+path.writer(:hash) do |io|
+  io << {"name" => "Jack Jones", "login" => "jjones", "notes" => "Not written"}
+  io << {"login" => "jsmith", "name" => "Jill Smith"}
+end
+path.read
+# => "login,name\njjones,Jack Jones\njsmith,Jill Smith\n"
+~~~
 
 Now lets write the same data into a JSON file, then read it to see what it looks like:
 ~~~ruby