iostreams 2.1.0 → 2.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d74053d56262767ee38729e8fc3810416b340ec7e1a318c37a95579f392a92d6
-  data.tar.gz: 7e6e2fa0088656a52a8fd9c45c78b96ed8b2c324e9f9379a091fcfdd3939dde9
+  metadata.gz: fedc839021ed6f9515ed91d584477debad754f05476169ee1bff4d0b5b161ee6
+  data.tar.gz: 8ac9d35f276306b6eb3feff0667229e0b8e5c61b7b37b05682c7a192f201a713
 SHA512:
-  metadata.gz: a492f9f89787eae3675fcf0aef1a1fbc9f67e461a0c1d0c9422888fa4cc66277c55d39248041b79a50de1b0d321a51a013445379de93404b0f85105d48914962
-  data.tar.gz: 2e6303a89a55bbbda4a9c73a19bdd6780823633afa4391d2fe3071ba8ca85e360a08214e76ed2e869182d8ed94388ed0454769b5d6dc41e540abf704d1c8a794
+  metadata.gz: 24aa6dd4c78acaed7bcc50d3c807a328e8f6828714fd17f8af245d68d32d8e5cc8e8810938e5f3f4ed66295369fc786b5c158fa2ef9a2d6037760a732bc35326
+  data.tar.gz: a2bd8ce524d37ab054de318b16176203515da7e24bf6907711fd521f5e7043ef55114c87529595a7fec727c73138b29baab55e80da1d20dee0c5b231dfabbe63

data/docs/config.md

@@ -85,6 +85,15 @@ IOStreams.root(:downloads).to_s
 # => "s3://my-app-bucket-name/downloads"
 ~~~
 
+The whole process shares a root, so set the streams, options, encoding, format and schema of a file on the path
+joined to the root, rather than on the root itself. A path joined to a root keeps the arguments of the root, such as
+its credentials or S3 options, but not the streams, options, encoding, format or schema set on the root:
+~~~ruby
+IOStreams.join("daily.csv.pgp", root: :uploads).option(:pgp, passphrase: "secret").each(:hash) do |record|
+  p record
+end
+~~~
+
 ## temp_dir
 
 When working with large files the standard temp file system location can be too small to handle downloading large
@@ -146,12 +155,7 @@ IOStreams also logs warnings for behavior that will change in the next major ver
 
 ## enforce_column_restrictions
 
-Applies `allowed_columns`, `required_columns` and `skip_unknown` to every input when reading
-records, including JSON records, so that renaming an uploaded file from `.csv` to `.json` cannot
-bypass them. Set it in an initializer:
-
-~~~ruby
-IOStreams.enforce_column_restrictions = true
-~~~
-
-Default: false. It will default to true in v3.0. See [Header options](formats#header-options).
+Deprecated in v2.2, and removed in v3.0, with the column options that it applies to every input,
+such as `allowed_columns`. A [schema](formats#schema) applies to every input instead, including the
+keys of each JSON record, see
+[Moving the column options to a schema](upgrading#moving-the-column-options-to-a-schema).

data/docs/copy_files.md

@@ -59,7 +59,7 @@ Custom stream conversions can be applied to both the source and the target in a
 Here the source is read as binary and the target is PGP encrypted:
 
 ~~~ruby
-source = IOStreams.path("source_file").stream(:encode, encoding: "BINARY")
+source = IOStreams.path("source_file").encoding("BINARY")
 IOStreams.path("target_file.pgp").option(:pgp, passphrase: "hello").copy_from(source)
 ~~~
 
@@ -72,4 +72,22 @@ IOStreams.path("target_file.json").copy_from(source, mode: :hash)
 ~~~
 
 Notes:
-* `mode` accepts `:line`, `:array`, or `:hash`, and only applies when `convert` is `true`.
+* `mode` accepts `:line`, `:array`, or `:hash`, and only applies when `convert` is `true`. Any other mode raises
+  `ArgumentError` before the source or the target is opened.
+* Without a `mode`, a copy into a file of another format, such as `people.csv` into `people.json` or `people.psv`,
+  copies the text of the source as it is, so that the target holds CSV text. This is deprecated, and reports a
+  deprecation warning, since IOStreams 3.0 converts the rows of such a copy into the format of the target, as the
+  `:array` mode does: supply `mode: :array` to convert the rows, or `mode: :hash` when either file is JSON, whose rows
+  are records, or `mode: :line` to copy the text as it is, which give the same result in both versions. A copy within
+  a format, such as `.csv.gz` into `.csv`, from or to a file whose format is not known, such as an IO or a `.txt` file,
+  or with `#raw`, is copied as it is in both, and so is a spreadsheet into CSV, since its text is CSV.
+
+## When a copy fails
+
+* The source is opened before the target, so a source that does not exist, or cannot be read, raises before the target
+  is changed.
+* A copy that raises before the target is opened, such as for a schema file that does not exist, leaves the target as
+  it was.
+* When the copy fails part way, the target is not left with part of the file: a local file is removed, and an S3 or
+  SFTP file is only uploaded once the copy has written all of it, so an existing S3 object is kept.
+* A move only deletes the source once the copy completed.

data/docs/extensions.md

@@ -42,7 +42,9 @@ IOStreams.path("sample.csv.bz2").option(:bz2, small: true).read
 
 Options are strict, so an option that a stream does not accept raises an `ArgumentError`.
 See [Streams](streams#reading-and-writing-need-separate-options). The exception is BZip2, which
-ignores any other option and logs a warning. In v3.0 it will raise an `ArgumentError` too.
+ignores any other option and logs a warning. One that neither reading nor writing takes also reports a
+deprecation warning when it is set, since in v3.0 it will raise an `ArgumentError` too, while v3.0
+ignores an option of the other direction, as BZip2 does.
 
 ## Reading an Excel Spreadsheet
 
@@ -56,23 +58,35 @@ end
 ~~~
 
 Notes:
+* Its rows are read as CSV, so leave the format of the path unset, or set it to `format(:csv)`.
 * Since the underlying `creek` gem operates on files, when reading from a stream (for example S3 or HTTP)
   the contents are first downloaded into a temp file.
 * Writing xlsx files is not supported.
 
 ## Character encoding
 
-The special `:encode` stream converts the character encoding of the data being read or written.
-It is applied with `option` or `stream` rather than a file name extension:
+The encode stream converts the character encoding of the text that the application reads or writes.
+Set it with `encoding`, which keeps the streams that the file name implies, and also works for a stream without a
+file name, or with streams set with `stream`:
 
 ~~~ruby
 IOStreams.path("sample.csv.gz").
-  option(:encode, encoding: "UTF-8", cleaner: :printable, replace: "").
+  encoding("UTF-8", cleaner: :printable, replace: "").
   each do |line|
     puts line
   end
 ~~~
 
+The encode stream is applied closest to the application, so above the data is decompressed first, and then
+converted. Each call to `encoding` merges its options with those already set, and `encoding` without any
+arguments returns the encoding that was set, or nil. An option that the encode stream does not take, or a value
+that it does not accept, such as an encoding that Ruby does not know, raises `ArgumentError` when it is set, before
+any file is written.
+
+`encoding` replaces `option(:encode, ...)`, `stream(:encode, ...)` and a file name ending in `.encode`, which also
+apply the encode stream. They are deprecated: each still works, and reports a deprecation warning, see
+[Upgrading](upgrading#deprecated-and-what-replaces-it).
+
 Options:
 
 * `encoding: [String|Encoding]`