tableschema 0.3.1 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 846c3cf9cf67190ece4602c3ecc26124789d6bfe
-  data.tar.gz: 6c2f2101cb63bad02b025281be54d85da91dcdcf
+  metadata.gz: be0ea32c71fc75dd1acca11a181b3fe8a7b69e33
+  data.tar.gz: e73959a568fd604b31fbe72376b9f2987095602c
 SHA512:
-  metadata.gz: 90f9e6af27235f1cf4e8509c1133c33840a695c2cb000ca70ab1bff5bf8e8e0cea8aec311fc704a58fbb13ea9cab7b58fb4758d73fa0bbb8f79cd4ea4ecdb6b0
-  data.tar.gz: abb5bbab98a9431458b4962ede074149e16da506d7366f9857dc4cc6b85f876a2fd7e0fa0990f8da71c7d81566d8f7c298e971cba55b9a611bea4c683fb12d3c
+  metadata.gz: 2795a5b5d62696987588e9dfd7970c5539b68385d8e20eba1f92249645a7a72d338c5a44e96514531e562f7fdec0a40b49c3fd095665a027a4763e21e43a0fb9
+  data.tar.gz: e145c763e64f6384cadf6d8b3fdfeae01b562c50a1243def427036f81d2f6f39c95dd22837cdddc0844e2e4287b2aae71d24b57101af63b62908090093abcf11

data/.rubocop.yml

@@ -0,0 +1,21 @@
+AllCops:
+  DisabledByDefault: true
+  Exclude:
+    - 'lib/tableschema/exceptions.rb'
+
+Security:
+  Enabled: true
+
+Lint:
+  Enabled: true
+
+Style/HashSyntax:
+  Enabled: true
+  EnforcedStyle: ruby19_no_mixed_keys
+
+Style/MutableConstant:
+  Enabled: true
+
+Metrics/CyclomaticComplexity:
+  Max: 10
+  Severity: error

data/.travis.yml

@@ -1,9 +1,23 @@
 ---
 language: ruby
+
 rvm:
 - 2.3.1
 - 2.4.1
-before_install: gem install bundler -v 1.11.2
+
+before_install:
+  gem install bundler -v 1.11.2
+
+install:
+  - bundle
+  - gem install rubocop
+
+script:
+  - rake spec
+
+after_success:
+  - rubocop
+
 deploy:
   provider: rubygems
   api_key:

data/README.md

@@ -33,24 +33,25 @@ Since version 0.3 the library was renamed `tableschema` and has a gem with the s
 The gem `jsontableschema` is no longer maintained. Here are the steps to transition your code to `tableschema`:
 
 1. Replace
-   ```ruby
-   gem 'jsontableschema'
-   ```
-  with
 
-  ```ruby
-  gem 'tableschema', '0.3.0'
-  ```
+    ```ruby
+    gem 'jsontableschema'
+    ```
+    with
+
+    ```ruby
+    gem 'tableschema', '0.3.0'
+    ```
 
 2. Replace module name `JsonTableSchema` with module name `TableSchema`. For example:
 
-  ```ruby
-  JsonTableSchema::Table.infer_schema(csv)
-  ```
-  with
-  ```ruby
-  TableSchema::Table.infer_schema(csv)
-  ```
+    ```ruby
+    JsonTableSchema::Table.infer_schema(csv)
+    ```
+    with
+    ```ruby
+    TableSchema::Table.infer_schema(csv)
+    ```
 
 ## Usage
 
@@ -60,27 +61,40 @@ Validate and cast data from a CSV as described by a schema.
 
 ```ruby
 schema = {
-    "fields": [
+    fields: [
         {
-            "name" => "id",
-            "title" => "Identifier",
-            "type" => "integer"
+            name: 'id',
+            title: 'Identifier',
+            type: 'integer'
         },
         {
-            "name" => "title",
-            "title" => "Title",
-            "type" => "string"
+            name: 'title',
+            title: 'Title',
+            type: 'string'
         }
     ]
-} # Can also be a URL or a path
+}
 
-csv = 'https://github.com/frictionlessdata/tableschema-rb/raw/master/spec/fixtures/simple_data.csv' # Can also be a url or array of arrays
+csv = 'https://github.com/frictionlessdata/tableschema-rb/raw/master/spec/fixtures/simple_data.csv'
 
 table = TableSchema::Table.new(csv, schema)
-table.rows
+
+# Iterate through rows
+table.iter{ |row| print row }
+# [1, "foo"]
+# [2, "bar"]
+# [3, "baz"]
+
+# Read the entire CSV in memory
+table.read
 #=> [[1,'foo'],[2,'bar'],[3,'baz']]
 ```
 
+Both `iter` and `read` take the optional parameters:
+- `row_limit`: integer, default `nil` - stop at this many rows
+- `cast`: boolean, default `true` - cast values for each row
+- `keyed`: boolean, default: `false` - return the rows as Hashes with headers as keys
+
 ### Infer a schema
 
 If you don't have a schema for a CSV, and want to generate one, you can infer a schema like so:
@@ -90,95 +104,69 @@ csv = 'https://github.com/frictionlessdata/tableschema-rb/raw/master/spec/fixtur
 
 table = TableSchema::Table.infer_schema(csv)
 table.schema
-#=> {"fields"=>[{"name"=>"id", "title"=>"", "description"=>"", "type"=>"string", "format"=>"default"}, {"name"=>"title", "title"=>"", "description"=>"", "type"=>"string", "format"=>"default"}]}
+#=> {:fields=>[{:name=>"id", :title=>"", :description=>"", :type=>"integer", :format=>"default", :constraints=>{}}, {:name=>"title", :title=>"", :description=>"", :type=>"string", :format=>"default", :constraints=>{}}]}
 ```
 
-### Validate a schema
+### Build a Schema
 
-To validate that a schema meets the JSON Table Schema spec, you can pass a schema to the initializer like so:
+You can also build a schema from scratch or modify an existing one:
 
 ```ruby
-schema_hash = {
-  "fields" => [
-      {
-          "name" => "id"
-      },
-      {
-          "name" => "height"
-      }
-  ]
-}
-
-schema = TableSchema::Schema.new(schema_hash)
-schema.valid?
-#=> true
+schema = TableSchema::Schema.new({
+  fields: [],
+})
+
+# Add a field
+schema.add_field({
+  name: 'id',
+  type: 'string',
+  constraints: {
+    required: true,
+  }
+})
+
+# Remove a field
+schema.remove_field('id')
 ```
 
-You can also pass a file path or URL to the initializer:
+`add_field` will ignore the updates if the updated version of the the schema fails [validation](#validate-a-schema).
+If you wish to prevent an invalid schema from being created or updated by raising validation errors, you can pass the `strict: true` argument to the Schema initializer:
 
 ```ruby
-schema = TableSchema::Schema.new('http://example.org/schema.json')
-schema.valid?
-#=> true
+schema = TableSchema::Schema.new(schema_hash, strict: true)
 ```
 
-If the schema is invalid, you can access the errors via the `messages` attribute
+There are multiple methods to inspect a schema:
 
 ```ruby
 schema_hash = {
-  "fields" => [
+  fields: [
     {
-      "name"=>"id",
-      "title"=>"Identifier",
-      "type"=>"integer"
-   },
-   {
-     "name"=>"title",
-     "title"=>"Title",
-     "type"=>"string"
-    }
-  ],
- "primaryKey"=>"identifier"
-}
-
-schema.valid?
-#=> false
-schema.messages
-#=> ["The JSON Table Schema primaryKey value `identifier` is not found in any of the schema's field names"]
-```
-
-## Schema Model
-
-You can also access the schema via a Ruby model, with some useful methods for interaction:
-
-```ruby
-schema_hash = {
-  "fields" => [
-      {
-          "name" => "id",
-          "type" => "string",
-          "constraints" => {
-            "required" => true,
-          }
+      name: 'id',
+      type: 'string',
+      constraints: {
+        required: true,
       },
-      {
-          "name" => "height",
-          "type" => "number"
-      }
+    },
+    {
+      name: 'height',
+      type: 'number',
+    },
+    {
+      name: 'state',
+    },
   ],
-  "primaryKey" => "id",
-  "foreignKeys" => [
+  primaryKey: 'id',
+  foreignKeys: [
     {
-        "fields" => "state",
-        "reference" => {
-            "datapackage" => "http://data.okfn.org/data/mydatapackage/",
-            "resource" => "the-resource",
-            "fields" => "state_id"
-        }
-    }
+      fields: 'state',
+      reference: {
+          resource: 'the-resource',
+          fields: 'state_id',
+      },
+    },
   ]
 }
-
 schema = TableSchema::Schema.new(schema_hash)
 
 schema.headers
@@ -186,79 +174,126 @@ schema.headers
 schema.required_headers
 #=> ["id"]
 schema.fields
-#=> [{"name"=>"id", "constraints"=>{"required"=>true}, "type"=>"string", "format"=>"default"}, {"name"=>"height", "type"=>"number", "format"=>"default"}]
+#=> [{:name=>"id", :type=>"string", :constraints=>{:required=>true}, :format=>"default"}, {:name=>"height", :type=>"number", :format=>"default", :constraints=>{}}]
 schema.primary_keys
 #=> ["id"]
 schema.foreign_keys
-#=> [{"fields" => "state", "reference" => { "datapackage" => "http://data.okfn.org/data/mydatapackage/", "resource" => "the-resource", "fields" => "state_id" } } ]
+# => [{:fields=>"state", :reference=>{:resource=>"the-resource", :fields=>"state_id"}}]
 schema.get_field('id')
-#=> {"name"=>"id", "constraints"=>{"required"=>true}, "type"=>"string", "format"=>"default"}
+# => {:name=>"id", :type=>"string", :constraints=>{:required=>true}, :format=>"default"}
 schema.has_field?('foo')
 #=> false
 schema.get_type('id')
 #=> 'string'
 schema.get_fields_by_type('string')
-#=> [{"name"=>"id", "constraints"=>{"required"=>true}, "type"=>"string", "format"=>"default"}, {"name"=>"height", "type"=>"string", "format"=>"default"}]
+# => [{:name=>"id", :type=>"string", :constraints=>{:required=>true}, :format=>"default"}, {:name=>"state", :type=>"string", :format=>"default", :constraints=>{}}]
 schema.get_constraints('id')
-#=> {"required" => true}
-schema.cast_row(['string', '10.0'])
-#=> ['string', 10.0]
-schema.cast([['foo', '12.0'],['bar', '10.0']])
-#=> [['foo', 12.0],['bar', 10.0]]
+# => {:required=>true}
+```
+
+#### Cast row
+
+To check if a given set of values complies with the schema, you can use `cast_row`:
+
+```
+schema.cast_row(['string', '10.0', 'State'])
+#=> ['string', 10.0, 'State']
+```
+
+By default the converter will fail on the first error it finds. However, by passing `fail_fast: false` as the second argument the errors will be collected into an `exception.errors` attribute for you to review later. For example:
+
+```ruby
+row = [3, 'nan', 'State']
+
+schema.cast_row(row)
+#=> TableSchema::InvalidCast: 3 is not a string
+begin
+  schema.cast_row(row, fail_fast: false)
+rescue TableSchema::MultipleInvalid => exception
+  exception.errors
+end
+#=> #<Set: {#<TableSchema::InvalidCast: 3 is not a string>,
+            #<TableSchema::InvalidCast: nan is not a number>}>
 ```
 
-When casting a row (using `cast_row`), or a number of rows (using `cast`), by default the converter will fail on the first error it finds. If you pass `false` as the second argument, the errors will be collected into a `errors` attribute for you to review later. For example:
+### Validate a schema
+
+To make sure a schema complies with [Table Schema spec](https://specs.frictionlessdata.io/table-schema), we validate each custom schema against the
+official [Table Schema schema](https://specs.frictionlessdata.io/schemas/table-schema.json):
 
 ```ruby
 schema_hash = {
-  "fields" => [
-      {
-          "name" => "id",
-          "type" => "string",
-          "constraints" => {
-            "required" => true,
-          }
-      },
-      {
-          "name" => "height",
-          "type" => "number"
-      }
+  fields: [
+      { name: 'id' },
   ]
 }
-
 schema = TableSchema::Schema.new(schema_hash)
+schema.validate
+#=> true
+```
 
-rows = [
-  ['foo', 'notanumber'],
-  ['bar', 'notanumber'],
-  ['wrong column count']
-]
+If the schema is invalid, you can access the errors via the `errors` attribute
 
-schema.cast(rows)
-#=> TableSchema::InvalidCast: notanumber is not a number
-schema.cast(rows, false)
-#=> TableSchema::MultipleInvalid
+```ruby
+schema_hash = {
+  fields: [
+    {
+      name: 'id',
+      title: 'Identifier',
+      type: 'integer'
+    },
+    {
+      name: 'title',
+      title: 'Title',
+      type: 'string'
+    }
+  ],
+  primaryKey: 'identifier'
+}
+
+schema = TableSchema::Schema.new(schema_hash)
+schema.validate
+#=> false
 schema.errors
-#=> [#<TableSchema::InvalidCast: notanumber is not a number>, #<TableSchema::InvalidCast: notanumber is not a number>, #<TableSchema::ConversionError: The number of items to convert (1) does not match the number of headers in the schema (2)>]
+#=> #<Set: {"The TableSchema primaryKey value `identifier` is not found in any of the schema's field names"}>
+
+# Raise error if validation fails
+schema.validate!
+#=> TableSchema::SchemaException: The TableSchema primaryKey value `identifier` is not found in any of the schema's field names
 ```
 
 ## Field
 
+Data values can be cast to native Ruby objects with a Field instance. This allows formats and constraints to be defined for the field in the [field descriptor](https://specs.frictionlessdata.io/table-schema/#field-descriptors):
+
 ```ruby
 # Init field
-field = TableSchema::Field.new({'type': 'number'})
+field = TableSchema::Field.new({
+  name: 'over_1700',
+  type: 'number',
+  constraints: {
+    minimum: '1700',
+  },
+})
 
 # Cast a value
 field.cast_value('12345')
 #=> 12345.0
 ```
 
-Data values can be cast to native Ruby objects with a Field instance. Type instances can be initialized with f[ield descriptors](http://dataprotocols.org/json-table-schema/#field-descriptors). This allows formats and constraints to be defined.
+Casting a value will check the value is of the expected `type`, is in the correct `format`, and complies with any `constraints` imposed in the descriptor.
 
-Casting a value will check the value is of the expected type, is in the correct format, and complies with any constraints imposed by a schema. E.g. a date value (in ISO 8601 format) can be cast with a DateType instance. Values that can't be cast will raise an `InvalidCast` exception.
+Value that can't be cast will raise an `InvalidCast` exception.
 
 Casting a value that doesn't meet the constraints will raise a `ConstraintError` exception.
 
+```ruby
+field.cast_value('nan')
+#=> TableSchema::InvalidCast: nan is not a number
+field.cast_value('1200')
+#=> TableSchema::ConstraintError: The field `over_1700` must not be less than 1700
+```
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.