portable 1.0.0.pre.alpha.1 → 1.0.0.pre.alpha.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 112f223aca7d26dda5b4e9c93b1e83174885e941806de503c2d15be37047cdc1
-  data.tar.gz: 7d9764151388559dd0ac999b69fa4bed16ff9d02fdb796944ff4d22b30b5056d
+  metadata.gz: ab583e6beab687062935129db41650e353f23ec7ca2af9c610f49d8ac60a4811
+  data.tar.gz: 5999aab0a2a8bb37337d0923a2c06bffe9876dd8e8b4b879b799112090b4f393
 SHA512:
-  metadata.gz: 0bb98a4e53e669c13fbf361d2b3e5be4572ce2b42ab99e43b555ecc8db7c89c25cd28b653b61bc1b3ab2ace6688886ba0f1d45f04d4d1bcb3b178d7183adad6a
-  data.tar.gz: 14ac7eebf7b0af0b6553317e1d9f1f51757554ad43b9f0827a036ed12ff9ff9b67c5e2feb5db03566cce6ce43783ad8a24e0aed42a2a5367e306283531cb0214
+  metadata.gz: b485fead56c787bc169135b0c084289fcc690c49a9bface4f5d394c5494ed1139096ea5da5916001a2a845a1817c057f40d8d498d005288958e1792ebfa96b72
+  data.tar.gz: 93a3912588240c7c3172e06f47d8226627e50016ab9444a43c4d60aa536919542e8cbf35cb6e7369a226e15796b1b25e9af77e168dcb16a3c1a8a6193a8b481b

data/.rubocop.yml

@@ -20,7 +20,7 @@ Metrics/MethodLength:
   Max: 30
 
 Metrics/AbcSize:
-  Max: 16
+  Max: 20
 
 Metrics/ClassLength:
   Max: 125

data/.travis.yml

@@ -2,8 +2,6 @@ env:
   global:
     - CC_TEST_REPORTER_ID=f40f0e6f9946420b05b247f1640b2f5fcef181ca86659a2e71c747f790fcecdd
 language: ruby
-services:
-  - mysql
 rvm:
   # Build on the latest stable of all supported Rubies (https://www.ruby-lang.org/en/downloads/):
   - 2.5.8

data/README.md

@@ -2,18 +2,9 @@
 
 [![Gem Version](https://badge.fury.io/rb/portable.svg)](https://badge.fury.io/rb/portable) [![Build Status](https://travis-ci.org/bluemarblepayroll/portable.svg?branch=master)](https://travis-ci.org/bluemarblepayroll/portable) [![Maintainability](https://api.codeclimate.com/v1/badges/4b47ce94b0c9d889e648/maintainability)](https://codeclimate.com/github/bluemarblepayroll/portable/maintainability) [![Test Coverage](https://api.codeclimate.com/v1/badges/4b47ce94b0c9d889e648/test_coverage)](https://codeclimate.com/github/bluemarblepayroll/portable/test_coverage) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-This library provides a configuration layer that allows you to express transformations, using [Realize](https://github.com/bluemarblepayroll/realize), and will write the transformed data down to disk.  Essentially it is meant to be the transformation and load steps within a larger ETL system. We currently use this in production paired up with [Dbee](https://github.com/bluemarblepayroll/dbee) to go from configurable data model + query to file.
+Portable is a virtual document object modeling library.  Out of the box is provides a CSV writer but others for other formats like Microsoft Excel could easily be implemented and used.
 
-Current limitations:
-
-1. Only supports CSV with limited options
-2. Only supports writing to local file system.
-
-Future extension considerations:
-
-1. Support Excel and richer formatting, sheets, etc.
-2. Expand CSV options: delimiter, forcing quotes, etc.
-3. Support PDF
+This library utilizes [the Realize library](https://github.com/bluemarblepayroll/realize) that allows you to express transformation pipelines.  Essentially this library is meant to be the transformation and load steps within a larger ETL system. We currently use this in production paired up with [Dbee](https://github.com/bluemarblepayroll/dbee) for data sources to go from configurable data model + query to file.
 
 ## Installation
 
@@ -33,36 +24,38 @@ bundle add portable
 
 ### Getting Started Writing CSV Files
 
-Consider the following data set as an array of hashes:
+Consider the following data provider/source:
 
 ````ruby
 patients = [
   { first: 'Marky', last: 'Mark', dob: '2000-04-05' },
   { first: 'Frank', last: 'Rizzo', dob: '1930-09-22' }
 ]
+
+data_provider = Portable::Data::Provider.new(
+  data_sources: {
+    data_rows: patients,
+    fields: %i[first last dob]
+  }
+)
 ````
 
-We could configure an export like so:
+**Note:** Data::Provider and Data::Source objects are pretty basic, on purpose, so they can be easily re-implemented based on an application's specific needs.
+
+We could configure the most basic document like so:
 
 ````ruby
-document = {
-  data_table: {
-    columns: [
-      { header: :first },
-      { header: :last },
-      { header: :dob }
-    ]
-  }
-}
+document = nil # or {} or Portable::Document.new
 ````
 
-And execute the export against the example dataset in order to generate a CSV file:
+The above document says I would like a document with one sheet, and since I did not provide a data_table specification, I would like all the fields emitted from the data source.
 
-````ruby
-writer   = Portable::Csv::Writer.new(export)
-filename = File.join('tmp', 'patients.csv')
+Combining a document + writer + data provider yields a set of documents (it may be more than one if the writer does not know how to write intra-file sheets, i.e. CSV files.)
 
-writer.open(filename) { |writer| writer.write_all(patients) }
+````ruby
+writer    = Portable::Writers::Csv.new(document)
+name      = File.join('tmp', 'patients.csv')
+written   = writer.write!(filename: name, data_provider: data_provider)
 ````
 
 We should now have a CSV file at tmp/patients.csv that looks like this:
@@ -80,29 +73,33 @@ Let's expand our CSV example above with different headers and date formatting:
 
 ````ruby
 document = {
-  data_table: {
-    columns: [
-      {
-        header: 'First Name',
-        transformers: [
-          { type: 'r/value/resolve', key: :first }
-        ]
-      },
-      {
-        header: 'Last Name',
-        transformers: [
-          { type: 'r/value/resolve', key: :last }
-        ]
-      },
-      {
-        header: 'Date of Birth',
-        transformers: [
-          { type: 'r/value/resolve', key: :dob },
-          { type: 'r/format/date', output_format: '%m/%d/%Y' },
+  sheets: [
+    {
+      data_table: {
+        columns: [
+          {
+            header: 'First Name',
+            transformers: [
+              { type: 'r/value/resolve', key: :first }
+            ]
+          },
+          {
+            header: 'Last Name',
+            transformers: [
+              { type: 'r/value/resolve', key: :last }
+            ]
+          },
+          {
+            header: 'Date of Birth',
+            transformers: [
+              { type: 'r/value/resolve', key: :dob },
+              { type: 'r/format/date', output_format: '%m/%d/%Y' },
+            ]
+          }
         ]
       }
-    ]
-  }
+    }
+  ]
 }
 ````
 
@@ -117,52 +114,51 @@ Realize is also [pluggable](https://github.com/bluemarblepayroll/realize#pluggin
 
 ### Options
 
-Each writer can have its own set of options.
+Each writer can choose how and which options to support.
 
 #### CSV Options
 
 The following options are available for customizing CSV documents:
 
-* byte_order_mark: (optional, default is nothing): This option will write out a byte order mark identifying the encoding for the file.  This is useful for ensuring applications like Microsoft Excel open CSV files properly.  See Portable::Csv::ByteOrderMark constants for acceptable values.
+* byte_order_mark: (optional, default is nothing): This option will write out a byte order mark identifying the encoding for the file.  This is useful for ensuring applications like Microsoft Excel open CSV files properly.  See Portable::Modeling::ByteOrderMark constants for acceptable values.
 
-### Custom Header/Footer Rows
+### Static Header/Footer Rows
 
-The main document model can also include statically defined rows to place either at the header (above data table) or footer (below data table) locations.  For example:
+The main document model can also include statically defined rows to place either at the header (above data table) or footer (below data table) locations.  You can also have the data_source inject static header and footer rows as well.  For example:
 
 ````ruby
-document = {
-  data_table: {
-    columns: [
-      {
-        header: 'First Name',
-        transformers: [
-          { type: 'r/value/resolve', key: :first }
-        ]
-      },
-      {
-        header: 'Last Name',
-        transformers: [
-          { type: 'r/value/resolve', key: :last }
-        ]
-      },
-      {
-        header: 'Date of Birth',
-        transformers: [
-          { type: 'r/value/resolve', key: :dob },
-          { type: 'r/format/date', output_format: '%m/%d/%Y' },
-        ]
-      }
+patients = [
+  { first: 'Marky', last: 'Mark', dob: '2000-04-05' },
+  { first: 'Frank', last: 'Rizzo', dob: '1930-09-22' }
+]
+
+data_provider = Portable::Data::Provider.new(
+  data_sources: {
+    data_rows: patients,
+    fields: %i[first last dob],
+    header_rows: [
+      %w[FIRST_START LAST_START DOB_START]
+    ],
+    footer_rows: [
+      %w[FIRST_END LAST_END DOB_END]
     ]
-  },
-  header_rows: [
-    [ 'Run Date', '04/05/2000' ],
-    [ 'Run By', 'Hops the Bunny' ],
-    [],
-    [ 'BEGIN' ]
-  ],
-  header_rows: [
-    [ 'END' ]
-  ],
+  }
+)
+
+document = {
+  sheets: [
+    {
+      header_rows: [
+        [ 'Run Date', '04/05/2000' ],
+        [ 'Run By', 'Hops the Bunny' ],
+        [],
+        [ 'BEGIN' ]
+      ],
+      footer_rows: [
+        [ 'END' ]
+      ]
+    }
+  ]
 }
 ````
 

data/lib/portable.rb

@@ -8,6 +8,7 @@
 #
 
 require 'acts_as_hashable'
+require 'benchmark'
 require 'csv'
 require 'fileutils'
 require 'forwardable'
@@ -15,7 +16,11 @@ require 'objectable'
 require 'realize'
 require 'time'
 
-require_relative 'portable/data_table'
+# Shared modules/classes
+require_relative 'portable/util'
+
+# Main implementation points
+require_relative 'portable/data'
 require_relative 'portable/document'
-require_relative 'portable/transformer'
+require_relative 'portable/rendering'
 require_relative 'portable/writers'

data/lib/portable/data.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'data/provider'

data/lib/portable/data/provider.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'source'
+
+module Portable
+  module Data
+    # Container of data sources that is inputted into a writer alongside a document.
+    # It contains all the data sources the writer will use to render a document.
+    class Provider
+      include Util::Pivotable
+      include Util::Uniqueness
+      acts_as_hashable
+
+      def initialize(data_sources: [])
+        sources               = Source.array(data_sources)
+        @data_sources_by_name = pivot_by_name(sources)
+
+        assert_no_duplicate_names(sources)
+
+        freeze
+      end
+
+      # Fail hard if we cannot identify which data source to use.  This should help prevent
+      # possible configuration issues (i.e. typos.)
+      def data_source(name)
+        data_sources_by_name[name.to_s] ||
+          raise(ArgumentError, "data source: '#{name}' cannot be found.")
+      end
+
+      private
+
+      attr_reader :data_sources_by_name
+    end
+  end
+end

data/lib/portable/data/source.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Data
+    # A single source of data.  This is meant to serve as an interface / example implementation
+    # with the intention of being re-implemented within applications.  For example, you may
+    # decide more database data sources would be better, so it could be connected to ORMs or
+    # other data adapters; all it really needs to provide is enumerables for each attribute.
+    class Source
+      acts_as_hashable
+
+      attr_reader :header_rows,
+                  :footer_rows,
+                  :data_rows,
+                  :fields,
+                  :name
+
+      # Individial header and footer rows are arrays, while individual data_rows is an object
+      # like a hash, Struct, OpenStruct, or really any PORO.
+      def initialize(name: '', header_rows: [], footer_rows: [], data_rows: [], fields: [])
+        @name        = name.to_s
+        @header_rows = header_rows || []
+        @footer_rows = footer_rows || []
+        @data_rows   = data_rows   || []
+        @fields      = fields      || []
+
+        freeze
+      end
+    end
+  end
+end

data/lib/portable/document.rb

@@ -7,29 +7,43 @@
 # LICENSE file in the root directory of this source tree.
 #
 
+require_relative 'modeling/options'
+require_relative 'modeling/sheet'
+
 module Portable
-  # Base document object model defining what all documents should include.
+  # Top-level object model for a renderable document.
   class Document
+    include Util::Pivotable
+    include Util::Uniqueness
     acts_as_hashable
-    extend Forwardable
-
-    attr_reader :data_table,
-                :footer_rows,
-                :header_rows
 
-    def_delegators :data_table,
-                   :columns,
-                   :headers,
-                   :include_headers?,
-                   :headers,
-                   :transform
+    attr_reader :options
 
-    def initialize(data_table: {}, footer_rows: [], header_rows: [])
-      @data_table  = Datagrid.make(data_table)
-      @footer_rows = footer_rows || []
-      @header_rows = header_rows || []
+    def initialize(sheets: [], options: {})
+      @sheets_by_name = make_unique_sheets_by_name(sheets)
+      @options        = Modeling::Options.make(options, nullable: false)
 
       freeze
     end
+
+    def sheet(name)
+      sheets_by_name.fetch(name.to_s)
+    end
+
+    def sheets
+      sheets_by_name.values
+    end
+
+    private
+
+    attr_reader :sheets_by_name
+
+    def make_unique_sheets_by_name(sheets)
+      sheets = Modeling::Sheet.array(sheets)
+      sheets << Modeling::Sheet.new if sheets.empty?
+
+      assert_no_duplicate_names(sheets)
+      pivot_by_name(sheets)
+    end
   end
 end

data/lib/portable/{csv → modeling}/byte_order_mark.rb

@@ -8,7 +8,7 @@
 #
 
 module Portable
-  module Csv
+  module Modeling
     # Define all acceptable byte order mark values.
     module ByteOrderMark
       UTF_8    = "\xEF\xBB\xBF"

data/lib/portable/modeling/column.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Modeling
+    # Defines all the options a column can contain.  The most basic would to just include a header
+    # (defaults to '').  If no transformers are defined then a simple resolver using the header
+    # will be used.  This works well for pass-through file writes.  Use the transformers to further
+    # customize each data point being written.
+    class Column
+      acts_as_hashable
+
+      DEFAULT_TRANSFORMER_TYPE = 'r/value/resolve'
+
+      attr_reader :header, :transformers
+
+      def initialize(header: '', transformers: [])
+        @header       = header.to_s
+        @transformers = Realize::Transformers.array(transformers)
+
+        @transformers << default_transformer if @transformers.empty?
+
+        freeze
+      end
+
+      private
+
+      def default_transformer
+        Realize::Transformers.make(type: DEFAULT_TRANSFORMER_TYPE, key: header)
+      end
+    end
+  end
+end

data/lib/portable/modeling/data_table.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'column'
+
+module Portable
+  module Modeling
+    # Defines all the options for the data grid within an export like columns, whether or not
+    # you want to include headers, and more.
+    class DataTable
+      acts_as_hashable
+
+      attr_reader :auto, :columns, :include_headers
+
+      alias include_headers? include_headers
+      alias auto? auto
+
+      def initialize(auto: true, columns: [], include_headers: true)
+        @auto            = auto || false
+        @columns         = Column.array(columns)
+        @include_headers = include_headers || false
+
+        freeze
+      end
+    end
+  end
+end

data/lib/portable/{csv → modeling}/options.rb

@@ -10,7 +10,7 @@
 require_relative 'byte_order_mark'
 
 module Portable
-  module Csv
+  module Modeling
     # Defines all the options for an export including static header rows, footer rows, and how
     # to draw the data table.
     class Options

data/lib/portable/modeling/sheet.rb

@@ -0,0 +1,62 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'data_table'
+
+module Portable
+  module Modeling
+    # Abstract concept modeling for the notion of a "sheet" in a "document".  This means different
+    # things given the writer.  For example, all writers should support multiple sheets but
+    # there is no internal representation of a "sheet" within a CSV, so each sheet will emit
+    # one file.
+    class Sheet
+      acts_as_hashable
+      extend Forwardable
+
+      def_delegators :data_table,
+                     :auto?,
+                     :columns,
+                     :include_headers?
+
+      attr_reader :data_source_name,
+                  :data_table,
+                  :footer_rows,
+                  :header_rows,
+                  :name
+
+      def initialize(
+        data_source_name: '',
+        data_table: nil,
+        footer_rows: [],
+        header_rows: [],
+        name: ''
+      )
+        @data_source_name = decide_data_source_name(data_source_name, name)
+        @name             = name.to_s
+        @data_table       = DataTable.make(data_table, nullable: false)
+        @footer_rows      = footer_rows || []
+        @header_rows      = header_rows || []
+
+        freeze
+      end
+
+      # Use exact name if possible, if not then use the sheet name or else use the
+      # "default" one (noted by a blank name).
+      def decide_data_source_name(data_source_name, sheet_name)
+        if !data_source_name.to_s.empty?
+          data_source_name.to_s
+        elsif !sheet_name.to_s.empty?
+          sheet_name.to_s
+        else
+          ''
+        end
+      end
+    end
+  end
+end

data/lib/portable/rendering.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'rendering/sheet'

data/lib/portable/rendering/row.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Rendering
+    # Internal intermediary class that knows how to combine columns specification
+    # instances with their respective Realize pipelines.
+    class Row # :nodoc: all
+      attr_reader :column_pipelines, :resolver
+
+      def initialize(columns, resolver: Objectable.resolver)
+        @resolver = resolver
+
+        @column_pipelines = columns.each_with_object({}) do |column, memo|
+          memo[column] = Realize::Pipeline.new(column.transformers, resolver: resolver)
+        end
+
+        freeze
+      end
+
+      def render(object, time)
+        column_pipelines.each_with_object({}) do |(column, pipeline), memo|
+          memo[column.header] = pipeline.transform(object, time)
+        end
+      end
+
+      def columns
+        column_pipelines.keys
+      end
+
+      def headers
+        columns.map(&:header)
+      end
+
+      def merge(other)
+        self.class.new(columns + other.columns, resolver: resolver)
+      end
+    end
+  end
+end

data/lib/portable/rendering/sheet.rb

@@ -0,0 +1,53 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'row'
+
+module Portable
+  module Rendering
+    # Understands the connection between a document's sheets and the internal row renderer
+    # necessary to render each sheet's data table.
+    class Sheet # :nodoc: all
+      attr_reader :document, :resolver
+
+      def initialize(document, resolver: Objectable.resolver)
+        @document = Document.make(document, nullable: false)
+        @resolver = resolver
+
+        @row_renderers = @document.sheets.each_with_object({}) do |sheet, memo|
+          memo[sheet.name] = Row.new(sheet.columns, resolver: resolver)
+        end
+
+        freeze
+      end
+
+      def row_renderer(sheet_name, fields)
+        sheet        = document.sheet(sheet_name)
+        row_renderer = row_renderers.fetch(sheet_name.to_s)
+
+        return row_renderer unless sheet.auto?
+
+        dynamic_row_renderer(fields).merge(row_renderer)
+      end
+
+      private
+
+      attr_reader :row_renderers
+
+      def fields_to_columns(fields)
+        fields = (fields || []).map { |f| { header: f.to_s } }
+        Modeling::Column.array(fields)
+      end
+
+      def dynamic_row_renderer(fields)
+        Row.new(fields_to_columns(fields), resolver: resolver)
+      end
+    end
+  end
+end

data/lib/portable/util.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'util/pivotable'
+require_relative 'util/uniqueness'

data/lib/portable/util/pivotable.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Util # :nodoc: all
+    # Mixes in helpers for asserting uniqueness across collections
+    module Pivotable
+      def pivot_by_name(array)
+        array.each_with_object({}) { |object, memo| memo[object.name] = object }
+      end
+    end
+  end
+end

data/lib/portable/util/uniqueness.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Util # :nodoc: all
+    # Mixes in helpers for asserting uniqueness across collections
+    module Uniqueness
+      class DuplicateNameError < StandardError; end
+
+      def assert_no_duplicate_names(array)
+        names = array.map { |a| a.name.downcase }
+
+        return if names.uniq.length == array.length
+
+        raise DuplicateNameError, "cannot contain duplicate names: #{names}"
+      end
+    end
+  end
+end

data/lib/portable/version.rb

@@ -8,5 +8,5 @@
 #
 
 module Portable
-  VERSION = '1.0.0-alpha.1'
+  VERSION = '1.0.0-alpha.6'
 end

data/lib/portable/writers.rb

@@ -7,4 +7,4 @@
 # LICENSE file in the root directory of this source tree.
 #
 
-require_relative 'csv/writer'
+require_relative 'writers/csv'

data/lib/portable/writers/base.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Writers
+    # Abstract base for all writers to share.
+    class Base
+      attr_reader :document,
+                  :sheet_renderer
+
+      def initialize(document, resolver: Objectable.resolver)
+        @document       = Document.make(document, nullable: false)
+        @sheet_renderer = Rendering::Sheet.new(@document, resolver: resolver)
+
+        freeze
+      end
+
+      private
+
+      def ensure_directory_exists(filename)
+        path = File.dirname(filename)
+
+        FileUtils.mkdir_p(path) unless File.exist?(path)
+      end
+    end
+  end
+end

data/lib/portable/writers/csv.rb

@@ -0,0 +1,85 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'base'
+require_relative 'result'
+
+module Portable
+  module Writers
+    # Can write documents to a CSV file.
+    class Csv < Base
+      def write!(filename:, data_provider: Data::Provider.new, time: Time.now.utc)
+        raise ArgumentError, 'filename is required' if filename.to_s.empty?
+
+        ensure_directory_exists(filename)
+
+        sheet_filenames = extrapolate_filenames(filename, document.sheets.length)
+
+        document.sheets.map.with_index do |sheet, index|
+          data_source    = data_provider.data_source(sheet.data_source_name)
+          sheet_filename = sheet_filenames[index]
+
+          time_in_seconds = Benchmark.measure do
+            write_sheet(sheet_filename, sheet, data_source, time)
+          end.real
+
+          Result.new(sheet_filename, time_in_seconds)
+        end
+      end
+
+      private
+
+      def write_sheet(sheet_filename, sheet, data_source, time)
+        CSV.open(sheet_filename, 'w') do |csv|
+          csv.to_io.write(document.options.byte_order_mark) if document.options.byte_order_mark?
+
+          write_head(csv, sheet, data_source)
+          write_data_table(csv, sheet, data_source, time)
+          write_foot(csv, sheet, data_source)
+        end
+      end
+
+      def write_head(csv, sheet, data_source)
+        sheet.header_rows.each { |row| csv << row }
+
+        data_source.header_rows.each { |row| csv << row }
+      end
+
+      def write_data_table(csv, sheet, data_source, time)
+        row_renderer = sheet_renderer.row_renderer(sheet.name, data_source.fields)
+
+        csv << row_renderer.headers if sheet.include_headers?
+
+        data_source.data_rows.each do |row|
+          csv << row_renderer.render(row, time).values
+        end
+      end
+
+      def write_foot(csv, sheet, data_source)
+        data_source.footer_rows.each { |row| csv << row }
+
+        sheet.footer_rows.each { |row| csv << row }
+      end
+
+      def extrapolate_filenames(filename, count)
+        dir      = File.dirname(filename)
+        ext      = File.extname(filename)
+        basename = File.basename(filename, ext)
+
+        (0..count).map do |index|
+          if index.positive?
+            File.join(dir, "#{basename}.#{index}#{ext}")
+          else
+            filename
+          end
+        end
+      end
+    end
+  end
+end

data/lib/portable/writers/result.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Writers
+    # Result return object from a Writer#write! call.
+    class Result
+      attr_reader :filename, :time_in_seconds
+
+      def initialize(filename, time_in_seconds)
+        @filename        = filename
+        @time_in_seconds = time_in_seconds
+
+        freeze
+      end
+    end
+  end
+end

data/portable.gemspec

@@ -5,10 +5,10 @@ require './lib/portable/version'
 Gem::Specification.new do |s|
   s.name        = 'portable'
   s.version     = Portable::VERSION
-  s.summary     = 'Transformable export writer'
+  s.summary     = 'Virtual Document Modeling and Rendering Engine'
 
   s.description = <<-DESCRIPTION
-    This library allows you to configure exports, using Realize pipelines, creating a transformation and writing layer.  It is meant to serve as an intermediary library within a much larger ETL framework.
+    Portable is a virtual document object modeling library.  Out of the box is provides a CSV writer but others for other formats like Microsoft Excel could easily be implemented and used.
   DESCRIPTION
 
   s.authors     = ['Matthew Ruggio']

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: portable
 version: !ruby/object:Gem::Version
-  version: 1.0.0.pre.alpha.1
+  version: 1.0.0.pre.alpha.6
 platform: ruby
 authors:
 - Matthew Ruggio
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-08-04 00:00:00.000000000 Z
+date: 2020-08-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: acts_as_hashable
@@ -150,9 +150,9 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.7.0
-description: "    This library allows you to configure exports, using Realize pipelines,
-  creating a transformation and writing layer.  It is meant to serve as an intermediary
-  library within a much larger ETL framework.\n"
+description: "    Portable is a virtual document object modeling library.  Out of
+  the box is provides a CSV writer but others for other formats like Microsoft Excel
+  could easily be implemented and used.\n"
 email:
 - mruggio@bluemarblepayroll.com
 executables: []
@@ -173,16 +173,26 @@ files:
 - bin/console
 - exe/.gitkeep
 - lib/portable.rb
-- lib/portable/column.rb
-- lib/portable/csv/byte_order_mark.rb
-- lib/portable/csv/document.rb
-- lib/portable/csv/options.rb
-- lib/portable/csv/writer.rb
-- lib/portable/data_table.rb
+- lib/portable/data.rb
+- lib/portable/data/provider.rb
+- lib/portable/data/source.rb
 - lib/portable/document.rb
-- lib/portable/transformer.rb
+- lib/portable/modeling/byte_order_mark.rb
+- lib/portable/modeling/column.rb
+- lib/portable/modeling/data_table.rb
+- lib/portable/modeling/options.rb
+- lib/portable/modeling/sheet.rb
+- lib/portable/rendering.rb
+- lib/portable/rendering/row.rb
+- lib/portable/rendering/sheet.rb
+- lib/portable/util.rb
+- lib/portable/util/pivotable.rb
+- lib/portable/util/uniqueness.rb
 - lib/portable/version.rb
 - lib/portable/writers.rb
+- lib/portable/writers/base.rb
+- lib/portable/writers/csv.rb
+- lib/portable/writers/result.rb
 - portable.gemspec
 homepage: https://github.com/bluemarblepayroll/portable
 licenses:
@@ -211,5 +221,5 @@ requirements: []
 rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
-summary: Transformable export writer
+summary: Virtual Document Modeling and Rendering Engine
 test_files: []

data/lib/portable/column.rb

@@ -1,37 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-module Portable
-  # Defines all the options a column can contain.  The most basic would to just include a header
-  # (defaults to '').  If no transformers are defined then a simple resolver using the header
-  # will be used.  This works well for pass-through file writes.  Use the transformers to further
-  # customize each data point being written.
-  class Column
-    acts_as_hashable
-
-    DEFAULT_TRANSFORMER_TYPE = 'r/value/resolve'
-
-    attr_reader :header, :transformers
-
-    def initialize(header: '', transformers: [])
-      @header       = header.to_s
-      @transformers = Realize::Transformers.array(transformers)
-
-      @transformers << default_transformer if @transformers.empty?
-
-      freeze
-    end
-
-    private
-
-    def default_transformer
-      Realize::Transformers.make(type: DEFAULT_TRANSFORMER_TYPE, key: header)
-    end
-  end
-end

data/lib/portable/csv/document.rb

@@ -1,34 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-require_relative 'options'
-
-module Portable
-  module Csv
-    # Defines all the options for an export including static header rows, footer rows, and how
-    # to draw the data table.
-    class Document < Portable::Document
-      attr_reader :options
-
-      def_delegators :options,
-                     :byte_order_mark,
-                     :byte_order_mark?
-
-      def initialize(data_table: {}, footer_rows: [], header_rows: [], options: {})
-        @options = Options.make(options)
-
-        super(
-          data_table: data_table,
-          footer_rows: footer_rows,
-          header_rows: header_rows
-        )
-      end
-    end
-  end
-end

data/lib/portable/csv/writer.rb

@@ -1,109 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-require_relative 'document'
-
-module Portable
-  module Csv
-    # Main API for writing files.  There are two main patterns to choose from:
-    #   1. calling #open, #write, and #close manually.
-    #   2. calling #open and passing a block and having #close automatically called.
-    class Writer
-      class AlreadyOpenError < StandardError; end
-      class NotOpenError < StandardError; end
-
-      attr_reader :csv, :document, :time, :transformer
-
-      def initialize(document, resolver: Objectable.resolver, time: Time.now.utc)
-        @document    = Document.make(document, nullable: false)
-        @time        = time || Time.now.utc
-        @transformer = Transformer.new(@document.columns, resolver: resolver)
-      end
-
-      def open?
-        !csv.nil?
-      end
-
-      # Will raise a AlreadyOpenError exception if a writer has already been opened but
-      # not yet closed.
-      def open(filename)
-        raise AlreadyOpenError, 'writer is already open' if open?
-
-        initialize_csv(filename)
-        write_head
-
-        if block_given?
-          yield self
-          close
-        end
-
-        self
-      end
-
-      # Will raise a NotOpenError exception if a writer has not yet been opened.
-      def write_all(objects = [])
-        raise NotOpenError, 'writer is not open' unless open?
-
-        objects.each { |o| write(o) }
-
-        self
-      end
-
-      # Will raise a NotOpenError exception if a writer has not yet been opened.
-      def write(object = {})
-        raise NotOpenError, 'writer is not open' unless open?
-
-        csv << transformer.transform(object, time).values
-
-        self
-      end
-
-      # Will raise a NotOpenError exception if a writer has not yet been opened.
-      def close
-        raise NotOpenError, 'writer is not open' unless open?
-
-        write_foot
-
-        @csv.close
-        @csv = nil
-        self
-      end
-
-      private
-
-      def ensure_directory_exists(filename)
-        path = File.dirname(filename)
-
-        FileUtils.mkdir_p(path) unless File.exist?(path)
-      end
-
-      def initialize_csv(filename)
-        ensure_directory_exists(filename)
-
-        @csv = CSV.open(filename, 'w')
-
-        csv.to_io.write(document.byte_order_mark) if document.byte_order_mark?
-      end
-
-      def write_head
-        raw_write(document.header_rows)
-
-        csv << document.headers if document.include_headers?
-      end
-
-      def write_foot
-        raw_write(document.footer_rows)
-      end
-
-      def raw_write(rows)
-        rows.each { |row| csv << row }
-      end
-    end
-  end
-end

data/lib/portable/data_table.rb

@@ -1,39 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-require_relative 'column'
-
-module Portable
-  # Defines all the options for the data grid within an export like columns, whether or not
-  # you want to include headers, and more.
-  class Datagrid
-    acts_as_hashable
-
-    attr_reader :columns
-
-    def initialize(columns: [], include_headers: true)
-      @columns         = Column.array(columns)
-      @include_headers = include_headers || false
-
-      freeze
-    end
-
-    def include_headers?
-      include_headers
-    end
-
-    def headers
-      columns.map(&:header)
-    end
-
-    private
-
-    attr_reader :include_headers
-  end
-end

data/lib/portable/transformer.rb

@@ -1,30 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-module Portable
-  # Internal intermediary class that knows how to combine columns specification instances with their
-  # respective Realize pipelines.
-  class Transformer # :nodoc: all
-    attr_reader :column_pipelines
-
-    def initialize(columns, resolver: Objectable.resolver)
-      @column_pipelines = columns.each_with_object({}) do |column, memo|
-        memo[column] = Realize::Pipeline.new(column.transformers, resolver: resolver)
-      end
-
-      freeze
-    end
-
-    def transform(object, time)
-      column_pipelines.each_with_object({}) do |(column, pipeline), memo|
-        memo[column.header] = pipeline.transform(object, time)
-      end
-    end
-  end
-end