portable 1.0.0.pre.alpha → 1.0.0.pre.alpha.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 06ca8f7296a55a271a94e228c6e47f7c8d572e42ed43ea7f03b3dc9547b05fc1
-  data.tar.gz: ebc9dc0c2f3867c468e27e4976a9f8a9304f739449b228b520a97cd14ec23b2f
+  metadata.gz: 7b9fa7e8020098fb9dd2d9e2c9c984e0d71ff0aa02d4f568879bb013ae65b8d4
+  data.tar.gz: 1e01f15d35797f4954b5a4861b6a8d44e54f3571fc81cd73c5ff2c08ac8ef550
 SHA512:
-  metadata.gz: 1497db394e1dab96acbe34327a683e5e3390f0912461c8ae2e312def8996b7add5f025012e9871ccb3193014d123e23ba2993625af9a462e86756f00d00b1550
-  data.tar.gz: 5e0ea223a2d1894075e6fcf07b0eecd6a7242e37263ba7bea4e01ffefe22f95c7bcaad18fef35e74ae4b7435efc8012636dcb1e0d10a349c802327726a65dcf1
+  metadata.gz: 97d59eb7d25b5ea806d908995f8978c8f711d280bd9c8b1e2dd0c09ca173f1c2fcf132f828e2218c17f3637fc6d92e8f7e12b827212ae404c153b2c5722cbbeb
+  data.tar.gz: 3878dc2b2f38b9de6cc90c6644e3913379cecefd21adef98b35adf9cbcce3bf0fae4832491de6f3a063477b2d64416778c55e4f0383820cdfc75970d0ece5152

data/.rubocop.yml

@@ -20,7 +20,7 @@ Metrics/MethodLength:
   Max: 30
 
 Metrics/AbcSize:
-  Max: 16
+  Max: 20
 
 Metrics/ClassLength:
   Max: 125

data/README.md

@@ -2,18 +2,9 @@
 
 [![Gem Version](https://badge.fury.io/rb/portable.svg)](https://badge.fury.io/rb/portable) [![Build Status](https://travis-ci.org/bluemarblepayroll/portable.svg?branch=master)](https://travis-ci.org/bluemarblepayroll/portable) [![Maintainability](https://api.codeclimate.com/v1/badges/4b47ce94b0c9d889e648/maintainability)](https://codeclimate.com/github/bluemarblepayroll/portable/maintainability) [![Test Coverage](https://api.codeclimate.com/v1/badges/4b47ce94b0c9d889e648/test_coverage)](https://codeclimate.com/github/bluemarblepayroll/portable/test_coverage) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-This library provides a configuration layer that allows you to express transformations, using [Realize](https://github.com/bluemarblepayroll/realize), and will write the transformed data down to disk.  Essentially it is meant to be the transformation and load steps within a larger ETL system. We currently use this in production paired up with [Dbee](https://github.com/bluemarblepayroll/dbee) to go from configurable data model + query to file.
+Portable is a virtual document object modeling library.  Out of the box is provides a CSV writer but others for other formats like Microsoft Excel could easily be implemented and used.
 
-Current limitations:
-
-1. Only supports CSV with limited options
-2. Only supports writing to local file system.
-
-Future extension considerations:
-
-1. Support Excel and richer formatting, sheets, etc.
-2. Expand CSV options: delimiter, forcing quotes, etc.
-3. Support PDF
+This library utilizes [the Realize library](https://github.com/bluemarblepayroll/realize) that allows you to express transformation pipelines.  Essentially this library is meant to be the transformation and load steps within a larger ETL system. We currently use this in production paired up with [Dbee](https://github.com/bluemarblepayroll/dbee) for data sources to go from configurable data model + query to file.
 
 ## Installation
 
@@ -31,40 +22,40 @@ bundle add portable
 
 ## Examples
 
-### Getting Started with Exports
+### Getting Started Writing CSV Files
 
-Consider the following data set as an array of hashes:
+Consider the following data provider/source:
 
 ````ruby
 patients = [
   { first: 'Marky', last: 'Mark', dob: '2000-04-05' },
   { first: 'Frank', last: 'Rizzo', dob: '1930-09-22' }
 ]
+
+data_provider = Portable::Data::Provider.new(
+  data_sources: {
+    data_rows: patients,
+    fields: %i[first last dob]
+  }
+)
 ````
 
-We could configure an export like so:
+**Note:** Data::Provider and Data::Source objects are pretty basic, on purpose, so they can be easily re-implemented based on an application's specific needs.
+
+We could configure the most basic document like so:
 
 ````ruby
-export = {
-  columns: [
-    { header: :first },
-    { header: :last },
-    { header: :dob }
-  ]
-}
+document = nil # or {} or Portable::Document.new
 ````
 
-And execute the export against the example dataset in order to generate a CSV file:
+The above document says I would like a document with one sheet, and since I did not provide a data_table specification, I would like all the fields emitted from the data source.
+
+Combining a document + writer + data provider yields a set of documents (it may be more than one if the writer does not know how to write intra-file sheets, i.e. CSV files.)
 
 ````ruby
-writer   = Portable::Writer.new(export)
-filename = File.join('tmp', 'patients.csv')
-
-writer.open(filename) do |writer|
-  patients.each do |patient|
-    writer.write(object: patient)
-  end
-end
+writer    = Portable::Writers::Csv.new(document)
+name      = File.join('tmp', 'patients.csv')
+written   = writer.write!(filename: name, data_provider: data_provider)
 ````
 
 We should now have a CSV file at tmp/patients.csv that looks like this:
@@ -78,29 +69,35 @@ Frank | Rizzo | 1930-09-22
 
 This library uses Realize under the hood, so you have the option of configuring any transformation pipeline for each column.  Reviewing [Realize's list of transformers](https://github.com/bluemarblepayroll/realize#transformer-gallery) is recommended to see what is available.
 
-Let's expand our example above with different headers and date formatting:
+Let's expand our CSV example above with different headers and date formatting:
 
 ````ruby
-export = {
-  columns: [
+document = {
+  sheets: [
     {
-      header: 'First Name',
-      transformers: [
-        { type: 'r/value/resolve', key: :first }
-      ]
-    },
-    {
-      header: 'Last Name',
-      transformers: [
-        { type: 'r/value/resolve', key: :last }
-      ]
-    },
-    {
-      header: 'Date of Birth',
-      transformers: [
-        { type: 'r/value/resolve', key: :dob },
-        { type: 'r/format/date', output_format: '%m/%d/%Y' },
-      ]
+      data_table: {
+        columns: [
+          {
+            header: 'First Name',
+            transformers: [
+              { type: 'r/value/resolve', key: :first }
+            ]
+          },
+          {
+            header: 'Last Name',
+            transformers: [
+              { type: 'r/value/resolve', key: :last }
+            ]
+          },
+          {
+            header: 'Date of Birth',
+            transformers: [
+              { type: 'r/value/resolve', key: :dob },
+              { type: 'r/format/date', output_format: '%m/%d/%Y' },
+            ]
+          }
+        ]
+      }
     }
   ]
 }
@@ -115,6 +112,70 @@ Frank      | Rizzo     | 09/22/1930
 
 Realize is also [pluggable](https://github.com/bluemarblepayroll/realize#plugging-in-transformers), so you are able to create your own and plug them directly into Realize.
 
+### Options
+
+Each writer can choose how and which options to support.
+
+#### CSV Options
+
+The following options are available for customizing CSV documents:
+
+* byte_order_mark: (optional, default is nothing): This option will write out a byte order mark identifying the encoding for the file.  This is useful for ensuring applications like Microsoft Excel open CSV files properly.  See Portable::Modeling::ByteOrderMark constants for acceptable values.
+
+### Static Header/Footer Rows
+
+The main document model can also include statically defined rows to place either at the header (above data table) or footer (below data table) locations.  You can also have the data_source inject static header and footer rows as well.  For example:
+
+````ruby
+patients = [
+  { first: 'Marky', last: 'Mark', dob: '2000-04-05' },
+  { first: 'Frank', last: 'Rizzo', dob: '1930-09-22' }
+]
+
+data_provider = Portable::Data::Provider.new(
+  data_sources: {
+    data_rows: patients,
+    fields: %i[first last dob],
+    header_rows: [
+      %w[FIRST_START LAST_START DOB_START]
+    ],
+    footer_rows: [
+      %w[FIRST_END LAST_END DOB_END]
+    ]
+  }
+)
+
+document = {
+  sheets: [
+    {
+      header_rows: [
+        [ 'Run Date', '04/05/2000' ],
+        [ 'Run By', 'Hops the Bunny' ],
+        [],
+        [ 'BEGIN' ]
+      ],
+      footer_rows: [
+        [ 'END' ]
+      ]
+    }
+  ]
+}
+````
+
+Using this document configuration would yield a CSV with four "header rows" at the top, one "data table header row", two data rows, and one "footer row".  This is not easily illustrated in Markdown, but this would be the result:
+
+````
+Run Date   | 04/05/2000
+Run By     | Hops the Bunny
+
+BEGIN
+First Name | Last Name | Date of Birth
+---------- | --------- | -------------
+Marky      | Mark      | 04/05/2000
+Frank      | Rizzo     | 09/22/1930
+END
+````
+
 ## Contributing
 
 ### Development Environment Configuration

data/lib/portable.rb

@@ -8,10 +8,19 @@
 #
 
 require 'acts_as_hashable'
+require 'benchmark'
 require 'csv'
 require 'fileutils'
+require 'forwardable'
 require 'objectable'
 require 'realize'
 require 'time'
 
-require_relative 'portable/writer'
+# Shared modules/classes
+require_relative 'portable/uniqueness'
+
+# Main implementation points
+require_relative 'portable/data'
+require_relative 'portable/document'
+require_relative 'portable/rendering'
+require_relative 'portable/writers'

data/lib/portable/data.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'data/provider'

data/lib/portable/data/provider.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'source'
+
+module Portable
+  module Data
+    # Container of data sources that is inputted into a writer alongside a document.
+    # It contains all the data sources the writer will use to render a document.
+    class Provider
+      include Uniqueness
+      acts_as_hashable
+
+      DEFAULT_NAME = ''
+
+      def initialize(data_sources: [])
+        sources               = Source.array(data_sources)
+        @data_sources_by_name = pivot_by_name(sources)
+
+        assert_no_duplicate_names(sources)
+
+        freeze
+      end
+
+      # Use exact name if possible, if not then use the "default" one (noted by a blank name).
+      # Fail hard if we cannot identify which data source to use.  This should help prevent
+      # possible configuration issues (i.e. typos.)
+      def data_source(name)
+        data_sources_by_name[name.to_s] ||
+          data_sources_by_name[DEFAULT_NAME] ||
+          raise(ArgumentError, "data source: '#{name}' cannot be found.")
+      end
+
+      private
+
+      attr_reader :data_sources_by_name
+
+      def pivot_by_name(data_sources)
+        data_sources.each_with_object({}) do |data_source, memo|
+          memo[data_source.name] = data_source
+        end
+      end
+    end
+  end
+end

data/lib/portable/data/source.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Data
+    # A single source of data.  This is meant to serve as an interface / example implementation
+    # with the intention of being re-implemented within applications.  For example, you may
+    # decide more database data sources would be better, so it could be connected to ORMs or
+    # other data adapters; all it really needs to provide is enumerables for each attribute.
+    class Source
+      acts_as_hashable
+
+      attr_reader :header_rows,
+                  :footer_rows,
+                  :data_rows,
+                  :fields,
+                  :name
+
+      # Individial header and footer rows are arrays, while individual data_rows is an object
+      # like a hash, Struct, OpenStruct, or really any PORO.
+      def initialize(name: '', header_rows: [], footer_rows: [], data_rows: [], fields: [])
+        @name        = name.to_s
+        @header_rows = header_rows || []
+        @footer_rows = footer_rows || []
+        @data_rows   = data_rows   || []
+        @fields      = fields      || []
+
+        freeze
+      end
+    end
+  end
+end

data/lib/portable/document.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'modeling/options'
+require_relative 'modeling/sheet'
+
+module Portable
+  # Top-level object model for a renderable document.
+  class Document
+    include Uniqueness
+    acts_as_hashable
+
+    attr_reader :sheets, :options
+
+    def initialize(sheets: [], options: {})
+      @sheets = Modeling::Sheet.array(sheets)
+      @sheets << Modeling::Sheet.new if @sheets.empty?
+      @options = Modeling::Options.make(options)
+
+      assert_no_duplicate_names(@sheets)
+
+      freeze
+    end
+  end
+end

data/lib/portable/modeling/byte_order_mark.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Modeling
+    # Define all acceptable byte order mark values.
+    module ByteOrderMark
+      UTF_8    = "\xEF\xBB\xBF"
+      UTF_16BE = "\xFE\xFF"
+      UTF_16LE = "\xFF\xFE"
+      UTF_32BE = "\x00\x00\xFE\xFF"
+      UTF_32LE = "\xFE\xFF\x00\x00"
+
+      class << self
+        def resolve(value)
+          value ? const_get(value.to_s.upcase.to_sym) : nil
+        end
+      end
+    end
+  end
+end

data/lib/portable/modeling/column.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Modeling
+    # Defines all the options a column can contain.  The most basic would to just include a header
+    # (defaults to '').  If no transformers are defined then a simple resolver using the header
+    # will be used.  This works well for pass-through file writes.  Use the transformers to further
+    # customize each data point being written.
+    class Column
+      acts_as_hashable
+
+      DEFAULT_TRANSFORMER_TYPE = 'r/value/resolve'
+
+      attr_reader :header, :transformers
+
+      def initialize(header: '', transformers: [])
+        @header       = header.to_s
+        @transformers = Realize::Transformers.array(transformers)
+
+        @transformers << default_transformer if @transformers.empty?
+
+        freeze
+      end
+
+      private
+
+      def default_transformer
+        Realize::Transformers.make(type: DEFAULT_TRANSFORMER_TYPE, key: header)
+      end
+    end
+  end
+end

data/lib/portable/modeling/data_table.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'column'
+
+module Portable
+  module Modeling
+    # Defines all the options for the data grid within an export like columns, whether or not
+    # you want to include headers, and more.
+    class DataTable
+      acts_as_hashable
+
+      attr_reader :columns
+
+      def initialize(columns: [])
+        @columns = Column.array(columns)
+
+        freeze
+      end
+    end
+  end
+end

data/lib/portable/modeling/options.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'byte_order_mark'
+
+module Portable
+  module Modeling
+    # Defines all the options for an export including static header rows, footer rows, and how
+    # to draw the data table.
+    class Options
+      acts_as_hashable
+
+      attr_reader :byte_order_mark
+
+      def initialize(byte_order_mark: nil)
+        @byte_order_mark = ByteOrderMark.resolve(byte_order_mark)
+
+        freeze
+      end
+
+      def byte_order_mark?
+        !byte_order_mark.nil?
+      end
+    end
+  end
+end

data/lib/portable/modeling/sheet.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'data_table'
+
+module Portable
+  module Modeling
+    # Abstract concept modeling for the notion of a "sheet" in a "document".  This means different
+    # things given the writer.  For example, all writers should support multiple sheets but
+    # there is no internal representation of a "sheet" within a CSV, so each sheet will emit
+    # one file.
+    class Sheet
+      acts_as_hashable
+
+      attr_reader :data_table,
+                  :footer_rows,
+                  :header_rows,
+                  :name,
+                  :include_headers
+
+      def initialize(
+        data_table: nil,
+        footer_rows: [],
+        header_rows: [],
+        name: '',
+        include_headers: true
+      )
+        @name            = name.to_s
+        @data_table      = DataTable.make(data_table)
+        @footer_rows     = footer_rows || []
+        @header_rows     = header_rows || []
+        @include_headers = include_headers || false
+
+        freeze
+      end
+
+      def include_headers?
+        include_headers
+      end
+    end
+  end
+end

data/lib/portable/rendering.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'rendering/sheet'

data/lib/portable/rendering/row.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Rendering
+    # Internal intermediary class that knows how to combine columns specification
+    # instances with their respective Realize pipelines.
+    class Row # :nodoc: all
+      attr_reader :column_pipelines
+
+      def initialize(columns, resolver: Objectable.resolver)
+        @column_pipelines = columns.each_with_object({}) do |column, memo|
+          memo[column] = Realize::Pipeline.new(column.transformers, resolver: resolver)
+        end
+
+        freeze
+      end
+
+      def render(object, time)
+        column_pipelines.each_with_object({}) do |(column, pipeline), memo|
+          memo[column.header] = pipeline.transform(object, time)
+        end
+      end
+
+      def columns
+        column_pipelines.keys
+      end
+
+      def headers
+        columns.map(&:header)
+      end
+    end
+  end
+end

data/lib/portable/rendering/sheet.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'row'
+
+module Portable
+  module Rendering
+    # Understands the connection between a document's sheets and the internal row renderer
+    # necessary to render each sheet's data table.
+    class Sheet # :nodoc: all
+      attr_reader :document, :resolver
+
+      def initialize(document, resolver: Objectable.resolver)
+        @document = Document.make(document, nullable: false)
+        @resolver = resolver
+
+        @row_renderers = @document.sheets.each_with_object({}) do |sheet, memo|
+          next unless sheet.data_table
+
+          memo[sheet.name] = Row.new(sheet.data_table.columns, resolver: resolver)
+        end
+
+        freeze
+      end
+
+      def row_renderer(sheet_name, fields)
+        row_renderers.fetch(sheet_name, dynamic_row_renderer(fields))
+      end
+
+      private
+
+      attr_reader :row_renderers
+
+      def dynamic_row_renderer(fields)
+        fields  = (fields || []).map { |f| { header: f.to_s } }
+        columns = Modeling::Column.array(fields)
+
+        Row.new(columns, resolver: resolver)
+      end
+    end
+  end
+end

data/lib/portable/uniqueness.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  # Mixes in helpers for asserting uniqueness across collections
+  module Uniqueness
+    class DuplicateNameError < StandardError; end
+
+    def assert_no_duplicate_names(array)
+      names = array.map { |a| a.name.downcase }
+
+      return if names.uniq.length == array.length
+
+      raise DuplicateNameError, "cannot contain duplicate names: #{names}"
+    end
+  end
+end

data/lib/portable/version.rb

@@ -8,5 +8,5 @@
 #
 
 module Portable
-  VERSION = '1.0.0-alpha'
+  VERSION = '1.0.0-alpha.5'
 end

data/lib/portable/writers.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'writers/csv'

data/lib/portable/writers/base.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Writers
+    # Abstract base for all writers to share.
+    class Base
+      attr_reader :document,
+                  :sheet_renderer
+
+      def initialize(document, resolver: Objectable.resolver)
+        @document       = Document.make(document, nullable: false)
+        @sheet_renderer = Rendering::Sheet.new(@document, resolver: resolver)
+
+        freeze
+      end
+
+      private
+
+      def ensure_directory_exists(filename)
+        path = File.dirname(filename)
+
+        FileUtils.mkdir_p(path) unless File.exist?(path)
+      end
+    end
+  end
+end

data/lib/portable/writers/csv.rb

@@ -0,0 +1,85 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+require_relative 'base'
+require_relative 'result'
+
+module Portable
+  module Writers
+    # Can write documents to a CSV file.
+    class Csv < Base
+      def write!(filename:, data_provider: Data::Provider.new, time: Time.now.utc)
+        raise ArgumentError, 'filename is required' if filename.to_s.empty?
+
+        ensure_directory_exists(filename)
+
+        sheet_filenames = extrapolate_filenames(filename, document.sheets.length)
+
+        document.sheets.map.with_index do |sheet, index|
+          data_source    = data_provider.data_source(sheet.name)
+          sheet_filename = sheet_filenames[index]
+
+          time_in_seconds = Benchmark.measure do
+            write_sheet(sheet_filename, sheet, data_source, time)
+          end.real
+
+          Result.new(sheet_filename, time_in_seconds)
+        end
+      end
+
+      private
+
+      def write_sheet(sheet_filename, sheet, data_source, time)
+        CSV.open(sheet_filename, 'w') do |csv|
+          csv.to_io.write(document.options.byte_order_mark) if document.options.byte_order_mark?
+
+          write_head(csv, sheet, data_source)
+          write_data_table(csv, sheet, data_source, time)
+          write_foot(csv, sheet, data_source)
+        end
+      end
+
+      def write_head(csv, sheet, data_source)
+        sheet.header_rows.each { |row| csv << row }
+
+        data_source.header_rows.each { |row| csv << row }
+      end
+
+      def write_data_table(csv, sheet, data_source, time)
+        row_renderer = sheet_renderer.row_renderer(sheet.name, data_source.fields)
+
+        csv << row_renderer.headers if sheet.include_headers?
+
+        data_source.data_rows.each do |row|
+          csv << row_renderer.render(row, time).values
+        end
+      end
+
+      def write_foot(csv, sheet, data_source)
+        data_source.footer_rows.each { |row| csv << row }
+
+        sheet.footer_rows.each { |row| csv << row }
+      end
+
+      def extrapolate_filenames(filename, count)
+        dir      = File.dirname(filename)
+        ext      = File.extname(filename)
+        basename = File.basename(filename, ext)
+
+        (0..count).map do |index|
+          if index.positive?
+            File.join(dir, "#{basename}.#{index}#{ext}")
+          else
+            filename
+          end
+        end
+      end
+    end
+  end
+end

data/lib/portable/writers/result.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+#
+# Copyright (c) 2020-present, Blue Marble Payroll, LLC
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+#
+
+module Portable
+  module Writers
+    # Result return object from a Writer#write! call.
+    class Result
+      attr_reader :filename, :time_in_seconds
+
+      def initialize(filename, time_in_seconds)
+        @filename        = filename
+        @time_in_seconds = time_in_seconds
+
+        freeze
+      end
+    end
+  end
+end

data/portable.gemspec

@@ -5,10 +5,10 @@ require './lib/portable/version'
 Gem::Specification.new do |s|
   s.name        = 'portable'
   s.version     = Portable::VERSION
-  s.summary     = 'Transformable export writer'
+  s.summary     = 'Virtual Document Modeling and Rendering Engine'
 
   s.description = <<-DESCRIPTION
-    This library allows you to configure exports, using Realize pipelines, creating a transformation and writing layer.  It is meant to serve as an intermediary library within a much larger ETL framework.
+    Portable is a virtual document object modeling library.  Out of the box is provides a CSV writer but others for other formats like Microsoft Excel could easily be implemented and used.
   DESCRIPTION
 
   s.authors     = ['Matthew Ruggio']

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: portable
 version: !ruby/object:Gem::Version
-  version: 1.0.0.pre.alpha
+  version: 1.0.0.pre.alpha.5
 platform: ruby
 authors:
 - Matthew Ruggio
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-08-03 00:00:00.000000000 Z
+date: 2020-08-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: acts_as_hashable
@@ -150,9 +150,9 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.7.0
-description: "    This library allows you to configure exports, using Realize pipelines,
-  creating a transformation and writing layer.  It is meant to serve as an intermediary
-  library within a much larger ETL framework.\n"
+description: "    Portable is a virtual document object modeling library.  Out of
+  the box is provides a CSV writer but others for other formats like Microsoft Excel
+  could easily be implemented and used.\n"
 email:
 - mruggio@bluemarblepayroll.com
 executables: []
@@ -173,11 +173,24 @@ files:
 - bin/console
 - exe/.gitkeep
 - lib/portable.rb
-- lib/portable/column.rb
-- lib/portable/export.rb
-- lib/portable/transformer.rb
+- lib/portable/data.rb
+- lib/portable/data/provider.rb
+- lib/portable/data/source.rb
+- lib/portable/document.rb
+- lib/portable/modeling/byte_order_mark.rb
+- lib/portable/modeling/column.rb
+- lib/portable/modeling/data_table.rb
+- lib/portable/modeling/options.rb
+- lib/portable/modeling/sheet.rb
+- lib/portable/rendering.rb
+- lib/portable/rendering/row.rb
+- lib/portable/rendering/sheet.rb
+- lib/portable/uniqueness.rb
 - lib/portable/version.rb
-- lib/portable/writer.rb
+- lib/portable/writers.rb
+- lib/portable/writers/base.rb
+- lib/portable/writers/csv.rb
+- lib/portable/writers/result.rb
 - portable.gemspec
 homepage: https://github.com/bluemarblepayroll/portable
 licenses:
@@ -206,5 +219,5 @@ requirements: []
 rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
-summary: Transformable export writer
+summary: Virtual Document Modeling and Rendering Engine
 test_files: []

data/lib/portable/column.rb

@@ -1,37 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-module Portable
-  # Defines all the options a column can contain.  The most basic would to just include a header
-  # (defaults to '').  If no transformers are defined then a simple resolver using the header
-  # will be used.  This works well for pass-through file writes.  Use the transformers to further
-  # customize each data point being written.
-  class Column
-    acts_as_hashable
-
-    DEFAULT_TRANSFORMER_TYPE = 'r/value/resolve'
-
-    attr_reader :header, :transformers
-
-    def initialize(header: '', transformers: [])
-      @header       = header.to_s
-      @transformers = Realize::Transformers.array(transformers)
-
-      @transformers << default_transformer if @transformers.empty?
-
-      freeze
-    end
-
-    private
-
-    def default_transformer
-      Realize::Transformers.make(type: DEFAULT_TRANSFORMER_TYPE, key: header)
-    end
-  end
-end

data/lib/portable/export.rb

@@ -1,39 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-require_relative 'column'
-
-module Portable
-  # Defines all the options for an export like columns, whether or not you want to include
-  # headers, and more.
-  class Export
-    acts_as_hashable
-
-    module Bom
-      UTF8 = "\uFEFF"
-    end
-    include Bom
-
-    attr_reader :bom, :columns, :include_headers
-
-    alias include_headers? include_headers
-
-    def initialize(bom: nil, columns: [], include_headers: true)
-      @bom             = bom ? Bom.const_get(bom.to_s.upcase.to_sym) : nil
-      @columns         = Column.array(columns)
-      @include_headers = include_headers || false
-
-      freeze
-    end
-
-    def headers
-      columns.map(&:header)
-    end
-  end
-end

data/lib/portable/transformer.rb

@@ -1,32 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-require_relative 'export'
-
-module Portable
-  # Internal intermediary class that knows how to combine columns specification instances with their
-  # respective Realize pipelines.
-  class Transformer # :nodoc: all
-    attr_reader :column_pipelines
-
-    def initialize(columns, resolver: Objectable.resolver)
-      @column_pipelines = columns.each_with_object({}) do |column, memo|
-        memo[column] = Realize::Pipeline.new(column.transformers, resolver: resolver)
-      end
-
-      freeze
-    end
-
-    def transform(object, time)
-      column_pipelines.each_with_object({}) do |(column, pipeline), memo|
-        memo[column.header] = pipeline.transform(object, time)
-      end
-    end
-  end
-end

data/lib/portable/writer.rb

@@ -1,82 +0,0 @@
-# frozen_string_literal: true
-
-#
-# Copyright (c) 2020-present, Blue Marble Payroll, LLC
-#
-# This source code is licensed under the MIT license found in the
-# LICENSE file in the root directory of this source tree.
-#
-
-require_relative 'export'
-require_relative 'transformer'
-
-module Portable
-  # Main API for writing files.  There are two main patterns to choose from:
-  #   1. calling #open, #write, and #close manually.
-  #   2. calling #open and passing a block and having #close automatically called.
-  class Writer
-    class AlreadyOpenError < StandardError; end
-    class NotOpenError < StandardError; end
-
-    attr_reader :csv, :export, :transformer
-
-    def initialize(export, resolver: Objectable.resolver)
-      @export      = Export.make(export, nullable: false)
-      @transformer = Transformer.new(@export.columns, resolver: resolver)
-    end
-
-    def open?
-      !csv.nil?
-    end
-
-    # Will raise a AlreadyOpenError exception if a writer has already been opened but
-    # not yet closed.
-    def open(filename)
-      raise AlreadyOpenError, 'writer is already open' if open?
-
-      initialize_csv(filename)
-
-      if block_given?
-        yield self
-        close
-      end
-
-      self
-    end
-
-    # Will raise a NotOpenError exception if a writer has not yet been opened.
-    def write(object: {}, time: Time.now.utc)
-      raise NotOpenError, 'writer is not open' unless open?
-
-      csv << transformer.transform(object, time).values
-
-      self
-    end
-
-    # Will raise a NotOpenError exception if a writer has not yet been opened.
-    def close
-      raise NotOpenError, 'writer is not open' unless open?
-
-      @csv.close
-      @csv = nil
-      self
-    end
-
-    private
-
-    def ensure_directory_exists(filename)
-      path = File.dirname(filename)
-
-      FileUtils.mkdir_p(path) unless File.exist?(path)
-    end
-
-    def initialize_csv(filename)
-      ensure_directory_exists(filename)
-
-      @csv = CSV.open(filename, 'w')
-
-      csv.to_io.write(export.bom) if export.bom
-      csv << export.headers       if export.include_headers?
-    end
-  end
-end