csv_plus_plus 0.1.1 → 0.1.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6ef86258faa82b75114acc715e9cb68bb312dc55194d533270fce0a796e02362
-  data.tar.gz: 9316f43dd4169a779efc6d8f447002434c4fc5984064b1f22f1448baee0f4522
+  metadata.gz: abee03db0f21f9d8c5d3cfb4ec81dbd748568f878458f355cab6db5fc458a509
+  data.tar.gz: 6c4e95fa1c780a2d009eaeb58fd48b28c2dd96826b22be02f84d5fc8a06ad31e
 SHA512:
-  metadata.gz: 8637e6c3dec41bf747e5fe1f6eb4d3227247e196ef530bbac96e964f6cdbe911d39bed9941ab7e116f00ea2547cf5050a609f078af8dab8f28a6e0f7746e776d
-  data.tar.gz: 57acb6edef384a31a41ea244841d2977d31817d87735944336688f5961d6eb22c97e5b88683d753c1163239f397936411a7836eafe1aef6e9ed6a612e846fd68
+  metadata.gz: 59f2cb756bdfd20f95f252aa1c07fb4b994def21f07d8bc4e5a976b22f0b439fb950e1723fdb34753baf23527e5950d244f1386e9231c6318b35de3974305ba4
+  data.tar.gz: 38b6d6d960aa17978adf2ee1724333754e7cdc86b8e41fa58522357448db8448d5441fb5961d3758af94d617b94b21b197b5e4d2d4eb2dba0cbc39da586c4413

data/README.md

@@ -9,101 +9,56 @@ A tool that allows you to programatically author spreadsheets in your favorite t
 
 A `csvpp` file consists of a (optional) code section and a CSV section separated by `---`.  In the code section you can define variables and functions that can be used in the CSV below it.  For example:
 
+###### **`mystocks.csvpp`**
 ```
 fees := 0.50 # my broker charges $0.50 a trade
 
-price := cellref(C)
-quantity := cellref(D)
+price := celladjacent(C)
+quantity := celladjacent(D)
 
 def profit() (price * quantity) - fees
 
 ---
 ![[format=bold/align=center]]Date,Ticker,Price,Quantity,Total,Fees
-![[expand]],[[format=bold]],,,"=PROFIT()",$$fees
+![[expand]],[[format=bold]],,,"=profit()",$$fees
 ```
 
-## Variables
-
-Variables can be defined in the code section by giving a name (a combination of letters, numbers and underscores ) the expression `:=` and followed with a value.
-
-### Built-in Variables
-
-* `$$rownum` - The current row number.  The first row of the spreadsheet starts at 1.  Can be used anywhere and it's value will evaluate to the current row being processed.
-
-## Functions
-
-### Built-in Functions
-* `cellref(CELL)` - Returns a reference to the `CELL` relative to the current row.  If the current `$$rownum` is `2`, then `CELLREF("C")` returns  a reference to cell `C2`.
-
-## Modifiers
-
-Modifiers can change the formatting of a cell or row, apply validation, change alignment, etc. All of the normal rules of CSV apply, with the addition that each cell can have modifiers (specified in `[[`/`]]` for cells and `![[`/`]]` for rows):
-
-```
-foo,[[...]]bar,baz
-```
-
-specifying formatting or various other modifiers to the cell.  Additionally a row can start with:
-
-```
-![[...]]foo,bar,baz
-```
-
-which will apply that modifier to all cells in the row.
-
-### Examples
-
-* Align the second cell left, align the last cell to the center and make it bold and italicized:
+And can be compiled into a `.xlsx` file by:
 
 ```
-Date,[[align=left]]Amount,Quantity,[[align=center/format=bold italic]]Price
+$ csv++ -n 'My Stock Tracker' -o mystocks.xlsx mystocks.csvpp
 ```
 
-* Underline and center-align an entire row:
+See the [Language Reference](./docs/LANGUAGE_REFERENCE.md) for a full explanation of features.
 
-```
-![[align=center/format=underline]]Date,Amount,Quantity,Price
-```
+## Installing
 
-* A header for the first row, then some formulas that repeat for each row for the rest of the spreadsheet:
+Just install it via rubygems (homebrew and debian packages are in the works):
 
-```
-![[align=center/format=bold]]Date,Price,Quantity,Profit
-![[expand=1:]],,,"=MULTIPLY(cellref(B), cellref(C))"
-```
+`$ gem install csv_plus_plus`
 
-## Setup (Google Sheets)
+or if you want the very latest changes, clone this repository and run:
 
-Just install it via rubygems (homebrew and debian packages are in the works):
+`$ rake gem:install`
 
-`$ gem install csv_plus_plus`
+### [Setting Up Google Sheets](./docs/README_GOOGLE_SHEETS.md)
 
-### Publishing to Google Sheets
+## Examples
 
-* Go to the [GCP developers console](https://console.cloud.google.com/projectselector2/apis/credentials?pli=1&supportedpurview=project), create a service account and export keys for it to `~/.config/gcloud/application_default_credentials.json`
-* "Share" the spreadsheet with the email associated with the service account
+Take a look at the [examples](./examples/) directory for a bunch of example `.csvpp` files.
 
 ## CLI Arguments
 
 ```
 Usage: csv++ [options]
+    -h, --help                       Show help information
     -b, --backup                     Create a backup of the spreadsheet before applying changes.
-    -g, --google-sheet-id SHEET_ID   The id of the sheet - you can extract this from the URL: https://docs.google.com/spreadsheets/d/< ... SHEET_ID ... >/edit#gid=0
     -c, --create                     Create the sheet if it doesn't exist.  It will use --sheet-name if specified
+    -g, --google-sheet-id SHEET_ID   The id of the sheet - you can extract this from the URL: https://docs.google.com/spreadsheets/d/< ... SHEET_ID ... >/edit#gid=0
     -k, --key-values KEY_VALUES      A comma-separated list of key=values which will be made available to the template
     -n, --sheet-name SHEET_NAME      The name of the sheet to apply the template to
+    -o, --output OUTPUT_FILE         The file to write to (must be .csv, .ods, .xls)
     -v, --verbose                    Enable verbose output
     -x, --offset-columns OFFSET      Apply the template offset by OFFSET cells
     -y, --offset-rows OFFSET         Apply the template offset by OFFSET rows
-    -h, --help                       Show help information
-```
-
-## Usage Examples
-
-```
-# apply my_taxes_template.csvpp to an existing Google Sheet with name "Taxes 2022"
-$ csv++ --sheet-name "Taxes 2022" --sheet-id "[...]" my_taxes_template.csvpp
-
-# take input from stdin, supply a variable ($$rate = 1) and apply to the "Stocks" spreadsheet
-$ cat stocks.csvpp | csv++ -k "rate=1" -n "Stocks" -i "[...]"
 ```

data/{CHANGELOG.md → docs/CHANGELOG.md}

@@ -1,3 +1,20 @@
+## v0.1.3
+
+- Proper scoping of variables defined within an expand modifier
+- Types via Sorbet
+- Fix formula insertion on Excel
+- Fix modifier string quoting
+- Fix broken Yard doc generation
+- Fix: multiple modifiers on the same row weren't being handled 
+
+## v0.1.2
+
+- var=... modifier which allows binding a variable to a cell
+- Improved error handling and messages
+- Moving in a direction that allows for the context-dependent aspects of modifiers
+- Fixes a bug with creating a new excel spreadsheet
+- Docs & tests
+
 ## v0.1.1
 
 - Better support for the various infix operators (+,-,/,*,^,%,=,<,etc)

data/lib/csv_plus_plus/benchmarked_compiler.rb

@@ -0,0 +1,112 @@
+# typed: strict
+# frozen_string_literal: true
+
+module CSVPlusPlus
+  # Extend a +Compiler+ class and add benchmark timings
+  #
+  # @attr_reader timings [Array<Benchmark::Tms>] +Benchmark+ timings that have been accumulated by each step of
+  #   compilation
+  # @attr_reader benchmark [Benchmark::Report] A +Benchmark+ instance
+  class BenchmarkedCompiler < ::CSVPlusPlus::Compiler
+    extend ::T::Sig
+
+    sig { returns(::Benchmark::Report) }
+    attr_reader :benchmark
+
+    sig { returns(::T::Array[::Benchmark::Tms]) }
+    attr_reader :timings
+
+    sig do
+      params(
+        options: ::CSVPlusPlus::Options,
+        runtime: ::CSVPlusPlus::Runtime::Runtime,
+        block: ::T.proc.params(compiler: ::CSVPlusPlus::Compiler).void
+      ).void
+    end
+    # Instantiate a +::Compiler+ that can benchmark (time) it's stages. For better or worse, the only way that they
+    # Benchmark library exposes it's +::Benchmark::Report+ is via a block, so this code also has to wrap with one
+    #
+    # @param options [Options]
+    # @param runtime [Runtime]
+    def self.with_benchmarks(options:, runtime:, &block)
+      ::Benchmark.benchmark(::Benchmark::CAPTION, 25, ::Benchmark::FORMAT, '> Total') do |x|
+        # compiler.extend(self)
+        compiler = new(benchmark: x, options:, runtime:)
+        block.call(compiler)
+        [compiler.timings.reduce(:+)]
+      end
+    end
+
+    sig do
+      params(
+        benchmark: ::Benchmark::Report,
+        options: ::CSVPlusPlus::Options,
+        runtime: ::CSVPlusPlus::Runtime::Runtime
+      ).void
+    end
+    # @param benchmark [::Benchmark::Report]
+    def initialize(benchmark:, options:, runtime:)
+      super(options:, runtime:)
+
+      @benchmark = ::T.let(benchmark, ::Benchmark::Report)
+      @timings = ::T.let([], ::T::Array[::Benchmark::Tms])
+    end
+
+    sig { override.params(block: ::T.proc.params(runtime: ::CSVPlusPlus::Runtime::Runtime).void).void }
+    # Time the Compiler#outputting! stage
+    # rubocop:disable Naming/BlockForwarding
+    def outputting!(&block)
+      time_stage('Writing the spreadsheet') { super(&block) }
+    end
+    # rubocop:enable Naming/BlockForwarding
+
+    protected
+
+    sig { override.void }
+    def parse_code_section!
+      time_stage('Parsing code section') { super }
+    end
+
+    sig { override.returns(::T::Array[::CSVPlusPlus::Row]) }
+    def parse_csv_section!
+      time_stage('Parsing CSV section') { super }
+    end
+
+    sig { override.params(block: ::T.proc.void).void }
+    # rubocop:disable Naming/BlockForwarding
+    def expanding!(&block)
+      time_stage('Expanding rows') { super(&block) }
+    end
+    # rubocop:enable Naming/BlockForwarding
+
+    sig { override.params(block: ::T.proc.void).void }
+    # rubocop:disable Naming/BlockForwarding
+    def bind_all_vars!(&block)
+      time_stage('Binding [[var=]]') { super(&block) }
+    end
+    # rubocop:enable Naming/BlockForwarding
+
+    sig do
+      override
+        .params(template: ::CSVPlusPlus::Template)
+        .returns(::T::Array[::T::Array[::CSVPlusPlus::Entities::Entity]])
+    end
+    def resolve_all_cells!(template)
+      time_stage('Resolving each cell') { super(template) }
+    end
+
+    private
+
+    sig do
+      type_parameters(:R).params(
+        stage: ::String,
+        block: ::T.proc.returns(::T.type_parameter(:R))
+      ).returns(::T.nilable(::T.type_parameter(:R)))
+    end
+    def time_stage(stage, &block)
+      ret = ::T.let(nil, ::T.nilable(::T.type_parameter(:R)))
+      @timings << ::T.unsafe(@benchmark.report(stage) { ret = block.call })
+      ret
+    end
+  end
+end

data/lib/csv_plus_plus/cell.rb

@@ -1,8 +1,6 @@
+# typed: strict
 # frozen_string_literal: true
 
-require_relative './language/cell_value.tab'
-require_relative './modifier'
-
 module CSVPlusPlus
   # A cell of a template
   #
@@ -11,9 +9,27 @@ module CSVPlusPlus
   # @attr_reader index [Integer] The cell's index (starts at 0)
   # @attr_reader modifier [Modifier] The modifier for this cell
   class Cell
-    attr_accessor :ast, :row_index
-    attr_reader :index, :modifier
+    extend ::T::Sig
+
+    sig { returns(::T.nilable(::CSVPlusPlus::Entities::Entity)) }
+    attr_accessor :ast
+
+    sig { returns(::Integer) }
+    attr_accessor :row_index
+
+    sig { returns(::Integer) }
+    attr_reader :index
 
+    sig { returns(::CSVPlusPlus::Modifier::Modifier) }
+    attr_reader :modifier
+
+    sig do
+      params(
+        value: ::T.nilable(::String),
+        runtime: ::CSVPlusPlus::Runtime::Runtime,
+        modifier: ::CSVPlusPlus::Modifier::Modifier
+      ).returns(::CSVPlusPlus::Cell)
+    end
     # Parse a +value+ into a Cell object.
     #
     # @param value [String] A string value which should already have been processed through a CSV parser
@@ -23,45 +39,51 @@ module CSVPlusPlus
     # @return [Cell]
     def self.parse(value, runtime:, modifier:)
       new(value:, row_index: runtime.row_index, index: runtime.cell_index, modifier:).tap do |c|
-        c.ast = ::CSVPlusPlus::Language::CellValueParser.new.parse(value, runtime)
+        c.ast = ::T.unsafe(::CSVPlusPlus::Parser::CellValue.new).parse(value, runtime)
       end
     end
 
-    # @param row_index [Integer] The cell's row index (starts at 0)
+    sig do
+      params(
+        index: ::Integer,
+        modifier: ::CSVPlusPlus::Modifier::Modifier,
+        row_index: ::Integer,
+        value: ::T.nilable(::String)
+      ).void
+    end
     # @param index [Integer] The cell's index (starts at 0)
-    # @param value [String] A string value which should already have been processed through a CSV parser
     # @param modifier [Modifier] A modifier to apply to this cell
-    def initialize(row_index:, index:, value:, modifier:)
+    # @param row_index [Integer] The cell's row index (starts at 0)
+    # @param value [String] A string value which should already have been processed through a CSV parser
+    def initialize(index:, modifier:, row_index:, value:)
       @value = value
       @modifier = modifier
       @index = index
       @row_index = row_index
     end
 
+    sig { returns(::T.nilable(::String)) }
     # The +@value+ (cleaned up some)
     #
-    # @return [String]
+    # @return [::String]
+    # TODO: is this used?
     def value
-      return if @value.nil? || @value.strip.empty?
+      stripped = @value&.strip
 
-      @value.strip
+      stripped&.empty? ? nil : stripped
     end
 
-    # @return [String]
-    def to_s
-      "Cell(index: #{@index}, row_index: #{@row_index}, value: #{@value}, modifier: #{@modifier})"
-    end
-
-    # A compiled final representation of the cell.  This can only happen after all cell have had
-    # variables and functions resolved.
+    sig { params(runtime: ::CSVPlusPlus::Runtime::Runtime).returns(::T.nilable(::String)) }
+    # A compiled final representation of the cell.  This can only happen after all cell have had variables and functions
+    # resolved.
+    #
+    # @param runtime [Runtime]
     #
-    # @return [String]
-    def to_csv
+    # @return [::String]
+    def evaluate(runtime)
       return value unless @ast
 
-      # This looks really simple but we're relying on each node of the AST to define #to_s such that calling
-      # this at the top will recursively print the tree (as a well-formatted spreadsheet formula)
-      "=#{@ast}"
+      "=#{@ast.evaluate(runtime)}"
     end
   end
 end

data/lib/csv_plus_plus/cli.rb

@@ -1,71 +1,96 @@
+# typed: strict
 # frozen_string_literal: true
 
-require 'optparse'
-
 module CSVPlusPlus
   # Handle running the application with the given CLI flags
   #
   # @attr options [Options, nil] The parsed CLI options
   class CLI
+    extend ::T::Sig
+
+    sig { returns(::CSVPlusPlus::Options) }
     attr_accessor :options
 
+    sig { void }
     # Handle CLI flags and launch the compiler
     #
     # @return [CLI]
     def self.launch_compiler!
       cli = new
-      cli.parse_options!
       cli.main
     rescue ::StandardError => e
-      cli.handle_error(e)
+      ::T.must(cli).handle_error(e)
       exit(1)
     end
 
+    sig { void }
+    # Initialize and parse the CLI flags provided to the program
+    def initialize
+      @options = ::T.let(::CSVPlusPlus::Options.new, ::CSVPlusPlus::Options)
+      parse_options!
+    end
+
+    sig { void }
     # Compile the given template using the given CLI flags
     def main
-      parse_options! unless @options
       ::CSVPlusPlus.apply_template_to_sheet!(::ARGF.read, ::ARGF.filename, @options)
     end
 
+    sig { params(error: ::StandardError).void }
     # Nicely handle a given error.  How it's handled depends on if it's our error and if @options.verbose
     #
     # @param error [CSVPlusPlus::Error, Google::Apis::ClientError, StandardError]
     def handle_error(error)
+      # make sure that we're on a newline (verbose mode might be in the middle of printing a benchmark)
+      puts("\n\n") if @options.verbose
+
       case error
-      when ::CSVPlusPlus::Error
+      when ::CSVPlusPlus::Error::Error
         handle_internal_error(error)
       when ::Google::Apis::ClientError
         handle_google_error(error)
       else
-        # TODO: more if verbose?
-        warn(error.message)
+        unhandled_error(error)
       end
     end
 
+    private
+
+    sig { void }
     # Handle the supplied command line options, setting +@options+ or throw an error if anything is invalid
     def parse_options!
-      @options = ::CSVPlusPlus::Options.new
       option_parser.parse!
       validate_options
     rescue ::OptionParser::InvalidOption => e
-      raise(::CSVPlusPlus::Error, e.message)
+      raise(::CSVPlusPlus::Error::Error, e.message)
     end
 
-    # @return [String]
-    def to_s
-      "CLI(options: #{options})"
-    end
+    sig { params(error: ::StandardError).void }
+    # An error was thrown that we weren't planning on
+    def unhandled_error(error)
+      warn(
+        <<~ERROR_MESSAGE)
+          An unexpected error was encountered.  Please try running again with --verbose and
+          reporting the error at: https://github.com/patrickomatic/csv-plus-plus/issues/new'
+        ERROR_MESSAGE
 
-    private
+      return unless @options.verbose
 
+      warn(error.full_message)
+      warn("Cause: #{error.cause}") if error.cause
+    end
+
+    sig { params(error: ::CSVPlusPlus::Error::Error).void }
     def handle_internal_error(error)
-      if error.is_a?(::CSVPlusPlus::Language::SyntaxError)
+      case error
+      when ::CSVPlusPlus::Error::SyntaxError
         warn(@options.verbose ? error.to_verbose_trace : error.to_trace)
       else
         warn(error.message)
       end
     end
 
+    sig { params(error: ::Google::Apis::ClientError).void }
     def handle_google_error(error)
       warn("Error making Google Sheets API request: #{error.message}")
       return unless @options.verbose
@@ -73,14 +98,16 @@ module CSVPlusPlus
       warn("#{error.status_code} Error making Google API request [#{error.message}]: #{error.body}")
     end
 
+    sig { void }
     def validate_options
       error_message = @options.validate
       return if error_message.nil?
 
       puts(option_parser)
-      raise(::CSVPlusPlus::Error, error_message)
+      raise(::CSVPlusPlus::Error::Error, error_message)
     end
 
+    sig { returns(::OptionParser) }
     def option_parser
       ::OptionParser.new do |parser|
         parser.on('-h', '--help', 'Show help information') do

data/lib/csv_plus_plus/cli_flag.rb

@@ -1,7 +1,6 @@
+# typed: true
 # frozen_string_literal: true
 
-require_relative './google_options'
-
 module CSVPlusPlus
   # Individual CLI flags that a user can supply
   #

data/lib/csv_plus_plus/color.rb

@@ -1,3 +1,4 @@
+# typed: strict
 # frozen_string_literal: true
 
 module CSVPlusPlus
@@ -7,19 +8,49 @@ module CSVPlusPlus
   # attr_reader green_hex [String] The green value in hex ("FF", "00", "AF", etc)
   # attr_reader red_hex [String] The red value in hex ("FF", "00", "AF", etc)
   class Color
-    attr_reader :red_hex, :green_hex, :blue_hex
+    extend ::T::Sig
 
-    # create an instance from a string like "#FFF" or "#FFFFFF"
+    sig { returns(::String) }
+    attr_reader :red_hex
+
+    sig { returns(::String) }
+    attr_reader :green_hex
+
+    sig { returns(::String) }
+    attr_reader :blue_hex
+
+    HEX_STRING_REGEXP = /^#?([0-9a-f]{1,2})([0-9a-f]{1,2})([0-9a-f]{1,2})/i
+    public_constant :HEX_STRING_REGEXP
+
+    sig { params(hex_string: ::String).returns(::T::Boolean) }
+    # Is +hex_string+ a valid hexadecimal color code? This function will accept input like the 6-digit format: #FF00FF,
+    # 00AABB and the shorter 3-digit format: #FFF, 0FA.
+    #
+    # @param hex_string [::String] The string to see if it's valid hex string
+    #
+    # @return [boolean]
+    def self.valid_hex_string?(hex_string)
+      !(hex_string.strip =~ ::CSVPlusPlus::Color::HEX_STRING_REGEXP).nil?
+    end
+
+    sig { params(hex_string: ::String).void }
+    # Create an instance from a string like "#FFF" or "#FFFFFF"
     #
     # @param hex_string [String] The hex string input to parse
+    # rubocop:disable Metrics/CyclomaticComplexity
     def initialize(hex_string)
-      @red_hex, @green_hex, @blue_hex = hex_string
-                                        .gsub(/^#?/, '')
-                                        .match(/([0-9a-f]{1,2})([0-9a-f]{1,2})([0-9a-f]{1,2})/i)
+      red_hex, green_hex, blue_hex = hex_string.strip.match(::CSVPlusPlus::Color::HEX_STRING_REGEXP)
                                         &.captures
                                         &.map { |s| s.length == 1 ? s + s : s }
+      raise(::CSVPlusPlus::Error::Error, "Invalid color: #{hex_string}") unless red_hex && green_hex && blue_hex
+
+      @red_hex = ::T.let(red_hex, ::String)
+      @green_hex = ::T.let(green_hex, ::String)
+      @blue_hex = ::T.let(blue_hex, ::String)
     end
+    # rubocop:enable Metrics/CyclomaticComplexity
 
+    sig { returns(::Float) }
     # The percent (decimal between 0-1) of red
     #
     # @return [Numeric]
@@ -27,6 +58,7 @@ module CSVPlusPlus
       hex_to_percent(@red_hex)
     end
 
+    sig { returns(::Float) }
     # The percent (decimal between 0-1) of green
     #
     # @return [Numeric]
@@ -34,6 +66,7 @@ module CSVPlusPlus
       hex_to_percent(@green_hex)
     end
 
+    sig { returns(::Float) }
     # The percent (decimal between 0-1) of blue
     #
     # @return [Numeric]
@@ -41,18 +74,15 @@ module CSVPlusPlus
       hex_to_percent(@blue_hex)
     end
 
+    sig { returns(::String) }
     # Create a hex representation of the color (without a '#')
     #
-    # @return [String]
+    # @return [::String]
     def to_hex
       [@red_hex, @green_hex, @blue_hex].join
     end
 
-    # @return [String]
-    def to_s
-      "Color(r: #{@red_hex}, g: #{@green_hex}, b: #{@blue_hex})"
-    end
-
+    sig { params(other: ::Object).returns(::T::Boolean) }
     # @return [boolean]
     def ==(other)
       other.is_a?(self.class) &&
@@ -63,6 +93,7 @@ module CSVPlusPlus
 
     private
 
+    sig { params(hex: ::String).returns(::Float) }
     def hex_to_percent(hex)
       hex.to_i(16) / 255.0
     end