tomlrb 2.0.4 → 3.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 79334a64c6114d7a83fd6446645fa3f83f5995fc92bb6e51c12579ea20846c8a
-  data.tar.gz: 007e015700fe6a9e739ba29d3840125c1ab859ed9039329843444cf5f4a735a1
+  metadata.gz: e6628b6e0662be0698c44c3dab186115a3424bfd9fe94dde4cb7c6e308d079ce
+  data.tar.gz: 81f84bf3dd98784ce68b7b6b7945461e381edb7234e84c86778f7d9eef2d8fab
 SHA512:
-  metadata.gz: 6b5090126db023bb2182a4dfd724d56c8f6cf9db33f775de2dedb14aaf4dca5844a4b00211bda3c8cecf68296e778dbe18be4e68486f112113f4dd2a4720df05
-  data.tar.gz: 5816f7c67269a251e71c37a4202f549b0356dcbfd5746ead8203638be1bbc9984a5847b167c225582ada7025f62d51bde71fce5742fa20d6f3762a1d7393730f
+  metadata.gz: 5bcefe0976b4840098240fd353784c886d58c1e1711a3130a8946751e85a670e170dceecd38ddb2557f06dd2d21ad1e96927abe0f52d1b17c855db6b1576ea29
+  data.tar.gz: bcdfd0a6f4638667e06d4228681b9cc764f849a16fb111125e3423910a00576f52e50c6c8749e0250b530d28ca4983e8e1eb0c9c2b513bf99da70123f6b0eaab

data/CHANGELOG.md

@@ -0,0 +1,103 @@
+### 3.0.0 - 2026-10-15
+
+Breaking changes:
+
+* Require Ruby 3.3 or later
+* Read documents as TOML 1.1 by default. Every valid TOML 1.0 document parses unchanged; 1.1 additionally accepts `\e` and `\xHH` escapes, times without seconds, and inline tables that span lines, contain comments or end with a comma. `version: "1.0"` keeps rejecting those
+* Fractional seconds are exact `Rational`s instead of `Float`s: `Time`, `LocalDateTime` and `LocalTime` values carry `56.123`, not `56.122999…`
+* Remove `Tomlrb::Scanner`, `Tomlrb::Handler`, `Tomlrb::StringUtils`, `Tomlrb::GeneratedParser`, `Tomlrb::Keys` and `Tomlrb::Key`; `Tomlrb::Parser.new` now takes the document string. Key redefinitions raise `Tomlrb::ParseError` as before
+* `require "tomlrb"` no longer loads `time` and `stringio`
+
+Features:
+
+* TOML 1.1 support, selected with `version: "1.1"` (default) or `"1.0"`
+
+Security:
+
+* Nesting depth of arrays and inline tables is limited only by memory: they are parsed with an explicit stack
+
+Performance:
+
+* Replace the Racc grammar with a hand-written single-pass parser; Racc is no longer used. Parsing executes 2.2-2.5x fewer instructions than perfect_toml 0.9.1 and 12-14x fewer than 2.0.4 (2.6-2.9x and 14-16x with YJIT)
+* Multi-line strings are scanned by searching for the closing quotes, a backslash or a control character: a 1 MB `"""` or `'''` string parses 25x faster, peak memory no longer grows by about 48 MB per MB of string, and quote-dense content (`ab"cd""ef`) no longer costs a step per quote
+
+Conformance (every TOML 1.0 and 1.1 case of toml-test 2.2.0 passes):
+
+* An escaped backslash right before a closing `"""` (`"""x\\"""`) ends the string
+* Reject a carriage return not followed by a line feed in multi-line strings, and trim a CRLF (not just LF) right after an opening `"""` or `'''`
+* Parse bare keys that start with `true`, `false`, `inf` or `nan` (`trueval`, `inf-x`) or look like dates and numbers (`1979-05-27a`, `0x1f`, `[[1e5]]`), and reject keys containing `:`
+* Reject empty values in inline tables (`{a = }`), inline tables that are not assigned to a key, and dotted keys that extend an inline table
+* Reject spaced array-of-tables brackets (`[ [a]]`, `[[a] ]`) on indented and CRLF lines too, and stop rejecting multi-line arrays whose line starts with `[ [`
+* Separate date and time only with `T`, `t` or a space, accept an offset only after a time, and reject negative years and days that do not exist (`1979-06-31T00:00:00-00:00`)
+* Reject floats with leading zeros (`01e5`) and accept underscores in exponents (`1e1_0`)
+
+Other:
+
+* Parse errors report the line and column (in characters)
+* `benchmarks/matrix.rb` measures time, allocations and peak memory over a set of generated workloads; `benchmarks/bench.rb` accepts several files
+* `LocalDate`, `LocalDateTime` and `LocalTime` validate their fields arithmetically and build their `Time` on first use
+* Ship RBS type signatures in `sig/`
+
+### 2.0.5 - 2026-10-15
+
+* Fix exponential backtracking (ReDoS) when scanning unterminated basic and literal strings ([GHSA-gq8g-3x53-v8fh](https://github.com/fbernier/tomlrb/security/advisories/GHSA-gq8g-3x53-v8fh))
+* Fix `SystemStackError` and quadratic scan time on long runs of spaces/tabs ([GHSA-mc56-vpwf-3m26](https://github.com/fbernier/tomlrb/security/advisories/GHSA-mc56-vpwf-3m26))
+* Literal strings no longer treat `\'` as an escaped quote, per the TOML spec
+* Add SECURITY.md: report vulnerabilities through GitHub private vulnerability reporting
+* Declare racc as a runtime dependency: Ruby 3.3 made it a bundled gem, so under Bundler `require "tomlrb"` failed unless the app's Gemfile pulled racc in
+* Stop shipping the unused `parser.y` grammar source in the gem
+
+Both vulnerabilities were reported by [@tsigouris007](https://github.com/tsigouris007), advised by [@kpatsakis](https://github.com/kpatsakis).
+
+### 2.0.4 - 2025-12-18
+
+* Update Racc to v1.8.1
+* Code cleanup
+* Fix conformance to TOML v1.0.0
+
+### 2.0.3 - 2022-05-28
+
+* Enable symbolize_keys: true for inline tables
+
+### 2.0.2 - 2022-05-25
+
+* Conform to TOML v1.0.0
+
+### 2.0.1 - 2020-12-20
+
+* Fix error with table nested under array-of-tables
+* Some performance improvements
+
+### 2.0.0 - 2020-11-26
+
+* Conform to TOML v1.0.0-rc3
+
+#### Breanking Changes ####
+
+TOML v0.5.0 introduced new value types: Local Date-Time, Local Date and Local Time which represent time without time zone information. Tomlrb also introduced corresponding classes starting from v2.0.0. By this change, some table values such as `2020-11-24T20:32:18`, `2020-11-24` or `20:32:18` are not treated as strings but as new the classes `Tomlrb::LocalDateTime`, `Tomlrb::LocalDate` and `Tomlrb::LocalTime` respectively. You can get the string values by calling `#to_s` method on any of those classes. Additionally, You can also get `Time` objects with the `#to_time` method. See [API documentation](https://www.rubydoc.info/gems/tomlrb) for the methods' details.
+
+### 1.3.0 - 2020-03-19
+
+* Fix error with falsy table values
+
+### 1.2.9 - 2019-11-22
+
+* Fixes and cleanups for ruby 2.7
+
+### 1.2.8 - 2018-12-18
+
+* Reduce gem size by excluding tests (tas50)
+* Make integer and float parsing closer to the spec (sgarciac)
+
+### 1.2.7 - 2018-07-12
+
+* Datetime should be UTC when no offset or timezone are specified
+
+### 1.2.6 - 2017-10-23
+
+* Fix issue where an unclosed table could make the parsing loop infinitely.
+* Proper string escaping
+
+### 1.1.3 - 2015-11-24
+
+* Bare integers can be used as keys as per the spec

data/README.md

@@ -0,0 +1,85 @@
+# Tomlrb
+
+[![CI](https://github.com/fbernier/tomlrb/actions/workflows/ci.yml/badge.svg)](https://github.com/fbernier/tomlrb/actions/workflows/ci.yml)
+[![Gem Version](https://img.shields.io/gem/v/tomlrb)](https://rubygems.org/gems/tomlrb)
+
+A fast, pure-Ruby [TOML](https://toml.io) parser with no runtime dependencies, for [TOML 1.1](https://toml.io/en/v1.1.0) and [1.0](https://toml.io/en/v1.0.0). It passes every case of the official [toml-test](https://github.com/toml-lang/toml-test) suite for both versions.
+
+Requires Ruby 3.3 or later. Runs on CRuby (with or without YJIT), JRuby and TruffleRuby.
+
+## Installation
+
+```sh
+bundle add tomlrb
+```
+
+## Usage
+
+```ruby
+require "tomlrb"
+
+Tomlrb.parse(%(title = "TOML"\n[owner]\nname = "Tom"))
+# => {"title" => "TOML", "owner" => {"name" => "Tom"}}
+
+Tomlrb.load_file("config.toml", symbolize_keys: true)
+# => {title: "TOML", owner: {name: "Tom"}}
+```
+
+`Tomlrb.parse` also accepts an IO. Invalid documents raise `Tomlrb::ParseError` with the line and column; impossible dates and times raise `ArgumentError`.
+
+Documents are read as TOML 1.1, which accepts every valid TOML 1.0 document unchanged. Pass `version: "1.0"` to also reject what 1.1 added (`\e` and `\xHH` escapes, times without seconds, inline tables spanning lines or ending with a comma), for example when a file must stay readable by a 1.0-only parser:
+
+```ruby
+Tomlrb.parse("t = 07:32")                 # => {"t" => #<Tomlrb::LocalTime: 07:32:00>}
+Tomlrb.parse("t = 07:32", version: "1.0") # raises Tomlrb::ParseError
+```
+
+TOML values map to Ruby as follows:
+
+| TOML | Ruby |
+|---|---|
+| string | `String` |
+| integer | `Integer` |
+| float | `Float` (`inf`, `nan` included) |
+| boolean | `true`, `false` |
+| offset date-time | `Time` |
+| local date-time | `Tomlrb::LocalDateTime` |
+| local date | `Tomlrb::LocalDate` |
+| local time | `Tomlrb::LocalTime` |
+| array | `Array` |
+| table, inline table | `Hash` |
+
+Fractional seconds are exact (`Rational`), and the local types convert with `#to_time`.
+
+Type signatures for the public API are in [`sig/`](sig/tomlrb.rbs).
+
+## Benchmark
+
+`ruby benchmarks/bench.rb [file...]` compares against [toml-rb](https://github.com/emancu/toml-rb) and [perfect_toml](https://github.com/mame/perfect_toml) when they are installed. Each iteration parses every given file (`benchmarks/example.toml` by default, a 4.6 KB TOML 1.0 document using every value type and table form, which all three parsers accept).
+
+`ruby [--yjit] benchmarks/matrix.rb` runs one parser over generated workloads, one per input shape (plain, escaped and 1 MB strings, plain and quote-dense multi-line strings, signed and unsigned numbers, floats, inline tables, date-times, many small documents), and reports time, Ruby allocations, peak RSS growth and a digest of the results per workload. `--lib DIR` loads tomlrb from another checkout, so equal digests show that two versions parse alike; `--parser perfect_toml` measures perfect_toml instead. Inputs keep the known encoding code range a string from `File.read` has; `--unknown-coderange` also times the UTF-8 validation of a string whose code range Ruby has not computed.
+
+CPU instructions per parse (fewer is better), measured with `perf stat` on CRuby 3.3 without YJIT. With YJIT tomlrb's lead grows: 2.6-2.9x fewer instructions than perfect_toml and 14-16x fewer than 2.0.4. The crates.io corpus is the first 300 `Cargo.toml` files of a local Cargo registry (`~/.cargo/registry/src/*/*/Cargo.toml`, sorted by path) that all three parsers accept.
+
+| Input | tomlrb | perfect_toml 0.9.1 | tomlrb 2.0.4 |
+|---|---|---|---|
+| `benchmarks/example.toml` (4.6 KB) | 3.0 M | 6.6 M | 34.1 M |
+| 300 crates.io `Cargo.toml` files (622 KB) | 243 M | 559 M | 3,100 M |
+| rust-lang/rust `Cargo.lock` at `56df459` (158 KB) | 48 M | 121 M | 687 M |
+
+`ruby benchmarks/profile.rb [file]` records a [Vernier](https://github.com/jhawthorn/vernier) profile.
+
+## Development
+
+```sh
+bin/setup            # fetch the toml-test submodule and install gems
+bundle exec rake     # tests, Standard (lint), RBS validation
+bundle exec rake toml_test   # the official toml-test runner (needs Go)
+bin/console          # IRB with tomlrb loaded
+```
+
+Releases are published from GitHub Actions with [trusted publishing](https://guides.rubygems.org/trusted-publishing/): bump `lib/tomlrb/version.rb`, add the CHANGELOG entry, then run the `release` workflow.
+
+## Thanks
+
+Thanks to [@jpbougie](https://github.com/jpbougie) for the crash course on the Chomsky hierarchy and general tips.

data/lib/tomlrb/local_date.rb

@@ -1,32 +1,43 @@
-# frozen-string-literal: true
-
-require 'forwardable'
+# frozen_string_literal: true
 
 module Tomlrb
+  # A TOML local date (no time, no offset), e.g. `1979-05-27`.
   class LocalDate
-    extend Forwardable
+    DAYS_IN_MONTH = [nil, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31].freeze
+
+    # @return [Boolean] whether year, month and day (Integers) name a real calendar day
+    def self.valid_date?(year, month, day)
+      return false unless month.between?(1, 12) && day >= 1
 
-    def_delegators :@time, :year, :month, :day
+      day <= DAYS_IN_MONTH[month] ||
+        (month == 2 && day == 29 && year % 4 == 0 && (year % 100 != 0 || year % 400 == 0))
+    end
 
+    attr_reader :year, :month, :day
+
+    # The Time behind #to_time is built on first use: parsing creates many of these
+    # and building a Time costs more than validating the fields.
     def initialize(year, month, day)
-      @time = Time.utc(year, month, day, 0, 0, 0)
-      raise ArgumentError, "Invalid Local Date: #{year}-#{month}-#{day}" unless day.to_i == @time.day && month.to_i == @time.month && year.to_i == @time.year
+      @year = year.to_i
+      @month = month.to_i
+      @day = day.to_i
+      raise ArgumentError, "Invalid Local Date: #{self}" unless LocalDate.valid_date?(@year, @month, @day)
     end
 
     # @param offset see {LocalDateTime#to_time}
     # @return [Time] 00:00:00 of the date
-    def to_time(offset = '-00:00')
-      return @time if offset == '-00:00'
-      Time.new(year, month, day, 0, 0, 0, offset)
+    def to_time(offset = "-00:00")
+      return @time ||= Time.utc(@year, @month, @day, 0, 0, 0) if offset == "-00:00"
+
+      Time.new(@year, @month, @day, 0, 0, 0, offset)
     end
 
     def to_s
-      @time.strftime('%F')
+      format("%04d-%02d-%02d", @year, @month, @day)
     end
 
     def ==(other)
-      other.is_a?(self.class) &&
-        to_time == other.to_time
+      other.is_a?(self.class) && @year == other.year && @month == other.month && @day == other.day
     end
 
     def inspect

data/lib/tomlrb/local_date_time.rb

@@ -1,44 +1,57 @@
-# frozen-string-literal: true
+# frozen_string_literal: true
 
-require 'forwardable'
+require "forwardable"
+require_relative "local_date"
+require_relative "local_time"
 
 module Tomlrb
+  # A TOML local date-time (no offset), e.g. `1979-05-27T07:32:00.999999`.
   class LocalDateTime
     extend Forwardable
 
-    def_delegators :@time, :year, :month, :day, :hour, :min, :sec, :usec, :nsec
+    attr_reader :year, :month, :day, :hour, :min
 
-    def initialize(year, month, day, hour, min, sec) # rubocop:disable Metrics/ParameterLists
-      @time = Time.utc(year, month, day, hour, min, sec)
-      raise ArgumentError, "Invalid Local Date-Time: #{year}-#{month}-#{day}T#{hour}:#{min}:#{sec}" unless min.to_i == @time.min && hour.to_i == @time.hour && day.to_i == @time.day && month.to_i == @time.month && year.to_i == @time.year
+    def_delegators :to_time, :sec, :usec, :nsec
 
+    # Fields are validated here; the Time behind them is built on first use.
+    #
+    # @param sec [Integer, Rational, Float] seconds, fraction included
+    def initialize(year, month, day, hour, min, sec) # rubocop:disable Metrics/ParameterLists
+      @year = year.to_i
+      @month = month.to_i
+      @day = day.to_i
+      @hour = hour.to_i
+      @min = min.to_i
       @sec = sec
+      raise ArgumentError, "Invalid Local Date-Time: #{self}" unless
+        LocalDate.valid_date?(@year, @month, @day) && @hour < 24 && @min < 60 && sec < 60
     end
 
-    # @param offset [String, Symbol, Numeric, nil] time zone offset.
-    #   * when +String+, must be '+HH:MM' format, '-HH:MM' format, 'UTC', 'A'..'I' or 'K'..'Z'. Arguments excluding '+-HH:MM' are supporeted at Ruby >= 2.7.0
-    #   * when +Symbol+, must be +:dst+(for summar time for local) or +:std+(for standard time).
-    #   * when +Numeric+, it is time zone offset in second.
-    #   * when +nil+, local time zone offset is used.
-    # @return [Time]
-    def to_time(offset = '-00:00')
-      return @time if offset == '-00:00'
-      Time.new(year, month, day, hour, min, @sec, offset)
+    # @param offset [String, Integer, nil] UTC offset as accepted by Time.new
+    #   ("+09:00", "UTC", "Z", seconds east of UTC, or nil for the local zone)
+    # @return [Time] this date-time at the given offset (UTC by default)
+    def to_time(offset = "-00:00")
+      return @time ||= Time.utc(@year, @month, @day, @hour, @min, @sec) if offset == "-00:00"
+
+      Time.new(@year, @month, @day, @hour, @min, @sec, offset)
     end
 
     def to_s
-      frac = (@sec - sec)
-      frac_str = frac.zero? ? '' : frac.to_s[1..-1]
-      @time.strftime('%FT%T') << frac_str
+      format("%04d-%02d-%02dT", @year, @month, @day) << LocalTime.format(@hour, @min, @sec)
     end
 
     def ==(other)
-      other.is_a?(self.class) &&
-        to_time == other.to_time
+      other.is_a?(self.class) && @year == other.year && @month == other.month && @day == other.day &&
+        @hour == other.hour && @min == other.min && @sec == other.seconds
     end
 
     def inspect
       "#<#{self.class}: #{self}>"
     end
+
+    protected
+
+    # Seconds with their fraction; #sec is the whole part only, like Time#sec.
+    def seconds = @sec
   end
 end

data/lib/tomlrb/local_time.rb

@@ -1,41 +1,61 @@
-# frozen-string-literal: true
+# frozen_string_literal: true
 
-require 'forwardable'
+require "forwardable"
 
 module Tomlrb
+  # A TOML local time (no date, no offset), e.g. `07:32:00.999999`.
   class LocalTime
     extend Forwardable
 
-    def_delegators :@time, :hour, :min, :sec, :usec, :nsec
+    attr_reader :hour, :min
 
+    def_delegators :time, :sec, :usec, :nsec
+
+    # @return [String] `hh:mm:ss`, plus `.` and the fraction digits if `sec` has any
+    def self.format(hour, min, sec)
+      whole = sec.floor
+      fraction = sec - whole
+      text = Kernel.format("%02d:%02d:%02d", hour, min, whole)
+      fraction.zero? ? text : text << Kernel.format("%.9f", fraction)[1..].sub(/0+\z/, "")
+    end
+
+    # Fields are validated here; the Time behind them is built on first use.
+    #
+    # @param sec [Integer, Rational, Float] seconds, fraction included
     def initialize(hour, min, sec)
-      @time = Time.utc(0, 1, 1, hour, min, sec)
-      raise ArgumentError, "Invalid Local Time: #{hour}-#{min}-#{sec}" unless min.to_i == @time.min && hour.to_i == @time.hour
+      @hour = hour.to_i
+      @min = min.to_i
       @sec = sec
+      raise ArgumentError, "Invalid Local Time: #{self}" unless @hour < 24 && @min < 60 && sec < 60
     end
 
-    # @param year [Integer]
-    # @param month [Integer]
-    # @param day [Integer]
     # @param offset see {LocalDateTime#to_time}
-    # @return [Time] the time of the date specified by params
-    def to_time(year, month, day, offset = '-00:00')
-      Time.new(year, month, day, hour, min, @sec, offset)
+    # @return [Time] this time on the given day
+    def to_time(year, month, day, offset = "-00:00")
+      Time.new(year, month, day, @hour, @min, @sec, offset)
     end
 
     def to_s
-      frac = (@sec - sec)
-      frac_str = frac.zero? ? '' : frac.to_s[1..-1]
-      @time.strftime('%T') << frac_str
+      LocalTime.format(@hour, @min, @sec)
     end
 
     def ==(other)
-      other.is_a?(self.class) &&
-        @time == other.to_time(0, 1, 1)
+      other.is_a?(self.class) && @hour == other.hour && @min == other.min && @sec == other.seconds
     end
 
     def inspect
       "#<#{self.class}: #{self}>"
     end
+
+    protected
+
+    # Seconds with their fraction; #sec is the whole part only, like Time#sec.
+    def seconds = @sec
+
+    private
+
+    def time
+      @time ||= Time.utc(0, 1, 1, @hour, @min, @sec)
+    end
   end
 end