dbf 5.5.0 → 5.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ff36c9c3d6b7a558cd5bf0f703f64f0cb7b22cbf6b4c52b843c855328a712a0a
-  data.tar.gz: ac0f725043e7dc9102a32381f6041e54ebd852a67a264778ae58b2a59430b01e
+  metadata.gz: e9d4250b08de3a7a96f24fff3f55e5457524288951b9d0ca8ffc8127aae2c790
+  data.tar.gz: 78d946bd48943b8d6527aac3e982017133480d8263bc5025d2c8d3a85407185e
 SHA512:
-  metadata.gz: 269035188f9741d80575efca78ae0bda153c4f27c79dbe3f3e6221fc714be22eb2fb9ec39899dcfc9d489f76d60bfaf1f892d6eeaaeea80fa406f5f401e909d7
-  data.tar.gz: 6203ede51060bad9d193b15bb98d814aa870d99dd29d6eb8ddd4cb2b5e04f078f522fb7f3051befcbebe11be86a13307da38f9981bbcb704c5307ebdb16dd6a3
+  metadata.gz: b2e613d19239abb6df53c941c952da129bb77ff6cfbcc7b91cdeac10d9f8ad28724102c20ad6f53533959e0adc1f0f159ffadd0811f4997b623633d5c9fa2edf
+  data.tar.gz: 5964675e521e61be0d3da141c642cfc13c0e71b232ae2a4070053d0764b3e3ee1ca653a786ef70f3001b98f24b4eaf05eee9c4a81a86fd14b2506a939ea2eba3

data/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## 5.6.0
+
+### Fixed
+
+- Security (CWE-200, CWE-400): memo auto-discovery now looks only for a memo with exactly the table's basename (`.fpt`/`.dbt`, any case), by scanning the directory rather than globbing. Previously `1.dbf` could pick up `12.fpt`, and glob metacharacters in an uploaded filename were expanded
+- Security (CWE-400): column parsing stops at `header_length` and drops columns that do not fit within `record_length`, so a small crafted file can no longer declare thousands of overlapping columns and cause quadratic decoding work and output
+- Security (CWE-838): string values that need no transcoding have invalid bytes replaced, as transcoded values already did, so they no longer raise in JSON generation, HTML escaping, or database inserts
+- Security (CWE-248): a DateTime (`T`) column narrower than 8 bytes decodes to nil instead of raising `NoMethodError`
+- Security (CWE-915): column names (and Visual FoxPro `.dbc` table names) such as `to_str`, `to_ary`, `to_hash`, `to_proc`, or `to_partial_path` are no longer exposed as methods, so a crafted file cannot hijack implicit conversions or Rails rendering; those columns stay readable via `#[]` and `#attributes`
+- Security (CWE-129): `Table#record` returns nil for a negative index instead of reading header bytes as a record or raising `Errno::EINVAL`
+- Security (CWE-150): CLI terminal output also replaces UTF-8 encoded C1 control characters (U+0080–U+009F, e.g. the single-character CSI U+009B)
+- Security (CWE-400): a table declares no more columns than its records have bytes, so zero-length columns can no longer bypass the column bound (an 84 KB file previously produced 2,000 nil cells per one-byte record: a 120 MB CSV and 34 s of work)
+- Security (CWE-248): a dBase III/IV memo pointer too large to seek to decodes to nil instead of raising `RangeError`
+- Tables with the Macintosh Central European language driver (`0x97`) decode through a vendored code page table instead of raising `Encoding::ConverterNotFoundError`, since Ruby has no converter for `macCentEuro`
+- dBase III memos end at the `0x1A` terminator; bytes left over after it in the same block are no longer appended to the memo
+- Visual FoxPro long column names from a `.dbc` container are no longer decoded a second time with the table's code page, which garbled non-ASCII names (`Café` became `Café`)
+- Security (CWE-248): `Table.new` raises only `DBF::Error` subclasses for a bad path or encoding, so rescuing `DBF::Error` is enough. A directory or unreadable file (data or memo) raises the new `DBF::FileAccessError` instead of `Errno::EISDIR`/`Errno::EACCES`. A missing memo path, or a path containing a null byte, raises `DBF::FileNotFoundError` instead of `Errno::ENOENT`/`ArgumentError`. An unknown encoding, or one Ruby cannot transcode such as UTF-7, raises the new `DBF::UnsupportedEncodingError` when the table is opened, rather than `ArgumentError` or `Encoding::ConverterNotFoundError` on the first read. The data file is closed if its memo cannot be opened
+- Security (CWE-209): `DBF::FileNotFoundError` and `DBF::FileAccessError` messages name only the file (`file not found: data.dbf`), not its directory, so an app that shows error messages to users does not reveal server paths. The full path is available as `#path`. Both now inherit from the new `DBF::FileError`
+- The `dbf` command reports a file that cannot be opened (such as a directory) as an error and exits 1, instead of printing a stack trace
+
+### Changed
+
+- Code that rescued `ArgumentError`, `Errno::*` or `Encoding::ConverterNotFoundError` from `Table.new` should rescue `DBF::Error` (or the specific subclasses above) instead. Passing something other than a path or IO still raises `ArgumentError`
+- File error messages no longer include the directory (`file not found: data.dbf` rather than `file not found: /srv/uploads/data.dbf`). Use `DBF::FileError#path` for the full path
+
+### Added
+
+- Security (CWE-400): optional `max_memo_size:` for `DBF::Table.new`, `DBF::Table.open` and `DBF::Database::Foxpro.new`. Each memo is cut off after that many bytes, and no more than that is read from the memo file, so a small table whose records all point at one large (or unterminated) memo can no longer produce output far larger than the upload. Memos are read in full by default
+- `rake fuzz`: coverage-guided mutation fuzzer for the read path (`spec/fuzz/fuzz.rb`), also run as a non-blocking CI job
+- A warning when column descriptors are ignored because they do not fit within the header length or record length, so a file written with wrong length fields no longer loses columns silently. The warning holds no text from the file, and is silenced when `$VERBOSE` is nil
+- README: "Processing untrusted files" section with guidance for accepting DBF uploads, including how to strip Unicode bidirectional controls and zero-width characters, which the library passes through
+
 ## 5.5.0
 
 ### Added
@@ -15,7 +47,6 @@
 - Docs site (dbf.infused.org): YARD docs are now built and deployed by CI instead of being committed to the repository; the Pages workflow's deprecated v2/v3 actions (whose artifact backend was shut down in early 2025) are updated to current versions, repairing the broken deploy
 - Dependabot updates for GitHub Actions and gems (weekly), a security policy (SECURITY.md) with private vulnerability reporting, and contributor documentation (CONTRIBUTING.md)
 - Automated releases: pushing a `vX.Y.Z` tag now builds and publishes the gem to RubyGems.org from CI via Trusted Publishing (OIDC) — no long-lived API key. `bundler/gem_tasks` is loaded in the Rakefile, providing `rake build` / `rake release`. Git tags resume at v5.4.0; versions 4.0.0–5.3.0 were released without tags
-
 - Ractor safety: all library constants are verified Ractor-shareable and reading/exporting tables inside non-main Ractors is covered by specs
 - CLI: `-j` exports records as a JSON array and `-J` as JSON Lines (one record per line, streamed in constant memory); deleted records are skipped, control bytes are escaped by JSON itself, and binary cells are scrubbed instead of raising
 

data/README.md

@@ -398,6 +398,46 @@ Data type descriptions
 * O = Double
 * + = Autoincrement
 
+## Processing untrusted files
+
+The parser is hardened against malformed files: crafted headers cannot force
+huge allocations, unbounded loops, path traversal, or crashes from truncated
+data. If you accept DBF uploads (for example in a web application), also:
+
+* Treat every column name and value as untrusted text. HTML-escape it before
+  rendering it, and escape JSON before embedding it in a `<script>` tag.
+* Pass the upload as a `StringIO` (with an explicit memo `StringIO` if there is
+  one) rather than as a path, so memo files are never looked up in a shared
+  upload directory.
+* Limit upload size and processing time. Many records can point at the same
+  large memo, so exported output can be far larger than the uploaded files.
+  Pass `max_memo_size:` to cut off each memo after that many bytes:
+
+  ```ruby
+  DBF::Table.new(upload, memo_upload, max_memo_size: 64 * 1024)
+  ```
+
+  `DBF::Database::Foxpro.new` takes the same option.
+* Rescue `DBF::Error`, and also `StandardError` if a malformed file must never
+  fail a request.
+* `Table#to_csv` prefixes a `'` to string cells that begin with `=`, `+`, `-`,
+  `@`, tab, or carriage return so spreadsheets do not run them as formulas.
+* Text is passed through as it is, including Unicode bidirectional controls
+  (such as U+202E RIGHT-TO-LEFT OVERRIDE) and zero-width characters. These
+  make displayed text differ from the underlying data (in a web page, a CSV
+  export, or `dbf` terminal output), but some real data needs them, so the
+  library does not remove them. Strip or flag them yourself where that
+  matters:
+
+  ```ruby
+  value.gsub(/[​-‏‪-‮⁠-⁤⁦-⁩]/, '')
+  ```
+
+* Column descriptors that do not fit within the header length or record
+  length are ignored, and the library prints a warning (silenced when
+  `$VERBOSE` is nil). If a file you trust loses columns, its header has wrong
+  length fields.
+
 ## Limitations
 
 * DBF is read-only

data/lib/dbf/cli.rb

@@ -22,6 +22,9 @@ module DBF
     CONTROL_BYTES = /[\x00-\x1F\x7F]/n
     # The same, but keeping CR and LF so CSV row separators survive.
     CONTROL_BYTES_KEEPING_NEWLINES = /[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]/n
+    # C1 controls (U+0080..U+009F) as UTF-8 bytes. Some terminals honor
+    # these, e.g. U+009B as a single-character CSI.
+    UTF8_C1_CONTROLS = /\xC2[\x80-\x9F]/n
 
     # Replaces terminal control bytes. Substitution happens on a byte copy so
     # a value whose bytes are invalid in its encoding cannot raise here.
@@ -31,7 +34,10 @@ module DBF
     # @return [String]
     def self.sanitize(value, pattern = CONTROL_BYTES)
       string = value.to_s
-      string.b.gsub(pattern, '?').force_encoding(string.encoding)
+      encoding = string.encoding
+      bytes = string.b.gsub(pattern, '?')
+      bytes = bytes.gsub(UTF8_C1_CONTROLS, '?') if encoding == Encoding::UTF_8
+      bytes.force_encoding(encoding)
     end
 
     # Wraps an IO so text written to an interactive terminal is stripped of
@@ -94,8 +100,8 @@ module DBF
         send(action, filename) if action
       end
       0
-    rescue DBF::FileNotFoundError => e
-      @stderr.puts "DBF::FileNotFoundError: #{e.message}"
+    rescue DBF::FileError => e
+      @stderr.puts "#{e.class}: #{e.message}"
       1
     end
 

data/lib/dbf/column.rb

@@ -15,7 +15,7 @@ module DBF
 
     attr_reader :name, :type, :length, :decimal
 
-    # rubocop:disable Style/MutableConstant
+    # rubocop:disable-next Style/MutableConstant
     TYPE_CAST_CLASS = {
       N: ColumnType::Number,
       I: ColumnType::SignedLong,
@@ -29,7 +29,6 @@ module DBF
       G: ColumnType::General,
       :+ => ColumnType::AutoIncrement
     }
-    # rubocop:enable Style/MutableConstant
     TYPE_CAST_CLASS.default = ColumnType::String
     TYPE_CAST_CLASS.freeze
 
@@ -53,6 +52,17 @@ module DBF
 
     def encoding = @table.encoding
 
+    # Returns a copy of the column with a different name. The name is used
+    # as given rather than decoded from the table's code page, so it must
+    # already be a decoded string, such as a Visual FoxPro long name read
+    # from the .dbc container.
+    #
+    # @param name [String]
+    # @return [DBF::Column]
+    def with_name(name) # :nodoc:
+      dup.tap { |column| column.rename(name) }
+    end
+
     # @param value [String]
     def type_cast(value)
       type_cast_class.type_cast(value)
@@ -84,6 +94,14 @@ module DBF
       @underscored_name ||= name.gsub(/([a-z\d])([A-Z])/, '\1_\2').tr('-', '_').downcase
     end
 
+    protected
+
+    def rename(name) # :nodoc:
+      @name = name
+      @underscored_name = nil
+      validate_name
+    end
+
     private
 
     def clean(value) # :nodoc:

data/lib/dbf/column_builder.rb

@@ -2,22 +2,46 @@
 
 module DBF
   class ColumnBuilder
-    def initialize(table, data, version_config)
+    def initialize(table, data, version_config, header_length:, record_length:)
       @table = table
       @data = data
       @version_config = version_config
+      @header_length = header_length
+      # One byte of every record is the deletion flag
+      @max_width = record_length - 1
     end
 
     def build
       safe_seek do
         @data.seek(@version_config.header_size)
+        width = 0
         [].tap do |columns|
           until end_of_record?
+            count = columns.size
+            # Descriptors live inside the header. Stop at header_length so a
+            # crafted file cannot declare thousands of columns over its
+            # record area.
+            if past_header?
+              warn_ignored(count, "header length of #{@header_length}")
+              break
+            end
+
             args = @version_config.read_column_args(@table, @data)
             # A descriptor truncated by EOF is returned as nil, or unpacks to
             # a nil length; stop rather than constructing an invalid column
             # (which would crash on nil < 0).
-            break if args.nil? || args[3].nil?
+            length = args&.[](3)
+            break if length.nil?
+
+            # Columns longer than the record would read into the following
+            # records; stop so a crafted file cannot multiply decoding work.
+            # Zero-length columns take no record bytes, so also allow at most
+            # one column per record byte.
+            width += length
+            if width > @max_width || count >= @max_width
+              warn_ignored(count, "record length of #{@max_width + 1}")
+              break
+            end
 
             columns << Column.new(*args)
           end
@@ -27,6 +51,18 @@ module DBF
 
     private
 
+    def past_header?
+      @data.pos + @version_config.column_layout.first > @header_length
+    end
+
+    # Columns that do not fit are dropped rather than read, so say so: a file
+    # written with a wrong header or record length would otherwise lose
+    # columns silently. The message holds no text from the file, so a
+    # crafted column name cannot inject lines into a log.
+    def warn_ignored(kept, limit)
+      warn "DBF: ignored the columns after the first #{kept}: they do not fit within the #{limit} bytes"
+    end
+
     def end_of_record?
       safe_seek do
         byte = @data.read(1)

data/lib/dbf/column_type.rb

@@ -121,7 +121,8 @@ module DBF
       # @param value [String]
       def type_cast(value)
         days, msecs = value.unpack('l2')
-        return nil if days.nil? || days.zero?
+        # A column narrower than 8 bytes leaves msecs nil
+        return nil if days.nil? || msecs.nil? || days.zero?
 
         secs = (msecs / 1000).to_i
         ::DateTime.jd(days, (secs / 3600).to_i, (secs / 60).to_i % 60, secs % 60).to_time
@@ -170,7 +171,13 @@ module DBF
       private
 
       def encode(value)
-        return value.force_encoding(encoding) unless @needs_encode
+        unless @needs_encode
+          # No transcoding happens here, so replace invalid bytes the same
+          # way Encoder.encode does rather than handing callers a string that
+          # raises in JSON generation, HTML escaping, or a database insert.
+          value.force_encoding(encoding)
+          return value.valid_encoding? ? value : value.scrub
+        end
 
         Encoder.encode(value, encoding, @target_encoding)
       end

data/lib/dbf/database/foxpro.rb

@@ -20,13 +20,16 @@ module DBF
       #  contacts = db.contacts.record(0)
       #
       # @param path [String]
-      def initialize(path)
+      # @param max_memo_size [optional Integer] Bytes to read from each memo
+      #   of the container and its tables; see DBF::Table.new
+      def initialize(path, max_memo_size: nil)
         @path = path
         @dirname = File.dirname(@path)
-        @db = DBF::Table.new(@path)
+        @max_memo_size = max_memo_size
+        @db = DBF::Table.new(@path, max_memo_size:)
         @tables = extract_dbc_data
       rescue Errno::ENOENT
-        raise DBF::FileNotFoundError, "file not found: #{path}"
+        raise FileHandler.not_found(path)
       end
 
       def table_names
@@ -38,7 +41,7 @@ module DBF
       # @param name [String]
       # @return [DBF::Table]
       def table(name)
-        Table.new(table_path(name), long_names: @tables[name])
+        Table.new(table_path(name), long_names: @tables[name], max_memo_size: @max_memo_size)
       end
 
       # Searches the database directory for the table's dbf file
@@ -71,15 +74,21 @@ module DBF
 
       def method_missing(method, *args) # :nodoc:
         name = method.to_s
-        table_names.index(name) ? table(name) : super
+        table_method?(name) ? table(name) : super
       end
 
       def respond_to_missing?(method, *)
-        table_names.index(method.to_s) || super
+        table_method?(method.to_s) || super
       end
 
       private
 
+      # Table names come from the container file, so implicit-conversion
+      # method names are never dispatched to a table (see Record).
+      def table_method?(name) # :nodoc:
+        !Record::RESERVED_METHODS.include?(name) && table_names.include?(name)
+      end
+
       # This method extracts the data from the database container. This is
       # just an ordinary table with a treelike structure. Field definitions
       # are in the same order as in the linked tables but only the long name
@@ -108,17 +117,18 @@ module DBF
     class Table < DBF::Table
       attr_reader :long_names
 
-      def initialize(path, long_names:)
+      def initialize(path, long_names:, max_memo_size: nil)
         @long_names = long_names
-        super(path)
+        super(path, max_memo_size:)
       end
 
+      # Replaces each column's short name with its long name from the
+      # container. Long names were decoded when the .dbc was read, so they
+      # are not decoded again with this table's code page.
       def build_columns # :nodoc:
-        # modify the column definitions to use the long names as the
-        # columnname property is readonly, recreate the column definitions
         super.each_with_index.map do |column, index|
-          long_name = long_names[index] || column.name
-          Column.new(self, long_name, column.type, column.length, column.decimal)
+          long_name = long_names[index]
+          long_name ? column.with_name(long_name) : column
         end
       end
     end

data/lib/dbf/encoder.rb

@@ -4,12 +4,15 @@ module DBF
   # Transcodes strings from a source encoding to a target encoding.
   #
   # Most DBF code pages map to encodings built into Ruby and are converted
-  # with String#encode. A few (Mazovia cp620, Kamenický cp895) have no Ruby
-  # equivalent and are decoded through vendored byte-to-Unicode tables.
+  # with String#encode. A few (Mazovia cp620, Kamenický cp895, Macintosh
+  # Central European) have no Ruby converter and are decoded through
+  # vendored byte-to-Unicode tables.
   module Encoder
     # Byte-to-Unicode tables generated from
-    # https://github.com/SheetJS/js-codepage/blob/master/codepages/620.TBL and
-    # https://github.com/SheetJS/js-codepage/blob/master/codepages/895.TBL
+    # https://github.com/SheetJS/js-codepage/blob/master/codepages/620.TBL,
+    # https://github.com/SheetJS/js-codepage/blob/master/codepages/895.TBL and
+    # (Mac Central European) Python's mac_latin2 codec, which matches iconv's
+    # MACCENTRALEUROPE
 
     # Mazovia (Polish) MS-DOS
     CP620_TO_UNICODE = [
@@ -83,9 +86,46 @@ module DBF
       0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2, 0x25A0, 0x00A0
     ].freeze
 
+    # Macintosh Central European (Ruby has the encoding but no converter)
+    MACCENTEURO_TO_UNICODE = [
+      0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007,
+      0x0008, 0x0009, 0x000A, 0x000B, 0x000C, 0x000D, 0x000E, 0x000F,
+      0x0010, 0x0011, 0x0012, 0x0013, 0x0014, 0x0015, 0x0016, 0x0017,
+      0x0018, 0x0019, 0x001A, 0x001B, 0x001C, 0x001D, 0x001E, 0x001F,
+      0x0020, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025, 0x0026, 0x0027,
+      0x0028, 0x0029, 0x002A, 0x002B, 0x002C, 0x002D, 0x002E, 0x002F,
+      0x0030, 0x0031, 0x0032, 0x0033, 0x0034, 0x0035, 0x0036, 0x0037,
+      0x0038, 0x0039, 0x003A, 0x003B, 0x003C, 0x003D, 0x003E, 0x003F,
+      0x0040, 0x0041, 0x0042, 0x0043, 0x0044, 0x0045, 0x0046, 0x0047,
+      0x0048, 0x0049, 0x004A, 0x004B, 0x004C, 0x004D, 0x004E, 0x004F,
+      0x0050, 0x0051, 0x0052, 0x0053, 0x0054, 0x0055, 0x0056, 0x0057,
+      0x0058, 0x0059, 0x005A, 0x005B, 0x005C, 0x005D, 0x005E, 0x005F,
+      0x0060, 0x0061, 0x0062, 0x0063, 0x0064, 0x0065, 0x0066, 0x0067,
+      0x0068, 0x0069, 0x006A, 0x006B, 0x006C, 0x006D, 0x006E, 0x006F,
+      0x0070, 0x0071, 0x0072, 0x0073, 0x0074, 0x0075, 0x0076, 0x0077,
+      0x0078, 0x0079, 0x007A, 0x007B, 0x007C, 0x007D, 0x007E, 0x007F,
+      0x00C4, 0x0100, 0x0101, 0x00C9, 0x0104, 0x00D6, 0x00DC, 0x00E1,
+      0x0105, 0x010C, 0x00E4, 0x010D, 0x0106, 0x0107, 0x00E9, 0x0179,
+      0x017A, 0x010E, 0x00ED, 0x010F, 0x0112, 0x0113, 0x0116, 0x00F3,
+      0x0117, 0x00F4, 0x00F6, 0x00F5, 0x00FA, 0x011A, 0x011B, 0x00FC,
+      0x2020, 0x00B0, 0x0118, 0x00A3, 0x00A7, 0x2022, 0x00B6, 0x00DF,
+      0x00AE, 0x00A9, 0x2122, 0x0119, 0x00A8, 0x2260, 0x0123, 0x012E,
+      0x012F, 0x012A, 0x2264, 0x2265, 0x012B, 0x0136, 0x2202, 0x2211,
+      0x0142, 0x013B, 0x013C, 0x013D, 0x013E, 0x0139, 0x013A, 0x0145,
+      0x0146, 0x0143, 0x00AC, 0x221A, 0x0144, 0x0147, 0x2206, 0x00AB,
+      0x00BB, 0x2026, 0x00A0, 0x0148, 0x0150, 0x00D5, 0x0151, 0x014C,
+      0x2013, 0x2014, 0x201C, 0x201D, 0x2018, 0x2019, 0x00F7, 0x25CA,
+      0x014D, 0x0154, 0x0155, 0x0158, 0x2039, 0x203A, 0x0159, 0x0156,
+      0x0157, 0x0160, 0x201A, 0x201E, 0x0161, 0x015A, 0x015B, 0x00C1,
+      0x0164, 0x0165, 0x00CD, 0x017D, 0x017E, 0x016A, 0x00D3, 0x00D4,
+      0x016B, 0x016E, 0x00DA, 0x016F, 0x0170, 0x0171, 0x0172, 0x0173,
+      0x00DD, 0x00FD, 0x0137, 0x017B, 0x0141, 0x017C, 0x0122, 0x02C7
+    ].freeze
+
     CUSTOM_TABLES = {
       'cp620' => CP620_TO_UNICODE,
-      'cp895' => CP895_TO_UNICODE
+      'cp895' => CP895_TO_UNICODE,
+      'maccenteuro' => MACCENTEURO_TO_UNICODE
     }.freeze
 
     # Is the encoding a code page without a Ruby Encoding, decoded through
@@ -97,6 +137,21 @@ module DBF
       encoding.is_a?(::String) && CUSTOM_TABLES.key?(encoding.downcase)
     end
 
+    # Raise unless strings in the encoding can be transcoded to Unicode, so
+    # an unknown name or a dummy encoding such as UTF-7 fails up front with
+    # a DBF::Error rather than with ArgumentError or
+    # Encoding::ConverterNotFoundError on the first read.
+    #
+    # @param encoding [String, Encoding]
+    # @raise [DBF::UnsupportedEncodingError]
+    def self.check(encoding)
+      return if custom?(encoding)
+
+      (+'').force_encoding(encoding).encode(Encoding::UTF_8)
+    rescue ArgumentError, TypeError, Encoding::ConverterNotFoundError
+      raise DBF::UnsupportedEncodingError, "unsupported encoding: #{encoding}"
+    end
+
     # Transcode a string from source_encoding to target_encoding
     #
     # @param string [String]

data/lib/dbf/errors.rb

@@ -6,7 +6,30 @@ module DBF
   class Error < StandardError
   end
 
-  class FileNotFoundError < Error
+  # A file cannot be opened. The message names only the file, not its
+  # directory, so an app that shows error messages to users does not reveal
+  # server paths; #path has the full path for logging.
+  class FileError < Error
+    # @return [String, nil] the full path of the file
+    attr_reader :path
+
+    def initialize(message = nil, path: nil)
+      super(message)
+      @path = path
+    end
+  end
+
+  class FileNotFoundError < FileError
+  end
+
+  # A file exists but cannot be opened, e.g. it is a directory or is not
+  # readable.
+  class FileAccessError < FileError
+  end
+
+  # The encoding passed to Table.new is unknown, or Ruby cannot transcode
+  # it to Unicode.
+  class UnsupportedEncodingError < Error
   end
 
   class NoColumnsDefined < Error

data/lib/dbf/file_handler.rb

@@ -9,7 +9,7 @@ module DBF
       when StringIO
         data
       when String
-        File.open(data, 'rb')
+        open_file(data)
       else
         raise ArgumentError, 'data must be a file path or an IO-like object responding to #read and #seek' unless data.respond_to?(:read) && data.respond_to?(:seek)
 
@@ -18,17 +18,41 @@ module DBF
         data.binmode if data.respond_to?(:binmode)
         data
       end
-    rescue Errno::ENOENT
-      raise DBF::FileNotFoundError, "file not found: #{data}"
     end
 
-    def open_memo(data, memo, memo_class, version)
+    # Opens a file for binary reading, raising DBF errors instead of the
+    # system errors File.open raises, so callers need only rescue DBF::Error.
+    # A path containing a null byte cannot name a file, so it is not found.
+    # A directory opens without error but fails on the first read, so it is
+    # rejected here.
+    def open_file(path)
+      raise Errno::EISDIR if File.directory?(path)
+
+      File.open(path, 'rb')
+    rescue Errno::ENOENT, Errno::ENOTDIR, ArgumentError
+      raise not_found(path)
+    rescue SystemCallError => e
+      reason = SystemCallError.new(nil, e.errno).message
+      raise DBF::FileAccessError.new("cannot open file: #{display_name(path)} (#{reason})", path:)
+    end
+
+    def not_found(path)
+      DBF::FileNotFoundError.new("file not found: #{display_name(path)}", path:)
+    end
+
+    # The file name without its directory, for error messages. A null byte,
+    # which File.basename rejects, is shown as "?".
+    def display_name(path)
+      File.basename(path.to_s.tr("\x00", '?'))
+    end
+
+    def open_memo(data, memo, memo_class, version, max_size: nil)
       if memo
         meth = memo.is_a?(String) ? :open : :new
-        memo_class.send(meth, memo, version)
+        memo_class.send(meth, memo, version, max_size:)
       elsif (path = data_path(data))
-        found = Dir.glob(memo_search_path(path)).first
-        found && memo_class.open(found, version)
+        found = find_memo(path)
+        found && memo_class.open(found, version, max_size:)
       end
     end
 
@@ -38,10 +62,23 @@ module DBF
       data.path if data.respond_to?(:path)
     end
 
-    def memo_search_path(path)
+    MEMO_EXTENSIONS = %w[fpt dbt].freeze
+
+    # Finds the memo file sitting next to the data file: same basename, with
+    # an .fpt or .dbt extension in any case. The directory is scanned rather
+    # than globbed so glob metacharacters in an uploaded filename cannot be
+    # expanded, and the basename must match exactly so data.dbf never picks
+    # up an unrelated memo such as data2.fpt.
+    def find_memo(path)
       dirname = File.dirname(path)
       basename = File.basename(path, '.*')
-      "#{dirname}/#{basename}*.{fpt,FPT,dbt,DBT}"
+      entries = Dir.children(dirname).sort.each_with_object({}) do |child, index|
+        index[child.downcase(:ascii)] ||= child
+      end
+      entry = MEMO_EXTENSIONS.filter_map { |ext| entries["#{basename}.#{ext}".downcase(:ascii)] }.first
+      entry && File.join(dirname, entry)
+    rescue SystemCallError
+      nil
     end
   end
 end

data/lib/dbf/memo/base.rb

@@ -6,19 +6,25 @@ module DBF
       BLOCK_HEADER_SIZE = 8
       BLOCK_SIZE = 512
 
-      def self.open(filename, version)
-        new(File.open(filename, 'rb'), version)
+      def self.open(filename, version, max_size: nil)
+        new(FileHandler.open_file(filename), version, max_size:)
       end
 
-      def initialize(data, version)
+      # @param max_size [Integer, nil] bytes to read from each memo; longer
+      #   memos are cut off. nil reads every memo in full.
+      def initialize(data, version, max_size: nil)
         @data = data
         @version = version
+        @max_size = max_size
       end
 
       def get(start_block)
         return nil unless start_block > 0
 
         build_memo start_block
+      rescue RangeError
+        # A corrupt memo pointer too large to seek to has no memo
+        nil
       end
 
       def close
@@ -31,7 +37,12 @@ module DBF
 
       private
 
-      attr_reader :data
+      attr_reader :data, :max_size
+
+      # Bounds a memo length by max_size, when one is set
+      def limit(length) # :nodoc:
+        max_size && length > max_size ? max_size : length
+      end
 
       def offset(start_block) # :nodoc:
         start_block * block_size

data/lib/dbf/memo/dbase3.rb

@@ -3,6 +3,9 @@
 module DBF
   module Memo
     class Dbase3 < Base
+      # Memo text up to, and whether it reached, the 0x1A terminator
+      TERMINATOR_PATTERN = /\A([^\x1A]*)(\x1A)?/n
+
       def build_memo(start_block) # :nodoc:
         data.seek offset(start_block)
         memo_string = +''
@@ -12,11 +15,22 @@ module DBF
           # than crashing on nil.gsub.
           break if block.nil?
 
-          block = block.gsub(/(\000|\032)/, '')
-          memo_string << block
-          break if block.size < BLOCK_SIZE
+          # The memo ends at its 0x1A terminator; bytes after it in the same
+          # block are leftovers, not memo text.
+          text, terminated = TERMINATOR_PATTERN.match(block).captures
+          text = text.delete("\000")
+          memo_string << text
+          break if terminated || text.size < BLOCK_SIZE || full?(memo_string)
         end
-        memo_string
+        memo_string.byteslice(0, limit(memo_string.bytesize))
+      end
+
+      private
+
+      # A memo with no terminator runs to the end of the file, so stop
+      # reading blocks once max_size bytes have been read.
+      def full?(memo_string) # :nodoc:
+        max_size && memo_string.bytesize >= max_size
       end
     end
   end

data/lib/dbf/memo/dbase4.rb

@@ -11,7 +11,7 @@ module DBF
         header = data.read(BLOCK_HEADER_SIZE)
         return nil unless header && header.bytesize == BLOCK_HEADER_SIZE
 
-        length = header.unpack1('x4L')
+        length = limit(header.unpack1('x4L'))
 
         # Bound the read by the bytes remaining so a crafted 32-bit length
         # field cannot force a ~4 GiB allocation from a small memo file.

data/lib/dbf/memo/foxpro.rb

@@ -14,7 +14,7 @@ module DBF
         memo_type, memo_size, memo_string = block.unpack('NNa*')
         return nil unless memo_type == 1 && memo_size.to_i.positive?
 
-        read_memo_content(memo_string, memo_size)
+        read_memo_content(memo_string, limit(memo_size))
       rescue IOError, SystemCallError, RangeError
         nil
       end

data/lib/dbf/record.rb

@@ -3,6 +3,15 @@
 module DBF
   # An instance of DBF::Record represents a row in the DBF file
   class Record
+    # Column names come from the file, so they are never exposed as these
+    # methods: Ruby (and Rails) call them implicitly to convert or render an
+    # object, which would let a crafted file control how a record behaves.
+    # Such columns remain readable through #[] and #attributes.
+    RESERVED_METHODS = %w[
+      to_str to_ary to_hash to_proc to_int to_io to_path to_regexp to_open
+      to_model to_partial_path to_param to_key
+    ].freeze
+
     # Initialize a new DBF::Record
     #
     # @param data [String, StringIO] data
@@ -112,7 +121,7 @@ module DBF
 
     def method_missing(method, *args) # :nodoc:
       key = method.to_s
-      if (index = underscored_column_names.index(key))
+      if (index = column_method_index(key))
         if @context.column_offsets && !@to_a
           column_value(index)
         else
@@ -124,7 +133,13 @@ module DBF
     end
 
     def respond_to_missing?(method, *) # :nodoc:
-      underscored_column_names.include?(method.to_s) || super
+      !column_method_index(method.to_s).nil? || super
+    end
+
+    def column_method_index(key) # :nodoc:
+      return nil if RESERVED_METHODS.include?(key)
+
+      underscored_column_names.index(key)
     end
 
     def underscored_column_names # :nodoc:

data/lib/dbf/table.rb

@@ -41,6 +41,9 @@ module DBF
     #   table = DBF::Table.new 'data.dbf', nil, 'cp437'
     #   table = DBF::Table.new 'data.dbf', 'memo.dbt', Encoding::US_ASCII
     #
+    #   # reading at most the first 64 KB of each memo
+    #   table = DBF::Table.new 'data.dbf', max_memo_size: 65_536
+    #
     # Opens a table like .new, but when given a block, yields the table,
     # closes it when the block returns, and returns the block's value —
     # the same contract as File.open.
@@ -50,8 +53,8 @@ module DBF
     #   end
     #
     # Takes the same arguments as .new. Without a block, equivalent to .new.
-    def self.open(data, memo = nil, encoding = nil, name: nil)
-      table = new(data, memo, encoding, name: name)
+    def self.open(data, memo = nil, encoding = nil, name: nil, max_memo_size: nil)
+      table = new(data, memo, encoding, name:, max_memo_size:)
       return table unless block_given?
 
       begin
@@ -64,11 +67,15 @@ module DBF
     # @param data [String, StringIO, IO] data Path to the dbf file or an IO-like object
     # @param memo [optional String, StringIO, IO] memo Path to the memo file or an IO-like object
     # @param encoding [optional String, Encoding] encoding Name of the encoding or an Encoding object
-    def initialize(data, memo = nil, encoding = nil, name: nil)
+    # @param max_memo_size [optional Integer] Bytes to read from each memo;
+    #   longer memos are cut off. Every memo is read in full by default.
+    def initialize(data, memo = nil, encoding = nil, name: nil, max_memo_size: nil)
+      Encoder.check(encoding) if encoding
+      validate_max_memo_size(max_memo_size)
       @data = FileHandler.open_data(data)
       @user_encoding = encoding
       @encoding = determine_encoding
-      @memo = FileHandler.open_memo(data, memo, version_config.memo_class, version)
+      @memo = open_memo(data, memo, max_memo_size)
       @name = name
       yield self if block_given?
     end
@@ -148,6 +155,9 @@ module DBF
     # @return [DBF::Record, NilClass]
     def record(index)
       raise DBF::NoColumnsDefined, 'The DBF file has no columns defined' if columns.empty?
+      # A negative index would read header bytes as a record (or raise
+      # Errno::EINVAL); treat it, like any index past the end, as absent.
+      return nil unless index >= 0 && index < record_count
 
       seek_to_record(index)
       return nil if deleted_record?
@@ -237,8 +247,23 @@ module DBF
       @user_encoding || header.encoding || Encoding.default_external
     end
 
+    # Close a data file opened from a path if its memo cannot be opened, so
+    # the failed Table does not leak a file handle.
+    def open_memo(data, memo, max_size) # :nodoc:
+      FileHandler.open_memo(data, memo, version_config.memo_class, version, max_size:)
+    rescue DBF::Error
+      @data.close if data.is_a?(String)
+      raise
+    end
+
+    def validate_max_memo_size(max_memo_size) # :nodoc:
+      return if max_memo_size.nil? || (max_memo_size.is_a?(Integer) && max_memo_size.positive?)
+
+      raise ArgumentError, 'max_memo_size must be a positive Integer or nil'
+    end
+
     def build_columns # :nodoc:
-      ColumnBuilder.new(self, @data, version_config).build
+      ColumnBuilder.new(self, @data, version_config, header_length:, record_length:).build
     end
 
     def deleted_record? # :nodoc:

data/lib/dbf/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DBF
-  VERSION = '5.5.0'
+  VERSION = '5.6.0'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dbf
 version: !ruby/object:Gem::Version
-  version: 5.5.0
+  version: 5.6.0
 platform: ruby
 authors:
 - Keith Morrison
@@ -84,7 +84,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.9
+rubygems_version: 4.0.20
 specification_version: 4
 summary: Read xBase files
 test_files: []