tabula-rb 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 80dbf2efc9afdaa82c43fed84838b003121a9ec2af277621b7be59f3394840a3
+  data.tar.gz: 8db614e4185f5fbd5e5b969e1ee76469a6d883542e83942121f18dace8eb80c3
+SHA512:
+  metadata.gz: 79e3c2de05740e98a587710dec2426e8d5294e928771136a51f22e81ed9eecb1b1d6018bf2dfc83dfb97c1ac64bbc2e6c5660f863e7cce4252b644ef9b495670
+  data.tar.gz: 52280f8b0a1fd27ea842bab9f250274d42854b141e7f056d00e7902eb38024bd77ed498c336ed1810fb94ca253787520408f9d29d82fde52b507bf98246be465

data/.rspec

@@ -0,0 +1,3 @@
+--require spec_helper
+--format documentation
+--color

data/.rubocop.yml

@@ -0,0 +1,39 @@
+require:
+  - rubocop-rspec
+
+AllCops:
+  TargetRubyVersion: 3.1
+  NewCops: enable
+  SuggestExtensions: false
+
+Style/Documentation:
+  Enabled: false
+
+Style/FrozenStringLiteralComment:
+  Enabled: true
+
+Metrics/ClassLength:
+  Max: 200
+
+Metrics/MethodLength:
+  Max: 30
+
+Metrics/AbcSize:
+  Max: 30
+
+Metrics/BlockLength:
+  Exclude:
+    - "spec/**/*"
+    - "*.gemspec"
+
+Layout/LineLength:
+  Max: 120
+
+RSpec/MultipleExpectations:
+  Max: 5
+
+RSpec/ExampleLength:
+  Max: 15
+
+RSpec/NestedGroups:
+  Max: 4

data/CHANGELOG.md

@@ -0,0 +1,59 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] - 2024
+
+Initial stable release of tabula-rb, a pure Ruby port of tabula-java.
+
+### Added
+
+- **Table Extraction**: Extract tables from PDF files using two modes:
+  - Lattice mode: For PDFs with visible ruling lines/borders
+  - Stream mode: For PDFs without visible borders (uses text positioning)
+  - Auto mode: Tries lattice first, falls back to stream
+
+- **Output Formats**:
+  - CSV (with customizable separator and quoting)
+  - TSV
+  - JSON (with optional pretty-printing and metadata)
+  - Markdown (GitHub-flavored, with alignment options)
+
+- **Command Line Interface**:
+  - Extract tables from multiple PDF files
+  - Page selection (individual pages, ranges, or all)
+  - Area extraction (specify top, left, bottom, right coordinates)
+  - Column boundary specification
+  - Auto-detection of table areas (`--guess`)
+  - Password-protected PDF support
+
+- **Text Handling**:
+  - Proper UTF-8 encoding support
+  - Right-to-left (RTL) text support (Arabic, Hebrew, etc.)
+  - Merged text runs for proper word/phrase extraction
+
+- **PDF Features**:
+  - Support for rotated pages
+  - Password-protected PDF support
+  - Ruling line detection from PDF graphics stream
+
+- **Core Geometry**:
+  - Rectangle, Point, and Ruling primitives
+  - Spatial indexing for efficient text lookup
+  - Cohen-Sutherland line clipping algorithm
+  - Projection profile analysis
+
+- **Detection Algorithms**:
+  - Spreadsheet detection (ruling-based)
+  - Nurminen detection algorithm for table area detection
+
+- **Configuration**:
+  - Customizable tolerance thresholds for text merging
+  - Configurable cell detection parameters
+
+### Notes
+
+- PDFs without drawn ruling lines require stream mode (lattice mode needs visible cell borders)

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Tabula Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,176 @@
+# Tabula
+
+[![Gem Version](https://badge.fury.io/rb/tabula-rb.svg)](https://rubygems.org/gems/tabula-rb)
+
+A Ruby library for extracting tables from PDF files.
+
+This is a pure Ruby port of [tabula-java](https://github.com/tabulapdf/tabula-java), the open-source library that powers [Tabula](https://tabula.technology/). It implements the same extraction algorithms and produces compatible output, allowing you to extract tables from PDFs without requiring Java or JRuby.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'tabula-rb'
+```
+
+And then execute:
+
+```bash
+bundle install
+```
+
+Or install it directly:
+
+```bash
+gem install tabula-rb
+```
+
+## Usage
+
+### Library
+
+```ruby
+require 'tabula'
+
+# Extract all tables from a PDF
+tables = Tabula.extract("document.pdf")
+
+# Each table can be converted to different formats
+tables.each do |table|
+  puts table.to_a.inspect    # Array of arrays
+  puts table.to_csv          # CSV string
+end
+
+# Extract from specific pages
+tables = Tabula.extract("document.pdf", pages: [1, 2, 3])
+
+# Use lattice mode (for PDFs with ruling lines/borders)
+tables = Tabula.extract("document.pdf", method: :lattice)
+
+# Use stream mode (for PDFs without ruling lines)
+tables = Tabula.extract("document.pdf", method: :stream)
+
+# Extract a specific area (top, left, bottom, right in points)
+tables = Tabula.extract("document.pdf", area: [0, 0, 500, 800])
+
+# Auto-detect table areas
+tables = Tabula.extract("document.pdf", guess: true)
+
+# Password-protected PDFs
+tables = Tabula.extract("document.pdf", password: "secret")
+```
+
+### Output Formats
+
+```ruby
+tables = Tabula.extract("document.pdf")
+
+# CSV
+Tabula::Writers::CSVWriter.to_string(tables)
+
+# TSV
+Tabula::Writers::TSVWriter.to_string(tables)
+
+# JSON
+Tabula::Writers::JSONWriter.to_string(tables)
+Tabula::Writers::JSONWriter.to_string(tables, pretty: true)
+
+# Markdown
+Tabula::Writers::MarkdownWriter.to_string(tables)
+Tabula::Writers::MarkdownWriter.to_string(tables, alignment: :center)
+```
+
+### Command Line
+
+```bash
+# Basic extraction (outputs CSV to stdout)
+tabula document.pdf
+
+# Specify output format
+tabula -f CSV document.pdf
+tabula -f TSV document.pdf
+tabula -f JSON document.pdf
+tabula -f MARKDOWN document.pdf
+
+# Write to file
+tabula -o output.csv document.pdf
+
+# Extract specific pages
+tabula -p 1,2,3 document.pdf
+tabula -p 1-5 document.pdf
+tabula -p all document.pdf
+
+# Force extraction mode
+tabula -l document.pdf  # Lattice mode (ruling lines)
+tabula -t document.pdf  # Stream mode (text positions)
+
+# Extract specific area
+tabula -a 0,0,500,800 document.pdf
+
+# Auto-detect table areas
+tabula -g document.pdf
+
+# Password-protected PDF
+tabula -s mypassword document.pdf
+```
+
+Full CLI options:
+
+```
+Usage: tabula [OPTIONS] <pdf_file> [<pdf_file> ...]
+
+Options:
+    -a, --area AREA          Extraction area (top,left,bottom,right)
+    -c, --columns COLUMNS    Column boundaries (comma-separated x coordinates)
+    -f, --format FORMAT      Output format: CSV, TSV, JSON, MARKDOWN (default: CSV)
+    -g, --guess              Guess table areas (use detection algorithm)
+    -l, --lattice            Force lattice mode (use ruling lines)
+    -t, --stream             Force stream mode (use text positions)
+    -p, --pages PAGES        Pages to extract (e.g., '1,2,3' or '1-5' or 'all')
+    -o, --output FILE        Output file (default: stdout)
+    -s, --password PASSWORD  PDF password
+        --debug              Show debug information
+    -v, --version            Show version
+    -h, --help               Show this help
+```
+
+## Extraction Modes
+
+### Lattice Mode (`-l` / `:lattice`)
+
+Best for tables with visible borders/ruling lines. The algorithm detects cell boundaries by finding intersections of horizontal and vertical lines drawn in the PDF.
+
+### Stream Mode (`-t` / `:stream`)
+
+Best for tables without visible borders. The algorithm infers table structure from text positioning, looking for gaps between text elements to determine column boundaries.
+
+### Auto Mode (default)
+
+Tries lattice mode first. If no tables are found, falls back to stream mode.
+
+## Requirements
+
+- Ruby 3.1+
+- pdf-reader gem (automatically installed as dependency)
+
+## Development
+
+After checking out the repo, run:
+
+```bash
+bundle install
+bundle exec rspec
+```
+
+## License
+
+MIT License. See [LICENSE](LICENSE) for details.
+
+## Acknowledgments
+
+This gem is a Ruby port of [tabula-java](https://github.com/tabulapdf/tabula-java) by the [Tabula](https://tabula.technology/) team. The extraction algorithms, test fixtures, and expected behaviors are derived from the original Java implementation.
+
+Special thanks to:
+- [Manuel Aristarán](https://github.com/jazzido) and the Tabula team for creating the original tabula-java
+- The [pdf-reader](https://github.com/yob/pdf-reader) gem maintainers for the excellent PDF parsing library

data/Rakefile

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+require 'rubocop/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]
+
+namespace :spec do
+  RSpec::Core::RakeTask.new(:unit) do |t|
+    t.pattern = 'spec/{core,text,table,pdf,extractors,detectors,writers,algorithms}/**/*_spec.rb'
+  end
+
+  RSpec::Core::RakeTask.new(:integration) do |t|
+    t.pattern = 'spec/integration/**/*_spec.rb'
+  end
+end
+
+namespace :doc do
+  require 'yard'
+  YARD::Rake::YardocTask.new do |t|
+    t.files = ['lib/**/*.rb']
+    t.options = ['--output-dir', 'doc']
+  end
+end

data/exe/tabula

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'tabula'
+require 'tabula/cli'
+
+exit Tabula::CLI.run(ARGV)

data/lib/tabula/algorithms/cohen_sutherland_clipping.rb

@@ -0,0 +1,94 @@
+# frozen_string_literal: true
+
+module Tabula
+  # Cohen-Sutherland line clipping algorithm.
+  # Clips a line segment to a rectangular region.
+  module CohenSutherlandClipping
+    # Region codes for Cohen-Sutherland algorithm
+    INSIDE = 0b0000
+    LEFT   = 0b0001
+    RIGHT  = 0b0010
+    BOTTOM = 0b0100
+    TOP    = 0b1000
+
+    class << self
+      # Clip a ruling to a rectangular region
+      # @param ruling [Ruling] the line segment to clip
+      # @param rect [Rectangle] the clipping region
+      # @return [Ruling, nil] clipped ruling, or nil if entirely outside
+      def clip(ruling, rect)
+        x1 = ruling.x1
+        y1 = ruling.y1
+        x2 = ruling.x2
+        y2 = ruling.y2
+
+        min_x = rect.left
+        max_x = rect.right
+        min_y = rect.top
+        max_y = rect.bottom
+
+        code1 = compute_code(x1, y1, min_x, max_x, min_y, max_y)
+        code2 = compute_code(x2, y2, min_x, max_x, min_y, max_y)
+
+        loop do
+          # Both endpoints inside - trivially accept
+          return Ruling.new(x1, y1, x2, y2) if (code1 | code2).zero?
+
+          # Both endpoints share an outside region - trivially reject
+          return nil if (code1 & code2).nonzero?
+
+          # At least one endpoint is outside, select it
+          code_out = code1.nonzero? ? code1 : code2
+
+          # Find intersection point
+          x, y = find_intersection(x1, y1, x2, y2, code_out, min_x, max_x, min_y, max_y)
+
+          # Replace the outside point
+          if code_out == code1
+            x1 = x
+            y1 = y
+            code1 = compute_code(x1, y1, min_x, max_x, min_y, max_y)
+          else
+            x2 = x
+            y2 = y
+            code2 = compute_code(x2, y2, min_x, max_x, min_y, max_y)
+          end
+        end
+      end
+
+      private
+
+      def compute_code(x, y, min_x, max_x, min_y, max_y)
+        code = INSIDE
+        code |= LEFT if x < min_x
+        code |= RIGHT if x > max_x
+        code |= TOP if y < min_y
+        code |= BOTTOM if y > max_y
+        code
+      end
+
+      def find_intersection(x1, y1, x2, y2, code_out, min_x, max_x, min_y, max_y)
+        x = 0.0
+        y = 0.0
+        dx = x2 - x1
+        dy = y2 - y1
+
+        if (code_out & BOTTOM).nonzero?
+          x = x1 + (dx * (max_y - y1) / dy)
+          y = max_y
+        elsif (code_out & TOP).nonzero?
+          x = x1 + (dx * (min_y - y1) / dy)
+          y = min_y
+        elsif (code_out & RIGHT).nonzero?
+          y = y1 + (dy * (max_x - x1) / dx)
+          x = max_x
+        elsif (code_out & LEFT).nonzero?
+          y = y1 + (dy * (min_x - x1) / dx)
+          x = min_x
+        end
+
+        [x, y]
+      end
+    end
+  end
+end

data/lib/tabula/algorithms/projection_profile.rb

@@ -0,0 +1,109 @@
+# frozen_string_literal: true
+
+module Tabula
+  # Projection profile analysis for detecting table structure.
+  # Computes histograms of text element positions to find gaps.
+  class ProjectionProfile
+    attr_reader :min_value, :max_value, :bins
+
+    # @param elements [Array<Rectangle>] elements to analyze
+    # @param orientation [Symbol] :horizontal or :vertical
+    # @param bin_size [Float] size of histogram bins
+    def initialize(elements, orientation:, bin_size: 1.0)
+      @orientation = orientation
+      @bin_size = bin_size
+      @bins = Hash.new(0)
+      @min_value = Float::INFINITY
+      @max_value = -Float::INFINITY
+
+      compute_profile(elements)
+    end
+
+    # Find gaps in the projection profile
+    # @param min_gap_size [Float] minimum gap size to detect
+    # @return [Array<Array<Float>>] array of [start, end] gap ranges
+    def find_gaps(min_gap_size: 3.0)
+      return [] if @bins.empty?
+
+      gaps = []
+      gap_start = nil
+      last_filled = nil
+
+      (min_bin..max_bin).each do |bin|
+        value = @bins[bin]
+
+        if value.positive?
+          if gap_start && last_filled
+            gap_end = bin * @bin_size
+            gap_size = gap_end - gap_start
+            gaps << [gap_start, gap_end] if gap_size >= min_gap_size
+          end
+          gap_start = nil
+          last_filled = (bin * @bin_size) + @bin_size
+        elsif last_filled && gap_start.nil?
+          gap_start = last_filled
+        end
+      end
+
+      gaps
+    end
+
+    # Get midpoints of gaps (useful for column detection)
+    # @param min_gap_size [Float] minimum gap size
+    # @return [Array<Float>] gap midpoint positions
+    def gap_midpoints(min_gap_size: 3.0)
+      find_gaps(min_gap_size: min_gap_size).map { |start, stop| (start + stop) / 2.0 }
+    end
+
+    # Get value at a specific position
+    # @param position [Float] position to query
+    # @return [Integer] count at that position
+    def [](position)
+      bin = (position / @bin_size).floor
+      @bins[bin]
+    end
+
+    # Check if a position is in a gap
+    # @param position [Float] position to check
+    # @param min_gap_size [Float] minimum gap size
+    # @return [Boolean] true if position is in a gap
+    def in_gap?(position, min_gap_size: 3.0)
+      find_gaps(min_gap_size: min_gap_size).any? do |gap_start, gap_end|
+        position.between?(gap_start, gap_end)
+      end
+    end
+
+    private
+
+    def compute_profile(elements)
+      elements.each do |element|
+        if @orientation == :horizontal
+          # For horizontal profile, we project onto the X axis
+          add_range(element.left, element.right)
+          @min_value = [@min_value, element.top].min
+          @max_value = [@max_value, element.bottom].max
+        else
+          # For vertical profile, we project onto the Y axis
+          add_range(element.top, element.bottom)
+          @min_value = [@min_value, element.left].min
+          @max_value = [@max_value, element.right].max
+        end
+      end
+    end
+
+    def add_range(start_pos, end_pos)
+      start_bin = (start_pos / @bin_size).floor
+      end_bin = (end_pos / @bin_size).floor
+
+      (start_bin..end_bin).each { |bin| @bins[bin] += 1 }
+    end
+
+    def min_bin
+      @bins.keys.min || 0
+    end
+
+    def max_bin
+      @bins.keys.max || 0
+    end
+  end
+end