kreuzberg 5.0.0.pre.rc.1 → 5.0.0.pre.rc.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1a69253c4b216e8cde6ee74a09384688fb16d3d6feffc038f6ce08799aa8e4d1
-  data.tar.gz: 6341d37dc7bfe4073461b89ae53ec5ccb6924f95745190299a3809ed2e3514ec
+  metadata.gz: 6b530ee610c625a8bd7e88f12e281aee8d5814d9ad7d26d6c2c839db83fc1563
+  data.tar.gz: 3ad761c213024ce192541e198da052302d66de1f7e682e872043fdd02d53dad9
 SHA512:
-  metadata.gz: c3127d63e11cd51f5b961d9ccc15b822af486f16ae3fc67cd651d3403f6192f6d12ea550070d2ce4c45bc2a0472e1e4ff930853315c03f6a32a91883e7ae7f59
-  data.tar.gz: 2b4a0bb3d9a44a2d66c7b58f4b91f2d51bd7ef59e5d0f69bd3561e012580c3548372f0ae007f9a9b07381509992bf01dca80116d3f828a3b54a44396e9c73e5d
+  metadata.gz: 183448573cf8e5c42f4ea9a620a1d4e390871d82750b9b703d066ad7146dc8f3f67b27da0afb5e45714544f00c53e4beaf54024406b8cc44c4f7a8ca96bb3266
+  data.tar.gz: 6a2e7a72318b0ddb2de2044940d7ef4ab84d2f20dd19bee0ff6ddcb2479ef5e1d5cf5118a83586214d4ba379a320d4bcc23ca04406c31f174b4b05752f6a8713

data/LICENSE

@@ -0,0 +1,93 @@
+Elastic License 2.0 (ELv2)
+
+Copyright 2025-2026 Kreuzberg, Inc.
+
+Acceptance
+
+By using the software, you agree to all of the terms and conditions below.
+
+Copyright License
+
+The licensor grants you a non-exclusive, royalty-free, worldwide,
+non-sublicensable, non-transferable license to use, copy, distribute, make
+available, and prepare derivative works of the software, in each case subject to
+the limitations and conditions below.
+
+Limitations
+
+You may not provide the software to third parties as a hosted or managed
+service, where the service provides users with access to any substantial set of
+the features or functionality of the software.
+
+You may not move, change, disable, or circumvent the license key functionality
+in the software, and you may not remove or obscure any functionality in the
+software that is protected by the license key.
+
+You may not alter, remove, or obscure any licensing, copyright, or other notices
+of the licensor in the software. Any use of the licensor's trademarks is subject
+to applicable law.
+
+Patents
+
+The licensor grants you a license, under any patent claims the licensor can
+license, or becomes able to license, to make, have made, use, sell, offer for
+sale, import and have imported the software, in each case subject to the
+limitations and conditions in this license. This license does not cover any
+patent claims that you cause to be infringed by modifications or additions to the
+software. If you or your company make any written claim that the software
+infringes or contributes to infringement of any patent, your patent license for
+the software granted under these terms ends immediately. If your company makes
+such a claim, your patent license ends immediately for work on behalf of your
+company.
+
+Notices
+
+You must ensure that anyone who gets a copy of any part of the software from you
+also gets a copy of these terms.
+
+If you modify the software, you must include in any modified copies of the
+software prominent notices stating that you have modified the software.
+
+No Other Rights
+
+These terms do not imply any licenses other than those expressly granted in
+these terms.
+
+Termination
+
+If you use the software in violation of these terms, such use is not licensed,
+and your licenses will automatically terminate. If the licensor provides you with
+a notice of your violation, and you cease all violation of this license no later
+than 30 days after you receive that notice, your licenses will be reinstated
+retroactively. However, if you violate these terms after such reinstatement, any
+additional violation of these terms will cause your licenses to terminate
+automatically and permanently.
+
+No Liability
+
+As far as the law allows, the software comes as is, without any warranty or
+condition, and the licensor will not be liable to you for any damages arising out
+of these terms or the use or nature of the software, under any kind of legal
+claim.
+
+Definitions
+
+The licensor is the entity offering these terms, and the software is the
+software the licensor makes available under these terms, including any portion
+of it.
+
+you refers to the individual or entity agreeing to these terms.
+
+your company is any legal entity, sole proprietorship, or other kind of
+organization that you work for, plus all organizations that have control over,
+are under the control of, or are under common control with that organization.
+control means ownership of substantially all the assets of an entity, or the
+power to direct its management and policies by vote, contract, or otherwise.
+Control can be direct or indirect.
+
+your licenses are all the licenses granted to you for the software under these
+terms.
+
+use means anything you do with the software requiring one of your licenses.
+
+trademark means trademarks, service marks, and similar rights.

data/README.md

@@ -0,0 +1,467 @@
+# Kreuzberg for Ruby
+
+<div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0;">
+  <a href="https://github.com/kreuzberg-dev/alef">
+    <img src="https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6" alt="Bindings">
+  </a>
+  <!-- Language Bindings -->
+  <a href="https://crates.io/crates/kreuzberg">
+    <img src="https://img.shields.io/crates/v/kreuzberg?label=Rust&color=007ec6" alt="Rust">
+  </a>
+  <a href="https://pypi.org/project/kreuzberg/">
+    <img src="https://img.shields.io/pypi/v/kreuzberg?label=Python&color=007ec6" alt="Python">
+  </a>
+  <a href="https://www.npmjs.com/package/@kreuzberg/node">
+    <img src="https://img.shields.io/npm/v/@kreuzberg/node?label=Node.js&color=007ec6" alt="Node.js">
+  </a>
+  <a href="https://www.npmjs.com/package/@kreuzberg/wasm">
+    <img src="https://img.shields.io/npm/v/@kreuzberg/wasm?label=WASM&color=007ec6" alt="WASM">
+  </a>
+  <a href="https://central.sonatype.com/artifact/dev.kreuzberg/kreuzberg">
+    <img src="https://img.shields.io/maven-central/v/dev.kreuzberg/kreuzberg?label=Java&color=007ec6" alt="Java">
+  </a>
+  <a href="https://github.com/kreuzberg-dev/kreuzberg/tree/main/packages/go/v5">
+    <img src="https://img.shields.io/github/v/tag/kreuzberg-dev/kreuzberg?label=Go&color=007ec6&filter=v5*" alt="Go">
+  </a>
+  <a href="https://www.nuget.org/packages/Kreuzberg/">
+    <img src="https://img.shields.io/nuget/v/Kreuzberg?label=C%23&color=007ec6" alt="C#">
+  </a>
+  <a href="https://packagist.org/packages/kreuzberg/kreuzberg">
+    <img src="https://img.shields.io/packagist/v/kreuzberg/kreuzberg?label=PHP&color=007ec6" alt="PHP">
+  </a>
+  <a href="https://rubygems.org/gems/kreuzberg">
+    <img src="https://img.shields.io/gem/v/kreuzberg?label=Ruby&color=007ec6" alt="Ruby">
+  </a>
+  <a href="https://hex.pm/packages/kreuzberg">
+    <img src="https://img.shields.io/hexpm/v/kreuzberg?label=Elixir&color=007ec6" alt="Elixir">
+  </a>
+  <a href="https://kreuzberg-dev.r-universe.dev/kreuzberg">
+    <img src="https://img.shields.io/badge/R-kreuzberg-007ec6" alt="R">
+  </a>
+  <a href="https://pub.dev/packages/kreuzberg">
+    <img src="https://img.shields.io/pub/v/kreuzberg?label=Dart&color=007ec6" alt="Dart">
+  </a>
+  <a href="https://central.sonatype.com/artifact/dev.kreuzberg/kreuzberg-android">
+    <img src="https://img.shields.io/maven-central/v/dev.kreuzberg/kreuzberg-android?label=Kotlin&color=007ec6" alt="Kotlin">
+  </a>
+  <a href="https://github.com/kreuzberg-dev/kreuzberg/tree/main/packages/swift">
+    <img src="https://img.shields.io/badge/Swift-SPM-007ec6" alt="Swift">
+  </a>
+  <a href="https://github.com/kreuzberg-dev/kreuzberg/tree/main/packages/zig">
+    <img src="https://img.shields.io/badge/Zig-package-007ec6" alt="Zig">
+  </a>
+  <a href="https://github.com/kreuzberg-dev/kreuzberg/releases">
+    <img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI">
+  </a>
+  <a href="https://github.com/kreuzberg-dev/kreuzberg/pkgs/container/kreuzberg">
+    <img src="https://img.shields.io/badge/Docker-ghcr.io-007ec6?logo=docker&logoColor=white" alt="Docker">
+  </a>
+  <a href="https://github.com/kreuzberg-dev/kreuzberg/pkgs/container/charts%2Fkreuzberg">
+    <img src="https://img.shields.io/badge/Helm-ghcr.io-007ec6?logo=helm&logoColor=white" alt="Helm">
+  </a>
+
+  <!-- Project Info -->
+  <a href="https://github.com/kreuzberg-dev/kreuzberg/blob/main/LICENSE">
+    <img src="https://img.shields.io/badge/License-Elastic--2.0-007ec6" alt="License">
+  </a>
+  <a href="https://docs.kreuzberg.dev">
+    <img src="https://img.shields.io/badge/Docs-kreuzberg-007ec6" alt="Documentation">
+  </a>
+  <a href="https://huggingface.co/Kreuzberg">
+    <img src="https://img.shields.io/badge/Hugging%20Face-Kreuzberg-007ec6" alt="Hugging Face">
+  </a>
+</div>
+
+<div align="center" style="margin: 24px 0 0;">
+  <a href="https://kreuzberg.dev">
+    <img alt="Kreuzberg" src="https://github.com/user-attachments/assets/419fc06c-8313-4324-b159-4b4d3cfce5c0" />
+  </a>
+</div>
+
+<div align="center" style="display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px;">
+  <a href="https://discord.gg/xt9WY3GnKR">
+    <img height="22" src="https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord&logoColor=white" alt="Join Discord">
+  </a>
+  <a href="https://docs.kreuzberg.dev/demo.html">
+    <img height="22" src="https://img.shields.io/badge/Live%20Demo-Open-007ec6?logo=webassembly&logoColor=white" alt="Live Demo">
+  </a>
+</div>
+
+Extract text, tables, images, and metadata from 90+ file formats and 300+ programming languages including PDF, Office documents, and images. Ruby bindings with idiomatic Ruby API and native performance.
+
+## What This Package Provides
+
+- **Ruby-native extraction** — idiomatic Ruby objects over the shared Rust document engine.
+- **Structured results** — text, tables, images, metadata, language detection, chunks, and warnings.
+- **OCR support** — Tesseract and PaddleOCR through the same configuration model as other bindings.
+- **Cross-binding parity** — output matches the Python, Node.js, Go, Java, .NET, PHP, Elixir, R, Dart, Swift, Zig, WASM, and C FFI packages.
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem 'kreuzberg'
+```
+
+Then execute:
+
+```bash
+bundle install
+```
+
+Or install it directly:
+
+```bash
+gem install kreuzberg
+```
+
+## Quick Start
+
+### Basic Usage
+
+```ruby
+require 'kreuzberg'
+
+# Simple synchronous extraction
+result = Kreuzberg.extract_file("document.pdf")
+puts result.content
+```
+
+### Async Extraction
+
+```ruby
+require 'kreuzberg'
+
+# Using Fiber for concurrency (Ruby 3.0+)
+Fiber.new do
+  result = Kreuzberg.extract_file_async("document.pdf")
+  puts result.content
+end.resume
+```
+
+### Batch Processing
+
+```ruby
+require 'kreuzberg'
+
+files = ["doc1.pdf", "doc2.docx", "doc3.xlsx"]
+
+results = files.map { |file| Kreuzberg.extract_file(file) }
+
+results.each do |result|
+  puts "Content length: #{result.content.length}"
+end
+```
+
+## Configuration
+
+```ruby
+require 'kreuzberg'
+
+config = Kreuzberg::ExtractionConfig.new(
+  use_cache: true,
+  enable_quality_processing: true,
+  ocr: Kreuzberg::OcrConfig.new(
+    backend: 'tesseract',
+    language: 'eng'
+  )
+)
+
+result = Kreuzberg.extract_file("document.pdf", config: config)
+puts result.content
+```
+
+## OCR Support
+
+### Tesseract Configuration
+
+```ruby
+require 'kreuzberg'
+
+config = Kreuzberg::ExtractionConfig.new(
+  ocr: Kreuzberg::OcrConfig.new(
+    backend: 'tesseract',
+    language: 'eng',
+    tesseract_config: Kreuzberg::TesseractConfig.new(
+      psm: 6,
+      enable_table_detection: true
+    )
+  )
+)
+
+result = Kreuzberg.extract_file("scanned.pdf", config: config)
+puts result.content
+```
+
+## Table Extraction
+
+```ruby
+require 'kreuzberg'
+
+config = Kreuzberg::ExtractionConfig.new(
+  ocr: Kreuzberg::OcrConfig.new(
+    backend: 'tesseract',
+    tesseract_config: Kreuzberg::TesseractConfig.new(
+      enable_table_detection: true
+    )
+  )
+)
+
+result = Kreuzberg.extract_file("invoice.pdf", config: config)
+
+result.tables.each_with_index do |table, index|
+  puts "Table #{index}:"
+  puts table.markdown
+end
+```
+
+## Metadata Extraction
+
+```ruby
+require 'kreuzberg'
+
+result = Kreuzberg.extract_file("document.pdf")
+
+# PDF metadata
+if result.metadata[:pdf]
+  pdf_meta = result.metadata[:pdf]
+  puts "Title: #{pdf_meta[:title]}"
+  puts "Author: #{pdf_meta[:author]}"
+  puts "Pages: #{pdf_meta[:page_count]}"
+end
+
+# Detected languages
+puts "Languages: #{result.detected_languages}"
+
+# Images
+if result.images
+  puts "Images found: #{result.images.count}"
+end
+```
+
+## Text Chunking
+
+```ruby
+require 'kreuzberg'
+
+config = Kreuzberg::ExtractionConfig.new(
+  chunking: Kreuzberg::ChunkingConfig.new(
+    max_chars: 1000,
+    max_overlap: 200
+  )
+)
+
+result = Kreuzberg.extract_file("long_document.pdf", config: config)
+
+result.chunks.each_with_index do |chunk, index|
+  puts "Chunk #{index}: #{chunk.length} characters"
+end
+```
+
+## Password-Protected PDFs
+
+```ruby
+require 'kreuzberg'
+
+config = Kreuzberg::ExtractionConfig.new(
+  pdf_options: Kreuzberg::PdfConfig.new(
+    passwords: ["password1", "password2"]
+  )
+)
+
+result = Kreuzberg.extract_file("protected.pdf", config: config)
+puts result.content
+```
+
+## Language Detection
+
+```ruby
+require 'kreuzberg'
+
+config = Kreuzberg::ExtractionConfig.new(
+  language_detection: Kreuzberg::LanguageDetectionConfig.new(
+    enabled: true
+  )
+)
+
+result = Kreuzberg.extract_file("multilingual.pdf", config: config)
+puts "Detected languages: #{result.detected_languages}"
+```
+
+## API Reference
+
+### Main Methods
+
+- `Kreuzberg.extract_file(path, config: nil)` – Extract from file
+- `Kreuzberg.extract_file_async(path, config: nil)` – Async extraction
+- `Kreuzberg.extract_bytes(data, mime_type, config: nil)` – Extract from bytes
+- `Kreuzberg.batch_extract_files(paths, config: nil)` – Batch processing
+
+### Configuration Classes
+
+- `ExtractionConfig` – Main configuration
+- `OcrConfig` – OCR settings
+- `TesseractConfig` – Tesseract-specific options
+- `ChunkingConfig` – Text chunking settings
+- `PdfConfig` – PDF-specific options
+- `LanguageDetectionConfig` – Language detection settings
+
+### Result Object
+
+- `content` – Extracted text
+- `metadata` – File metadata as Hash
+- `tables` – Array of ExtractedTable objects
+- `detected_languages` – Array of language codes
+- `chunks` – Array of text chunks
+- `images` – Array of extracted images (if enabled)
+
+## System Requirements
+
+### Ruby Version
+
+- **Ruby 3.2.0 or higher** (including Ruby 4.x)
+- Ruby 4.0+ is fully supported with no code changes required
+- Magnus bindings compile successfully on all supported Ruby versions
+
+### Required
+
+- Rust toolchain (for native extension compilation)
+
+### Optional
+
+```bash
+# Tesseract OCR
+brew install tesseract          # macOS
+sudo apt-get install tesseract-ocr  # Ubuntu/Debian
+```
+
+### Ruby 4.0 Compatibility
+
+Kreuzberg is fully compatible with Ruby 4.0 (released December 25, 2025) and later. Key Ruby 4.0 features that work seamlessly:
+
+- **Ruby Box** - Improved memory efficiency and performance
+- **ZJIT Compiler** - Enhanced JIT compilation for faster execution
+- **Ractor Improvements** - Better multi-threaded document processing
+- **Set Promoted to Core** - No changes needed for Kreuzberg
+
+All tests pass with Ruby 4.0.1 with 100% compatibility. The gem compiles without any breaking changes.
+
+## Development
+
+Clone and setup:
+
+```bash
+git clone https://github.com/kreuzberg-dev/kreuzberg.git
+cd kreuzberg
+bundle install
+```
+
+Run tests:
+
+```bash
+rake test
+```
+
+## Troubleshooting
+
+### Native extension compilation error
+
+Ensure build tools are installed:
+
+```bash
+# macOS
+xcode-select --install
+
+# Ubuntu/Debian
+sudo apt-get install build-essential ruby-dev
+
+# Windows (via RubyInstaller)
+ridk install
+```
+
+### "Could not find Kreuzberg"
+
+Reinstall the gem:
+
+```bash
+gem uninstall kreuzberg
+gem install kreuzberg --no-document
+```
+
+### OCR not working
+
+Verify Tesseract is installed:
+
+```bash
+tesseract --version
+```
+
+## Examples
+
+### Process Directory of PDFs
+
+```ruby
+require 'kreuzberg'
+require 'pathname'
+
+Dir.glob("documents/*.pdf").each do |file|
+  puts "Processing: #{file}"
+  result = Kreuzberg.extract_file(file)
+  puts "  Content length: #{result.content.length}"
+  puts "  Language: #{result.detected_languages}"
+end
+```
+
+### Extract and Parse Structured Data
+
+```ruby
+require 'kreuzberg'
+require 'json'
+
+result = Kreuzberg.extract_file("data.pdf")
+
+# Parse content as JSON (if applicable)
+begin
+  data = JSON.parse(result.content)
+  puts "Parsed data: #{data}"
+rescue JSON::ParserError
+  puts "Content is not JSON"
+end
+```
+
+### Save Extracted Images
+
+```ruby
+require 'kreuzberg'
+
+config = Kreuzberg::ExtractionConfig.new(
+  images: Kreuzberg::ImageExtractionConfig.new(
+    extract_images: true
+  )
+)
+
+result = Kreuzberg.extract_file("document.pdf", config: config)
+
+result.images&.each_with_index do |image, index|
+  File.write("image_#{index}.png", image.data)
+end
+```
+
+## Documentation
+
+For comprehensive documentation, visit [https://kreuzberg.dev](https://kreuzberg.dev)
+
+## Part of Kreuzberg.dev
+
+- [Kreuzberg Cloud](https://github.com/kreuzberg-dev/kreuzberg-cloud) — managed extraction API with SDKs, dashboards, and observability.
+- [kreuzcrawl](https://github.com/kreuzberg-dev/kreuzcrawl) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
+- [html-to-markdown](https://github.com/kreuzberg-dev/html-to-markdown) — fast, lossless HTML→Markdown engine.
+- [liter-llm](https://github.com/kreuzberg-dev/liter-llm) — universal LLM API client with native bindings for 14 languages and 143 providers.
+- [tree-sitter-language-pack](https://github.com/kreuzberg-dev/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.
+- [alef](https://github.com/kreuzberg-dev/alef) — the polyglot binding generator that produces this README and all per-language bindings.
+- [Discord](https://discord.gg/xt9WY3GnKR) — community, roadmap, announcements.
+
+## License
+
+Elastic-2.0 License - see [LICENSE](../../LICENSE) for details.