acroforge 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 700f9e41b86a58ef21a6f5d045907c1df97ae55c9d2728a465cd7b0f3ecf828e
+  data.tar.gz: 9859e634dbf6236e17ee1392ccf1239c8edf494be6072272218f6e2c58092c33
+SHA512:
+  metadata.gz: 29c17e2e6958e5772cab51aa25a11187cc8d1f08cc9ccf7b2add174abacee05682b44f3f38f54dd50e6755084d5ccd5949407373da0d5d9a0e81de4257ed2348
+  data.tar.gz: eebeb0e39a636cba72f879cb6b4d31af3a670a01ae3692a86e47d5d248eb8a20ba656ded433f7c17f1fdf4b8d29041c4751bb2b78f2fff141c660798bbd1f805

data/CHANGELOG.md

@@ -0,0 +1,11 @@
+# CHANGELOG
+
+## [0.1.0] - 2026-05-26
+
+## Added
+
+- `AcroForge::Engine`: PDF AcroForm compilation with spatial heuristic and DI'd schema/overrides.
+- `AcroForge::Schema`: load, dump, normalize, and infer schemas (YAML or JSON).
+- `AcroForge::Relabeler`: propose and apply field renames via heuristic-derived YAML mappings.
+- `AcroForge::Validator`: type validation for payloads.
+- CLI: `acroforge schema|relabel|compile|bootstrap|version|help`.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026-present Maxwell Nana Forson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,217 @@
+# AcroForge
+
+A Ruby toolkit for working with PDF AcroForms, especially the broken ones.
+
+AcroForge reads, validates, relabels, and fills PDF forms. Its standout feature is the **relabeling workflow**: when a vendor ships you a fillable PDF whose internal field names look like `page0_field6`, `Text101`, or worse, AcroForge runs a spatial heuristic to figure out what each field is _actually_ for, writes its proposal to a human-reviewable YAML file, and then permanently renames the AcroForm fields once you've approved the mapping. The result is a PDF you can fill programmatically without ever again writing `pdf.fields["page0_field6"] = "Alice"`.
+
+It works on any AcroForm PDF: loan applications, school admission forms, government paperwork, internal HR templates. Nothing in the gem is domain-specific.
+
+## Requirements
+
+- Ruby `>= 2.7`
+- [HexaPDF](https://hexapdf.gettalong.org/) `~> 1.0` (runtime dependency, installed automatically)
+
+## Installation
+
+```bash
+gem install acroforge
+```
+
+The `acroforge` command lands on your `PATH` automatically. RubyGems handles this the same way it does for `rails`, `bundle`, or any other Ruby CLI; tools like mise, rbenv, asdf, and rvm pick up the new binary through their shim layer without further configuration.
+
+To use AcroForge as a library inside another Ruby project, add it to that project's `Gemfile`:
+
+```ruby
+gem "acroforge"
+```
+
+See the [Installation guide](https://lzcorp-solutions.github.io/acroforge/installation) for troubleshooting `PATH` issues on non-standard Ruby setups.
+
+## Quick start: the relabeling workflow
+
+Given a PDF with garbage-named fields:
+
+```bash
+# 1. Generate a starter schema (advisory; the heuristic's best guess at canonical keys)
+$ acroforge schema infer broken_form.pdf --out schema.yml
+
+# 2. Generate a draft mapping (per-field rename proposals, sorted by page/position)
+$ acroforge relabel propose broken_form.pdf --schema schema.yml --out mapping.yml
+
+# 3. Review mapping.yml in your editor: fix wrong proposals, fill in any blanks
+
+# 4. Apply the mapping: permanently renames the AcroForm fields in place
+$ acroforge relabel apply broken_form.pdf mapping.yml
+```
+
+After step 4, the PDF's internal field names are semantic (`full_name`, `email`, `gender`, ...) and you can fill the form programmatically with confidence.
+
+The shortcut for the first two steps:
+
+```bash
+$ acroforge bootstrap broken_form.pdf
+# writes schema.yml AND mapping.yml in one pass
+```
+
+## CLI
+
+```text
+acroforge schema infer <pdf>     [--out schema.yml] [--sections a,b,c]
+acroforge relabel propose <pdf>  [--out mapping.yml] [--schema schema.yml] [--merge|--overwrite]
+acroforge relabel apply <pdf> <mapping.yml>
+acroforge compile <pdf>          [--schema schema.yml]
+acroforge bootstrap <pdf>        [--schema-out s.yml] [--mapping-out m.yml]
+acroforge version
+acroforge help
+```
+
+| Subcommand        | What it does                                                                                                                                                                                                   |
+| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `schema infer`    | Runs the heuristic on a PDF and writes a starter schema (canonical key → type + variations). Advisory; you review and edit.                                                                                    |
+| `relabel propose` | Writes a YAML mapping file proposing a semantic name for every AcroForm field. Sorted by page → top-to-bottom → left-to-right. Default mode `--merge` preserves any `key`/`type` values you've already edited. |
+| `relabel apply`   | Reads a corrected mapping file and rewrites `field[:T]` / `field[:TU]` in the source PDF in place. Auto-disambiguates collisions (`full_name`, `full_name_1`, ...).                                            |
+| `compile`         | Diagnostic: runs the engine and prints mapped/unmapped counts. Useful for checking heuristic coverage without writing any files.                                                                               |
+| `bootstrap`       | Convenience: `schema infer` + `relabel propose` in one call.                                                                                                                                                   |
+
+Exit codes: `0` success, `1` user error (bad args, missing file), `2` validation error, `3` internal error.
+
+## Library API
+
+```ruby
+require "acroforge"
+
+# Compile a PDF and inspect what the heuristic found.
+engine = AcroForge::Engine.new(
+  "form.pdf",
+  schema: AcroForge::Schema.load("schema.yml"),   # or pass a Hash directly
+  overrides: {},                                  # optional per-PDF overrides
+  sections: ["Personal Details", "Loan Details"]  # optional section headers for scoping
+)
+result = engine.compile!
+# => { mapped: {...}, unmapped: [...], select_options: {...}, new_fields_detected: [...] }
+
+# Fill a form with a payload.
+engine.validate_payload!(full_name: "Alice", email: "alice@example.com")
+engine.fill!({ full_name: "Alice", email: "alice@example.com" }, "filled.pdf")
+
+# Generate a starter schema from a PDF.
+schema = AcroForge::Schema.infer("form.pdf")
+AcroForge::Schema.dump(schema, "schema.yml")
+
+# Run the relabeler programmatically.
+AcroForge::Relabeler.propose("form.pdf", out: "mapping.yml", schema: schema)
+AcroForge::Relabeler.apply!("form.pdf", "mapping.yml")
+
+# Validate individual values.
+AcroForge::Validator.valid?("alice@example.com", :email)  # => true
+AcroForge::Validator.valid?("not a date", :date)          # => false
+```
+
+### Errors
+
+- `AcroForge::ValidationError`: raised by `Engine#validate_payload!` on type mismatch.
+- `AcroForge::RelabelError`: raised by `Relabeler.apply!` on malformed mapping YAML, invalid key names, or missing AcroForm.
+
+## Schema format
+
+Schemas are YAML or JSON files in the "rich form":
+
+```yaml
+full_name:
+  type: string
+  variations:
+    - Full Name
+    - First Name
+    - Surname
+gender:
+  type: select
+  variations:
+    - Gender
+    - Sex
+  options:
+    - male
+    - female
+amount_requested:
+  type: money
+  variations:
+    - Amount Requested
+    - Loan Amount
+```
+
+**Field keys** (`full_name`, `gender`, ...) become Ruby symbols. **`type`** is one of `string | select | boolean | money | date | email | number`. **`variations`** are the human-readable label strings to look for on the page. **`options`** are the allowed select values (for `select` and `boolean` types).
+
+AcroForge also accepts a legacy "shorthand" form where the value is just an array of variations. `AcroForge::Schema.normalize` upgrades it to rich form on the way in:
+
+```ruby
+{
+  full_name: ["Full Name", "First Name", "Surname"],
+  dob:       ["Date of Birth", "DOB"]
+}
+```
+
+## Mapping file format
+
+`relabel propose` writes one of these. Edit the `key:` and `type:` values; the `meta:` blocks are advisory and get regenerated on the next `propose`.
+
+```yaml
+_meta:
+  source_pdf: broken_form.pdf
+  generated_at: 2026-05-26T14:32:11Z
+  acroforge_version: 0.1.0
+  total_fields: 98
+
+page0_field6:
+  key: full_name
+  type: string
+  meta:
+    raw_label: Full Name
+    confidence: high
+    section: personal_details
+    page: 0
+
+page0_field28:
+  key: full_name # collision: apply! renames this one to full_name_1
+  type: string
+  meta:
+    raw_label: Customer Name
+    confidence: medium
+    section: personal_details
+    page: 0
+
+page0_field99:
+  key: ~ # null = skip this field, leave its name unchanged
+  type: ~
+  meta:
+    raw_label: ~
+    confidence: none
+    section: ~
+    page: 3
+```
+
+`key` must match `/\A[a-z][a-z0-9_]*\z/`. Invalid keys cause `apply!` to raise `RelabelError` _before_ writing anything to the PDF.
+
+## How the heuristic works
+
+For each AcroForm field, AcroForge:
+
+1. Reads every text chunk on the page along with its bounding box.
+2. Scores nearby text against the field's widget rectangle using a mode-aware weighted heuristic (Grid-Lock, Inline Paragraph, or Standard Label depending on layout).
+3. Picks the best-scoring label, sanitises it into a snake-case key.
+4. If a `schema` is supplied, canonicalises the key against its `variations` lists.
+5. For radio groups and checkboxes, also discovers the option export values from the widget appearance states.
+
+You can inspect what it found via `engine.field_proposals` after `compile!`. That's the data structure the Relabeler consumes.
+
+## Development
+
+```bash
+bundle install
+bundle exec rspec        # run the test suite
+bundle exec standardrb   # lint
+```
+
+Synthetic test fixtures live in `spec/fixtures/`. To regenerate them, run `ruby spec/fixtures/build_fixtures.rb`.
+
+## License
+
+MIT.

data/Rakefile

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "standard/rake"
+
+task default: %i[spec standard]

data/acroforge.gemspec

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+require_relative "lib/acroforge/version"
+
+Gem::Specification.new do |spec|
+  spec.name = "acroforge"
+  spec.version = AcroForge::VERSION
+  spec.authors = ["Maxwell Nana Forson"]
+  spec.email = ["nanaforson@gmail.com"]
+
+  spec.summary = "PDF AcroForm engine with heuristic-assisted field relabeling."
+  spec.description = "Compile, fill, and relabel garbage-named AcroForm PDFs through a spatial heuristic and a human-reviewed mapping file."
+  spec.homepage = "https://github.com/Lzcorp-Solutions/acroforge"
+  spec.license = "MIT"
+  spec.required_ruby_version = ">= 2.7"
+
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = "#{spec.homepage}/tree/main"
+  spec.metadata["bug_tracker_uri"] = "#{spec.homepage}/issues"
+  spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/main/CHANGELOG.md"
+  spec.metadata["documentation_uri"] = "https://lzcorp-solutions.github.io/acroforge/"
+  spec.metadata["rubygems_mfa_required"] = "true"
+
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      (File.expand_path(f) == __FILE__) ||
+        f.start_with?(*%w[bin/ test/ spec/ features/ docs/ .git .github appveyor Gemfile]) ||
+        %w[.rspec .standard.yml .rubocop.yml Gemfile.lock].include?(f)
+    end
+  end
+
+  spec.bindir = "exe"
+  spec.executables = ["acroforge"]
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "hexapdf", "~> 1.0"
+end

data/exe/acroforge

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "acroforge/cli"
+exit AcroForge::CLI.run(ARGV)

data/lib/acroforge/all_text_processor.rb

@@ -0,0 +1,126 @@
+# frozen_string_literal: true
+
+require "hexapdf"
+
+module AcroForge
+  class AllTextProcessor < HexaPDF::Content::Processor
+    def initialize
+      super
+      @raw_chunks = []
+    end
+
+    def show_text(str)
+      process_text(str)
+    end
+
+    def show_text_with_positioning(arr)
+      process_text(arr)
+    end
+
+    def text_chunks
+      merged = merge_fragments(@raw_chunks)
+      # merge_fragments joins adjacent chunks with a literal " ", which can
+      # produce strings like "M o d e O f R e p a y m e n t" when the PDF
+      # rendered each glyph as a separate text object. Re-run normalization
+      # on the merged result so the spaced-letter collapse and other fragment
+      # fixes get a second chance to fire on the joined text.
+      merged.map { |c| c.merge(text: normalize_extracted_text(c[:text])) }
+    end
+
+    private
+
+    def process_text(data)
+      text_pos = decode_text_with_positioning(data)
+
+      str = normalize_extracted_text(text_pos.string)
+      return if str.empty?
+
+      @raw_chunks << {
+        text: str,
+        x_min: text_pos.lower_left[0],
+        y_min: text_pos.lower_left[1],
+        x_max: text_pos.upper_right[0],
+        y_max: text_pos.upper_right[1]
+      }
+    rescue HexaPDF::Error
+      nil
+    end
+
+    def normalize_extracted_text(raw_text)
+      str = raw_text.to_s
+      # Unicode NFKC normalization handles ligatures (fi -> fi), fullwidth
+      # letters, superscripts, and the rest of the Unicode "compatibility"
+      # subset in one pass. We guard against invalid encoding because PDFs
+      # occasionally smuggle raw bytes through.
+      str = str.unicode_normalize(:nfkc) if str.valid_encoding?
+      # NBSP to regular space (NFKC leaves NBSP alone)
+      str = str.tr(" ", " ")
+      # Curly quotes, en/em dashes, ellipsis, zero-width chars: NFKC doesn't
+      # touch these since they're separate codepoints, not compatibility forms.
+      AcroForge::Constants::UNICODE_REPLACEMENTS.each do |from, to|
+        str = str.gsub(from, to)
+      end
+      str = str.gsub(/\s+/, " ").strip
+
+      # Fix split apostrophes like "Customer ' s".
+      str = str.gsub(/\s+['’]\s+/, "'")
+
+      # Collapse only clear spaced-letter sequences like "C u s t o m e r".
+      str = str.gsub(/\b(?:\p{L}\s+){2,}\p{L}\b/) { |m| m.gsub(/\s+/, "") }
+
+      # Collapsing strips ALL whitespace, so a sequence like
+      # "L o a n T e n o r A p p r o v e d" becomes "LoanTenorApproved".
+      # Recover the word breaks from the surviving capital letters.
+      str = str.gsub(/([a-z])([A-Z])/, '\1 \2')
+
+      # PDFs often render punctuation as separate text objects too, so we end
+      # up with "( ForDisbursement )" or "No ." once everything else collapses.
+      # Tighten the spacing around those.
+      str = str.gsub(/\(\s+/, "(").gsub(/\s+\)/, ")")
+      str = str.gsub(/(\w)\s+([.,;:!?])/, '\1\2')
+
+      # Merge split fragments that commonly appear in broken PDF extraction,
+      # e.g. "c ertify", "h as", "th at", "o ther".
+      loop do
+        previous = str
+
+        # One-letter consonant + token: "c ertify" => "certify", "h as" => "has".
+        str = str.gsub(/\b([bcdfghjklmnpqrstvwxyz])\s+([a-z]{2,})\b/i, '\\1\\2')
+
+        # Common 2-letter heads: "th at" => "that", "ot her" => "other".
+        str = str.gsub(/\b(th|wh|ot|pr|tr|cl|gr|fr|br|dr|st|sp|ch)\s+([a-z]{2,})\b/i, '\\1\\2')
+
+        break if str == previous
+      end
+
+      str.gsub(/\s+/, " ").strip
+    end
+
+    def merge_fragments(chunks)
+      sorted = chunks.sort_by { |c| [-c[:y_min], c[:x_min]] }
+      merged = []
+
+      sorted.each do |chunk|
+        if merged.empty?
+          merged << chunk
+        else
+          last = merged.last
+
+          if (last[:y_min] - chunk[:y_min]).abs < 8 &&
+              (chunk[:x_min] - last[:x_max]) < 20 &&
+              (chunk[:x_min] - last[:x_max]) > -5 &&
+              !last[:text].strip.end_with?(":")
+
+            last[:text] += " " + chunk[:text]
+            last[:x_max] = chunk[:x_max]
+            last[:y_min] = [last[:y_min], chunk[:y_min]].min
+            last[:y_max] = [last[:y_max], chunk[:y_max]].max
+          else
+            merged << chunk
+          end
+        end
+      end
+      merged
+    end
+  end
+end

data/lib/acroforge/annotator.rb

@@ -0,0 +1,137 @@
+# frozen_string_literal: true
+
+require "yaml"
+require "hexapdf"
+
+module AcroForge
+  # Overlays a labeled badge on every AcroForm field in a PDF so a human
+  # can correlate cryptic field names (page0_field6) to what's visible on
+  # the page. The badge optionally shows the proposed semantic key from a
+  # mapping file, with colour-coding for mapped vs. unmapped entries.
+  module Annotator
+    module_function
+
+    MAPPED_COLOR = "1f7a3a"  # green: in mapping with a key set
+    UNMAPPED_COLOR = "c2410c"  # amber: in mapping with key: nil
+    MISSING_COLOR = "6b7280"  # gray: not in mapping at all
+    BARE_COLOR = "1f3a8a"  # blue: no mapping supplied at all
+
+    def annotate(pdf_path, out:, mapping: nil)
+      entries = mapping ? load_mapping(mapping) : nil
+
+      doc = HexaPDF::Document.open(pdf_path)
+      form = doc.acro_form(create: false)
+      raise RelabelError, "PDF has no AcroForm: #{pdf_path}" unless form
+
+      annotated = 0
+      mapped_count = 0
+      unmapped_count = 0
+      missing_count = 0
+
+      form.each_field do |field|
+        field.each_widget do |widget|
+          next unless widget[:Rect]
+          page = find_page_for_widget(doc, widget)
+          next unless page
+
+          original_name = field.full_field_name
+          entry = entries&.[](original_name)
+
+          color, label = if entries.nil?
+            [BARE_COLOR, original_name]
+          elsif entry.nil?
+            missing_count += 1
+            [MISSING_COLOR, "#{original_name} (not in mapping)"]
+          elsif entry["key"].nil? || entry["key"].to_s.empty?
+            unmapped_count += 1
+            [UNMAPPED_COLOR, "#{original_name} (no key)"]
+          else
+            mapped_count += 1
+            [MAPPED_COLOR, "#{original_name} -> #{entry["key"]}"]
+          end
+
+          draw_badge(page, widget[:Rect], label, color)
+          annotated += 1
+        end
+      end
+
+      doc.write(out)
+
+      {
+        annotated: annotated,
+        mapped: mapped_count,
+        unmapped: unmapped_count,
+        missing: missing_count,
+        out_path: out
+      }
+    end
+
+    def find_page_for_widget(doc, widget)
+      doc.pages.find { |page| page[:Annots]&.include?(widget) }
+    end
+
+    def load_mapping(arg)
+      return arg if arg.is_a?(Hash)
+      data = YAML.load_file(arg) || {}
+      data.reject { |k, _| k.to_s.start_with?("_") }
+    end
+
+    # Draw a coloured badge near the field rectangle showing the label text.
+    #
+    # Placement heuristic:
+    #   - Text input (wide rectangle with enough height): badge goes INSIDE
+    #     the field's empty input area. This is "free space" before the form
+    #     is filled and never collides with the form's own labels or
+    #     neighboring fields.
+    #   - Small field (checkbox / radio): too small to host a badge inside,
+    #     so the badge sits ABOVE the field. The form's label for these is
+    #     typically to the right of the box, so above doesn't collide.
+    def draw_badge(page, rect, label, color_hex)
+      x1, y1, x2, y2 = rect.to_a
+      width = x2 - x1
+      height = y2 - y1
+
+      canvas = page.canvas(type: :overlay)
+
+      # Outline around the field so each badge is tied visually to its field,
+      # even when the badge sits inside an empty input.
+      canvas.save_graphics_state do
+        canvas.stroke_color(color_hex)
+        canvas.line_width(0.75)
+        canvas.rectangle(x1 - 1, y1 - 1, width + 2, height + 2).stroke
+      end
+
+      page_box = page.box(:media)
+      badge_h = 10
+      char_width = 3.6
+      max_width = page_box.width - x1 - 4
+      badge_w = [(label.length * char_width) + 6, max_width].min
+
+      fits_inside = width > height * 1.5 && height >= badge_h + 2
+
+      badge_x, badge_y = if fits_inside
+        # Inside the field, top-aligned, clipped to the field width
+        inside_w = [badge_w, width - 2].min
+        [x1 + 1, y2 - badge_h - 1].tap { |_| badge_w = inside_w }
+      else
+        # Above the small field; clamp to page top if needed
+        above_y = y2 + 1
+        below_y = y1 - badge_h - 1
+        candidate = (above_y + badge_h <= page_box.top) ? above_y : below_y
+        [x1, candidate.clamp(0, page_box.top - badge_h)]
+      end
+
+      canvas.save_graphics_state do
+        canvas.fill_color(color_hex)
+        canvas.opacity(fill_alpha: fits_inside ? 0.85 : 0.9)
+        canvas.rectangle(badge_x, badge_y, badge_w, badge_h).fill
+      end
+
+      canvas.save_graphics_state do
+        canvas.fill_color("ffffff")
+        canvas.font("Helvetica", size: 6.5)
+        canvas.text(label, at: [badge_x + 3, badge_y + 2.5])
+      end
+    end
+  end
+end