acro_that 0.1.1 → 0.1.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7cccc792d205578fc981258a0b171e4906b3f9c945ac2abd664447054251f940
-  data.tar.gz: 4544010bb0642b5c88cd9b5cc95a0c97d018109ecbe19648807419590d3f5e8b
+  metadata.gz: 56f6be44d023bdedaf254cc097d18791f284793d0c79c07847fd462c0643cc91
+  data.tar.gz: b649d290433fac6a6113a74ca47bfde60e31687f6e93b85e30c32053ee416b3a
 SHA512:
-  metadata.gz: 5a43d6c18e6babead9223d93f661032ff324ee45fae76646d2d8c37528ea4ddf294b5c5240ddc67d894c53aaff05d2245baff6c9c6f7813d9dad83c0584e5cea
-  data.tar.gz: 17052dece8b5a7700c4d9f822b2ba7e2f2b8d0ded537a173d38b2e7190ffc0b6d2d9f86d51848b8d7dd99cd35015cda99299dfab0b7f87a5df1d0404e087cbe8
+  metadata.gz: b27c5ec88a2cfdec49750d9d3316979c90c0815461eb6281b4cf602dc6533c8761c8fba7fb49407256b9ade87f2f3731e8f9d121c4ef089d08be2901107cc295
+  data.tar.gz: f1b41cef40049564af8f081547460d72e787a721f676db4c80cd74eb21844b48364ac3b871e6ad2839505908d9e5be3b8d643c16812bd91ae6d0e858bcfb51eb

data/CHANGELOG.md

@@ -5,6 +5,22 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.3] - 2025-01-XX
+
+### Fixed
+- Fixed bug where fields added to multi-page PDFs were all placed on the same page. Fields now correctly appear on their specified pages when using the `page` option in `add_field`.
+
+### Changed
+- Refactored page-finding logic to eliminate code duplication across `Document` and `AddField` classes
+- Unified page discovery through `Document#find_all_pages` and `Document#find_page_by_number` methods
+- Updated all page detection patterns to use centralized `DictScan.is_page?` utility method
+
+### Added
+- `DictScan.is_page?` utility method for consistent page object detection across the codebase
+- `Document#find_all_pages` private method for unified page discovery in document order
+- `Document#find_page_by_number` private method for finding pages by page number
+- Exposed `find_page_by_number` through `Base` module for use in action classes
+
 ## [0.1.1] - 2025-10-31
 
 ### Added

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    acro_that (0.1.0)
+    acro_that (0.1.2)
       chunky_png (~> 1.4)
 
 GEM

data/README.md

@@ -189,6 +189,31 @@ flattened_doc = AcroThat::Document.flatten_pdf("input.pdf", "output.pdf")
 flattened_bytes = AcroThat::Document.flatten_pdf("input.pdf")
 ```
 
+#### Clearing Fields
+
+The `clear` and `clear!` methods allow you to completely remove unwanted fields by rewriting the entire PDF:
+
+```ruby
+doc = AcroThat::Document.new("form.pdf")
+
+# Remove all fields matching a pattern
+doc.clear!(remove_pattern: /^text-/)
+
+# Keep only specific fields
+doc.clear!(keep_fields: ["Name", "Email"])
+
+# Remove specific fields
+doc.clear!(remove_fields: ["OldField1", "OldField2"])
+
+# Use a block to determine which fields to keep
+doc.clear! { |name| !name.start_with?("temp_") }
+
+# Write the cleared PDF
+doc.write("cleared.pdf", flatten: true)
+```
+
+**Note:** Unlike `remove_field`, which uses incremental updates, `clear` completely rewrites the PDF to exclude unwanted fields. This is more efficient when removing many fields and ensures complete removal. See [Clearing Fields Documentation](docs/cleaning_fields.md) for detailed information.
+
 ### API Reference
 
 #### `AcroThat::Document.new(path_or_io)`
@@ -282,6 +307,30 @@ AcroThat::Document.flatten_pdf("input.pdf", "output.pdf")
 flattened_doc = AcroThat::Document.flatten_pdf("input.pdf")
 ```
 
+#### `#clear(options = {})` and `#clear!(options = {})`
+Removes unwanted fields by rewriting the entire PDF. `clear` returns cleared PDF bytes without modifying the document, while `clear!` modifies the document in-place. Options include:
+
+- `keep_fields`: Array of field names to keep (all others removed)
+- `remove_fields`: Array of field names to remove
+- `remove_pattern`: Regex pattern - fields matching this are removed
+- Block: Given field name, return `true` to keep, `false` to remove
+
+```ruby
+# Remove all fields
+cleared = doc.clear(remove_pattern: /.*/)
+
+# Remove fields matching pattern (in-place)
+doc.clear!(remove_pattern: /^text-/)
+
+# Keep only specific fields
+doc.clear!(keep_fields: ["Name", "Email"])
+
+# Use block to filter fields
+doc.clear! { |name| !name.match?(/^[a-f0-9-]{30,}/) }
+```
+
+**Note:** This completely rewrites the PDF (like `flatten`), so it's more efficient than using `remove_field` multiple times. See [Clearing Fields Documentation](docs/cleaning_fields.md) for detailed information.
+
 ### Field Object
 
 Each field returned by `#list_fields` is a `Field` object with the following attributes and methods:

data/docs/README.md

@@ -38,6 +38,17 @@ Explains how PDF object streams work and how `AcroThat` parses them:
 
 **Key insight:** Object streams compress multiple objects together, but parsing them is still **text traversal**—once decompressed, it's just parsing space-separated numbers and extracting substrings by offset.
 
+### [Clearing Fields](./cleaning_fields.md)
+
+Documentation for the `clear` and `clear!` methods:
+- How to remove unwanted fields completely
+- Difference between `clear` and `remove_field`
+- Pattern matching and field selection
+- Removing orphaned widget references
+- Best practices for clearing PDFs
+
+**Key insight:** `clear` rewrites the entire PDF to exclude unwanted fields, ensuring complete removal rather than just marking fields as deleted.
+
 ## Common Themes
 
 Throughout all documentation, you'll see these recurring themes:
@@ -54,6 +65,7 @@ Throughout all documentation, you'll see these recurring themes:
 1. Start with [PDF Structure](./pdf_structure.md) to understand PDFs at a high level
 2. Read [DictScan Explained](./dict_scan_explained.md) to see how text traversal works
 3. Read [Object Streams](./object_streams.md) to understand compression features
+4. Read [Clearing Fields](./cleaning_fields.md) to learn how to remove unwanted fields
 
 **If you're debugging:**
 - [DictScan Explained](./dict_scan_explained.md) has function-by-function walkthroughs

data/docs/clear_fields.md

@@ -0,0 +1,202 @@
+# Clearing Fields with `clear` and `clear!`
+
+The `clear` method allows you to completely remove unwanted form fields from a PDF by rewriting the entire document, rather than using incremental updates. This is useful when you want to:
+
+- Remove multiple layers of added fields
+- Clear a PDF that has accumulated many unwanted fields
+- Get back to a base file without certain fields
+- Remove orphaned or invalid field references
+
+Unlike `remove_field`, which uses incremental updates, `clear` rewrites the entire PDF (similar to `flatten`) but excludes the unwanted fields entirely. This ensures that:
+
+- Field objects are completely removed (not just marked as deleted)
+- Widget annotations are removed from page `/Annots` arrays
+- Orphaned widget references are cleaned up
+- The AcroForm `/Fields` array is updated
+- All references to removed fields are eliminated
+
+## Methods
+
+### `clear(options = {})`
+
+Returns a new PDF with unwanted fields removed. Does not modify the current document.
+
+**Options:**
+- `keep_fields`: Array of field names to keep (all others removed)
+- `remove_fields`: Array of field names to remove
+- `remove_pattern`: Regex pattern - fields matching this are removed
+- Block: Given field name, return `true` to keep, `false` to remove
+
+### `clear!(options = {})`
+
+Same as `clear`, but modifies the current document in-place. Mutates the document instance.
+
+## Usage Examples
+
+### Remove All Fields
+
+```ruby
+doc = AcroThat::Document.new("form.pdf")
+
+# Remove all fields
+cleared_pdf = doc.clear(remove_pattern: /.*/)
+
+# Or in-place
+doc.clear!(remove_pattern: /.*/)
+```
+
+### Remove Fields Matching a Pattern
+
+```ruby
+# Remove all fields starting with "text-"
+doc.clear!(remove_pattern: /^text-/)
+
+# Remove UUID-like generated fields
+doc.clear! { |name| !(name =~ /text-/ || name =~ /^[a-f0-9]{20,}/) }
+```
+
+### Keep Only Specific Fields
+
+```ruby
+# Keep only these fields, remove all others
+doc.clear!(keep_fields: ["Name", "Email", "Phone"])
+
+# Write the cleared PDF
+doc.write("cleared.pdf", flatten: true)
+```
+
+### Remove Specific Fields
+
+```ruby
+# Remove specific unwanted fields
+doc.clear!(remove_fields: ["OldField1", "OldField2", "GeneratedField3"])
+```
+
+### Complex Selection with Block
+
+```ruby
+# Remove all fields except those matching certain criteria
+doc.clear! do |field_name|
+  # Keep fields that don't look generated
+  !field_name.start_with?("text-") && 
+  !field_name.match?(/^[a-f0-9]{20,}/)
+end
+```
+
+## How It Works
+
+The `clear` method:
+
+1. **Identifies fields to remove** based on the provided criteria (pattern, list, or block)
+
+2. **Finds related widgets** for each field to be removed:
+   - Widgets that reference the field via `/Parent`
+   - Widgets that have the same name via `/T`
+
+3. **Collects objects to write**, excluding:
+   - Field objects that should be removed
+   - Widget annotation objects that should be removed
+
+4. **Updates AcroForm structure**:
+   - Removes field references from the `/Fields` array
+   - Handles both inline and indirect array references
+
+5. **Clears page annotations**:
+   - Removes widget references from page `/Annots` arrays
+   - Removes orphaned widget references (widgets pointing to non-existent fields)
+   - Removes references to widgets that don't exist in the cleared PDF
+
+6. **Rewrites the entire PDF** from scratch (like `flatten`) with only the selected objects
+
+## Key Differences from `remove_field`
+
+| Feature | `remove_field` | `clear` |
+|---------|---------------|---------|
+| Update Type | Incremental update | Complete rewrite |
+| Object Removal | Marks as deleted | Completely excluded |
+| PDF Structure | Preserves all objects | Only includes selected objects |
+| Use Case | Remove one/a few fields | Remove many fields or clean up |
+| Performance | Fast (append only) | Slower (full rewrite) |
+
+## Best Practices
+
+1. **Use `clear` when removing many fields**: If you need to remove a large number of fields, `clear` is more efficient and produces cleaner output.
+
+2. **Always flatten after clearing**: Since `clear` rewrites the PDF, consider using `write(..., flatten: true)` to ensure compatibility with all PDF viewers:
+
+```ruby
+doc.clear!(remove_pattern: /^text-/)
+doc.write("output.pdf", flatten: true)
+```
+
+3. **Combine with field addition**: After clearing, you can add new fields:
+
+```ruby
+doc.clear!(remove_pattern: /.*/)
+doc.add_field("NewField", value: "Value", x: 100, y: 500, width: 200, height: 20, page: 1)
+doc.write("output.pdf", flatten: true)
+```
+
+4. **Use patterns for generated fields**: If you have fields with predictable naming patterns (e.g., UUID-based names), use regex patterns:
+
+```ruby
+# Remove all UUID-like fields
+doc.clear!(remove_pattern: /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}/)
+
+# Remove all fields containing "temp" or "test"
+doc.clear!(remove_pattern: /temp|test/i)
+```
+
+## Technical Details
+
+### Orphaned Widget Removal
+
+The `clear` method automatically identifies and removes orphaned widget references:
+
+- **Non-existent widgets**: Widget references in `/Annots` arrays that point to objects that don't exist
+- **Orphaned widgets**: Widgets that reference parent fields that don't exist in the cleaned PDF
+
+This ensures that page annotation arrays don't contain invalid references that could confuse PDF viewers.
+
+### Page Detection
+
+The method correctly identifies actual page objects (`/Type /Page`) and avoids matching page container objects (`/Type /Pages`), ensuring widgets are properly associated with the correct page.
+
+### AcroForm Structure
+
+The method properly handles both:
+- **Inline `/Fields` arrays**: Arrays directly in the AcroForm dictionary
+- **Indirect `/Fields` arrays**: Arrays referenced as separate objects
+
+Both are updated to remove references to deleted fields.
+
+## Example: Complete Clearing Workflow
+
+```ruby
+require 'acro_that'
+
+# Load PDF with many unwanted fields
+doc = AcroThat::Document.new("messy_form.pdf")
+
+# Remove all generated/UUID-like fields
+doc.clear! { |name| 
+  # Keep only fields that look intentional
+  !name.match?(/^[a-f0-9-]{30,}/) &&  # Not UUID-like
+  !name.start_with?("temp_") &&       # Not temporary
+  !name.empty?                         # Not empty
+}
+
+# Add new fields
+doc.add_field("Name", value: "", x: 100, y: 700, width: 200, height: 20, page: 1, type: :text)
+doc.add_field("Email", value: "", x: 100, y: 670, width: 200, height: 20, page: 1, type: :text)
+
+# Write cleared and updated PDF
+doc.write("cleared_form.pdf", flatten: true)
+```
+
+## See Also
+
+- [`flatten` and `flatten!`](./README.md#flattening-pdfs) - Similar rewrite approach for removing incremental updates
+- [`remove_field`](../README.md#remove_field) - Incremental removal of single fields
+- [Main README](../README.md) - General usage and API reference
+

data/issues/README.md

@@ -0,0 +1,38 @@
+# Code Review Issues
+
+This folder contains documentation of code cleanup and refactoring opportunities found in the codebase.
+
+## Files
+
+- **[refactoring-opportunities.md](./refactoring-opportunities.md)** - Detailed list of code duplication and refactoring opportunities
+
+## Summary
+
+### High Priority Issues
+1. **Widget Matching Logic** - Duplicated across 6+ locations
+2. **/Annots Array Manipulation** - Complex logic duplicated in 3 locations
+
+### Medium Priority Issues
+3. **Page-Finding Logic** - Similar logic in 4+ methods
+4. **Box Parsing Logic** - Repeated code blocks for 5 box types
+
+### Low Priority Issues
+5. Duplicated `next_fresh_object_number` implementation
+6. Object reference extraction pattern duplication
+7. Unused method: `get_widget_rect_dimensions`
+8. Base64 decoding logic duplication
+
+## Quick Stats
+
+- **8 refactoring opportunities** identified
+- **6+ locations** with widget matching duplication
+- **3 locations** with /Annots array manipulation duplication
+- **1 unused method** found
+
+## Next Steps
+
+1. Review [refactoring-opportunities.md](./refactoring-opportunities.md) for detailed information
+2. Prioritize refactoring based on maintenance needs
+3. Create test coverage before refactoring
+4. Refactor incrementally, starting with high-priority items
+

data/issues/refactoring-opportunities.md

@@ -0,0 +1,269 @@
+# Refactoring Opportunities
+
+This document identifies code duplication and unused methods that could be refactored to improve maintainability.
+
+## 1. Duplicated Page-Finding Logic
+
+### Issue
+Multiple methods have similar logic for finding page objects in a PDF document.
+
+### Locations
+- `Document#list_pages` (lines 75-104)
+- `Document#collect_pages_from_tree` (lines 691-712)
+- `Document#find_page_number_for_ref` (lines 714-728)
+- `AddField#find_page_ref` (lines 155-211)
+
+### Pattern
+The pattern `body.include?("/Type /Page") || body =~ %r{/Type\s*/Page(?!s)\b}` appears in multiple places with slight variations.
+
+### Suggested Refactor
+Create a shared module or utility methods in `DictScan`:
+- `DictScan.is_page?(body)` - Check if a body represents a page object
+- `Document#find_all_pages` - Unified method to find all page objects
+- `Document#find_page_by_number(page_num)` - Find a specific page by number
+
+### Benefits
+- Single source of truth for page detection logic
+- Easier to maintain and update page-finding behavior
+- Consistent page ordering across methods
+
+---
+
+## 2. Duplicated Widget-Matching Logic
+
+### Issue
+Multiple methods have similar logic for finding widgets that belong to a field. Widgets can be matched by:
+1. `/Parent` reference pointing to the field
+2. `/T` (field name) matching the field name
+
+### Locations
+- `Document#list_fields` (lines 222-327) - Finds widgets and matches them to fields
+- `Document#clear` (lines 472-495) - Finds widgets for removed fields
+- `UpdateField#update_widget_annotations_for_field` (lines 220-247) - Finds widgets by /Parent
+- `UpdateField#update_widget_names_for_field` (lines 249-280) - Finds widgets by /Parent and /T
+- `RemoveField#remove_widget_annotations_from_pages` (lines 55-103) - Finds widgets by /Parent and /T
+- `AddSignatureAppearance#find_widget_annotation` (lines 164-206) - Finds widgets by /Parent
+
+### Pattern
+The pattern of checking `/Parent` reference and matching by `/T` field name is repeated throughout.
+
+### Suggested Refactor
+Create utility methods in `Base` or a new `WidgetMatcher` module:
+- `find_widgets_by_parent(field_ref)` - Find widgets with /Parent pointing to field_ref
+- `find_widgets_by_name(field_name)` - Find widgets with /T matching field_name
+- `find_widgets_for_field(field_ref, field_name)` - Find all widgets for a field (by parent or name)
+
+### Benefits
+- Centralized widget matching logic
+- Consistent widget finding behavior
+- Easier to extend matching criteria
+
+---
+
+## 3. Duplicated /Annots Array Manipulation
+
+### Issue
+Multiple methods handle adding or removing widget references from page `/Annots` arrays. The logic needs to handle:
+1. Inline `/Annots` arrays: `/Annots [...]`
+2. Indirect `/Annots` arrays: `/Annots X Y R` (reference to separate array object)
+
+### Locations
+- `AddField#add_widget_to_page` (lines 213-275) - Adds widget to /Annots
+- `RemoveField#remove_widget_from_page_annots` (lines 125-155) - Removes widget from /Annots
+- `Document#clear` (lines 555-633) - Removes widgets from /Annots during cleanup
+
+### Pattern
+All three methods have similar conditional logic:
+```ruby
+if page_body =~ %r{/Annots\s*\[(.*?)\]}m
+  # Handle inline array
+elsif page_body =~ %r{/Annots\s+(\d+)\s+(\d+)\s+R}
+  # Handle indirect array
+else
+  # Create new /Annots array
+end
+```
+
+### Suggested Refactor
+Extend `DictScan` with methods:
+- `DictScan.add_to_annots_array(page_body, widget_ref)` - Unified method to add widget to /Annots
+- `DictScan.remove_from_annots_array(page_body, widget_ref)` - Unified method to remove widget from /Annots
+- `DictScan.get_annots_array(page_body)` - Extract /Annots array (handles both inline and indirect)
+
+### Benefits
+- Single implementation of /Annots manipulation logic
+- Consistent handling of edge cases
+- Easier to test /Annots operations
+
+---
+
+## 4. Duplicated Box Parsing Logic
+
+### Issue
+`Document#list_pages` has repeated code blocks for parsing different box types (MediaBox, CropBox, ArtBox, BleedBox, TrimBox).
+
+### Locations
+- `Document#list_pages` (lines 120-165)
+
+### Pattern
+Each box type uses identical logic:
+```ruby
+if body =~ %r{/MediaBox\s*\[(.*?)\]}
+  box_values = ::Regexp.last_match(1).scan(/[-+]?\d*\.?\d+/).map(&:to_f)
+  if box_values.length == 4
+    llx, lly, urx, ury = box_values
+    media_box = { llx: llx, lly: lly, urx: urx, ury: ury }
+  end
+end
+```
+
+### Suggested Refactor
+Create a helper method:
+```ruby
+def parse_box(body, box_type)
+  pattern = %r{/#{box_type}\s*\[(.*?)\]}
+  return nil unless body =~ pattern
+  
+  box_values = ::Regexp.last_match(1).scan(/[-+]?\d*\.?\d+/).map(&:to_f)
+  return nil unless box_values.length == 4
+  
+  llx, lly, urx, ury = box_values
+  { llx: llx, lly: lly, urx: urx, ury: ury }
+end
+```
+
+### Benefits
+- Reduces code duplication from ~45 lines to ~10 lines per box type
+- Easier to add new box types
+- Consistent parsing logic
+
+---
+
+## 5. Duplicated next_fresh_object_number Implementation
+
+### Issue
+The `next_fresh_object_number` method is implemented identically in two places.
+
+### Locations
+- `Document#next_fresh_object_number` (lines 730-739)
+- `Base#next_fresh_object_number` (lines 28-37)
+
+### Pattern
+Both methods have identical implementation:
+```ruby
+def next_fresh_object_number
+  max_obj_num = 0
+  resolver.each_object do |ref, _|
+    max_obj_num = [max_obj_num, ref[0]].max
+  end
+  patches.each do |p|
+    max_obj_num = [max_obj_num, p[:ref][0]].max
+  end
+  max_obj_num + 1
+end
+```
+
+### Suggested Refactor
+- Remove `Document#next_fresh_object_number` - it's only called within `Document` but could use `Base`'s implementation
+- Or: Document already has access to resolver and patches, so remove duplication by making Document use Base's method
+
+### Benefits
+- Single implementation
+- Consistent object numbering logic
+
+---
+
+## 6. Unused Methods
+
+### Issue
+Some methods are defined but never called.
+
+### Locations
+- `AddSignatureAppearance#get_widget_rect_dimensions` (lines 218-223)
+  - Defined but never used
+  - `extract_rect` is used instead, which provides the same information
+
+### Suggested Refactor
+- Remove `get_widget_rect_dimensions` if it's truly unused
+- Or: Verify if it was intended for future use and document it
+
+### Benefits
+- Cleaner codebase
+- Less confusion about which method to use
+
+---
+
+## 7. Duplicated Base64 Decoding Logic
+
+### Issue
+`AddSignatureAppearance` has two similar methods for decoding base64 data.
+
+### Locations
+- `AddSignatureAppearance#decode_base64_data_uri` (lines 101-106)
+- `AddSignatureAppearance#decode_base64_if_needed` (lines 108-119)
+
+### Pattern
+Both methods handle base64 decoding, with slightly different logic. Could potentially be unified.
+
+### Suggested Refactor
+- Consider merging into a single method that handles both cases
+- Or: Document the distinction if both are needed
+
+### Benefits
+- Simpler API
+- Less code duplication
+
+---
+
+## 8. Duplicated Regex Pattern for Object Reference
+
+### Issue
+The pattern for extracting object references `(\d+)\s+(\d+)\s+R` appears in many places.
+
+### Locations
+Throughout the codebase, used in:
+- Extracting `/Parent` references
+- Extracting `/P` (page) references  
+- Extracting `/Pages` references
+- Extracting `/Fields` array references
+- And many more...
+
+### Suggested Refactor
+Create a utility method:
+```ruby
+def DictScan.extract_object_ref(str)
+  # Extract object reference from string
+  # Returns [obj_num, gen_num] or nil
+end
+```
+
+### Benefits
+- Consistent reference extraction
+- Easier to update if PDF reference format changes
+- More readable code
+
+---
+
+## Priority Recommendations
+
+### High Priority
+1. **Widget Matching Logic (#2)** - Most duplicated, used in many critical operations
+2. **/Annots Array Manipulation (#3)** - Complex logic that's error-prone when duplicated
+
+### Medium Priority
+3. **Page-Finding Logic (#1)** - Used in multiple places, but less frequently
+4. **Box Parsing Logic (#4)** - Simple duplication, easy to refactor
+
+### Low Priority
+5. **next_fresh_object_number (#5)** - Simple duplication
+6. **Object Reference Extraction (#8)** - Could improve consistency
+7. **Unused Methods (#6)** - Cleanup task
+8. **Base64 Decoding (#7)** - Minor duplication
+
+---
+
+## Notes
+- All refactoring should be accompanied by tests to ensure behavior doesn't change
+- Consider backward compatibility if any methods are moved between modules
+- Some duplication may be intentional for performance reasons (avoid method call overhead) - evaluate before refactoring
+