parse-bench 1.0.0__tar.gz → 1.0.2__tar.gz

{parse_bench-1.0.0 → parse_bench-1.0.2}/CHANGELOG.md

@@ -4,6 +4,85 @@ All notable changes to `parse-bench` are recorded here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
 [Semantic Versioning](https://semver.org/).
 
+## [1.0.2] - 2026-09-04
+
+Ports two fixes that landed in the internal harness while 1.0.1 was being cut.
+
+### Scoring (changes evaluation numbers)
+- `chart_data_point`: number parsing accepts accounting negatives (`(4)`,
+  `$(4)`, `($4)`, trailing minus), the `bn` billion suffix, and no longer
+  reads a comma followed by a space as a decimal separator. A numeric rule
+  value no longer fuzzy-matches a compound cell (`249` vs `249, 188`); a cell
+  that carries one numeric token plus an axis unit (`68.7 days`) does match.
+  A repeated column header after a body section label counts as local scope
+  only when the complete pair precedes the candidate.
+
+### Added
+- Granular pages carry a `cells` layer; the LlamaParse adapter reads the
+  provider-neutral `granular_layers` on normalized pages before falling back
+  to the grounded-page payload.
+
+## [1.0.1] - 2026-09-03
+
+Harness-facing release: no evaluation number changes. Everything here lets a
+downstream benchmark harness build on `parse-bench` without overriding
+built-in rule classes or patching package models at import time.
+
+### Fixed
+- `diagram_graph` rules with a `reference_image` never received the test case
+  or source file path from the evaluator, so the reference render was never
+  found. `ParseEvaluator` now forwards `source_file_path` and
+  `test_case_file_path` to the rule metric.
+
+### Added
+- `RuleBasedMetric._prepare_rule(rule, actual, kwargs)`: one hook, run inside
+  the per-rule timeout, that hands a freshly created rule its side inputs.
+  Injection is attribute-driven, so any rule (built-in or extension) that
+  declares `raw_output`, `source_file_path` or `test_case_path` receives the
+  matching `compute` keyword argument. Subclasses extend it for
+  harness-specific inputs.
+- `ChartTableCache`: chart rules on one document share a single parsed-table
+  pass. Pass `chart_table_cache=` to `compute` to read the parse back.
+- Page-parallel GriTS pairwise scoring: `GriTSMetric(pair_workers=N)`,
+  `ParseEvaluator(grits_pair_workers=N)`, or `BENCH_GRITS_PAIR_WORKERS`.
+  The evaluation runner splits the CPU budget so `doc_workers x pair_workers`
+  stays within the core count. Scores are identical to the sequential path
+  (guarded by `test_grits_perf_equivalence`).
+- `py.typed` marker: the package is now typed for downstream mypy.
+- Rotated-box geometry: `parse_bench.geometry.rotated_bbox` (`xywh_r` <->
+  polygon conversion, containment) and rotated polygon IoU / IoA in
+  `evaluation.metrics.layoutdet.iou` for datasets whose layout ground truth
+  carries an `r` rotation.
+
+### Layout adapters and schemas (changes layout numbers for the providers named)
+- Azure Document Intelligence: checkbox items carry `scope=mark`, so mark-scope
+  checkbox datasets score them; Textract and Azure DI adapters build granular
+  (line / word) pages for granular layout scoring.
+- LlamaParse: the adapter also matches results whose `ParseOutput` carries
+  `grounded_pages`, and builds granular pages from that payload before falling
+  back to the raw response; merged granular bboxes keep a shared rotation `r`.
+- Projection prefers `layout_pages[*].items` over the legacy flat
+  `predictions` list when any page has items (canonical labels coerced
+  directly), and a missing detector score projects as 0.0 in both paths.
+- `LayoutOutput` gains the `ParseOutput`-parity fields (`pages`,
+  `grounded_pages`, `job_id`); `LayoutDetectionModel` gains
+  `OPENAI_COMPATIBLE_VLM_LAYOUT`, `CHECKBOX_DETECTOR_YOLOV8`, `COHERE_PARSE_LAYOUT`.
+- Parse IR: `LineNumberIR`, `LinkIR`, `RevisionIR`, `GranularUnitIR` /
+  `GranularLayerIR`, `LayoutSegmentIR.r`, `ParseLayoutPageIR.links /
+  revisions / granular_layers` and `ParseOutput.grounded_pages`.
+- `register_pipeline_resolver` (in `parse_bench.extensions`): a harness with
+  its own pipeline registry can map `pipeline_name` to a provider key, so the
+  adapter and mapper registries resolve its pipelines too.
+- Adapter aliases: `anthropic_haiku` resolves to the Anthropic adapter.
+
+### Changed
+- `MetricValue` and `RunStat` accept extra fields (`extra="allow"`) so
+  provenance a harness attaches round-trips losslessly.
+- `ParseRuleInput` and `ParseTestRule.__init__` accept any `ParseRuleBase`
+  subclass, not only the closed built-in union, so extension rule classes
+  need no cast to call `super().__init__`.
+- `FieldCitation.bbox` is optional: a page-only citation carries `bbox=None`.
+
 ## [1.0.0] - 2026-09-02
 
 This release brings the public evaluator back to parity with the internal

{parse_bench-1.0.0 → parse_bench-1.0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: parse-bench
-Version: 1.0.0
+Version: 1.0.2
 Summary: ParseBench: a benchmark and evaluation harness for document parsing systems on real-world enterprise documents
 Project-URL: Homepage, https://parsebench.ai
 Project-URL: Repository, https://github.com/run-llama/ParseBench

{parse_bench-1.0.0 → parse_bench-1.0.2}/src/parse_bench/__init__.py

@@ -1,3 +1,3 @@
 """Document parsing evaluation system for competitive benchmarking."""
 
-__version__ = "1.0.0"
+__version__ = "1.0.2"

{parse_bench-1.0.0 → parse_bench-1.0.2}/src/parse_bench/evaluation/evaluators/parse.py

@@ -228,6 +228,7 @@ class ParseEvaluator(BaseEvaluator):
         enable_table_record_match: bool = True,
         enable_table_composite: bool = False,
         teds_variants: set[str] | None = None,
+        grits_pair_workers: int | None = None,
     ):
         """
         Initialize the ParseEvaluator.
@@ -243,6 +244,9 @@ class ParseEvaluator(BaseEvaluator):
         :param enable_structural_consistency: Enable structural consistency metric (default: True)
         :param teds_variants: Set of TEDS variant names to compute. Defaults to
             {TEDS_CONTENT} (standard TEDS only). Use ALL_TEDS_VARIANTS for all.
+        :param grits_pair_workers: Per-document page-parallelism width for GriTS
+            (the runner passes a CPU budget so ``doc_workers x pair_workers`` stays
+            within the core count). ``None`` defers to ``BENCH_GRITS_PAIR_WORKERS``.
         """
         self._enable_rule_based = enable_rule_based
         self._enable_text_similarity = enable_text_similarity
@@ -256,7 +260,7 @@ class ParseEvaluator(BaseEvaluator):
         logger.info("Chart LLM normalization mode: %s", get_normalization_mode().value)
         self._text_similarity_metric = TextSimilarityMetric()
         self._teds_metric = TEDSMetric(variants=teds_variants if teds_variants is not None else {TEDS_CONTENT})
-        self._grits_metric = GriTSMetric()
+        self._grits_metric = GriTSMetric(pair_workers=grits_pair_workers)
         self._header_accuracy_metric = HeaderAccuracyMetric()
         self._header_accuracy_generous_metric = HeaderAccuracyMetricGenerous()
         self._structural_consistency_metric = StructuralConsistencyMetric()
@@ -362,6 +366,11 @@ class ParseEvaluator(BaseEvaluator):
                     page=None,  # Document-level for now
                     raw_output=inference_result.raw_output,
                     parse_output=inference_result.output,
+                    source_file_path=inference_result.request.source_file_path,
+                    # Rules that read side files (reference renders next to the
+                    # test case) need the test case's own location: providers may
+                    # stage the input elsewhere.
+                    test_case_file_path=str(test_case.file_path),
                 )
                 metrics.append(rule_result)
                 if "judge_pass_rate" in rule_result.metadata: