rbx-proofreader 1.0.1__tar.gz → 1.1.0__tar.gz

rbx_proofreader-1.1.0/PKG-INFO

@@ -0,0 +1,160 @@
+Metadata-Version: 2.4
+Name: rbx-proofreader
+Version: 1.1.0
+Summary: Visual trade detection and OCR engine
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: easyocr>=1.7.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: opencv-python>=4.8.0
+Requires-Dist: Pillow>=10.0.0
+Requires-Dist: rapidfuzz>=3.0.0
+Requires-Dist: requests>=2.31.0
+Requires-Dist: torch>=2.0.0
+Requires-Dist: tqdm>=4.66.0
+Requires-Dist: transformers>=4.30.0
+Requires-Dist: ultralytics>=8.0.0
+Provides-Extra: train
+Requires-Dist: playwright>=1.40.0; extra == "train"
+Dynamic: license-file
+
+# Proofreader 🔍
+
+A high-speed vision pipeline for reading Roblox trade screenshots.
+
+[![PyPI](https://img.shields.io/pypi/v/rbx-proofreader?color=blue&label=PyPI)](https://pypi.org/project/rbx-proofreader/)
+[![Downloads](https://static.pepy.tech/badge/rbx-proofreader)](https://pepy.tech/project/rbx-proofreader)
+[![Python](https://img.shields.io/pypi/pyversions/rbx-proofreader?logo=python&logoColor=white&color=blue)](https://pypi.org/project/rbx-proofreader/)
+[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
+[![Build Status](https://github.com/lucacrose/proofreader/actions/workflows/build.yml/badge.svg)](https://github.com/lucacrose/proofreader/actions)
+[![GPU](https://img.shields.io/badge/GPU-CUDA-blueviolet)](https://developer.nvidia.com/cuda-zone)
+[![YOLOv11](https://img.shields.io/badge/model-YOLOv11-blueviolet)](https://github.com/ultralytics/ultralytics)
+
+Proofreader transforms unstructured screenshots of Roblox trades ("proofs", hence "proofreader") into structured Python dictionaries. By combining **YOLOv11** for object detection, **CLIP** for visual similarity, and **EasyOCR**, it achieves high accuracy across diverse UI themes, resolutions, and extensions.
+
+## Why Proofreader?
+
+Roblox trade screenshots are commonly used as proof in marketplaces, moderation workflows, and value analysis, yet they are manually verified and error-prone. Proofreader automates this process by converting screenshots into structured, verifiable data in milliseconds.
+
+
+## Example
+
+![Example](https://github.com/lucacrose/proofreader/raw/main/docs/assets/example.png)
+
+## ⚡ Performance
+
+Tested on an **RTX 5070** using $n=500$ real-world "worst-case" user screenshots (compressed, cropped, and varied UI).
+
+| Metric                  | Result (E2E)                |
+|:------------------------|:----------------------------|
+| Exact Match Accuracy    | 97.2% (95% CI: 95.4–98.5%)  |
+| Median latency          | 36.8 ms                     |
+| 95th percentile latency | 73.4 ms                     |
+
+> [!NOTE]
+> End-to-End **(E2E)** latency includes image loading, YOLO detection, spatial organization, CLIP similarity matching, and OCR fallback.
+
+## ✨ Key Features
+
+- **Sub-40ms Latency:** Optimized with "Fast-Path" logic that skips OCR for high-confidence visual matches, ensuring near-instant processing.
+
+- **Multi-modal decision engine:** Weighs visual embeddings against OCR text to resolve identities across 2,500+ distinct item classes.
+
+- **Fuzzy Logic Recovery:** Built-in string distance matching corrects OCR typos and text obscurations against a local asset database.
+
+- **Theme & Scale Agnostic:** Robust performance across various UI themes (Dark/Light), resolutions, and custom display scales.
+
+## 💻 Quick Start
+
+### Installation
+
+```bash
+pip install rbx-proofreader
+```
+
+> [!IMPORTANT]
+> **Hardware Acceleration:** Proofreader automatically detects NVIDIA GPUs. For sub-40ms performance, ensure you have the CUDA-enabled version of PyTorch installed. If a CPU-only environment is detected on a GPU-capable machine, the engine will provide the exact `pip` command to fix your environment.
+
+### Usage
+
+```py
+import proofreader
+
+# Extract metadata from a screenshot
+data = proofreader.get_trade_data("trade_proof.png")
+
+print(f"Items Out: {data['outgoing']['item_count']}")
+print(f"Robux In: {data['incoming']['robux_value']}")
+```
+
+> [!TIP]
+> **First Run:** On your first execution, Proofreader will automatically download the model weights and item database (~360MB). Subsequent runs will use the local cache for maximum speed.
+
+## 🧩 How it Works
+The model handles the inconsistencies of user-generated screenshots (varied crops, UI themes, and extensions) through a multi-stage process:
+
+1. **Detection:** YOLOv11 localizes item cards, thumbnails, and robux containers.
+
+2. **Spatial Organization:** Assigns child elements (names/values) to parents and determines trade side.
+
+3. **Identification:** CLIP performs similarity matching. High-confidence results become Resolved Items immediately.
+
+4. **Heuristic Judge:** Low-confidence visual matches trigger OCR and fuzzy-logic reconciliation.
+
+![Diagram](https://github.com/lucacrose/proofreader/raw/main/docs/assets/flow_diagram.png)
+
+## 📊 Data Schema
+The `get_trade_data()` function returns a structured dictionary containing `incoming` and `outgoing` trade sides.
+
+| Key | Type | Description |
+| :--- | :--- | :--- |
+| `item_count` | `int` | Number of distinct item boxes detected. |
+| `robux_value` | `int` | Total Robux parsed from the trade. |
+| `items` | `list` | List of `ResolvedItem` objects containing `id` and `name`. |
+
+**ResolvedItem Schema:**
+
+| Property | Type | Description |
+| :--- | :--- | :--- |
+| `id` | `int` | The official Roblox Asset ID. |
+| `name` | `str` | Canonical item name from the database. |
+
+## 🏗️ Development & Training
+To set up a custom training environment for the YOLO and CLIP models:
+
+```bash
+# 1. Clone and Install
+git clone https://github.com/lucacrose/proofreader.git
+cd proofreader
+pip install -e ".[train]"
+
+# 2. Initialize Database
+python scripts/setup_items.py
+
+# 3. Training
+# Place backgrounds in src/proofreader/train/emulator/backgrounds
+# Place HTML templates in src/proofreader/train/emulator/templates
+python scripts/train_models.py
+```
+
+> [!CAUTION]
+> **GPU Required:** Training is not recommended on a CPU. Final models save to `runs/train/weights/best.pt`. Rename to `yolo.pt` and move to `src/assets/weights`.
+
+## 🛠️ Tech Stack
+
+- **Vision:** YOLOv11 (Detection), CLIP (Embeddings), OpenCV (Processing)
+- **OCR:** EasyOCR
+- **Logic:** RapidFuzz (Fuzzy String Matching)
+- **Core:** Python 3.12, PyTorch, NumPy
+
+## 🤝 Contributing
+
+Contributions are welcome! Please open an issue or submit a pull request.
+
+## 📜 License
+
+This project is licensed under the MIT License.

rbx_proofreader-1.1.0/README.md

@@ -0,0 +1,136 @@
+# Proofreader 🔍
+
+A high-speed vision pipeline for reading Roblox trade screenshots.
+
+[![PyPI](https://img.shields.io/pypi/v/rbx-proofreader?color=blue&label=PyPI)](https://pypi.org/project/rbx-proofreader/)
+[![Downloads](https://static.pepy.tech/badge/rbx-proofreader)](https://pepy.tech/project/rbx-proofreader)
+[![Python](https://img.shields.io/pypi/pyversions/rbx-proofreader?logo=python&logoColor=white&color=blue)](https://pypi.org/project/rbx-proofreader/)
+[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
+[![Build Status](https://github.com/lucacrose/proofreader/actions/workflows/build.yml/badge.svg)](https://github.com/lucacrose/proofreader/actions)
+[![GPU](https://img.shields.io/badge/GPU-CUDA-blueviolet)](https://developer.nvidia.com/cuda-zone)
+[![YOLOv11](https://img.shields.io/badge/model-YOLOv11-blueviolet)](https://github.com/ultralytics/ultralytics)
+
+Proofreader transforms unstructured screenshots of Roblox trades ("proofs", hence "proofreader") into structured Python dictionaries. By combining **YOLOv11** for object detection, **CLIP** for visual similarity, and **EasyOCR**, it achieves high accuracy across diverse UI themes, resolutions, and extensions.
+
+## Why Proofreader?
+
+Roblox trade screenshots are commonly used as proof in marketplaces, moderation workflows, and value analysis, yet they are manually verified and error-prone. Proofreader automates this process by converting screenshots into structured, verifiable data in milliseconds.
+
+
+## Example
+
+![Example](https://github.com/lucacrose/proofreader/raw/main/docs/assets/example.png)
+
+## ⚡ Performance
+
+Tested on an **RTX 5070** using $n=500$ real-world "worst-case" user screenshots (compressed, cropped, and varied UI).
+
+| Metric                  | Result (E2E)                |
+|:------------------------|:----------------------------|
+| Exact Match Accuracy    | 97.2% (95% CI: 95.4–98.5%)  |
+| Median latency          | 36.8 ms                     |
+| 95th percentile latency | 73.4 ms                     |
+
+> [!NOTE]
+> End-to-End **(E2E)** latency includes image loading, YOLO detection, spatial organization, CLIP similarity matching, and OCR fallback.
+
+## ✨ Key Features
+
+- **Sub-40ms Latency:** Optimized with "Fast-Path" logic that skips OCR for high-confidence visual matches, ensuring near-instant processing.
+
+- **Multi-modal decision engine:** Weighs visual embeddings against OCR text to resolve identities across 2,500+ distinct item classes.
+
+- **Fuzzy Logic Recovery:** Built-in string distance matching corrects OCR typos and text obscurations against a local asset database.
+
+- **Theme & Scale Agnostic:** Robust performance across various UI themes (Dark/Light), resolutions, and custom display scales.
+
+## 💻 Quick Start
+
+### Installation
+
+```bash
+pip install rbx-proofreader
+```
+
+> [!IMPORTANT]
+> **Hardware Acceleration:** Proofreader automatically detects NVIDIA GPUs. For sub-40ms performance, ensure you have the CUDA-enabled version of PyTorch installed. If a CPU-only environment is detected on a GPU-capable machine, the engine will provide the exact `pip` command to fix your environment.
+
+### Usage
+
+```py
+import proofreader
+
+# Extract metadata from a screenshot
+data = proofreader.get_trade_data("trade_proof.png")
+
+print(f"Items Out: {data['outgoing']['item_count']}")
+print(f"Robux In: {data['incoming']['robux_value']}")
+```
+
+> [!TIP]
+> **First Run:** On your first execution, Proofreader will automatically download the model weights and item database (~360MB). Subsequent runs will use the local cache for maximum speed.
+
+## 🧩 How it Works
+The model handles the inconsistencies of user-generated screenshots (varied crops, UI themes, and extensions) through a multi-stage process:
+
+1. **Detection:** YOLOv11 localizes item cards, thumbnails, and robux containers.
+
+2. **Spatial Organization:** Assigns child elements (names/values) to parents and determines trade side.
+
+3. **Identification:** CLIP performs similarity matching. High-confidence results become Resolved Items immediately.
+
+4. **Heuristic Judge:** Low-confidence visual matches trigger OCR and fuzzy-logic reconciliation.
+
+![Diagram](https://github.com/lucacrose/proofreader/raw/main/docs/assets/flow_diagram.png)
+
+## 📊 Data Schema
+The `get_trade_data()` function returns a structured dictionary containing `incoming` and `outgoing` trade sides.
+
+| Key | Type | Description |
+| :--- | :--- | :--- |
+| `item_count` | `int` | Number of distinct item boxes detected. |
+| `robux_value` | `int` | Total Robux parsed from the trade. |
+| `items` | `list` | List of `ResolvedItem` objects containing `id` and `name`. |
+
+**ResolvedItem Schema:**
+
+| Property | Type | Description |
+| :--- | :--- | :--- |
+| `id` | `int` | The official Roblox Asset ID. |
+| `name` | `str` | Canonical item name from the database. |
+
+## 🏗️ Development & Training
+To set up a custom training environment for the YOLO and CLIP models:
+
+```bash
+# 1. Clone and Install
+git clone https://github.com/lucacrose/proofreader.git
+cd proofreader
+pip install -e ".[train]"
+
+# 2. Initialize Database
+python scripts/setup_items.py
+
+# 3. Training
+# Place backgrounds in src/proofreader/train/emulator/backgrounds
+# Place HTML templates in src/proofreader/train/emulator/templates
+python scripts/train_models.py
+```
+
+> [!CAUTION]
+> **GPU Required:** Training is not recommended on a CPU. Final models save to `runs/train/weights/best.pt`. Rename to `yolo.pt` and move to `src/assets/weights`.
+
+## 🛠️ Tech Stack
+
+- **Vision:** YOLOv11 (Detection), CLIP (Embeddings), OpenCV (Processing)
+- **OCR:** EasyOCR
+- **Logic:** RapidFuzz (Fuzzy String Matching)
+- **Core:** Python 3.12, PyTorch, NumPy
+
+## 🤝 Contributing
+
+Contributions are welcome! Please open an issue or submit a pull request.
+
+## 📜 License
+
+This project is licensed under the MIT License.

{rbx_proofreader-1.0.1 → rbx_proofreader-1.1.0}/pyproject.toml

@@ -4,17 +4,22 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "rbx-proofreader"
-version = "1.0.1"
+version = "1.1.0"
 description = "Visual trade detection and OCR engine"
 readme = "README.md"
 requires-python = ">=3.12"
 license = {text = "MIT"}
+
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+]
+
 dependencies = [
     "easyocr>=1.7.0",
     "numpy>=1.24.0",
     "opencv-python>=4.8.0",
     "Pillow>=10.0.0",
-    "playwright>=1.40.0",
     "rapidfuzz>=3.0.0",
     "requests>=2.31.0",
     "torch>=2.0.0",
@@ -23,6 +28,9 @@ dependencies = [
     "ultralytics>=8.0.0"
 ]
 
+[project.optional-dependencies]
+train = ["playwright>=1.40.0"]
+
 [tool.setuptools]
 package-dir = {"" = "src"}
 

{rbx_proofreader-1.0.1 → rbx_proofreader-1.1.0}/src/proofreader/core/config.py

@@ -5,13 +5,17 @@ from pathlib import Path
 # --- BASE PATHS ---
 # Resolves to the 'proofreader' root directory
 BASE_DIR = Path(__file__).resolve().parent.parent.parent
+BASE_URL = "https://github.com/lucacrose/proofreader/releases/latest/download"
 
 # --- ASSETS & MODELS ---
 ASSETS_PATH = BASE_DIR / "assets"
 MODEL_PATH = ASSETS_PATH / "weights" / "yolo.pt"
-DB_PATH = ASSETS_PATH / "db.json"
-CACHE_PATH = ASSETS_PATH / "embedding_bank.pt"
+DB_PATH = ASSETS_PATH / "item_database.json"
+CACHE_PATH = ASSETS_PATH / "item_embeddings_bank.pt"
 THUMBNAILS_DIR = ASSETS_PATH / "thumbnails"
+TRAIN_THUMBNAILS_DIR = ASSETS_PATH / "train_data"
+CLASS_MAP_PATH = ASSETS_PATH / "class_mapping.json"
+CLIP_BEST_PATH = ASSETS_PATH / "weights" / "clip.pt"
 
 # --- TRAINING & EMULATOR ---
 TRAIN_DIR = BASE_DIR / "proofreader" / "train"
@@ -26,11 +30,11 @@ DEFAULT_TEMPLATE = TEMPLATES_DIR / "trade_ui.html"
 
 # --- HYPERPARAMETERS (Training Settings) ---
 TRAINING_CONFIG = {
-    "epochs": 100,             # Number of times the model sees the whole dataset
+    "epochs": 240,             # Number of times the model sees the whole dataset
     "batch_size": 16,          # Number of images processed at once
     "img_size": 640,           # Standard YOLO resolution
-    "patience": 10,            # Stop early if no improvement for 10 epochs
-    "close_mosaic_epochs": 10  # Disable mosaic augmentation for the last N epochs
+    "patience": 20,            # Stop early if no improvement for 20 epochs
+    "close_mosaic_epochs": 32  # Disable mosaic augmentation for the last N epochs
 }
 
 # --- AUGMENTER PROBABILITIES AND GENERATOR SETTINGS ---
@@ -82,7 +86,7 @@ AUGMENTER_CONFIG = {
 
 # Robustness Thresholds
 FUZZY_MATCH_CONFIDENCE_THRESHOLD = 60.0
-VISUAL_MATCH_THRESHOLD = 0.88
+CERTAIN_VISUAL_CONF = 0.995
 
 # --- HARDWARE SETTINGS ---
 # Automatically detects if a GPU is available for faster training

rbx_proofreader-1.1.0/src/proofreader/core/matcher.py

@@ -0,0 +1,89 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import numpy as np
+import json
+import cv2
+from PIL import Image
+from torchvision import transforms
+from transformers import CLIPVisionModelWithProjection
+from typing import List
+from .schema import TradeLayout, ResolvedItem
+
+class CLIPItemEmbedder(nn.Module):
+    def __init__(self, num_classes, model_id="openai/clip-vit-base-patch32"):
+        super().__init__()
+        self.vision_encoder = CLIPVisionModelWithProjection.from_pretrained(model_id)
+        self.item_prototypes = nn.Embedding(num_classes, 512)
+        self.logit_scale = nn.Parameter(torch.ones([]) * 2.659)
+
+    def forward(self, pixel_values):
+        outputs = self.vision_encoder(pixel_values=pixel_values)
+        return F.normalize(outputs.image_embeds, p=2, dim=-1)
+
+class VisualMatcher:
+    def __init__(self, weights_path: str, mapping_path: str, item_db: List[dict], device: str = "cuda"):
+        self.device = device
+
+        with open(mapping_path, "r") as f:
+            self.class_to_idx = json.load(f)
+        self.idx_to_class = {v: k for k, v in self.class_to_idx.items()}
+
+        self.id_to_name = {str(i["id"]): i["name"] for i in item_db}
+        self.name_to_id = {str(i["name"]).lower().strip(): i["id"] for i in item_db}
+
+        num_classes = len(self.class_to_idx)
+        self.model = CLIPItemEmbedder(num_classes).to(self.device)
+        self.model.load_state_dict(torch.load(weights_path, map_location=self.device))
+        self.model.eval()
+
+        with torch.inference_mode():
+            self.bank_tensor = F.normalize(self.model.item_prototypes.weight, p=2, dim=-1)
+        
+        self.preprocess = transforms.Compose([
+            transforms.Resize((224, 224), interpolation=transforms.InterpolationMode.BICUBIC),
+            transforms.ToTensor(),
+            transforms.Normalize((0.48145466, 0.4578275, 0.40821073), 
+                                 (0.26862954, 0.26130258, 0.27577711)),
+        ])
+
+    def match_item_visuals(self, image: np.ndarray, layout: TradeLayout):
+        items_to_process: List[ResolvedItem] = []
+        crops = []
+
+        for side in (layout.outgoing.items, layout.incoming.items):
+            for item in side:
+                if item.thumb_box:
+                    x1, y1, x2, y2 = item.thumb_box.coords
+                    crop = image[y1:y2, x1:x2]
+                    if crop.size > 0:
+                        pil_img = Image.fromarray(cv2.cvtColor(crop, cv2.COLOR_BGR2RGB))
+                        processed_crop = self.preprocess(pil_img)
+                        crops.append(processed_crop)
+                        items_to_process.append(item)
+
+        if not crops:
+            return
+        
+        batch_tensor = torch.stack(crops).to(self.device)
+        
+        with torch.inference_mode():
+            query_features = self.model(batch_tensor)
+
+            logits = query_features @ self.bank_tensor.t() * self.model.logit_scale.exp()
+            topk_scores, topk_indices = logits.topk(k=5, dim=1)
+
+            probs = F.softmax(topk_scores.float(), dim=1)
+            
+            best_idx_in_topk = probs.argmax(dim=1)
+            best_indices = topk_indices[torch.arange(len(topk_indices)), best_idx_in_topk]
+            best_probs = probs[torch.arange(len(probs)), best_idx_in_topk]
+
+        
+        for i, item in enumerate(items_to_process):
+            visual_idx = best_indices[i].item()
+
+            visual_match_id_str = self.idx_to_class[visual_idx]
+
+            item.visual_id = int(visual_match_id_str)
+            item.visual_conf = float(best_probs[i].item())

rbx_proofreader-1.1.0/src/proofreader/core/ocr.py

@@ -0,0 +1,92 @@
+import cv2
+import easyocr
+import numpy as np
+import re
+from rapidfuzz import process, utils
+from .schema import TradeLayout
+from proofreader.core.config import FUZZY_MATCH_CONFIDENCE_THRESHOLD, OCR_LANGUAGES, OCR_USE_GPU
+
+class OCRReader:
+    def __init__(self, item_list, languages=OCR_LANGUAGES, gpu=OCR_USE_GPU):
+        self.reader = easyocr.Reader(languages, gpu=gpu)
+        self.item_names = [item["name"] for item in item_list]
+
+    def _fuzzy_match_name(self, raw_text: str, threshold: float = FUZZY_MATCH_CONFIDENCE_THRESHOLD) -> str:
+        if not raw_text or len(raw_text) < 2:
+            return raw_text
+        
+        match = process.extractOne(
+            raw_text, 
+            self.item_names, 
+            processor=utils.default_process
+        )
+
+        if match and match[1] >= threshold:
+            return match[0]
+        
+        return raw_text
+
+    def _clean_robux_text(self, raw_text: str) -> int:
+        cleaned = raw_text.upper().strip()
+        substitutions = {
+            ',': '', '.': '', ' ': '',
+            'S': '5', 'O': '0', 'I': '1', 
+            'L': '1', 'B': '8', 'G': '6'
+        }
+        for char, sub in substitutions.items():
+            cleaned = cleaned.replace(char, sub)
+        
+        digits = re.findall(r'\d+', cleaned)
+        return int("".join(digits)) if digits else 0
+
+    def process_layout(self, image: np.ndarray, layout: TradeLayout, skip_if=None):
+        all_items = layout.outgoing.items + layout.incoming.items
+        crops = []
+        target_refs = []
+        STD_H = 64 
+
+        for item in all_items:
+            if skip_if and skip_if(item):
+                continue
+
+            if item.name_box:
+                x1, y1, x2, y2 = item.name_box.coords
+                crop = image[max(0, y1-2):y2+2, max(0, x1-2):x2+2]
+                if crop.size > 0:
+                    gray = cv2.cvtColor(crop, cv2.COLOR_BGR2GRAY)
+                    h, w = gray.shape
+                    new_w = int(w * (STD_H / h))
+                    resized = cv2.resize(gray, (new_w, STD_H), interpolation=cv2.INTER_LINEAR)
+                    crops.append(resized)
+                    target_refs.append({'type': 'item', 'obj': item})
+        
+        for side in [layout.outgoing, layout.incoming]:
+            if side.robux and side.robux.value_box:
+                x1, y1, x2, y2 = side.robux.value_box.coords
+                crop = image[max(0, y1-2):y2+2, max(0, x1-2):x2+2]
+                if crop.size > 0:
+                    gray = cv2.cvtColor(crop, cv2.COLOR_BGR2GRAY)
+                    h, w = gray.shape
+                    new_w = int(w * (STD_H / h))
+                    resized = cv2.resize(gray, (new_w, STD_H), interpolation=cv2.INTER_LINEAR)
+                    crops.append(resized)
+                    target_refs.append({'type': 'robux', 'obj': side.robux})
+
+        if not crops:
+            return
+        
+        max_w = max(c.shape[1] for c in crops)
+        padded_crops = [cv2.copyMakeBorder(c, 0, 0, 0, max_w - c.shape[1], cv2.BORDER_CONSTANT, value=0) for c in crops]
+
+        batch_results = self.reader.readtext_batched(padded_crops, batch_size=len(padded_crops))
+
+        for i, res in enumerate(batch_results):
+            raw_text = " ".join([text_info[1] for text_info in res]).strip()
+            conf = np.mean([text_info[2] for text_info in res]) if res else 0.0
+            
+            target = target_refs[i]
+            if target['type'] == 'item':
+                target['obj'].text_name = raw_text
+                target['obj'].text_conf = float(conf)
+            else:
+                target['obj'].value = self._clean_robux_text(raw_text)

{rbx_proofreader-1.0.1 → rbx_proofreader-1.1.0}/src/proofreader/core/schema.py

@@ -15,6 +15,14 @@ class ResolvedItem:
     thumb_box: Optional[Box] = None
     name_box: Optional[Box] = None
 
+    visual_id: int = -1
+    visual_conf: float = 0
+
+    text_name: str = ""
+    text_conf: float = 0
+
+    _finalized: bool = False
+
 @dataclass
 class ResolvedRobux:
     value: int = 0