fh-pydantic-form 0.2.5__py3-none-any.whl → 0.3.0__py3-none-any.whl

{fh_pydantic_form-0.2.5.dist-info → fh_pydantic_form-0.3.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fh-pydantic-form
-Version: 0.2.5
+Version: 0.3.0
 Summary: a library to turn any pydantic BaseModel object into a fasthtml/monsterui input form
 Project-URL: Homepage, https://github.com/Marcura/fh-pydantic-form
 Project-URL: Repository, https://github.com/Marcura/fh-pydantic-form
@@ -55,10 +55,12 @@ Description-Content-Type: text/markdown
 10. [Disabling & Excluding Fields](#disabling--excluding-fields)
 11. [Refreshing & Resetting](#refreshing--resetting)
 12. [Label Colors](#label-colors)
-13. [Schema Drift Resilience](#schema-drift-resilience)
-14. [Custom Renderers](#custom-renderers)
-15. [API Reference](#api-reference)
-16. [Contributing](#contributing)
+13. [Metrics & Highlighting](#metrics--highlighting)
+14. [ComparisonForm](#comparisonform)
+15. [Schema Drift Resilience](#schema-drift-resilience)
+16. [Custom Renderers](#custom-renderers)
+17. [API Reference](#api-reference)
+18. [Contributing](#contributing)
 
 ## Purpose
 
@@ -522,6 +524,333 @@ form_renderer = PydanticForm(
 
 This can be useful for e.g. highlighting the values of different fields in a pdf with different highlighting colors matching the form input label color. 
 
+## Metrics & Highlighting
+
+`fh-pydantic-form` provides a powerful metrics system for visual highlighting of form fields based on extraction quality scores and confidence assessments. This is particularly useful for evaluating LLM structured output extraction, comparing generated data against ground truth, and building quality assessment interfaces.
+
+<img width="796" alt="image" src="https://github.com/user-attachments/assets/df2f8623-991d-45b1-80d8-5c7239187a74" />
+
+
+### Basic Metrics Usage
+
+```python
+from fh_pydantic_form import PydanticForm
+
+# Define metrics for your form fields
+metrics_dict = {
+    "title": {
+        "metric": 0.95,
+        "comment": "Excellent title quality - clear and engaging"
+    },
+    "rating": {
+        "metric": 0.3,
+        "comment": "Low rating needs attention"
+    },
+    "status": {
+        "metric": 0.0,
+        "comment": "Critical status issue - requires immediate review"
+    }
+}
+
+# Create form with metrics
+form_renderer = PydanticForm(
+    "my_form",
+    MyModel,
+    metrics_dict=metrics_dict
+)
+```
+
+### Metrics Dictionary Structure
+
+Each field can have the following metrics properties:
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `metric` | `float` or `str` | Numeric score (0.0-1.0) or string assessment |
+| `color` | `str` | Custom color (overrides automatic color-coding) |
+| `comment` | `str` | Tooltip text shown on hover |
+
+### Automatic Color Coding
+
+Numeric metrics are automatically color-coded:
+
+- **1.0**: Bright green (perfect score)
+- **0.5-1.0**: Medium green (good range)
+- **0.0-0.5**: Dark red (needs attention)
+- **0.0**: Bright red (critical issue)
+
+```python
+metrics_dict = {
+    "field1": {"metric": 1.0, "comment": "Perfect!"},      # Bright green
+    "field2": {"metric": 0.8, "comment": "Very good"},     # Medium green
+    "field3": {"metric": 0.3, "comment": "Needs work"},    # Dark red
+    "field4": {"metric": 0.0, "comment": "Critical"},      # Bright red
+}
+```
+
+### Custom Colors
+
+Override automatic colors with custom values:
+
+```python
+metrics_dict = {
+    "status": {
+        "metric": 0.0,
+        "color": "purple",  # Custom color overrides red
+        "comment": "Status requires special attention"
+    },
+    "priority": {
+        "metric": 1.0,
+        "color": "#FF6B35",  # Custom hex color
+        "comment": "High priority with custom highlight"
+    }
+}
+```
+
+### String Metrics
+
+Use string values for qualitative assessments:
+
+```python
+metrics_dict = {
+    "validation_status": {
+        "metric": "NEEDS_REVIEW",
+        "color": "#F59E0B",  # Amber color
+        "comment": "Requires human review"
+    },
+    "data_quality": {
+        "metric": "EXCELLENT",
+        "color": "#10B981",  # Green color
+        "comment": "Data quality exceeds standards"
+    }
+}
+```
+
+### Nested Field Metrics
+
+Support for nested objects and list items:
+
+```python
+metrics_dict = {
+    # Nested object fields
+    "author.name": {
+        "metric": 0.95,
+        "comment": "Author name perfectly formatted"
+    },
+    "author.email": {
+        "metric": 0.9,
+        "comment": "Email format excellent"
+    },
+    
+    # List item metrics
+    "tags[0]": {
+        "metric": 1.0,
+        "comment": "First tag is perfect"
+    },
+    "tags[1]": {
+        "metric": 0.8,
+        "comment": "Second tag very good"
+    },
+    
+    # Complex nested paths
+    "author.addresses[0].street": {
+        "metric": 1.0,
+        "comment": "Street address perfectly formatted"
+    },
+    "author.addresses[1].city": {
+        "metric": 0.1,
+        "color": "teal",
+        "comment": "City has verification problems"
+    }
+}
+```
+
+### Practical Use Cases
+
+**LLM Structured Output Evaluation:**
+```python
+# Evaluate LLM extraction quality against ground truth
+extraction_metrics = {
+    "product.name": {
+        "metric": 0.9,
+        "comment": "Name extracted with minor formatting issue: missing space"
+    },
+    "product.category": {
+        "metric": 0.0,
+        "comment": "Critical error: LLM misclassified Electronics instead of Sports"
+    },
+    "key_features": {
+        "metric": 0.6,
+        "comment": "LLM missed 2 of 5 key features from source text"
+    },
+    "extraction_confidence": {
+        "metric": 1.0,
+        "comment": "LLM confidence score accurately reflects actual performance"
+    }
+}
+```
+
+**Document Processing Quality:**
+```python
+# Highlight extraction quality from documents
+doc_extraction_metrics = {
+    "invoice_number": {
+        "metric": 1.0,
+        "comment": "Invoice number perfectly extracted from PDF"
+    },
+    "line_items": {
+        "metric": 0.75,
+        "comment": "3/4 line items extracted correctly"
+    },
+    "total_amount": {
+        "metric": 0.0,
+        "comment": "Amount extraction failed - currency symbol confusion"
+    }
+}
+```
+
+See `examples/metrics_example.py` for a comprehensive demonstration of all metrics features.
+
+## ComparisonForm
+
+The `ComparisonForm` component provides side-by-side comparison of two related forms, perfect for evaluating LLM structured output against ground truth, annotation correction workflows, and comparing extraction results.
+
+
+<img width="1177" alt="image" src="https://github.com/user-attachments/assets/75020059-0d4d-4519-9c71-70a082d3242e" />
+
+### Basic Usage
+
+```python
+from fh_pydantic_form import PydanticForm, ComparisonForm
+
+# Create two forms to compare
+left_form = PydanticForm(
+    "ground_truth",
+    ProductModel,
+    initial_values=annotated_ground_truth,
+    disabled=False  # Editable for annotation correction
+)
+
+right_form = PydanticForm(
+    "llm_output", 
+    ProductModel,
+    initial_values=llm_extracted_data,
+    disabled=True,  # Read-only LLM output
+    metrics_dict=extraction_quality_metrics
+)
+
+# Create comparison form
+comparison_form = ComparisonForm(
+    name="extraction_evaluation",
+    left_form=left_form,
+    right_form=right_form,
+    left_label="📝 Ground Truth (Editable)",
+    right_label="🤖 LLM Output (with Quality Scores)"
+)
+```
+
+### Required JavaScript
+
+Include the comparison form JavaScript in your app headers:
+
+```python
+from fh_pydantic_form import comparison_form_js
+
+app, rt = fh.fast_app(
+    hdrs=[
+        mui.Theme.blue.headers(),
+        comparison_form_js(),  # Required for comparison forms
+    ],
+    pico=False,
+    live=True,
+)
+```
+
+### Complete Example
+
+```python
+@rt("/")
+def get():
+    return fh.Div(
+        mui.Container(
+            mui.Card(
+                mui.CardHeader(
+                    fh.H1("LLM Extraction Evaluation")
+                ),
+                mui.CardBody(
+                    # Render the comparison form
+                    comparison_form.form_wrapper(
+                        fh.Div(
+                            comparison_form.render_inputs(),
+                            
+                            # Action buttons
+                            fh.Div(
+                                mui.Button(
+                                    "Update Ground Truth",
+                                    type="submit",
+                                    hx_post="/update_ground_truth",
+                                    hx_target="#result"
+                                ),
+                                comparison_form.left_reset_button("Reset Left"),
+                                comparison_form.left_refresh_button("Refresh Left"),
+                                cls="mt-4 flex gap-2"
+                            ),
+                            
+                            fh.Div(id="result", cls="mt-4")
+                        )
+                    )
+                )
+            )
+        )
+    )
+
+@rt("/update_ground_truth")
+async def post_update_ground_truth(req):
+    # Validate left form (ground truth side)
+    validated = await comparison_form.left_form.model_validate_request(req)
+    
+    # Process the ground truth update
+    return success_response(validated)
+
+# Register routes for both forms
+comparison_form.register_routes(app)
+```
+
+### Key Features
+- **Aligned fields** input fields are horizontally aligned for easy comparison.
+- **Synchronized Accordions**: Expanding/collapsing sections syncs between both forms
+- **Independent Controls**: Separate refresh and reset buttons for each side
+- **Metrics Integration**: Right side typically shows LLM output quality scores
+- **Flexible Layout**: Responsive design works on desktop and mobile
+- **Form Validation**: Standard validation works with either form
+
+### Common Patterns
+
+**LLM Output Evaluation:**
+```python
+# Left: Editable ground truth
+# Right: Read-only LLM output with extraction quality metrics
+truth_form = PydanticForm(..., disabled=False, metrics_dict={})
+llm_form = PydanticForm(..., disabled=True, metrics_dict=extraction_metrics)
+```
+
+**Document Extraction Comparison:**
+```python
+# Left: Manual annotation
+# Right: Automated LLM extraction
+manual_form = PydanticForm(..., initial_values=manual_annotation)
+auto_form = PydanticForm(..., initial_values=llm_extraction, metrics_dict=quality_scores)
+```
+
+**Annotation Correction Workflow:**
+```python
+# Left: Correctable ground truth
+# Right: LLM output with confidence scores
+ground_truth_form = PydanticForm(..., disabled=False)
+llm_output_form = PydanticForm(..., disabled=True, metrics_dict=confidence_scores)
+```
+
+See `examples/comparison_example.py` for a complete LLM extraction evaluation interface demonstration.
 
 ## Setting Initial Values
 
@@ -650,8 +979,19 @@ form_renderer = PydanticForm(
 | `label_colors` | `Optional[Dict[str, str]]` | `None` | Mapping of field names to CSS colors or Tailwind classes |
 | `exclude_fields` | `Optional[List[str]]` | `None` | List of field names to exclude from rendering (auto-injected on submission) |
 | `spacing` | `SpacingValue` | `"normal"` | Spacing theme: `"normal"`, `"compact"`, or `SpacingTheme` enum |
+| `metrics_dict` | `Optional[Dict[str, Dict]]` | `None` | Field metrics for highlighting and tooltips |
 
-### Key Methods
+### ComparisonForm Constructor
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `name` | `str` | Required | Unique identifier for the comparison form |
+| `left_form` | `PydanticForm` | Required | Form to display on the left side |
+| `right_form` | `PydanticForm` | Required | Form to display on the right side |
+| `left_label` | `str` | `"Left"` | Label for the left form |
+| `right_label` | `str` | `"Right"` | Label for the right form |
+
+### PydanticForm Methods
 
 | Method | Purpose |
 |--------|---------|
@@ -662,11 +1002,24 @@ form_renderer = PydanticForm(
 | `parse(form_dict)` | Parse raw form data into model-compatible dictionary |
 | `model_validate_request(req)` | Extract, parse, and validate form data from request |
 
+### ComparisonForm Methods
+
+| Method | Purpose |
+|--------|---------|
+| `render_inputs()` | Generate side-by-side form inputs |
+| `form_wrapper(content)` | Wrap content with comparison form structure |
+| `left_reset_button(text=None, **kwargs)` | Reset button for left form |
+| `right_reset_button(text=None, **kwargs)` | Reset button for right form |
+| `left_refresh_button(text=None, **kwargs)` | Refresh button for left form |
+| `right_refresh_button(text=None, **kwargs)` | Refresh button for right form |
+| `register_routes(app)` | Register HTMX endpoints for both forms |
+
 ### Utility Functions
 
 | Function | Purpose |
 |----------|---------|
 | `list_manipulation_js()` | JavaScript for list reordering and toggle functionality |
+| `comparison_form_js()` | JavaScript for comparison form accordion synchronization |
 | `default_dict_for_model(model_class)` | Generate default values for all fields in a model |
 | `default_for_annotation(annotation)` | Get sensible default for a type annotation |
 

fh_pydantic_form-0.3.0.dist-info/RECORD

@@ -0,0 +1,17 @@
+fh_pydantic_form/__init__.py,sha256=uNDN6UXIM25U7NazFi0Y9ivAeA8plERrRBk7TOd6P6M,4313
+fh_pydantic_form/color_utils.py,sha256=M0HSXX0i-lSHkcsgesxw7d3PEAnLsZ46i_STymZAM_k,18271
+fh_pydantic_form/comparison_form.py,sha256=TwlOCVmaY535zVzz01ZNSTpk5RkufQZEjAJuURwh5gQ,20986
+fh_pydantic_form/constants.py,sha256=-N9wzkibFNn-V6cO8iWTQ7_xBvwSr2hBdq-m3apmW4M,169
+fh_pydantic_form/defaults.py,sha256=Pwv46v7e43cykx4Pt01e4nw-6FBkHmPvTZK36ZTZqgA,6068
+fh_pydantic_form/field_renderers.py,sha256=iCCt8hKjfJedPwad5ASUzI6KpOhNW_4oxZRSShCoRtc,77581
+fh_pydantic_form/form_parser.py,sha256=mAsE5pE7A27k7zgHg4UD-P3HHQj9FmZneK_z68jLHbo,26117
+fh_pydantic_form/form_renderer.py,sha256=IHuO8cxbzuv0y5htiLp1csQ30ZRAnVP4v3qIMOvaZBM,36898
+fh_pydantic_form/list_path.py,sha256=AA8bmDmaYy4rlGIvQOOZ0fP2tgcimNUB2Re5aVGnYc8,5182
+fh_pydantic_form/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+fh_pydantic_form/registry.py,sha256=sufK-85ST3rc3Vu0XmjjjdTqTAqgHr_ZbMGU0xRgTK8,4996
+fh_pydantic_form/type_helpers.py,sha256=JUzHT8YrWj2_g7f_Wr2GL9i3BgP1zZftFrrO8xDPeis,7409
+fh_pydantic_form/ui_style.py,sha256=UPK5OBwUVVTLnfvQ-yKukz2vbKZaT_GauaNB7OGc-Uw,3848
+fh_pydantic_form-0.3.0.dist-info/METADATA,sha256=Ze11LbQeBJu43_sG3_8009PSRNwamWdeohk7bxe5EBY,37067
+fh_pydantic_form-0.3.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+fh_pydantic_form-0.3.0.dist-info/licenses/LICENSE,sha256=AOi2eNK3D2aDycRHfPRiuACZ7WPBsKHTV2tTYNl7cls,577
+fh_pydantic_form-0.3.0.dist-info/RECORD,,

fh_pydantic_form-0.2.5.dist-info/RECORD

@@ -1,15 +0,0 @@
-fh_pydantic_form/__init__.py,sha256=luxohu6NgZDC0nhSIyw5lJGP2A8JQ51Ge1Ga7DYDkF8,4048
-fh_pydantic_form/constants.py,sha256=-N9wzkibFNn-V6cO8iWTQ7_xBvwSr2hBdq-m3apmW4M,169
-fh_pydantic_form/defaults.py,sha256=Pwv46v7e43cykx4Pt01e4nw-6FBkHmPvTZK36ZTZqgA,6068
-fh_pydantic_form/field_renderers.py,sha256=VYvAmLsLhQttlg97g2KGg-VNlS4ohxrPN1O906EJM6I,54984
-fh_pydantic_form/form_parser.py,sha256=mAsE5pE7A27k7zgHg4UD-P3HHQj9FmZneK_z68jLHbo,26117
-fh_pydantic_form/form_renderer.py,sha256=cPd7NbaPOZC8cTvhEZOsy8sf5fH6FomrsR_r6KAFF54,34573
-fh_pydantic_form/list_path.py,sha256=AA8bmDmaYy4rlGIvQOOZ0fP2tgcimNUB2Re5aVGnYc8,5182
-fh_pydantic_form/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-fh_pydantic_form/registry.py,sha256=sufK-85ST3rc3Vu0XmjjjdTqTAqgHr_ZbMGU0xRgTK8,4996
-fh_pydantic_form/type_helpers.py,sha256=FH4yl5FW1KNKvfHzs8TKQinFTC-MUgqDvRTVfPHs1LM,6815
-fh_pydantic_form/ui_style.py,sha256=aIWDWbPBUAQ73nPC5AHZi5cnqA0SIp9ISWwsxFdXXdE,3776
-fh_pydantic_form-0.2.5.dist-info/METADATA,sha256=0xo0GN6Vj50q9Qb4vQufOaqAs8Fn1w3a3h58elpBbqA,26356
-fh_pydantic_form-0.2.5.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
-fh_pydantic_form-0.2.5.dist-info/licenses/LICENSE,sha256=AOi2eNK3D2aDycRHfPRiuACZ7WPBsKHTV2tTYNl7cls,577
-fh_pydantic_form-0.2.5.dist-info/RECORD,,