som-parser 0.3.0__tar.gz

som_parser-0.3.0/.gitignore

@@ -0,0 +1,22 @@
+/target
+*.swp
+*.swo
+*~
+.DS_Store
+report.md
+.claude/settings.local.json
+smoke/node_modules/
+sdk/node/node_modules/
+sdk/node/dist/
+sdk/python/*.egg-info/
+sdk/python/dist/
+sdk/python/build/
+__pycache__/
+*.pyc
+
+# MCP registry publisher tokens
+.mcpregistry_*
+.vercel
+packages/*/node_modules/
+packages/*/.venv/
+packages/*/dist/

som_parser-0.3.0/PKG-INFO

@@ -0,0 +1,144 @@
+Metadata-Version: 2.4
+Name: som-parser
+Version: 0.3.0
+Summary: Parse and query SOM (Semantic Object Model) - the structured web format for AI agents
+Project-URL: Homepage, https://plasmate.app
+Project-URL: Documentation, https://plasmate.app/docs/som-spec
+Project-URL: Repository, https://github.com/plasmate-labs/plasmate
+License-Expression: Apache-2.0
+Keywords: ai-agents,browser-automation,plasmate,semantic-object-model,som,web-scraping
+Requires-Python: >=3.9
+Requires-Dist: pydantic>=2.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# som-parser
+
+Parse and query [SOM (Semantic Object Model)](https://plasmate.app/docs/som-spec) output in Python. SOM is a structured JSON format that represents web pages as semantic regions and elements, designed for AI agents, browser automation, and web scraping. This library provides Pydantic v2 models for type-safe parsing, validation, and a rich set of query utilities to extract exactly what you need.
+
+## Install
+
+```bash
+pip install som-parser
+```
+
+## Quick Start
+
+### Parse Plasmate output
+
+```python
+import subprocess
+from som_parser import parse_som, from_plasmate
+
+# Parse a SOM JSON string or dict
+som = parse_som('{"som_version": "0.1", ...}')
+
+# Or parse raw Plasmate CLI output directly
+result = subprocess.run(["plasmate", "https://example.com"], capture_output=True, text=True)
+som = from_plasmate(result.stdout)
+
+print(som.title)       # "Example Domain"
+print(som.url)         # "https://example.com/"
+print(som.som_version) # "0.1"
+```
+
+### Find links
+
+```python
+from som_parser import parse_som, get_links, find_by_role
+
+som = parse_som(data)
+
+# Get all links as simple dicts
+for link in get_links(som):
+    print(f"{link['text']} -> {link['href']}")
+
+# Or find by role for full SomElement objects
+for el in find_by_role(som, "link"):
+    print(el.id, el.text, el.attrs.href)
+```
+
+### Get interactive elements
+
+```python
+from som_parser import parse_som, get_interactive_elements
+
+som = parse_som(data)
+for el in get_interactive_elements(som):
+    print(f"{el.id}: {el.role.value} - actions: {[a.value for a in el.actions]}")
+```
+
+### Convert to markdown
+
+```python
+from som_parser import parse_som, to_markdown
+
+som = parse_som(data)
+print(to_markdown(som))
+```
+
+### Use Pydantic models directly
+
+```python
+from som_parser import Som, SomElement, ElementRole
+
+# Validate and construct from a dict
+som = Som.model_validate(my_dict)
+
+# Access typed fields
+for region in som.regions:
+    for element in region.elements:
+        if element.role == ElementRole.LINK:
+            print(element.attrs.href)
+
+# Serialize back to JSON
+print(som.model_dump_json(indent=2))
+```
+
+## API Reference
+
+### Parser
+
+| Function | Description |
+|----------|-------------|
+| `parse_som(input: str \| dict) -> Som` | Parse JSON string or dict into a validated Som object |
+| `is_valid_som(input) -> bool` | Check if input conforms to the SOM schema |
+| `from_plasmate(json_output: str) -> Som` | Parse raw Plasmate CLI JSON output |
+
+### Query Utilities
+
+| Function | Description |
+|----------|-------------|
+| `get_all_elements(som) -> list[SomElement]` | Flatten all elements from all regions |
+| `find_by_role(som, role) -> list[SomElement]` | Find elements by role (enum or string) |
+| `find_by_id(som, id) -> SomElement \| None` | Find a single element by its SOM id |
+| `find_by_text(som, text, exact=False) -> list[SomElement]` | Search elements by text content |
+| `get_interactive_elements(som) -> list[SomElement]` | Get elements that have actions |
+| `get_links(som) -> list[dict]` | Extract all links as `{text, href, id}` dicts |
+| `get_forms(som) -> list[SomRegion]` | Get all form regions |
+| `get_inputs(som) -> list[SomElement]` | Get all input elements |
+| `get_headings(som) -> list[dict]` | Extract heading hierarchy as `{level, text, id}` |
+| `get_text(som) -> str` | Extract all visible text content |
+| `get_text_by_region(som) -> list[dict]` | Extract text grouped by region |
+| `get_compression_ratio(som) -> float` | Return `html_bytes / som_bytes` |
+| `to_markdown(som) -> str` | Convert SOM to readable markdown |
+| `filter_elements(som, predicate) -> list[SomElement]` | Generic filter with a callable |
+
+### Types
+
+All Pydantic v2 models are exported from the top level:
+
+- `Som`, `SomRegion`, `SomElement`, `SomElementAttrs`, `SomMeta`
+- `StructuredData`, `LinkElement`, `SelectOption`, `ListItem`
+- `RegionRole`, `ElementRole`, `ElementAction`, `SemanticHint` (enums)
+
+## Links
+
+- [SOM Spec](https://plasmate.app/docs/som-spec)
+- [Plasmate](https://plasmate.app)
+- [GitHub Repository](https://github.com/plasmate-labs/plasmate)
+
+## License
+
+Apache-2.0

som_parser-0.3.0/README.md

@@ -0,0 +1,129 @@
+# som-parser
+
+Parse and query [SOM (Semantic Object Model)](https://plasmate.app/docs/som-spec) output in Python. SOM is a structured JSON format that represents web pages as semantic regions and elements, designed for AI agents, browser automation, and web scraping. This library provides Pydantic v2 models for type-safe parsing, validation, and a rich set of query utilities to extract exactly what you need.
+
+## Install
+
+```bash
+pip install som-parser
+```
+
+## Quick Start
+
+### Parse Plasmate output
+
+```python
+import subprocess
+from som_parser import parse_som, from_plasmate
+
+# Parse a SOM JSON string or dict
+som = parse_som('{"som_version": "0.1", ...}')
+
+# Or parse raw Plasmate CLI output directly
+result = subprocess.run(["plasmate", "https://example.com"], capture_output=True, text=True)
+som = from_plasmate(result.stdout)
+
+print(som.title)       # "Example Domain"
+print(som.url)         # "https://example.com/"
+print(som.som_version) # "0.1"
+```
+
+### Find links
+
+```python
+from som_parser import parse_som, get_links, find_by_role
+
+som = parse_som(data)
+
+# Get all links as simple dicts
+for link in get_links(som):
+    print(f"{link['text']} -> {link['href']}")
+
+# Or find by role for full SomElement objects
+for el in find_by_role(som, "link"):
+    print(el.id, el.text, el.attrs.href)
+```
+
+### Get interactive elements
+
+```python
+from som_parser import parse_som, get_interactive_elements
+
+som = parse_som(data)
+for el in get_interactive_elements(som):
+    print(f"{el.id}: {el.role.value} - actions: {[a.value for a in el.actions]}")
+```
+
+### Convert to markdown
+
+```python
+from som_parser import parse_som, to_markdown
+
+som = parse_som(data)
+print(to_markdown(som))
+```
+
+### Use Pydantic models directly
+
+```python
+from som_parser import Som, SomElement, ElementRole
+
+# Validate and construct from a dict
+som = Som.model_validate(my_dict)
+
+# Access typed fields
+for region in som.regions:
+    for element in region.elements:
+        if element.role == ElementRole.LINK:
+            print(element.attrs.href)
+
+# Serialize back to JSON
+print(som.model_dump_json(indent=2))
+```
+
+## API Reference
+
+### Parser
+
+| Function | Description |
+|----------|-------------|
+| `parse_som(input: str \| dict) -> Som` | Parse JSON string or dict into a validated Som object |
+| `is_valid_som(input) -> bool` | Check if input conforms to the SOM schema |
+| `from_plasmate(json_output: str) -> Som` | Parse raw Plasmate CLI JSON output |
+
+### Query Utilities
+
+| Function | Description |
+|----------|-------------|
+| `get_all_elements(som) -> list[SomElement]` | Flatten all elements from all regions |
+| `find_by_role(som, role) -> list[SomElement]` | Find elements by role (enum or string) |
+| `find_by_id(som, id) -> SomElement \| None` | Find a single element by its SOM id |
+| `find_by_text(som, text, exact=False) -> list[SomElement]` | Search elements by text content |
+| `get_interactive_elements(som) -> list[SomElement]` | Get elements that have actions |
+| `get_links(som) -> list[dict]` | Extract all links as `{text, href, id}` dicts |
+| `get_forms(som) -> list[SomRegion]` | Get all form regions |
+| `get_inputs(som) -> list[SomElement]` | Get all input elements |
+| `get_headings(som) -> list[dict]` | Extract heading hierarchy as `{level, text, id}` |
+| `get_text(som) -> str` | Extract all visible text content |
+| `get_text_by_region(som) -> list[dict]` | Extract text grouped by region |
+| `get_compression_ratio(som) -> float` | Return `html_bytes / som_bytes` |
+| `to_markdown(som) -> str` | Convert SOM to readable markdown |
+| `filter_elements(som, predicate) -> list[SomElement]` | Generic filter with a callable |
+
+### Types
+
+All Pydantic v2 models are exported from the top level:
+
+- `Som`, `SomRegion`, `SomElement`, `SomElementAttrs`, `SomMeta`
+- `StructuredData`, `LinkElement`, `SelectOption`, `ListItem`
+- `RegionRole`, `ElementRole`, `ElementAction`, `SemanticHint` (enums)
+
+## Links
+
+- [SOM Spec](https://plasmate.app/docs/som-spec)
+- [Plasmate](https://plasmate.app)
+- [GitHub Repository](https://github.com/plasmate-labs/plasmate)
+
+## License
+
+Apache-2.0

som_parser-0.3.0/pyproject.toml

@@ -0,0 +1,24 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "som-parser"
+version = "0.3.0"
+description = "Parse and query SOM (Semantic Object Model) - the structured web format for AI agents"
+readme = "README.md"
+license = "Apache-2.0"
+requires-python = ">=3.9"
+dependencies = ["pydantic>=2.0"]
+keywords = ["som", "semantic-object-model", "web-scraping", "ai-agents", "browser-automation", "plasmate"]
+
+[project.optional-dependencies]
+dev = ["pytest>=7.0"]
+
+[project.urls]
+Homepage = "https://plasmate.app"
+Documentation = "https://plasmate.app/docs/som-spec"
+Repository = "https://github.com/plasmate-labs/plasmate"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

som_parser-0.3.0/som_parser/__init__.py

@@ -0,0 +1,72 @@
+"""som-parser: Parse and query SOM (Semantic Object Model) output."""
+
+from .types import (
+    ElementAction,
+    ElementRole,
+    LinkElement,
+    ListItem,
+    RegionRole,
+    SelectOption,
+    SemanticHint,
+    Som,
+    SomElement,
+    SomElementAttrs,
+    SomMeta,
+    SomRegion,
+    StructuredData,
+)
+from .parser import from_plasmate, is_valid_som, parse_som
+from .query import (
+    filter_elements,
+    find_by_id,
+    find_by_role,
+    find_by_text,
+    get_all_elements,
+    get_compression_ratio,
+    get_forms,
+    get_headings,
+    get_inputs,
+    get_interactive_elements,
+    get_links,
+    get_text,
+    get_text_by_region,
+    to_markdown,
+)
+
+__all__ = [
+    # Types
+    "ElementAction",
+    "ElementRole",
+    "LinkElement",
+    "ListItem",
+    "RegionRole",
+    "SelectOption",
+    "SemanticHint",
+    "Som",
+    "SomElement",
+    "SomElementAttrs",
+    "SomMeta",
+    "SomRegion",
+    "StructuredData",
+    # Parser
+    "from_plasmate",
+    "is_valid_som",
+    "parse_som",
+    # Query
+    "filter_elements",
+    "find_by_id",
+    "find_by_role",
+    "find_by_text",
+    "get_all_elements",
+    "get_compression_ratio",
+    "get_forms",
+    "get_headings",
+    "get_inputs",
+    "get_interactive_elements",
+    "get_links",
+    "get_text",
+    "get_text_by_region",
+    "to_markdown",
+]
+
+__version__ = "0.3.0"

som_parser-0.3.0/som_parser/parser.py

@@ -0,0 +1,79 @@
+"""Parse and validate SOM JSON."""
+
+from __future__ import annotations
+
+import json
+from typing import Any, Union
+
+from pydantic import ValidationError
+
+from .types import Som
+
+
+def parse_som(input: Union[str, dict]) -> Som:
+    """Parse a JSON string or dict into a validated Som object.
+
+    Args:
+        input: A JSON string or a dictionary conforming to the SOM schema.
+
+    Returns:
+        A validated Som instance.
+
+    Raises:
+        ValueError: If the input is not valid JSON.
+        ValidationError: If the input does not conform to the SOM schema.
+    """
+    if isinstance(input, str):
+        try:
+            data = json.loads(input)
+        except json.JSONDecodeError as e:
+            raise ValueError(f"Invalid JSON: {e}") from e
+    elif isinstance(input, dict):
+        data = input
+    else:
+        raise TypeError(f"Expected str or dict, got {type(input).__name__}")
+
+    return Som.model_validate(data)
+
+
+def is_valid_som(input: Any) -> bool:
+    """Check if input conforms to the SOM schema.
+
+    Args:
+        input: A JSON string, dict, or any other value.
+
+    Returns:
+        True if the input is valid SOM, False otherwise.
+    """
+    try:
+        parse_som(input)
+        return True
+    except (ValueError, ValidationError, TypeError):
+        return False
+
+
+def from_plasmate(json_output: str) -> Som:
+    """Parse raw Plasmate CLI JSON output into a Som object.
+
+    Plasmate CLI outputs JSON that may be the SOM directly or wrapped
+    in a container object with a ``som`` key.
+
+    Args:
+        json_output: Raw JSON string from Plasmate CLI.
+
+    Returns:
+        A validated Som instance.
+
+    Raises:
+        ValueError: If the output cannot be parsed.
+    """
+    try:
+        data = json.loads(json_output)
+    except json.JSONDecodeError as e:
+        raise ValueError(f"Invalid JSON from Plasmate: {e}") from e
+
+    # Handle wrapped output: {"som": {...}}
+    if isinstance(data, dict) and "som" in data and "som_version" not in data:
+        data = data["som"]
+
+    return Som.model_validate(data)

som_parser-0.3.0/som_parser/query.py

@@ -0,0 +1,221 @@
+"""Query, filter, and search utilities for SOM objects."""
+
+from __future__ import annotations
+
+from typing import Callable, Dict, List, Optional, Union
+
+from .types import ElementRole, RegionRole, Som, SomElement, SomRegion
+
+
+def _collect_elements(elements: List[SomElement]) -> List[SomElement]:
+    """Recursively collect all elements including nested children."""
+    result: List[SomElement] = []
+    for el in elements:
+        result.append(el)
+        if el.children:
+            result.extend(_collect_elements(el.children))
+    return result
+
+
+def get_all_elements(som: Som) -> List[SomElement]:
+    """Flatten all elements from all regions, including nested children."""
+    result: List[SomElement] = []
+    for region in som.regions:
+        result.extend(_collect_elements(region.elements))
+    return result
+
+
+def find_by_role(som: Som, role: Union[ElementRole, str]) -> List[SomElement]:
+    """Find all elements with a specific role.
+
+    Args:
+        som: The parsed SOM object.
+        role: An ElementRole enum value or string (e.g. "link").
+    """
+    if isinstance(role, str):
+        role = ElementRole(role)
+    return [el for el in get_all_elements(som) if el.role == role]
+
+
+def find_by_id(som: Som, id: str) -> Optional[SomElement]:
+    """Find an element by its SOM id. Returns None if not found."""
+    for el in get_all_elements(som):
+        if el.id == id:
+            return el
+    return None
+
+
+def find_by_text(
+    som: Som, text: str, *, exact: bool = False
+) -> List[SomElement]:
+    """Find elements containing text.
+
+    Args:
+        som: The parsed SOM object.
+        text: The text to search for.
+        exact: If True, match the full text exactly (case-sensitive).
+               If False (default), case-insensitive substring match.
+    """
+    results: List[SomElement] = []
+    for el in get_all_elements(som):
+        el_text = el.text or ""
+        el_label = el.label or ""
+        if exact:
+            if text == el_text or text == el_label:
+                results.append(el)
+        else:
+            text_lower = text.lower()
+            if text_lower in el_text.lower() or text_lower in el_label.lower():
+                results.append(el)
+    return results
+
+
+def get_interactive_elements(som: Som) -> List[SomElement]:
+    """Get all elements that have actions."""
+    return [el for el in get_all_elements(som) if el.actions]
+
+
+def get_links(som: Som) -> List[Dict[str, Optional[str]]]:
+    """Extract all links as dicts with text, href, and id."""
+    links: List[Dict[str, Optional[str]]] = []
+    for el in find_by_role(som, ElementRole.LINK):
+        href = el.attrs.href if el.attrs else None
+        links.append({"text": el.text, "href": href, "id": el.id})
+    return links
+
+
+def get_forms(som: Som) -> List[SomRegion]:
+    """Get all form regions."""
+    return [r for r in som.regions if r.role == RegionRole.FORM]
+
+
+def get_inputs(som: Som) -> List[SomElement]:
+    """Get all input elements (text_input, textarea, select, checkbox, radio)."""
+    input_roles = {
+        ElementRole.TEXT_INPUT,
+        ElementRole.TEXTAREA,
+        ElementRole.SELECT,
+        ElementRole.CHECKBOX,
+        ElementRole.RADIO,
+    }
+    return [el for el in get_all_elements(som) if el.role in input_roles]
+
+
+def get_headings(som: Som) -> List[Dict[str, object]]:
+    """Extract heading hierarchy as a list of dicts with level, text, and id."""
+    headings: List[Dict[str, object]] = []
+    for el in find_by_role(som, ElementRole.HEADING):
+        level = el.attrs.level if el.attrs and el.attrs.level is not None else 0
+        headings.append({"level": level, "text": el.text, "id": el.id})
+    return headings
+
+
+def get_text(som: Som) -> str:
+    """Extract all visible text content from the SOM."""
+    parts: List[str] = []
+    for el in get_all_elements(som):
+        if el.text:
+            parts.append(el.text)
+        elif el.label:
+            parts.append(el.label)
+    return "\n".join(parts)
+
+
+def get_text_by_region(som: Som) -> List[Dict[str, object]]:
+    """Extract text grouped by region."""
+    results: List[Dict[str, object]] = []
+    for region in som.regions:
+        texts: List[str] = []
+        for el in _collect_elements(region.elements):
+            if el.text:
+                texts.append(el.text)
+            elif el.label:
+                texts.append(el.label)
+        results.append({
+            "region_id": region.id,
+            "role": region.role.value,
+            "label": region.label,
+            "text": "\n".join(texts),
+        })
+    return results
+
+
+def get_compression_ratio(som: Som) -> float:
+    """Return html_bytes / som_bytes compression ratio."""
+    if som.meta.som_bytes == 0:
+        return float("inf")
+    return som.meta.html_bytes / som.meta.som_bytes
+
+
+def to_markdown(som: Som) -> str:
+    """Convert a SOM object to readable markdown."""
+    lines: List[str] = []
+    lines.append(f"# {som.title}")
+    lines.append(f"URL: {som.url}")
+    lines.append("")
+
+    for region in som.regions:
+        role_label = region.role.value.title()
+        if region.label:
+            lines.append(f"## {role_label}: {region.label}")
+        else:
+            lines.append(f"## {role_label}")
+        lines.append("")
+
+        for el in _collect_elements(region.elements):
+            _element_to_markdown(el, lines)
+
+        lines.append("")
+
+    return "\n".join(lines)
+
+
+def _element_to_markdown(el: SomElement, lines: List[str]) -> None:
+    """Append markdown for a single element."""
+    role = el.role
+
+    if role == ElementRole.HEADING:
+        level = el.attrs.level if el.attrs and el.attrs.level else 1
+        prefix = "#" * (level + 2)  # offset by 2 since region is ##
+        lines.append(f"{prefix} {el.text or ''}")
+    elif role == ElementRole.PARAGRAPH:
+        lines.append(el.text or "")
+        lines.append("")
+    elif role == ElementRole.LINK:
+        href = el.attrs.href if el.attrs else "#"
+        lines.append(f"[{el.text or ''}]({href})")
+    elif role == ElementRole.BUTTON:
+        lines.append(f"[Button: {el.text or ''}]")
+    elif role == ElementRole.IMAGE:
+        alt = el.attrs.alt if el.attrs else ""
+        src = el.attrs.src if el.attrs else ""
+        lines.append(f"![{alt}]({src})")
+    elif role in (ElementRole.TEXT_INPUT, ElementRole.TEXTAREA):
+        label = el.label or ""
+        placeholder = ""
+        if el.attrs and el.attrs.placeholder:
+            placeholder = f' placeholder="{el.attrs.placeholder}"'
+        lines.append(f"[Input: {label}{placeholder}]")
+    elif role == ElementRole.SELECT:
+        lines.append(f"[Select: {el.label or el.text or ''}]")
+    elif role in (ElementRole.CHECKBOX, ElementRole.RADIO):
+        checked = ""
+        if el.attrs and el.attrs.checked:
+            checked = "x"
+        lines.append(f"[{checked}] {el.text or el.label or ''}")
+    elif role == ElementRole.LIST:
+        if el.attrs and el.attrs.items:
+            for item in el.attrs.items:
+                lines.append(f"- {item.text}")
+    elif role == ElementRole.SEPARATOR:
+        lines.append("---")
+    else:
+        if el.text:
+            lines.append(el.text)
+
+
+def filter_elements(
+    som: Som, predicate: Callable[[SomElement], bool]
+) -> List[SomElement]:
+    """Filter all elements using a predicate function."""
+    return [el for el in get_all_elements(som) if predicate(el)]

som_parser-0.3.0/som_parser/types.py

@@ -0,0 +1,152 @@
+"""Pydantic v2 models for SOM (Semantic Object Model)."""
+
+from __future__ import annotations
+
+from enum import Enum
+from typing import Any, Dict, List, Optional
+
+from pydantic import BaseModel
+
+
+class RegionRole(str, Enum):
+    NAVIGATION = "navigation"
+    MAIN = "main"
+    ASIDE = "aside"
+    HEADER = "header"
+    FOOTER = "footer"
+    FORM = "form"
+    DIALOG = "dialog"
+    CONTENT = "content"
+
+
+class ElementRole(str, Enum):
+    LINK = "link"
+    BUTTON = "button"
+    TEXT_INPUT = "text_input"
+    TEXTAREA = "textarea"
+    SELECT = "select"
+    CHECKBOX = "checkbox"
+    RADIO = "radio"
+    HEADING = "heading"
+    IMAGE = "image"
+    LIST = "list"
+    TABLE = "table"
+    PARAGRAPH = "paragraph"
+    SECTION = "section"
+    SEPARATOR = "separator"
+
+
+class ElementAction(str, Enum):
+    CLICK = "click"
+    TYPE = "type"
+    CLEAR = "clear"
+    SELECT = "select"
+    TOGGLE = "toggle"
+
+
+class SemanticHint(str, Enum):
+    ACTIVE = "active"
+    BADGE = "badge"
+    CARD = "card"
+    COLLAPSED = "collapsed"
+    DANGER = "danger"
+    DISABLED = "disabled"
+    ERROR = "error"
+    EXPANDED = "expanded"
+    HERO = "hero"
+    HIDDEN = "hidden"
+    LARGE = "large"
+    LOADING = "loading"
+    MODAL = "modal"
+    NOTIFICATION = "notification"
+    PRIMARY = "primary"
+    REQUIRED = "required"
+    SECONDARY = "secondary"
+    SELECTED = "selected"
+    SMALL = "small"
+    STICKY = "sticky"
+    SUCCESS = "success"
+    WARNING = "warning"
+
+
+class SelectOption(BaseModel):
+    value: str
+    text: str
+    selected: Optional[bool] = None
+
+
+class ListItem(BaseModel):
+    text: str
+
+
+class SomElementAttrs(BaseModel):
+    href: Optional[str] = None
+    input_type: Optional[str] = None
+    value: Optional[str] = None
+    placeholder: Optional[str] = None
+    required: Optional[bool] = None
+    disabled: Optional[bool] = None
+    checked: Optional[bool] = None
+    group: Optional[str] = None
+    multiple: Optional[bool] = None
+    options: Optional[List[SelectOption]] = None
+    level: Optional[int] = None
+    alt: Optional[str] = None
+    src: Optional[str] = None
+    ordered: Optional[bool] = None
+    items: Optional[List[ListItem]] = None
+    headers: Optional[List[str]] = None
+    rows: Optional[List[List[str]]] = None
+    section_label: Optional[str] = None
+
+
+class SomElement(BaseModel):
+    id: str
+    role: ElementRole
+    text: Optional[str] = None
+    label: Optional[str] = None
+    actions: Optional[List[ElementAction]] = None
+    attrs: Optional[SomElementAttrs] = None
+    children: Optional[List[SomElement]] = None
+    hints: Optional[List[SemanticHint]] = None
+
+
+class SomRegion(BaseModel):
+    id: str
+    role: RegionRole
+    label: Optional[str] = None
+    action: Optional[str] = None
+    method: Optional[str] = None
+    elements: List[SomElement]
+
+
+class SomMeta(BaseModel):
+    html_bytes: int
+    som_bytes: int
+    element_count: int
+    interactive_count: int
+
+
+class LinkElement(BaseModel):
+    rel: str
+    href: str
+    type: Optional[str] = None
+    hreflang: Optional[str] = None
+
+
+class StructuredData(BaseModel):
+    json_ld: Optional[List[Dict[str, Any]]] = None
+    open_graph: Optional[Dict[str, str]] = None
+    twitter_card: Optional[Dict[str, str]] = None
+    meta: Optional[Dict[str, str]] = None
+    links: Optional[List[LinkElement]] = None
+
+
+class Som(BaseModel):
+    som_version: str
+    url: str
+    title: str
+    lang: str
+    regions: List[SomRegion]
+    meta: SomMeta
+    structured_data: Optional[StructuredData] = None

som_parser-0.3.0/tests/test_parser.py

@@ -0,0 +1,421 @@
+"""Tests for som-parser package."""
+
+import json
+
+import pytest
+
+from som_parser import (
+    ElementRole,
+    RegionRole,
+    Som,
+    SomElement,
+    filter_elements,
+    find_by_id,
+    find_by_role,
+    find_by_text,
+    from_plasmate,
+    get_all_elements,
+    get_compression_ratio,
+    get_forms,
+    get_headings,
+    get_inputs,
+    get_interactive_elements,
+    get_links,
+    get_text,
+    get_text_by_region,
+    is_valid_som,
+    parse_som,
+    to_markdown,
+)
+
+FIXTURE_SOM = {
+    "som_version": "0.1",
+    "url": "https://example.com/",
+    "title": "Example Domain",
+    "lang": "en",
+    "regions": [
+        {
+            "id": "r_nav",
+            "role": "navigation",
+            "elements": [
+                {
+                    "id": "e_1",
+                    "role": "link",
+                    "text": "Home",
+                    "actions": ["click"],
+                    "attrs": {"href": "/"},
+                },
+                {
+                    "id": "e_2",
+                    "role": "link",
+                    "text": "About",
+                    "actions": ["click"],
+                    "attrs": {"href": "/about"},
+                },
+            ],
+        },
+        {
+            "id": "r_content",
+            "role": "content",
+            "elements": [
+                {
+                    "id": "e_3",
+                    "role": "heading",
+                    "text": "Welcome",
+                    "attrs": {"level": 1},
+                },
+                {
+                    "id": "e_4",
+                    "role": "paragraph",
+                    "text": "This is a test page.",
+                },
+                {
+                    "id": "e_5",
+                    "role": "link",
+                    "text": "Learn more",
+                    "actions": ["click"],
+                    "attrs": {"href": "https://example.org"},
+                },
+                {
+                    "id": "e_6",
+                    "role": "image",
+                    "attrs": {"src": "/logo.png", "alt": "Logo"},
+                },
+            ],
+        },
+        {
+            "id": "r_form",
+            "role": "form",
+            "action": "/search",
+            "method": "GET",
+            "elements": [
+                {
+                    "id": "e_7",
+                    "role": "text_input",
+                    "label": "Search",
+                    "actions": ["type", "clear"],
+                    "attrs": {"input_type": "text", "placeholder": "Search..."},
+                },
+                {
+                    "id": "e_8",
+                    "role": "button",
+                    "text": "Go",
+                    "actions": ["click"],
+                },
+            ],
+        },
+    ],
+    "meta": {
+        "html_bytes": 5000,
+        "som_bytes": 800,
+        "element_count": 8,
+        "interactive_count": 5,
+    },
+}
+
+
+@pytest.fixture
+def som() -> Som:
+    return parse_som(FIXTURE_SOM)
+
+
+@pytest.fixture
+def som_json() -> str:
+    return json.dumps(FIXTURE_SOM)
+
+
+# --- Parser tests ---
+
+
+class TestParseSom:
+    def test_parse_dict(self, som: Som):
+        assert isinstance(som, Som)
+        assert som.title == "Example Domain"
+        assert som.url == "https://example.com/"
+        assert som.som_version == "0.1"
+        assert som.lang == "en"
+        assert len(som.regions) == 3
+
+    def test_parse_json_string(self, som_json: str):
+        result = parse_som(som_json)
+        assert isinstance(result, Som)
+        assert result.title == "Example Domain"
+
+    def test_parse_invalid_json_string(self):
+        with pytest.raises(ValueError, match="Invalid JSON"):
+            parse_som("not valid json {{{")
+
+    def test_parse_invalid_schema(self):
+        with pytest.raises(Exception):
+            parse_som({"not": "a som"})
+
+    def test_parse_wrong_type(self):
+        with pytest.raises(TypeError):
+            parse_som(42)  # type: ignore
+
+    def test_regions_parsed(self, som: Som):
+        assert som.regions[0].role == RegionRole.NAVIGATION
+        assert som.regions[1].role == RegionRole.CONTENT
+        assert som.regions[2].role == RegionRole.FORM
+
+    def test_elements_parsed(self, som: Som):
+        nav_elements = som.regions[0].elements
+        assert len(nav_elements) == 2
+        assert nav_elements[0].role == ElementRole.LINK
+        assert nav_elements[0].text == "Home"
+
+    def test_meta_parsed(self, som: Som):
+        assert som.meta.html_bytes == 5000
+        assert som.meta.som_bytes == 800
+        assert som.meta.element_count == 8
+        assert som.meta.interactive_count == 5
+
+    def test_form_region_attrs(self, som: Som):
+        form = som.regions[2]
+        assert form.action == "/search"
+        assert form.method == "GET"
+
+
+class TestIsValidSom:
+    def test_valid(self):
+        assert is_valid_som(FIXTURE_SOM) is True
+
+    def test_valid_string(self):
+        assert is_valid_som(json.dumps(FIXTURE_SOM)) is True
+
+    def test_invalid_dict(self):
+        assert is_valid_som({"bad": "data"}) is False
+
+    def test_invalid_string(self):
+        assert is_valid_som("nope") is False
+
+    def test_invalid_type(self):
+        assert is_valid_som(123) is False
+
+
+class TestFromPlasmate:
+    def test_direct_som(self):
+        result = from_plasmate(json.dumps(FIXTURE_SOM))
+        assert result.title == "Example Domain"
+
+    def test_wrapped_som(self):
+        wrapped = json.dumps({"som": FIXTURE_SOM})
+        result = from_plasmate(wrapped)
+        assert result.title == "Example Domain"
+
+    def test_invalid_json(self):
+        with pytest.raises(ValueError, match="Invalid JSON"):
+            from_plasmate("not json")
+
+
+# --- Query tests ---
+
+
+class TestGetAllElements:
+    def test_count(self, som: Som):
+        elements = get_all_elements(som)
+        assert len(elements) == 8
+
+    def test_all_have_ids(self, som: Som):
+        elements = get_all_elements(som)
+        ids = [el.id for el in elements]
+        assert ids == ["e_1", "e_2", "e_3", "e_4", "e_5", "e_6", "e_7", "e_8"]
+
+
+class TestFindByRole:
+    def test_links(self, som: Som):
+        links = find_by_role(som, ElementRole.LINK)
+        assert len(links) == 3
+
+    def test_string_role(self, som: Som):
+        links = find_by_role(som, "link")
+        assert len(links) == 3
+
+    def test_headings(self, som: Som):
+        headings = find_by_role(som, ElementRole.HEADING)
+        assert len(headings) == 1
+        assert headings[0].text == "Welcome"
+
+    def test_buttons(self, som: Som):
+        buttons = find_by_role(som, ElementRole.BUTTON)
+        assert len(buttons) == 1
+        assert buttons[0].text == "Go"
+
+    def test_no_results(self, som: Som):
+        tables = find_by_role(som, ElementRole.TABLE)
+        assert tables == []
+
+
+class TestFindById:
+    def test_found(self, som: Som):
+        el = find_by_id(som, "e_3")
+        assert el is not None
+        assert el.text == "Welcome"
+        assert el.role == ElementRole.HEADING
+
+    def test_not_found(self, som: Som):
+        assert find_by_id(som, "nonexistent") is None
+
+
+class TestFindByText:
+    def test_substring(self, som: Som):
+        results = find_by_text(som, "home")
+        assert len(results) == 1
+        assert results[0].id == "e_1"
+
+    def test_case_insensitive(self, som: Som):
+        results = find_by_text(som, "WELCOME")
+        assert len(results) == 1
+
+    def test_exact_match(self, som: Som):
+        results = find_by_text(som, "Home", exact=True)
+        assert len(results) == 1
+
+    def test_exact_no_match(self, som: Som):
+        results = find_by_text(som, "home", exact=True)
+        assert len(results) == 0
+
+    def test_label_match(self, som: Som):
+        results = find_by_text(som, "search")
+        assert len(results) == 1
+        assert results[0].id == "e_7"
+
+    def test_no_match(self, som: Som):
+        results = find_by_text(som, "xyznonexistent")
+        assert len(results) == 0
+
+
+class TestGetInteractiveElements:
+    def test_count(self, som: Som):
+        interactive = get_interactive_elements(som)
+        assert len(interactive) == 5
+
+    def test_all_have_actions(self, som: Som):
+        interactive = get_interactive_elements(som)
+        for el in interactive:
+            assert el.actions is not None
+            assert len(el.actions) > 0
+
+
+class TestGetLinks:
+    def test_links(self, som: Som):
+        links = get_links(som)
+        assert len(links) == 3
+        assert links[0] == {"text": "Home", "href": "/", "id": "e_1"}
+        assert links[1] == {"text": "About", "href": "/about", "id": "e_2"}
+        assert links[2] == {
+            "text": "Learn more",
+            "href": "https://example.org",
+            "id": "e_5",
+        }
+
+
+class TestGetForms:
+    def test_forms(self, som: Som):
+        forms = get_forms(som)
+        assert len(forms) == 1
+        assert forms[0].id == "r_form"
+        assert forms[0].action == "/search"
+
+
+class TestGetInputs:
+    def test_inputs(self, som: Som):
+        inputs = get_inputs(som)
+        assert len(inputs) == 1
+        assert inputs[0].id == "e_7"
+        assert inputs[0].role == ElementRole.TEXT_INPUT
+
+
+class TestGetHeadings:
+    def test_headings(self, som: Som):
+        headings = get_headings(som)
+        assert len(headings) == 1
+        assert headings[0] == {"level": 1, "text": "Welcome", "id": "e_3"}
+
+
+class TestGetText:
+    def test_text(self, som: Som):
+        text = get_text(som)
+        assert "Home" in text
+        assert "About" in text
+        assert "Welcome" in text
+        assert "This is a test page." in text
+        assert "Learn more" in text
+        assert "Search" in text
+        assert "Go" in text
+
+
+class TestGetTextByRegion:
+    def test_regions(self, som: Som):
+        regions = get_text_by_region(som)
+        assert len(regions) == 3
+        assert regions[0]["region_id"] == "r_nav"
+        assert regions[0]["role"] == "navigation"
+        assert "Home" in regions[0]["text"]
+
+    def test_content_region(self, som: Som):
+        regions = get_text_by_region(som)
+        content = regions[1]
+        assert content["role"] == "content"
+        assert "Welcome" in content["text"]
+        assert "This is a test page." in content["text"]
+
+
+class TestGetCompressionRatio:
+    def test_ratio(self, som: Som):
+        ratio = get_compression_ratio(som)
+        assert ratio == 5000 / 800
+        assert ratio == 6.25
+
+
+class TestToMarkdown:
+    def test_contains_title(self, som: Som):
+        md = to_markdown(som)
+        assert "# Example Domain" in md
+
+    def test_contains_url(self, som: Som):
+        md = to_markdown(som)
+        assert "URL: https://example.com/" in md
+
+    def test_contains_regions(self, som: Som):
+        md = to_markdown(som)
+        assert "## Navigation" in md
+        assert "## Content" in md
+        assert "## Form" in md
+
+    def test_contains_links(self, som: Som):
+        md = to_markdown(som)
+        assert "[Home](/)" in md
+        assert "[About](/about)" in md
+
+    def test_contains_heading(self, som: Som):
+        md = to_markdown(som)
+        assert "### Welcome" in md
+
+    def test_contains_paragraph(self, som: Som):
+        md = to_markdown(som)
+        assert "This is a test page." in md
+
+    def test_contains_image(self, som: Som):
+        md = to_markdown(som)
+        assert "![Logo](/logo.png)" in md
+
+    def test_contains_button(self, som: Som):
+        md = to_markdown(som)
+        assert "[Button: Go]" in md
+
+    def test_contains_input(self, som: Som):
+        md = to_markdown(som)
+        assert "Input: Search" in md
+
+
+class TestFilterElements:
+    def test_filter_by_actions(self, som: Som):
+        clickable = filter_elements(
+            som, lambda el: el.actions is not None and "click" in [a.value for a in el.actions]
+        )
+        assert len(clickable) == 4  # 3 links + 1 button
+
+    def test_filter_by_text(self, som: Som):
+        with_text = filter_elements(som, lambda el: el.text is not None)
+        assert len(with_text) == 6  # all except image and text_input