agent-tool-parser 0.1.0__tar.gz

agent_tool_parser-0.1.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,41 @@
+---
+name: Bug Report
+about: Create a report to help us improve agent-tool-parser
+title: "[BUG] "
+labels: bug
+assignees: ''
+---
+
+### Describe the Bug
+A clear and concise description of what the bug is.
+
+### Engine Mode
+- [ ] Pure Python (`AGENT_TOOL_PARSER_NO_EXT=1`)
+- [ ] Accelerated PyO3 (`_accelerated`)
+- [ ] Rust Core (`agent-tool-parser-core`)
+- [ ] Go bindings / C-ABI / WASM
+
+### Minimal Reproducible Example
+
+```python
+from agent_tool_parser import parse_tool_calls
+
+raw_model_output = """
+<paste sample model completion here>
+"""
+
+calls = parse_tool_calls(raw_model_output)
+print(calls)
+```
+
+### Expected Behavior
+A clear description of what tool calls, names, or arguments you expected to be parsed.
+
+### Actual Output or Error
+Include tracebacks or unexpected parsed outputs here.
+
+### Environment
+- OS: [e.g., Linux Ubuntu 24.04, macOS Sonoma, Windows 11]
+- Python version: [e.g., 3.11.8]
+- Rust version (if building from source): [e.g., 1.80.0]
+- Library version: [e.g., 0.1.0]

agent_tool_parser-0.1.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,25 @@
+---
+name: Feature Request / Format Extension
+about: Suggest an idea or new LLM tool call format for agent-tool-parser
+title: "[FEAT] "
+labels: enhancement
+assignees: ''
+---
+
+### Problem Description
+Is your feature request related to a specific model provider, format, or edge case? Please describe the problem.
+
+### Proposed Format Specification
+Provide sample outputs produced by the target model:
+
+```text
+<paste representative model output here>
+```
+
+### Desired ToolCall Extraction
+How should the sample above be normalized into a `ToolCall`?
+- **Name**: `...`
+- **Args**: `{...}`
+
+### Additional Context
+Link to official model documentation, technical report, or Hugging Face tokenizer configs if available.

agent_tool_parser-0.1.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,28 @@
+## Summary of Changes
+Provide a brief explanation of what this pull request does and why it is needed.
+
+## Dual-Engine Parity Verification
+`agent-tool-parser` enforces strict Dual-Engine Parity between the pure-Python reference engine and the compiled Rust core.
+
+- [ ] I have implemented all parsing/cleaning changes in both `src/agent_tool_parser/` and `crates/agent-tool-parser-core/`.
+- [ ] *OR* If parity is not technically possible, I have included the mandatory **Dual-Engine Parity Analysis** section below.
+
+### Dual-Engine Parity Analysis (only required if engines diverge)
+*Root Cause, Impact Assessment, and Mitigation Strategy.*
+
+---
+
+## Contributor Legal & DCO Checklist
+- [ ] All commits in this pull request are signed off (`git commit -s`) per the [Developer Certificate of Origin (DCO 1.1)](DCO.md).
+- [ ] I confirm that this submission contains no proprietary, leaked, or license-incompatible code.
+- [ ] I have read and agree to the [Contributing Guide](CONTRIBUTING.md) and Contributor Indemnification terms (Section 3.2.3).
+
+---
+
+## Test & Verification Checklist
+- [ ] Unit tests added to both `tests/test_parser.py` (Python) and `crates/agent-tool-parser-core/` (Rust).
+- [ ] `cargo test --all-targets` passes with zero errors.
+- [ ] Pure-Python test suite passes: `AGENT_TOOL_PARSER_NO_EXT=1 python -m unittest discover -s tests`
+- [ ] Accelerated PyO3 test suite passes: `python -m unittest discover -s tests`
+- [ ] Rust formatting & linting: `cargo fmt --all -- --check` and `cargo clippy --all-targets -- -D warnings`
+- [ ] Python formatting & linting: `ruff check .` and `ruff format --check .` and `mypy src`

agent_tool_parser-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,77 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+  workflow_dispatch:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install ruff mypy pytest
+          pip install -e .
+
+      - name: Run Ruff Linter & Format Check
+        run: |
+          ruff check src tests
+          ruff format --check src tests
+
+      - name: Run Mypy Type Checker
+        run: |
+          mypy src
+
+      - name: Run Unit Tests (Accelerated / Native if built)
+        run: |
+          python -m unittest discover -s tests
+
+      - name: Run Unit Tests (Pure-Python Universal Fallback)
+        run: |
+          AGENT_TOOL_PARSER_NO_EXT=1 python -m unittest discover -s tests
+
+  rust-test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: clippy, rustfmt
+          targets: wasm32-unknown-unknown
+      - name: Set up Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: "1.22"
+      - name: Run Rustfmt Check
+        run: cargo fmt --all -- --check
+      - name: Run Clippy
+        run: cargo clippy --workspace --all-targets -- -D warnings
+      - name: Run Cargo Tests
+        run: cargo test --all-targets
+      - name: Build C Library
+        run: cargo build --release -p agent-tool-parser-c
+      - name: Run Go Binding Tests
+        run: LD_LIBRARY_PATH="$(pwd)/target/release:$LD_LIBRARY_PATH" go test -v ./bindings/go/agenttoolparser/...
+      - name: Build WebAssembly Target
+        run: cargo build --target wasm32-unknown-unknown --release -p agent-tool-parser-wasm
+      - name: Set up Python for PyO3 Parity Verification
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Build PyO3 Native Extension & Run Dual-Engine Parity Tests
+        run: |
+          cargo build --release -p agent-tool-parser-py
+          cp target/release/lib_accelerated.so src/agent_tool_parser/_accelerated.so
+          PYTHONPATH=src python -m unittest discover -s tests -v

agent_tool_parser-0.1.0/.github/workflows/pages.yml

@@ -0,0 +1,41 @@
+name: Deploy Documentation to GitHub Pages
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - 'docs/**'
+      - '.github/workflows/pages.yml'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: "pages"
+  cancel-in-progress: false
+
+jobs:
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Setup Pages
+        uses: actions/configure-pages@v4
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: './docs'
+
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

agent_tool_parser-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,177 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - "v*"
+  workflow_dispatch:
+    inputs:
+      tag_name:
+        description: "Release tag (e.g. v0.1.0)"
+        required: false
+        default: ""
+
+permissions:
+  contents: write
+  id-token: write
+
+jobs:
+  build-python:
+    name: Build Python Package
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install build dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install build
+
+      - name: Build sdist and wheel
+        run: python -m build
+
+      - name: Upload Python distribution artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: python-dist
+          path: dist/*
+          if-no-files-found: error
+          retention-days: 1
+
+  build-c-binaries:
+    name: Build C Libraries (${{ matrix.target_name }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - os: ubuntu-latest
+            target_name: linux-x86_64
+            lib_file: libagent_tool_parser_c.so
+            is_windows: false
+          - os: macos-latest
+            target_name: macos-universal
+            lib_file: libagent_tool_parser_c.dylib
+            is_windows: false
+          - os: windows-latest
+            target_name: windows-x86_64
+            lib_file: agent_tool_parser_c.dll
+            is_windows: true
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+
+      - name: Build C shared library
+        run: cargo build --release -p agent-tool-parser-c
+
+      - name: Package C distribution (Unix)
+        if: ${{ !matrix.is_windows }}
+        run: |
+          mkdir -p c-dist/include c-dist/lib
+          cp crates/agent-tool-parser-c/include/agent_tool_parser.h c-dist/include/
+          cp target/release/${{ matrix.lib_file }} c-dist/lib/
+          cp LICENSE README.md c-dist/
+          tar -czvf agent-tool-parser-c-${{ matrix.target_name }}.tar.gz -C c-dist .
+
+      - name: Package C distribution (Windows)
+        if: ${{ matrix.is_windows }}
+        shell: pwsh
+        run: |
+          New-Item -ItemType Directory -Path c-dist/include, c-dist/lib -Force
+          Copy-Item crates/agent-tool-parser-c/include/agent_tool_parser.h c-dist/include/
+          Copy-Item target/release/${{ matrix.lib_file }} c-dist/lib/
+          Copy-Item LICENSE, README.md c-dist/
+          Compress-Archive -Path c-dist/* -DestinationPath agent-tool-parser-c-${{ matrix.target_name }}.zip
+
+      - name: Upload C binary artifacts (Unix)
+        if: ${{ !matrix.is_windows }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: c-binaries-${{ matrix.target_name }}
+          path: agent-tool-parser-c-${{ matrix.target_name }}.tar.gz
+          retention-days: 1
+
+      - name: Upload C binary artifacts (Windows)
+        if: ${{ matrix.is_windows }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: c-binaries-${{ matrix.target_name }}
+          path: agent-tool-parser-c-${{ matrix.target_name }}.zip
+          retention-days: 1
+
+  github-release:
+    name: Create GitHub Release
+    needs: [build-python, build-c-binaries]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Create release assets directory
+        run: mkdir -p release-assets
+
+      - name: Download all build artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: downloaded-artifacts
+
+      - name: Gather release assets
+        run: |
+          find downloaded-artifacts -type f -exec cp {} release-assets/ \;
+          cd release-assets
+          sha256sum * > SHA256SUMS.txt
+          cat SHA256SUMS.txt
+
+      - name: Publish GitHub Release
+        uses: softprops/action-gh-release@v2
+        with:
+          files: release-assets/*
+          generate_release_notes: true
+          draft: false
+          prerelease: false
+
+  publish-pypi:
+    name: Publish to PyPI
+    needs: [build-python, github-release]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Download Python distribution
+        uses: actions/download-artifact@v4
+        with:
+          name: python-dist
+          path: dist
+
+      - name: Publish package to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          packages-dir: dist/
+          skip-existing: true
+        continue-on-error: true
+
+  publish-crates:
+    name: Publish Rust Core Crate
+    needs: [github-release]
+    runs-on: ubuntu-latest
+    env:
+      CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+
+      - name: Publish to crates.io
+        if: env.CARGO_REGISTRY_TOKEN != ''
+        run: cargo publish -p agent-tool-parser-core
+        continue-on-error: true

agent_tool_parser-0.1.0/.gitignore

@@ -0,0 +1,25 @@
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.pytest_cache/
+dist/
+build/
+*.egg-info/
+.eggs/
+venv/
+.env
+
+# Rust build artifacts
+target/
+
+# Local editor & agent artifacts
+.antigravity/
+.agents/
+.mypy_cache/
+.ruff_cache/
+.DS_Store
+
+# Release and distribution packaging
+c-dist/
+release-assets/

agent_tool_parser-0.1.0/CHANGELOG.md

@@ -0,0 +1,41 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2026-09-12
+
+### Added
+- **Dual-Engine Architecture**:
+  - Pure-Python zero-dependency fallback engine with transparent native acceleration detection.
+  - High-throughput Rust core engine (`agent-tool-parser-core`) with sub-millisecond execution.
+  - PyO3 native module (`_accelerated`) with zero-copy Python deserialization.
+  - C-ABI dynamic library (`agent-tool-parser-c`) and standard C header (`agent_tool_parser.h`).
+  - WebAssembly bindings (`agent-tool-parser-wasm`) for browser and Node.js runtimes.
+  - Native Go binding module (`bindings/go/agenttoolparser`) using C-ABI via cgo.
+- **Heterogeneous Format Support**:
+  - OpenAI function call objects and `tool_calls` arrays (including multi-call batches).
+  - Anthropic Claude XML tags (`<tool_use>`, `<invoke>`, `<ant_tool_use>`) and parameter markup.
+  - DeepSeek DSML tags (`<｜DSML｜invoke>`) with unquoted and quoted attribute syntax.
+  - DeepSeek native streaming delimiter tokens (`<｜tool call begin｜>`).
+  - Qwen-Agent tokens (`<|action_start|>`, `<|action_name|>`, `<|action_args|>`).
+  - ChatGLM function call formats (`✿FUNCTION✿`).
+  - Llama 3.x Python AST expressions (`call:tool(arg=val)` and positional argument mappings).
+  - ReAct agent text patterns (`Action: ... Action Input: ...`).
+- **Resilient Cleaners & Heuristic Repairs**:
+  - Bracket-stack depth tracking for truncated JSON payloads, automatically balancing unclosed nested objects `{` and arrays `[` at stream EOF.
+  - Single-quoted JSON repair preserving inner double quotes.
+  - Two-step reasoning/thinking block extraction (`<think>...</think>`, `<thought>`, `<reasoning>`), safely stripping internal reasoning and preventing premature tool execution.
+  - CDATA unwrapping (`<![CDATA[...]]>`) preserving whitespace, indentation, and unified diffs.
+  - HTML entity unescaping (`&lt;`, `&gt;`, `&amp;`, `&quot;`, `&#39;`).
+- **Verification & Parity**:
+  - 98 Python unit tests passing on both pure-Python and accelerated engines.
+  - 56 Rust core integration tests passing with 100% format parity.
+  - 25 dedicated pathological, degenerate, and adversarial input tests.
+  - Complete ReDoS resilience test with 50,000 repeating characters executed in <10ms.
+- **Project Infrastructure**:
+  - Developer Certificate of Origin (DCO 1.1) enforcement policy and contributor indemnification.
+  - PEP 561 typing compliance (`py.typed`).
+  - GitHub Actions CI/CD workflows for automated multi-runtime testing and multi-platform binary releases.

agent_tool_parser-0.1.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,66 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at **boostcode.sg@gmail.com**. All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
+**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+**Community Impact**: A violation through a single incident or series of actions.
+**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
+
+### 3. Temporary Ban
+**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
+**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
+**Consequence**: A permanent ban from any sort of public interaction within the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 2.1,
+available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html).