streaming-json-parser 0.1.0__tar.gz → 0.2.0__tar.gz

streaming_json_parser-0.2.0/PKG-INFO

@@ -0,0 +1,214 @@
+Metadata-Version: 2.4
+Name: streaming-json-parser
+Version: 0.2.0
+Summary: High-performance JSON decoding and true incremental parsing for Python streams.
+Author-email: Aramis Facchinetti <aramis.facchinetti16@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/aramisfacchinetti/streaming-json-parser
+Project-URL: Repository, https://github.com/aramisfacchinetti/streaming-json-parser
+Project-URL: Issues, https://github.com/aramisfacchinetti/streaming-json-parser/issues
+Keywords: streaming,json,parser,llm,large language model,incremental parsing
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Operating System :: OS Independent
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: test
+Requires-Dist: pytest>=8.3.5; extra == "test"
+Requires-Dist: msgspec>=0.18; extra == "test"
+Requires-Dist: pydantic-core>=2.0; extra == "test"
+Requires-Dist: jiter>=0.12; extra == "test"
+Provides-Extra: accelerated
+Requires-Dist: msgspec>=0.18; extra == "accelerated"
+Requires-Dist: orjson>=3.9; extra == "accelerated"
+Requires-Dist: pysimdjson>=5; extra == "accelerated"
+Requires-Dist: ujson>=5; extra == "accelerated"
+Requires-Dist: python-rapidjson>=1.23; extra == "accelerated"
+Requires-Dist: yyjson>=4; extra == "accelerated"
+Provides-Extra: partial
+Requires-Dist: pydantic-core>=2.0; extra == "partial"
+Requires-Dist: jiter>=0.12; extra == "partial"
+Provides-Extra: benchmark
+Requires-Dist: ijson>=3.3; extra == "benchmark"
+Requires-Dist: partial-json-parser>=0.2; extra == "benchmark"
+Requires-Dist: partialjson>=1.1; extra == "benchmark"
+Requires-Dist: json-repair>=0.63; extra == "benchmark"
+Requires-Dist: jsonriver>=1.0; extra == "benchmark"
+Requires-Dist: untruncate-json>=1.1; extra == "benchmark"
+Requires-Dist: ujson>=5; extra == "benchmark"
+Requires-Dist: python-rapidjson>=1.23; extra == "benchmark"
+Requires-Dist: yyjson>=4; extra == "benchmark"
+Requires-Dist: pydantic-core>=2.0; extra == "benchmark"
+Requires-Dist: jiter>=0.12; extra == "benchmark"
+Provides-Extra: native-build
+Requires-Dist: maturin<2,>=1; extra == "native-build"
+Dynamic: license-file
+
+# Streaming JSON Parser
+
+High-performance JSON decoding and true incremental parsing for Python streams.
+The package chooses an appropriate backend for complete documents, NDJSON, typed
+decoding, selective extraction, and partial input. The strict incremental API
+owns the semantics that ordinary JSON decoders do not provide.
+
+The current release is `0.2.0` and is in beta while the public API settles.
+
+## Install
+
+```bash
+python -m pip install streaming-json-parser
+```
+
+Optional backend groups:
+
+```bash
+python -m pip install 'streaming-json-parser[accelerated]'
+python -m pip install 'streaming-json-parser[partial]'
+```
+
+The Rust extension is optional. When a compatible wheel is available, install
+it separately:
+
+```bash
+python -m pip install streaming-json-parser-native
+```
+
+The Python implementation remains functional without optional dependencies.
+
+## Choose An API
+
+| Workload | API |
+| --- | --- |
+| Complete JSON document | `decode_complete_json` |
+| Complete document with zero-copy view semantics | `decode_complete_json_view` |
+| Newline-delimited JSON | `decode_ndjson` or `StreamingJsonParser(framing="ndjson")` |
+| True incremental parsing | `StreamingJsonParser` |
+| Structural partial snapshots | `decode_structural_partial_json` |
+| Repeated selective extraction | `make_tuned_json_path_extractor` |
+
+There is no universal fastest decoder for every payload. The tuned factories
+calibrate compatible backends for a representative workload; the benchmark
+[scorecard](docs/current-api-scorecard.md) records the current evidence.
+
+## Complete Documents
+
+```python
+from streaming_json_parser import decode_complete_json
+
+value = decode_complete_json(b'{"name":"example","ok":true}')
+assert value == {"name": "example", "ok": True}
+```
+
+For a stable repeated workload, bind a decoder once:
+
+```python
+from streaming_json_parser import make_tuned_complete_json_decoder
+
+decode = make_tuned_complete_json_decoder(
+    sample=b'{"id":1,"name":"example"}',
+    payload_size_hint=1024,
+)
+value = decode(b'{"id":2,"name":"another"}')
+```
+
+## Incremental Streams
+
+`StreamingJsonParser` preserves state across chunks and reports
+one of `EMPTY`, `PARTIAL`, `COMPLETE`, or `INVALID`.
+
+```python
+from streaming_json_parser import StreamingJsonParser, ParseStatus
+
+parser = StreamingJsonParser()
+result = parser.feed(b'{"message":"hel')
+assert result.status is ParseStatus.PARTIAL
+assert result.value == {"message": "hel"}
+
+result = parser.feed(b'lo"}')
+assert result.status is ParseStatus.COMPLETE
+assert result.value == {"message": "hello"}
+```
+
+Call `finish()` when the input source ends. This is required for ambiguous root
+scalars and for an NDJSON stream whose final record has no trailing newline.
+
+```python
+parser = StreamingJsonParser()
+parser.consume("12")
+result = parser.finish()
+assert result.value == 12
+```
+
+For NDJSON, use `poll_many()` to drain complete records:
+
+```python
+parser = StreamingJsonParser(framing="ndjson")
+parser.consume(b'{"id":1}\n{"id":2}\n')
+assert parser.poll_many() == [{"id": 1}, {"id": 2}]
+```
+
+## Partial JSON
+
+Structural mode is useful when an unfinished string value does not need to be
+returned. It is a cumulative finisher, not a resumable strict state machine:
+
+```python
+from streaming_json_parser import decode_structural_partial_json
+
+assert decode_structural_partial_json('{"items":[1,2') == {"items": [1, 2]}
+assert decode_structural_partial_json('{"text":"hel') == {}
+assert decode_structural_partial_json('{"text":"hel', trailing_strings=True) == {
+    "text": "hel"
+}
+```
+
+Use `StreamingJsonParser(partial_mode="structural")` when the
+prefix arrives as many small chunks and a stateful parser is preferable.
+
+## Selective Extraction
+
+```python
+from streaming_json_parser import make_tuned_json_path_extractor
+
+extract = make_tuned_json_path_extractor(
+    ("meta", "name"),
+    ("meta", "count"),
+    framing="single",
+    sample={"meta": {"name": "example", "count": 1}},
+    payload_size_hint=1024,
+)
+assert extract(b'{"meta":{"name":"example","count":2}}') == ("example", 2)
+```
+
+## Development
+
+```bash
+python -m pip install -e '.[test]'
+pytest
+```
+
+Build and inspect release artifacts locally:
+
+```bash
+python -m build
+python -m twine check dist/*
+```
+
+Benchmark artifacts are optional and can be regenerated with:
+
+```bash
+make benchmark-artifacts
+make verify-benchmark-artifacts
+```
+
+The package supports Python 3.10 and later. It is distributed under the MIT
+license.

streaming_json_parser-0.2.0/README.md

@@ -0,0 +1,159 @@
+# Streaming JSON Parser
+
+High-performance JSON decoding and true incremental parsing for Python streams.
+The package chooses an appropriate backend for complete documents, NDJSON, typed
+decoding, selective extraction, and partial input. The strict incremental API
+owns the semantics that ordinary JSON decoders do not provide.
+
+The current release is `0.2.0` and is in beta while the public API settles.
+
+## Install
+
+```bash
+python -m pip install streaming-json-parser
+```
+
+Optional backend groups:
+
+```bash
+python -m pip install 'streaming-json-parser[accelerated]'
+python -m pip install 'streaming-json-parser[partial]'
+```
+
+The Rust extension is optional. When a compatible wheel is available, install
+it separately:
+
+```bash
+python -m pip install streaming-json-parser-native
+```
+
+The Python implementation remains functional without optional dependencies.
+
+## Choose An API
+
+| Workload | API |
+| --- | --- |
+| Complete JSON document | `decode_complete_json` |
+| Complete document with zero-copy view semantics | `decode_complete_json_view` |
+| Newline-delimited JSON | `decode_ndjson` or `StreamingJsonParser(framing="ndjson")` |
+| True incremental parsing | `StreamingJsonParser` |
+| Structural partial snapshots | `decode_structural_partial_json` |
+| Repeated selective extraction | `make_tuned_json_path_extractor` |
+
+There is no universal fastest decoder for every payload. The tuned factories
+calibrate compatible backends for a representative workload; the benchmark
+[scorecard](docs/current-api-scorecard.md) records the current evidence.
+
+## Complete Documents
+
+```python
+from streaming_json_parser import decode_complete_json
+
+value = decode_complete_json(b'{"name":"example","ok":true}')
+assert value == {"name": "example", "ok": True}
+```
+
+For a stable repeated workload, bind a decoder once:
+
+```python
+from streaming_json_parser import make_tuned_complete_json_decoder
+
+decode = make_tuned_complete_json_decoder(
+    sample=b'{"id":1,"name":"example"}',
+    payload_size_hint=1024,
+)
+value = decode(b'{"id":2,"name":"another"}')
+```
+
+## Incremental Streams
+
+`StreamingJsonParser` preserves state across chunks and reports
+one of `EMPTY`, `PARTIAL`, `COMPLETE`, or `INVALID`.
+
+```python
+from streaming_json_parser import StreamingJsonParser, ParseStatus
+
+parser = StreamingJsonParser()
+result = parser.feed(b'{"message":"hel')
+assert result.status is ParseStatus.PARTIAL
+assert result.value == {"message": "hel"}
+
+result = parser.feed(b'lo"}')
+assert result.status is ParseStatus.COMPLETE
+assert result.value == {"message": "hello"}
+```
+
+Call `finish()` when the input source ends. This is required for ambiguous root
+scalars and for an NDJSON stream whose final record has no trailing newline.
+
+```python
+parser = StreamingJsonParser()
+parser.consume("12")
+result = parser.finish()
+assert result.value == 12
+```
+
+For NDJSON, use `poll_many()` to drain complete records:
+
+```python
+parser = StreamingJsonParser(framing="ndjson")
+parser.consume(b'{"id":1}\n{"id":2}\n')
+assert parser.poll_many() == [{"id": 1}, {"id": 2}]
+```
+
+## Partial JSON
+
+Structural mode is useful when an unfinished string value does not need to be
+returned. It is a cumulative finisher, not a resumable strict state machine:
+
+```python
+from streaming_json_parser import decode_structural_partial_json
+
+assert decode_structural_partial_json('{"items":[1,2') == {"items": [1, 2]}
+assert decode_structural_partial_json('{"text":"hel') == {}
+assert decode_structural_partial_json('{"text":"hel', trailing_strings=True) == {
+    "text": "hel"
+}
+```
+
+Use `StreamingJsonParser(partial_mode="structural")` when the
+prefix arrives as many small chunks and a stateful parser is preferable.
+
+## Selective Extraction
+
+```python
+from streaming_json_parser import make_tuned_json_path_extractor
+
+extract = make_tuned_json_path_extractor(
+    ("meta", "name"),
+    ("meta", "count"),
+    framing="single",
+    sample={"meta": {"name": "example", "count": 1}},
+    payload_size_hint=1024,
+)
+assert extract(b'{"meta":{"name":"example","count":2}}') == ("example", 2)
+```
+
+## Development
+
+```bash
+python -m pip install -e '.[test]'
+pytest
+```
+
+Build and inspect release artifacts locally:
+
+```bash
+python -m build
+python -m twine check dist/*
+```
+
+Benchmark artifacts are optional and can be regenerated with:
+
+```bash
+make benchmark-artifacts
+make verify-benchmark-artifacts
+```
+
+The package supports Python 3.10 and later. It is distributed under the MIT
+license.

{streaming_json_parser-0.1.0 → streaming_json_parser-0.2.0}/pyproject.toml

@@ -4,24 +4,23 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "streaming-json-parser"
-version = "0.1.0"
+version = "0.2.0"
 authors = [
     { name = "Aramis Facchinetti", email = "aramis.facchinetti16@gmail.com" },
 ]
-description = "A streaming JSON parser that processes JSON data incrementally, handling partial states. Useful for incrementally parsing partial responses from streaming outputs of Large Language Models (LLMs)."
+description = "High-performance JSON decoding and true incremental parsing for Python streams."
 readme = "README.md"
-requires-python = ">=3.8"
-license = { text = "MIT" }
+requires-python = ">=3.10"
+license = "MIT"
 classifiers = [
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.8",
-    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
-    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
     "Operating System :: OS Independent",
-    "Development Status :: 5 - Production/Stable",
+    "Development Status :: 4 - Beta",
     "Intended Audience :: Developers",
     "Topic :: Software Development :: Libraries :: Python Modules",
     "Topic :: Text Processing",
@@ -35,13 +34,45 @@ keywords = ["streaming", "json", "parser", "llm", "large language model", "incre
 [project.optional-dependencies]
 test = [
     "pytest>=8.3.5",
+    "msgspec>=0.18",
+    "pydantic-core>=2.0",
+    "jiter>=0.12",
+]
+accelerated = [
+    "msgspec>=0.18",
+    "orjson>=3.9",
+    "pysimdjson>=5",
+    "ujson>=5",
+    "python-rapidjson>=1.23",
+    "yyjson>=4",
+]
+partial = [
+    "pydantic-core>=2.0",
+    "jiter>=0.12",
+]
+benchmark = [
+    "ijson>=3.3",
+    "partial-json-parser>=0.2",
+    "partialjson>=1.1",
+    "json-repair>=0.63",
+    "jsonriver>=1.0",
+    "untruncate-json>=1.1",
+    "ujson>=5",
+    "python-rapidjson>=1.23",
+    "yyjson>=4",
+    "pydantic-core>=2.0",
+    "jiter>=0.12",
+]
+native-build = [
+    "maturin>=1,<2",
 ]
 
 [project.urls]
 Homepage = "https://github.com/aramisfacchinetti/streaming-json-parser"
 Repository = "https://github.com/aramisfacchinetti/streaming-json-parser"
+Issues = "https://github.com/aramisfacchinetti/streaming-json-parser/issues"
 
 [tool.setuptools.packages.find]
 where = ["src"]
 include = ["streaming_json_parser*"]
-exclude = ["tests*"]
+exclude = ["tests*"]

streaming_json_parser-0.2.0/src/streaming_json_parser/__init__.py

@@ -0,0 +1,72 @@
+from .high_performance_parser import (
+    HighPerformanceStreamingJsonParser,
+    ParseResult,
+    ParseStatus,
+    decode_complete_json,
+    decode_complete_json_view,
+    decode_ndjson,
+    decode_ndjson_adaptive,
+    decode_structural_partial_json,
+    extract_complete_json_paths,
+    extract_complete_json_typed_paths,
+    extract_ndjson_paths,
+    extract_ndjson_paths_native,
+    extract_ndjson_typed_paths,
+    extract_tuned_complete_json_paths,
+    extract_tuned_json_paths,
+    extract_tuned_ndjson_paths,
+    make_complete_json_decoder,
+    make_complete_json_typed_path_extractor,
+    make_complete_json_view_decoder,
+    make_json_path_extractor,
+    make_ndjson_decoder,
+    make_ndjson_path_extractor,
+    make_ndjson_path_extractor_native,
+    make_ndjson_typed_path_extractor,
+    make_tuned_complete_json_decoder,
+    make_tuned_complete_json_path_extractor,
+    make_tuned_json_path_extractor,
+    make_tuned_ndjson_decoder,
+    make_tuned_ndjson_path_extractor,
+    make_tuned_structural_partial_decoder,
+)
+
+__version__ = "0.2.0"
+
+# The canonical public name points to the strict incremental implementation.
+StreamingJsonParser = HighPerformanceStreamingJsonParser
+
+__all__ = [
+    "StreamingJsonParser",
+    "__version__",
+    "HighPerformanceStreamingJsonParser",
+    "ParseResult",
+    "ParseStatus",
+    "decode_complete_json",
+    "decode_complete_json_view",
+    "decode_ndjson",
+    "decode_ndjson_adaptive",
+    "decode_structural_partial_json",
+    "extract_complete_json_paths",
+    "extract_complete_json_typed_paths",
+    "extract_ndjson_paths",
+    "extract_ndjson_paths_native",
+    "extract_ndjson_typed_paths",
+    "extract_tuned_complete_json_paths",
+    "extract_tuned_json_paths",
+    "extract_tuned_ndjson_paths",
+    "make_complete_json_decoder",
+    "make_complete_json_typed_path_extractor",
+    "make_complete_json_view_decoder",
+    "make_json_path_extractor",
+    "make_ndjson_decoder",
+    "make_ndjson_path_extractor",
+    "make_ndjson_path_extractor_native",
+    "make_ndjson_typed_path_extractor",
+    "make_tuned_complete_json_decoder",
+    "make_tuned_complete_json_path_extractor",
+    "make_tuned_json_path_extractor",
+    "make_tuned_ndjson_decoder",
+    "make_tuned_ndjson_path_extractor",
+    "make_tuned_structural_partial_decoder",
+]