partialjson 1.0.0__tar.gz → 1.2.0__tar.gz

partialjson-1.2.0/CHANGELOG.md

@@ -0,0 +1,117 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.2.0] - 2026-09-12
+
+No API changes. Every input that 1.1.0 parsed successfully parses to the same
+value in 1.2.0, verified by `tests/test_compat_1_1_0.py`, which runs the frozen
+1.1.0 parser next to the current one over every prefix of a corpus of documents.
+
+### Fixed
+
+- Numbers with an exponent (`1e5`, `2.5E-3`) inside an incomplete array or
+  object raised `JSONDecodeError`. They now parse; an exponent that has not
+  received its digits yet (`1e`, `1e-`) is dropped until it is complete.
+- In strict mode an unterminated string whose tail was an incomplete escape
+  (`"foo\`, `"foo\u00`) returned `""`, discarding text that had already
+  streamed. It now returns `"foo"`; only the unfinished escape is held back
+  (issue #8).
+- In strict mode a string cut between the two halves of a surrogate pair
+  (`"\ud83d`, half of an emoji) returned a lone surrogate, which raises
+  `UnicodeEncodeError` as soon as it is encoded. The high half is now held
+  back until its partner arrives.
+- The JSON5 parser raised on partial literals (`{"a": tr`, `[fals`, `[Inf`)
+  and on exponent numbers, and treated a comment that had only streamed its
+  first `/` as an unknown token. It now behaves like the JSON parser.
+- JSON5 string decoding no longer depends on whether the optional `json5`
+  package is installed; the same escapes (`\x41`, `\'`, line continuations,
+  surrogate pairs) decode the same way either way.
+- `bytes` and `bytearray` input, which `json.loads` accepts, no longer crash
+  the fallback parser with `AttributeError`. A chunk that ends in the middle
+  of a multi-byte UTF-8 character drops the incomplete bytes.
+- A leading UTF-8 byte-order mark no longer causes a `JSONDecodeError`.
+- `JSONParser.strict`, `.on_extra_token` and `.last_parse_reminding` are
+  readable and assignable again (assigning `strict` on a 1.x parser was
+  silently ignored), and the 0.x method names `parse_string`, `parse_number`,
+  `parse_array`, `parse_object`, `parse_true`, `parse_false`, `parse_null`
+  and `parse_space` are callable again.
+
+### Changed
+
+- The scanner works on string indexes instead of re-slicing the input at
+  every token, so a parse is linear in the input size. A 900 KB partial
+  document went from about 1 s to about 60 ms per `parse()` call.
+- A literal that is not a prefix of `true`/`false`/`null` (for example
+  `[trap]`, which 1.1.0 returned as `[True]`) now raises, matching
+  `json.loads`. Prefixes such as `[t`, `[tru` still parse.
+- `_JSON5Parser` is now a subclass of the JSON parser instead of a copy of it.
+- Packaging moved to `pyproject.toml` with `requires-python >= 3.8`,
+  classifiers and a `py.typed` marker; the package is fully type-annotated.
+- CI runs on Python 3.8 through 3.14, with and without the optional `json5`
+  dependency.
+
+## [1.1.0] - 2026-02-20
+
+### Added
+
+- JSON5 support: comments, unquoted keys, single-quoted strings, hex numbers,
+  `Infinity`/`NaN`, trailing commas. Available through
+  `create_json5_parser()` or `JSONParser(json5_enabled=True)`; install
+  `partialjson[json5]` for the optional `json5` fast path (issue #9).
+- `create_json_parser()` factory.
+
+## [1.0.0] - 2026-02
+
+### Added
+
+- `CITATION.cff`, `CODE_OF_CONDUCT.md`, `CONTRIBUTING.md`, JOSS paper draft,
+  GitHub Actions test workflow.
+
+### Changed
+
+- `JSONParser` became a thin facade over an internal implementation class.
+
+## [0.1.0] - 2025-01-28
+
+### Fixed
+
+- Incomplete escape sequences (`"\`, `"\u12`) at the end of a streamed
+  string no longer raise (issue #8).
+
+## [0.0.8] - 2024-08-03
+
+### Added
+
+- Support `strict` mode based on [this issue](https://github.com/iw4p/partialjson/issues/5)
+- Test cases for `parser_strict` and `parser_non_strict` to handle incomplete and complete JSON strings with newline characters.
+- Example usage of both strict and non-strict parsers in the unit tests.
+- Unit tests for various number, string, boolean, array, and object parsing scenarios.
+
+### Changed
+
+- Updated incomplete number parsing logic to ensure better error handling and test coverage.
+
+### Fixed
+
+- Fixed issue with parsing incomplete floating point numbers where the parser incorrectly returned an error.
+- Corrected string parsing logic to properly handle escape characters in strict mode.
+
+## [0.0.2] - 2023-11-24
+
+### Added
+
+### Changed
+
+### Fixed
+
+- json format
+
+## [0.0.1] - 2023-11-24
+
+### Added
+
+- Initial implementation of `JSONParser` with support for only strict mode.

partialjson-1.2.0/CITATION.cff

@@ -0,0 +1,13 @@
+cff-version: 1.2.0
+title: partialjson
+message: "If you use this software, please cite it as below."
+type: software
+authors:
+  - family-names: Akbarzadeh
+    given-names: Nima
+    orcid: "https://orcid.org/0009-0005-8143-8083"
+repository-code: "https://github.com/iw4p/partialjson"
+keywords: [python, json, parsing, streaming]
+version: 1.0.0
+date-released: 2025-10-05
+

partialjson-1.2.0/MANIFEST.in

@@ -0,0 +1,3 @@
+include LICENSE README.md CHANGELOG.md CITATION.cff
+graft tests
+global-exclude __pycache__ *.py[cod]

partialjson-1.2.0/PKG-INFO

@@ -0,0 +1,153 @@
+Metadata-Version: 2.4
+Name: partialjson
+Version: 1.2.0
+Summary: Parse incomplete or partial JSON, e.g. from a streaming LLM response
+Author-email: Nima Akbarzadeh <iw4p@protonmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/iw4p/partialjson
+Project-URL: Repository, https://github.com/iw4p/partialjson
+Project-URL: Changelog, https://github.com/iw4p/partialjson/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/iw4p/partialjson/issues
+Keywords: json,partial,incomplete,streaming,llm,openai,json5
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing
+Classifier: Typing :: Typed
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: json5
+Requires-Dist: json5; extra == "json5"
+Dynamic: license-file
+
+# PartialJson
+
+[![Partialjson](https://github.com/iw4p/partialjson/raw/main/images/partialjson.png)](https://pypi.org/project/partialjson/)
+
+## Parse Partial and incomplete JSON in python
+
+![](https://github.com/iw4p/partialjson/raw/main/images/partialjson.gif)
+
+### Parse Partial and incomplete JSON in python with just 3 lines of python code.
+
+[![PyPI version](https://img.shields.io/pypi/v/partialjson.svg)](https://pypi.org/project/partialjson)
+[![Supported Python versions](https://img.shields.io/pypi/pyversions/partialjson.svg)](#Installation)
+[![Downloads](https://pepy.tech/badge/partialjson)](https://pepy.tech/project/partialjson)
+
+## Example
+
+```python
+from partialjson import JSONParser
+parser = JSONParser()
+
+incomplete_json = '{"name": "John Doe", "age": 30, "is_student": false, "courses": ["Math", "Science"'
+print(parser.parse(incomplete_json))
+# {'name': 'John Doe', 'age': 30, 'is_student': False, 'courses': ['Math', 'Science']}
+```
+
+Problem with `\n`? Use `strict=False`:
+
+```python
+from partialjson import JSONParser
+parser = JSONParser(strict=False)
+
+incomplete_json = '{"name": "John\nDoe", "age": 30, "is_student": false, "courses": ["Math", "Science"'
+print(parser.parse(incomplete_json))
+# {'name': 'John\nDoe', 'age': 30, 'is_student': False, 'courses': ['Math', 'Science']}
+```
+
+### JSON5 support
+
+Use `create_json5_parser` or `JSONParser(json5_enabled=True)` for JSON5 (comments, unquoted keys, single quotes, etc.):
+
+```python
+from partialjson import create_json5_parser
+parser = create_json5_parser()
+
+incomplete_json5 = '{name: "Demo", version: 1.0, items: [1, 2, 3,]'
+print(parser.parse(incomplete_json5))
+# {'name': 'Demo', 'version': 1.0, 'items': [1, 2, 3]}
+```
+
+The optional `json5` dependency speeds up parsing of complete JSON5 documents: `pip install partialjson[json5]`. Partial documents parse the same way with or without it.
+
+### What you get while a string is still streaming
+
+Text that has already arrived is returned; only what cannot be decided yet is held back. With `strict=True` (the default) escapes are decoded and an unfinished escape or half an emoji is dropped until it is complete:
+
+```python
+parser.parse('{"msg": "caf\\u00')        # {'msg': 'caf'}
+parser.parse('{"msg": "caf\\u00e9"')      # {'msg': 'café'}
+parser.parse('{"msg": "hi \\ud83d"')      # {'msg': 'hi '}
+parser.parse('{"msg": "hi \\ud83d\\ude00"')  # {'msg': 'hi 😀'}
+```
+
+With `strict=False` the raw text of an unfinished string is returned untouched, backslashes included.
+
+### Extra tokens
+
+If the input contains a complete value followed by more text, the value is returned and the callback passed as `on_extra_token` is called with the input, the value and the leftover text. The default callback prints to stdout; pass `on_extra_token=None` to silence it, or read `parser.last_parse_reminding` afterwards.
+
+```python
+parser = JSONParser(on_extra_token=None)
+parser.parse('{"a": 1} trailing')   # {'a': 1}
+parser.last_parse_reminding         # ' trailing'
+```
+
+### Installation
+
+```sh
+$ pip install partialjson
+```
+
+Also can be found on [pypi](https://pypi.org/project/partialjson/)
+
+### How can I use it?
+
+- Install the package by pip package manager.
+- After installing, you can use it and call the library.
+
+## Testing
+
+```bash
+pip install -e '.[json5]'
+pip install -r requirements-dev.txt
+pytest -q
+```
+
+`tests/test_compat_1_1_0.py` runs the frozen 1.1.0 parser next to the current one over every prefix of a corpus of documents, so behaviour changes for existing users show up as test failures.
+
+## Citation
+
+If you use this software, please cite it using the metadata in `CITATION.cff`.
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=iw4p/partialjson&type=Date)](https://star-history.com/#iw4p/partialjson&Date)
+
+### Issues
+
+Feel free to submit issues and enhancement requests or contact me via [vida.page/nima](https://vida.page/nima).
+
+### Contributing
+
+Please refer to each project's style and contribution guidelines for submitting patches and additions. In general, we follow the "fork-and-pull" Git workflow.
+
+1.  **Fork** the repo on GitHub
+2.  **Clone** the project to your own machine
+3.  **Update the Version** inside `partialjson/__init__.py` and add a `CHANGELOG.md` entry
+4.  **Commit** changes to your own branch
+5.  **Push** your work back up to your fork
+6.  Submit a **Pull request** so that we can review your changes

partialjson-1.0.0/PKG-INFO → partialjson-1.2.0/README.md

@@ -1,22 +1,3 @@
-Metadata-Version: 2.4
-Name: partialjson
-Version: 1.0.0
-Summary: Parse incomplete or partial json
-Home-page: https://github.com/iw4p/partialjson
-Author: Nima Akbarzadeh
-Author-email: iw4p@protonmail.com
-License: MIT
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Dynamic: author
-Dynamic: author-email
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: license
-Dynamic: license-file
-Dynamic: summary
-
 # PartialJson
 
 [![Partialjson](https://github.com/iw4p/partialjson/raw/main/images/partialjson.png)](https://pypi.org/project/partialjson/)
@@ -34,18 +15,18 @@ Dynamic: summary
 ## Example
 
 ```python
-from partialjson.json_parser import JSONParser
+from partialjson import JSONParser
 parser = JSONParser()
 
 incomplete_json = '{"name": "John Doe", "age": 30, "is_student": false, "courses": ["Math", "Science"'
 print(parser.parse(incomplete_json))
-# {'name': 'John', 'age': 30, 'is_student': False, 'courses': ['Math', 'Science']}
+# {'name': 'John Doe', 'age': 30, 'is_student': False, 'courses': ['Math', 'Science']}
 ```
 
-Problem with `\n`? strict mode is here
+Problem with `\n`? Use `strict=False`:
 
 ```python
-from partialjson.json_parser import JSONParser
+from partialjson import JSONParser
 parser = JSONParser(strict=False)
 
 incomplete_json = '{"name": "John\nDoe", "age": 30, "is_student": false, "courses": ["Math", "Science"'
@@ -53,6 +34,44 @@ print(parser.parse(incomplete_json))
 # {'name': 'John\nDoe', 'age': 30, 'is_student': False, 'courses': ['Math', 'Science']}
 ```
 
+### JSON5 support
+
+Use `create_json5_parser` or `JSONParser(json5_enabled=True)` for JSON5 (comments, unquoted keys, single quotes, etc.):
+
+```python
+from partialjson import create_json5_parser
+parser = create_json5_parser()
+
+incomplete_json5 = '{name: "Demo", version: 1.0, items: [1, 2, 3,]'
+print(parser.parse(incomplete_json5))
+# {'name': 'Demo', 'version': 1.0, 'items': [1, 2, 3]}
+```
+
+The optional `json5` dependency speeds up parsing of complete JSON5 documents: `pip install partialjson[json5]`. Partial documents parse the same way with or without it.
+
+### What you get while a string is still streaming
+
+Text that has already arrived is returned; only what cannot be decided yet is held back. With `strict=True` (the default) escapes are decoded and an unfinished escape or half an emoji is dropped until it is complete:
+
+```python
+parser.parse('{"msg": "caf\\u00')        # {'msg': 'caf'}
+parser.parse('{"msg": "caf\\u00e9"')      # {'msg': 'café'}
+parser.parse('{"msg": "hi \\ud83d"')      # {'msg': 'hi '}
+parser.parse('{"msg": "hi \\ud83d\\ude00"')  # {'msg': 'hi 😀'}
+```
+
+With `strict=False` the raw text of an unfinished string is returned untouched, backslashes included.
+
+### Extra tokens
+
+If the input contains a complete value followed by more text, the value is returned and the callback passed as `on_extra_token` is called with the input, the value and the leftover text. The default callback prints to stdout; pass `on_extra_token=None` to silence it, or read `parser.last_parse_reminding` afterwards.
+
+```python
+parser = JSONParser(on_extra_token=None)
+parser.parse('{"a": 1} trailing')   # {'a': 1}
+parser.last_parse_reminding         # ' trailing'
+```
+
 ### Installation
 
 ```sh
@@ -69,11 +88,13 @@ Also can be found on [pypi](https://pypi.org/project/partialjson/)
 ## Testing
 
 ```bash
-pip install -e .
+pip install -e '.[json5]'
 pip install -r requirements-dev.txt
 pytest -q
 ```
 
+`tests/test_compat_1_1_0.py` runs the frozen 1.1.0 parser next to the current one over every prefix of a corpus of documents, so behaviour changes for existing users show up as test failures.
+
 ## Citation
 
 If you use this software, please cite it using the metadata in `CITATION.cff`.
@@ -92,7 +113,7 @@ Please refer to each project's style and contribution guidelines for submitting
 
 1.  **Fork** the repo on GitHub
 2.  **Clone** the project to your own machine
-3.  **Update the Version** inside **init**.py
+3.  **Update the Version** inside `partialjson/__init__.py` and add a `CHANGELOG.md` entry
 4.  **Commit** changes to your own branch
 5.  **Push** your work back up to your fork
 6.  Submit a **Pull request** so that we can review your changes

partialjson-1.2.0/partialjson/__init__.py

@@ -0,0 +1,24 @@
+"""
+Partial Json.
+
+Parse partial and incomplete JSON, such as a streaming LLM response, without
+crashing: ``JSONParser().parse('{"a": [1, 2')`` returns ``{"a": [1, 2]}``.
+"""
+
+from .json5_parser import create_json5_parser
+from .json_parser import JSONParser, create_json_parser
+
+__version__ = "1.2.0"
+__author__ = "Nima Akbarzadeh"
+__author_email__ = "iw4p@protonmail.com"
+__license__ = "MIT"
+__url__ = "https://github.com/iw4p/partialjson"
+
+PYPI_SIMPLE_ENDPOINT: str = "https://pypi.org/project/partialjson"
+
+__all__ = [
+    "PYPI_SIMPLE_ENDPOINT",
+    "JSONParser",
+    "create_json5_parser",
+    "create_json_parser",
+]

partialjson-1.2.0/partialjson/json5_parser.py

@@ -0,0 +1,235 @@
+"""JSON5 parser - extends JSON with comments, unquoted keys, single quotes, etc.
+
+Built on top of the JSON scanner in ``json_parser``; only the JSON5-specific
+pieces (whitespace and comments, identifiers, extra string escapes, hex and
+signed numbers, ``Infinity``/``NaN``, case-insensitive literals) are overridden.
+"""
+import json
+from types import ModuleType
+from typing import Any, ClassVar, FrozenSet, Optional, Tuple
+
+from .json_parser import (
+    _HEX,
+    _NO_KEY,
+    OnExtraToken,
+    ScanResult,
+    _default_on_extra_token,
+    _is_high_surrogate,
+    _is_low_surrogate,
+    _JSONParser,
+)
+
+json5: Optional[ModuleType]
+try:
+    import json5
+except ImportError:  # pragma: no cover - exercised via monkeypatching in tests
+    json5 = None
+
+__all__ = ["_default_on_extra_token", "create_json5_parser"]
+
+_JSON5_WHITESPACE = "\v\f\u00A0\u2028\u2029\uFEFF"
+_LINE_TERMINATORS = "\n\r\u2028\u2029"
+_SIMPLE_ESCAPES = {
+    "b": "\b",
+    "f": "\f",
+    "n": "\n",
+    "r": "\r",
+    "t": "\t",
+    "v": "\v",
+    "0": "\0",
+}
+
+
+def create_json5_parser(
+    strict: bool = True, on_extra_token: Optional[OnExtraToken] = None
+) -> "_JSON5Parser":
+    """Create a JSON5 parser."""
+    return _JSON5Parser(strict=strict, on_extra_token=on_extra_token)
+
+
+def _decode_json5_string(content: str) -> str:
+    """Decode the body of a JSON5 string literal (quotes already removed)."""
+    out = []
+    i = 0
+    n = len(content)
+    while i < n:
+        c = content[i]
+        if c != "\\":
+            out.append(c)
+            i += 1
+            continue
+        if i + 1 >= n:
+            raise ValueError("incomplete escape")
+        esc = content[i + 1]
+        if esc == "u":
+            hex4 = content[i + 2 : i + 6]
+            if len(hex4) < 4 or any(h not in _HEX for h in hex4):
+                raise ValueError("bad \\u escape")
+            code = int(hex4, 16)
+            i += 6
+            if _is_high_surrogate(hex4) and content[i : i + 2] == "\\u":
+                low = content[i + 2 : i + 6]
+                if len(low) == 4 and all(h in _HEX for h in low) and _is_low_surrogate(low):
+                    code = 0x10000 + ((code - 0xD800) << 10) + (int(low, 16) - 0xDC00)
+                    i += 6
+            out.append(chr(code))
+        elif esc == "x":
+            hex2 = content[i + 2 : i + 4]
+            if len(hex2) < 2 or any(h not in _HEX for h in hex2):
+                raise ValueError("bad \\x escape")
+            out.append(chr(int(hex2, 16)))
+            i += 4
+        elif esc == "\r":
+            i += 3 if content[i + 2 : i + 3] == "\n" else 2
+        elif esc in _LINE_TERMINATORS:
+            i += 2  # line continuation
+        elif esc in _SIMPLE_ESCAPES:
+            out.append(_SIMPLE_ESCAPES[esc])
+            i += 2
+        else:
+            out.append(esc)  # \' \" \\ \/ and any other escaped character
+            i += 2
+    return "".join(out)
+
+
+class _JSON5Parser(_JSONParser):
+    """JSON5 parser with comments, unquoted keys, single quotes, hex, Infinity, etc."""
+
+    _VALUE_START: ClassVar[FrozenSet[str]] = frozenset("[{\"'tfnTFNI+0123456789.-")
+
+    # ------------------------------------------------------------ fast path
+
+    def _loads(self, s: str) -> Any:
+        try:
+            return json.loads(s)
+        except (json.JSONDecodeError, ValueError):
+            if json5 is None:
+                raise
+            return json5.loads(s)
+
+    # ------------------------------------------------- whitespace & comments
+
+    def _is_space(self, s: str, i: int) -> bool:
+        c = s[i]
+        return c.isspace() or c in _JSON5_WHITESPACE
+
+    def _skip_space(self, s: str, i: int) -> int:
+        n = len(s)
+        while i < n:
+            c = s[i]
+            if c.isspace() or c in _JSON5_WHITESPACE:
+                i += 1
+            elif c == "/":
+                if i + 1 >= n:
+                    return n  # a comment that has only streamed its first '/'
+                nxt = s[i + 1]
+                if nxt == "/":
+                    i += 2
+                    while i < n and s[i] not in _LINE_TERMINATORS:
+                        i += 1
+                elif nxt == "*":
+                    end = s.find("*/", i + 2)
+                    if end == -1:
+                        return n  # unterminated block comment swallows the rest
+                    i = end + 2
+                else:
+                    break
+            else:
+                break
+        return i
+
+    # --------------------------------------------------------------- values
+
+    def _scan_value(self, s: str, i: int, e: BaseException) -> ScanResult:
+        c = s[i]
+        if c == "'":
+            return self._scan_string(s, i, e)
+        if c in "tT":
+            return self._scan_literal(s, i, "true", True, e)
+        if c in "fF":
+            return self._scan_literal(s, i, "false", False, e)
+        if c == "n":
+            return self._scan_literal(s, i, "null", None, e)
+        if c == "N":
+            return self._scan_n_literal(s, i, e)
+        if c in "+I":
+            return self._scan_number(s, i, e)
+        return super()._scan_value(s, i, e)
+
+    def _scan_key(self, s: str, i: int, e: BaseException) -> ScanResult:
+        if s[i] in "\"'":
+            return self._scan_string(s, i, e)
+        n = len(s)
+        j = i
+        while j < n and (s[j].isalnum() or s[j] in "_$"):
+            j += 1
+        if j == i:
+            return _NO_KEY, i
+        return s[i:j], j
+
+    # -------------------------------------------------------------- strings
+
+    def _scan_extra_escape(self, s: str, j: int, n: int) -> Tuple[bool, int]:
+        esc = s[j + 1]
+        if esc == "x":
+            hex2 = s[j + 2 : j + 4]
+            if all(h in _HEX for h in hex2):
+                if len(hex2) < 2:
+                    return True, 0  # \x or \xA at the end of the input
+                return False, 4
+            return False, 2
+        if esc == "\r" and s[j + 2 : j + 3] == "\n":
+            return False, 3
+        return False, 2
+
+    def _decode_incomplete(self, quote: str, content: str) -> Any:
+        try:
+            return _decode_json5_string(content)
+        except ValueError:
+            return ""
+
+    def _decode_complete(self, literal: str, quote: str) -> Any:
+        try:
+            return _decode_json5_string(literal[1:-1])
+        except ValueError:
+            return literal[1:-1]
+
+    # -------------------------------------------------------------- numbers
+
+    def _scan_number(self, s: str, i: int, e: BaseException) -> ScanResult:
+        n = len(s)
+        sign = 1
+        j = i
+        if s[j] in "+-":
+            sign = -1 if s[j] == "-" else 1
+            j += 1
+        if s[j : j + 2] in ("0x", "0X"):
+            k = j + 2
+            while k < n and s[k] in _HEX:
+                k += 1
+            if k == j + 2:
+                return s[i:k], n  # "0x" with no digits yet
+            return sign * int(s[j + 2 : k], 16), k
+        for word, value in (("Infinity", float("inf")), ("NaN", float("nan"))):
+            k = self._literal_matches(s, j, word)
+            if k and (k == len(word) or j + k >= n):
+                return sign * value, j + k
+        if s[i] == "+":
+            if j >= n:
+                return "+", n
+            return super()._scan_number(s, j, e)
+        return super()._scan_number(s, i, e)
+
+    # ------------------------------------------------------------- literals
+
+    def _literal_matches(self, s: str, i: int, word: str) -> int:
+        n = len(s)
+        k = 0
+        while i + k < n and k < len(word) and s[i + k].lower() == word[k].lower():
+            k += 1
+        return k
+
+    def _scan_n_literal(self, s: str, i: int, e: BaseException) -> ScanResult:
+        if s[i + 1 : i + 2].lower() == "a":
+            return self._scan_number(s, i, e)
+        return self._scan_literal(s, i, "null", None, e)