json-repair 0.30.0__tar.gz → 0.30.2__tar.gz

{json_repair-0.30.0/src/json_repair.egg-info → json_repair-0.30.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: json_repair
-Version: 0.30.0
+Version: 0.30.2
 Summary: A package to repair broken json strings
 Author-email: Stefano Baccianella <4247706+mangiucugna@users.noreply.github.com>
 License: MIT License
@@ -32,18 +32,20 @@ Keywords: JSON,REPAIR,LLM,PARSER
 Classifier: Programming Language :: Python :: 3
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
-Requires-Python: >=3.8
+Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 
 [![PyPI](https://img.shields.io/pypi/v/json-repair)](https://pypi.org/project/json-repair/)
-![Python version](https://img.shields.io/badge/python-3.8+-important)
+![Python version](https://img.shields.io/badge/python-3.9+-important)
 [![PyPI downloads](https://img.shields.io/pypi/dm/json-repair)](https://pypi.org/project/json-repair/)
 [![Github Sponsors](https://img.shields.io/github/sponsors/mangiucugna)](https://github.com/sponsors/mangiucugna)
+[![GitHub Repo stars](https://img.shields.io/github/stars/mangiucugna/json_repair?style=flat)](https://github.com/mangiucugna/json_repair/stargazers)
+
 
 This simple package can be used to fix an invalid json string. To know all cases in which this package will work, check out the unit test.
 
-Inspired by https://github.com/josdejong/jsonrepair
+![banner](banner.png)
 
 ---
 # Offer me a beer
@@ -54,6 +56,8 @@ If you find this library useful, you can help me by donating toward my monthly b
 # Demo
 If you are unsure if this library will fix your specific problem, or simply want your json validated online, you can visit the demo site on GitHub pages: https://mangiucugna.github.io/json_repair/
 
+Or hear an [audio deepdive generate by Google's NotebookLM](https://notebooklm.google.com/notebook/05312bb3-f6f3-4e49-a99b-bd51db64520b/audio) for an introduction to the module
+
 ---
 
 # Motivation
@@ -64,6 +68,11 @@ I searched for a lightweight python package that was able to reliably fix this p
 
 *So I wrote one*
 
+### Wouldn't GPT-4o Structured Output make this library outdated?
+
+As part of my job we use OpenAI APIs and we noticed that even with structured output sometimes the result isn't a fully valid json.
+So we still use this library to cover those outliers.
+
 # Supported use cases
 
 ### Fixing Syntax Errors in JSON
@@ -144,6 +153,26 @@ and another method to read from a file:
 
 Keep in mind that the library will not catch any IO-related exception and those will need to be managed by you
 
+### Non-Latin characters
+
+When working with non-Latin characters (such as Chinese, Japanese, or Korean), you need to pass `ensure_ascii=False` to `repair_json()` in order to preserve the non-Latin characters in the output.
+
+Here's an example using Chinese characters:
+
+    repair_json("{'test_chinese_ascii':'统一码'}")
+
+will return
+
+    {"test_chinese_ascii": "\u7edf\u4e00\u7801"}
+
+Instead passing `ensure_ascii=False`:
+
+    repair_json("{'test_chinese_ascii':'统一码'}", ensure_ascii=False)
+
+will return
+
+    {"test_chinese_ascii": "统一码"}
+
 ### Performance considerations
 If you find this library too slow because is using `json.loads()` you can skip that by passing `skip_json_loads=True` to `repair_json`. Like:
 
@@ -226,7 +255,7 @@ This module will parse the JSON file following the BNF definition:
     <object> ::= '{' [ <member> *(', ' <member>) ] '}' ; A sequence of 'members'
     <member> ::= <string> ': ' <json> ; A pair consisting of a name, and a JSON value
 
-If something is wrong (a missing parantheses or quotes for example) it will use a few simple heuristics to fix the JSON string:
+If something is wrong (a missing parentheses or quotes for example) it will use a few simple heuristics to fix the JSON string:
 - Add the missing parentheses if the parser believes that the array or object should be closed
 - Quote strings or add missing single quotes
 - Adjust whitespaces and remove line breaks

{json_repair-0.30.0 → json_repair-0.30.2}/README.md

@@ -1,11 +1,13 @@
 [![PyPI](https://img.shields.io/pypi/v/json-repair)](https://pypi.org/project/json-repair/)
-![Python version](https://img.shields.io/badge/python-3.8+-important)
+![Python version](https://img.shields.io/badge/python-3.9+-important)
 [![PyPI downloads](https://img.shields.io/pypi/dm/json-repair)](https://pypi.org/project/json-repair/)
 [![Github Sponsors](https://img.shields.io/github/sponsors/mangiucugna)](https://github.com/sponsors/mangiucugna)
+[![GitHub Repo stars](https://img.shields.io/github/stars/mangiucugna/json_repair?style=flat)](https://github.com/mangiucugna/json_repair/stargazers)
+
 
 This simple package can be used to fix an invalid json string. To know all cases in which this package will work, check out the unit test.
 
-Inspired by https://github.com/josdejong/jsonrepair
+![banner](banner.png)
 
 ---
 # Offer me a beer
@@ -16,6 +18,8 @@ If you find this library useful, you can help me by donating toward my monthly b
 # Demo
 If you are unsure if this library will fix your specific problem, or simply want your json validated online, you can visit the demo site on GitHub pages: https://mangiucugna.github.io/json_repair/
 
+Or hear an [audio deepdive generate by Google's NotebookLM](https://notebooklm.google.com/notebook/05312bb3-f6f3-4e49-a99b-bd51db64520b/audio) for an introduction to the module
+
 ---
 
 # Motivation
@@ -26,6 +30,11 @@ I searched for a lightweight python package that was able to reliably fix this p
 
 *So I wrote one*
 
+### Wouldn't GPT-4o Structured Output make this library outdated?
+
+As part of my job we use OpenAI APIs and we noticed that even with structured output sometimes the result isn't a fully valid json.
+So we still use this library to cover those outliers.
+
 # Supported use cases
 
 ### Fixing Syntax Errors in JSON
@@ -106,6 +115,26 @@ and another method to read from a file:
 
 Keep in mind that the library will not catch any IO-related exception and those will need to be managed by you
 
+### Non-Latin characters
+
+When working with non-Latin characters (such as Chinese, Japanese, or Korean), you need to pass `ensure_ascii=False` to `repair_json()` in order to preserve the non-Latin characters in the output.
+
+Here's an example using Chinese characters:
+
+    repair_json("{'test_chinese_ascii':'统一码'}")
+
+will return
+
+    {"test_chinese_ascii": "\u7edf\u4e00\u7801"}
+
+Instead passing `ensure_ascii=False`:
+
+    repair_json("{'test_chinese_ascii':'统一码'}", ensure_ascii=False)
+
+will return
+
+    {"test_chinese_ascii": "统一码"}
+
 ### Performance considerations
 If you find this library too slow because is using `json.loads()` you can skip that by passing `skip_json_loads=True` to `repair_json`. Like:
 
@@ -188,7 +217,7 @@ This module will parse the JSON file following the BNF definition:
     <object> ::= '{' [ <member> *(', ' <member>) ] '}' ; A sequence of 'members'
     <member> ::= <string> ': ' <json> ; A pair consisting of a name, and a JSON value
 
-If something is wrong (a missing parantheses or quotes for example) it will use a few simple heuristics to fix the JSON string:
+If something is wrong (a missing parentheses or quotes for example) it will use a few simple heuristics to fix the JSON string:
 - Add the missing parentheses if the parser believes that the array or object should be closed
 - Quote strings or add missing single quotes
 - Adjust whitespaces and remove line breaks

{json_repair-0.30.0 → json_repair-0.30.2}/pyproject.toml

@@ -3,7 +3,7 @@ requires = ["setuptools>=61.0"]
 build-backend = "setuptools.build_meta"
 [project]
 name = "json_repair"
-version = "0.30.0"
+version = "0.30.2"
 license = {file = "LICENSE"}
 authors = [
   { name="Stefano Baccianella", email="4247706+mangiucugna@users.noreply.github.com" },
@@ -11,7 +11,7 @@ authors = [
 description = "A package to repair broken json strings"
 keywords = ["JSON", "REPAIR", "LLM", "PARSER"]
 readme = "README.md"
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 classifiers = [
     "Programming Language :: Python :: 3",
     "License :: OSI Approved :: MIT License",

{json_repair-0.30.0 → json_repair-0.30.2}/src/json_repair/json_context.py

@@ -24,11 +24,9 @@ class JsonContext:
         Returns:
             None
         """
-        # If a value is provided update the context variable and save in stack
-        if value:
-            self.context.append(value)
-            self.current = value
-            self.empty = False
+        self.context.append(value)
+        self.current = value
+        self.empty = False
 
     def reset(self) -> None:
         """

{json_repair-0.30.0 → json_repair-0.30.2}/src/json_repair/json_parser.py

@@ -314,31 +314,49 @@ class JSONParser:
                 if next_c:
                     i += 1
                     # found a delimiter, now we need to check that is followed strictly by a comma or brace
+                    # or the string ended
                     i = self.skip_whitespaces_at(idx=i, move_main_index=False)
                     next_c = self.get_char_at(i)
-                    if next_c and next_c in [",", "}"]:
+                    if not next_c or next_c in [",", "}"]:
                         rstring_delimiter_missing = False
+                    else:
+                        # OK but this could still be some garbage at the end of the string
+                        # So we need to check if we find a new lstring_delimiter afterwards
+                        # If we do, this is a missing delimiter
+                        i = self.skip_to_character(character=lstring_delimiter, idx=i)
+                        next_c = self.get_char_at(i)
+                        if not next_c:
+                            rstring_delimiter_missing = False
                 else:
-                    # skip any whitespace first
-                    i = self.skip_whitespaces_at(idx=1, move_main_index=False)
-                    # We couldn't find any rstring_delimeter before the end of the string
-                    # check if this is the last string of an object and therefore we can keep going
-                    # make an exception if this is the last char before the closing brace
-                    j = self.skip_to_character(character="}", idx=i)
-                    if j - i > 1:
-                        # Ok it's not right after the comma
-                        # Let's ignore
-                        rstring_delimiter_missing = False
-                    # Check that j was not out of bound
-                    elif self.get_char_at(j):
-                        # Check for an unmatched opening brace in string_acc
-                        for c in reversed(string_acc):
-                            if c == "{":
-                                # Ok then this is part of the string
-                                rstring_delimiter_missing = False
-                                break
-                            elif c == "}":
-                                break
+                    # There could be a case in which even the next key:value is missing delimeters
+                    # because it might be a systemic issue with the output
+                    # So let's check if we can find a : in the string instead
+                    i = self.skip_to_character(character=":", idx=1)
+                    next_c = self.get_char_at(i)
+                    if next_c:
+                        # OK then this is a systemic issue with the output
+                        break
+                    else:
+                        # skip any whitespace first
+                        i = self.skip_whitespaces_at(idx=1, move_main_index=False)
+                        # We couldn't find any rstring_delimeter before the end of the string
+                        # check if this is the last string of an object and therefore we can keep going
+                        # make an exception if this is the last char before the closing brace
+                        j = self.skip_to_character(character="}", idx=i)
+                        if j - i > 1:
+                            # Ok it's not right after the comma
+                            # Let's ignore
+                            rstring_delimiter_missing = False
+                        # Check that j was not out of bound
+                        elif self.get_char_at(j):
+                            # Check for an unmatched opening brace in string_acc
+                            for c in reversed(string_acc):
+                                if c == "{":
+                                    # Ok then this is part of the string
+                                    rstring_delimiter_missing = False
+                                    break
+                                elif c == "}":
+                                    break
                 if rstring_delimiter_missing:
                     self.log(
                         "While parsing a string missing the left delimiter in object value context, we found a , or } and we couldn't determine that a right delimiter was present. Stopping here",

{json_repair-0.30.0 → json_repair-0.30.2}/src/json_repair/json_repair.py

@@ -41,10 +41,18 @@ def repair_json(
 ) -> Union[JSONReturnType, Tuple[JSONReturnType, List[Dict[str, str]]]]:
     """
     Given a json formatted string, it will try to decode it and, if it fails, it will try to fix it.
-    It will return the fixed string by default.
-    When `return_objects=True` is passed, it will return the decoded data structure instead.
-    When `skip_json_loads=True` is passed, it will not call the built-in json.loads() function
-    When `logging=True` is passed, it will return a tuple with the repaired json and a log of all repair actions
+
+    Args:
+        json_str (str, optional): The JSON string to repair. Defaults to an empty string.
+        return_objects (bool, optional): If True, return the decoded data structure. Defaults to False.
+        skip_json_loads (bool, optional): If True, skip calling the built-in json.loads() function to verify that the json is valid before attempting to repair. Defaults to False.
+        logging (bool, optional): If True, return a tuple with the repaired json and a log of all repair actions. Defaults to False.
+        json_fd (Optional[TextIO], optional): File descriptor for JSON input. Do not use! Use `from_file` or `load` instead. Defaults to None.
+        ensure_ascii (bool, optional): Set to False to avoid converting non-latin characters to ascii (for example when using chinese characters). Defaults to True. Ignored if `skip_json_loads` is True.
+        chunk_length (int, optional): Size in bytes of the file chunks to read at once. Ignored if `json_fd` is None. Do not use! Use `from_file` or `load` instead. Defaults to 1MB.
+
+    Returns:
+        Union[JSONReturnType, Tuple[JSONReturnType, List[Dict[str, str]]]]: The repaired JSON or a tuple with the repaired JSON and repair log.
     """
     parser = JSONParser(json_str, json_fd, logging, chunk_length)
     if skip_json_loads:
@@ -72,6 +80,14 @@ def loads(
     """
     This function works like `json.loads()` except that it will fix your JSON in the process.
     It is a wrapper around the `repair_json()` function with `return_objects=True`.
+
+    Args:
+        json_str (str): The JSON string to load and repair.
+        skip_json_loads (bool, optional): If True, skip calling the built-in json.loads() function to verify that the json is valid before attempting to repair. Defaults to False.
+        logging (bool, optional): If True, return a tuple with the repaired json and a log of all repair actions. Defaults to False.
+
+    Returns:
+        Union[JSONReturnType, Tuple[JSONReturnType, List[Dict[str, str]]]]: The repaired JSON object or a tuple with the repaired JSON object and repair log.
     """
     return repair_json(
         json_str=json_str,
@@ -90,6 +106,15 @@ def load(
     """
     This function works like `json.load()` except that it will fix your JSON in the process.
     It is a wrapper around the `repair_json()` function with `json_fd=fd` and `return_objects=True`.
+
+    Args:
+        fd (TextIO): File descriptor for JSON input.
+        skip_json_loads (bool, optional): If True, skip calling the built-in json.loads() function to verify that the json is valid before attempting to repair. Defaults to False.
+        logging (bool, optional): If True, return a tuple with the repaired json and a log of all repair actions. Defaults to False.
+        chunk_length (int, optional): Size in bytes of the file chunks to read at once. Defaults to 1MB.
+
+    Returns:
+        Union[JSONReturnType, Tuple[JSONReturnType, List[Dict[str, str]]]]: The repaired JSON object or a tuple with the repaired JSON object and repair log.
     """
     return repair_json(
         json_fd=fd,
@@ -108,20 +133,48 @@ def from_file(
 ) -> Union[JSONReturnType, Tuple[JSONReturnType, List[Dict[str, str]]]]:
     """
     This function is a wrapper around `load()` so you can pass the filename as string
+
+    Args:
+        filename (str): The name of the file containing JSON data to load and repair.
+        skip_json_loads (bool, optional): If True, skip calling the built-in json.loads() function to verify that the json is valid before attempting to repair. Defaults to False.
+        logging (bool, optional): If True, return a tuple with the repaired json and a log of all repair actions. Defaults to False.
+        chunk_length (int, optional): Size in bytes of the file chunks to read at once. Defaults to 1MB.
+
+    Returns:
+        Union[JSONReturnType, Tuple[JSONReturnType, List[Dict[str, str]]]]: The repaired JSON object or a tuple with the repaired JSON object and repair log.
     """
-    fd = open(filename)
-    jsonobj = load(
-        fd=fd,
-        skip_json_loads=skip_json_loads,
-        logging=logging,
-        chunk_length=chunk_length,
-    )
-    fd.close()
+    with open(filename) as fd:
+        jsonobj = load(
+            fd=fd,
+            skip_json_loads=skip_json_loads,
+            logging=logging,
+            chunk_length=chunk_length,
+        )
 
     return jsonobj
 
 
 def cli(inline_args: Optional[List[str]] = None) -> int:
+    """
+    Command-line interface for repairing and parsing JSON files.
+
+    Args:
+        inline_args (Optional[List[str]]): List of command-line arguments for testing purposes. Defaults to None.
+            - filename (str): The JSON file to repair
+            - -i, --inline (bool): Replace the file inline instead of returning the output to stdout.
+            - -o, --output TARGET (str): If specified, the output will be written to TARGET filename instead of stdout.
+            - --ensure_ascii (bool): Pass ensure_ascii=True to json.dumps(). Will pass False otherwise.
+            - --indent INDENT (int): Number of spaces for indentation (Default 2).
+
+    Returns:
+        int: Exit code of the CLI operation.
+
+    Raises:
+        Exception: Any exception that occurs during file processing.
+
+    Example:
+        >>> cli(['example.json', '--indent', '4'])
+    """
     parser = argparse.ArgumentParser(description="Repair and parse JSON files.")
     parser.add_argument("filename", help="The JSON file to repair")
     parser.add_argument(
@@ -167,14 +220,13 @@ def cli(inline_args: Optional[List[str]] = None) -> int:
         result = from_file(args.filename)
 
         if args.inline or args.output:
-            fd = open(args.output or args.filename, mode="w")
-            json.dump(result, fd, indent=args.indent, ensure_ascii=ensure_ascii)
-            fd.close()
+            with open(args.output or args.filename, mode="w") as fd:
+                json.dump(result, fd, indent=args.indent, ensure_ascii=ensure_ascii)
         else:
             print(json.dumps(result, indent=args.indent, ensure_ascii=ensure_ascii))
     except Exception as e:  # pragma: no cover
         print(f"Error: {str(e)}", file=sys.stderr)
-        sys.exit(1)
+        return 1
 
     return 0  # Success
 

{json_repair-0.30.0 → json_repair-0.30.2/src/json_repair.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: json_repair
-Version: 0.30.0
+Version: 0.30.2
 Summary: A package to repair broken json strings
 Author-email: Stefano Baccianella <4247706+mangiucugna@users.noreply.github.com>
 License: MIT License
@@ -32,18 +32,20 @@ Keywords: JSON,REPAIR,LLM,PARSER
 Classifier: Programming Language :: Python :: 3
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
-Requires-Python: >=3.8
+Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 
 [![PyPI](https://img.shields.io/pypi/v/json-repair)](https://pypi.org/project/json-repair/)
-![Python version](https://img.shields.io/badge/python-3.8+-important)
+![Python version](https://img.shields.io/badge/python-3.9+-important)
 [![PyPI downloads](https://img.shields.io/pypi/dm/json-repair)](https://pypi.org/project/json-repair/)
 [![Github Sponsors](https://img.shields.io/github/sponsors/mangiucugna)](https://github.com/sponsors/mangiucugna)
+[![GitHub Repo stars](https://img.shields.io/github/stars/mangiucugna/json_repair?style=flat)](https://github.com/mangiucugna/json_repair/stargazers)
+
 
 This simple package can be used to fix an invalid json string. To know all cases in which this package will work, check out the unit test.
 
-Inspired by https://github.com/josdejong/jsonrepair
+![banner](banner.png)
 
 ---
 # Offer me a beer
@@ -54,6 +56,8 @@ If you find this library useful, you can help me by donating toward my monthly b
 # Demo
 If you are unsure if this library will fix your specific problem, or simply want your json validated online, you can visit the demo site on GitHub pages: https://mangiucugna.github.io/json_repair/
 
+Or hear an [audio deepdive generate by Google's NotebookLM](https://notebooklm.google.com/notebook/05312bb3-f6f3-4e49-a99b-bd51db64520b/audio) for an introduction to the module
+
 ---
 
 # Motivation
@@ -64,6 +68,11 @@ I searched for a lightweight python package that was able to reliably fix this p
 
 *So I wrote one*
 
+### Wouldn't GPT-4o Structured Output make this library outdated?
+
+As part of my job we use OpenAI APIs and we noticed that even with structured output sometimes the result isn't a fully valid json.
+So we still use this library to cover those outliers.
+
 # Supported use cases
 
 ### Fixing Syntax Errors in JSON
@@ -144,6 +153,26 @@ and another method to read from a file:
 
 Keep in mind that the library will not catch any IO-related exception and those will need to be managed by you
 
+### Non-Latin characters
+
+When working with non-Latin characters (such as Chinese, Japanese, or Korean), you need to pass `ensure_ascii=False` to `repair_json()` in order to preserve the non-Latin characters in the output.
+
+Here's an example using Chinese characters:
+
+    repair_json("{'test_chinese_ascii':'统一码'}")
+
+will return
+
+    {"test_chinese_ascii": "\u7edf\u4e00\u7801"}
+
+Instead passing `ensure_ascii=False`:
+
+    repair_json("{'test_chinese_ascii':'统一码'}", ensure_ascii=False)
+
+will return
+
+    {"test_chinese_ascii": "统一码"}
+
 ### Performance considerations
 If you find this library too slow because is using `json.loads()` you can skip that by passing `skip_json_loads=True` to `repair_json`. Like:
 
@@ -226,7 +255,7 @@ This module will parse the JSON file following the BNF definition:
     <object> ::= '{' [ <member> *(', ' <member>) ] '}' ; A sequence of 'members'
     <member> ::= <string> ': ' <json> ; A pair consisting of a name, and a JSON value
 
-If something is wrong (a missing parantheses or quotes for example) it will use a few simple heuristics to fix the JSON string:
+If something is wrong (a missing parentheses or quotes for example) it will use a few simple heuristics to fix the JSON string:
 - Add the missing parentheses if the parser believes that the array or object should be closed
 - Quote strings or add missing single quotes
 - Adjust whitespaces and remove line breaks

{json_repair-0.30.0 → json_repair-0.30.2}/tests/test_json_repair.py

@@ -148,6 +148,8 @@ def test_object_edge_cases():
     assert repair_json("{'text': 'words{words in brackets}more words'}") == '{"text": "words{words in brackets}more words"}'
     assert repair_json('{text:words{words in brackets}}') == '{"text": "words{words in brackets}"}'
     assert repair_json('{text:words{words in brackets}m}') == '{"text": "words{words in brackets}m"}'
+    assert repair_json('{"key": "value, value2"```') == '{"key": "value, value2"}'
+    assert repair_json('{key:value,key2:value2}') == '{"key": "value", "key2": "value2"}'
 
 def test_number_edge_cases():
     assert repair_json(' - { "test_key": ["test_value", "test_value2"] }') == '{"test_key": ["test_value", "test_value2"]}'