json-repair 0.10.0 → 0.11.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6de36fcd3ab73ce63e1f9367d4e86dd7373dd7430b2dc87d451d75dd1f5fd685
-  data.tar.gz: 7221ab8c14253abf6cf4d40ff8fb8e6bd93c20504a8b5e6608c93a159420dc2f
+  metadata.gz: 288e3502829f51d11dbf2c3a9ab45f04dd44a1fa9ae5e00c1537f47c215aad96
+  data.tar.gz: 1ab99e5121ef3e73066157569bd85379a527cc1f44ecd7087d4f058133c4fcf0
 SHA512:
-  metadata.gz: 5083d8d8ada9a0b0a67beb8cb8ad1f1af53435e4e5c99632de5c68e19fd02d6dc85567b8c44cfc3068ae44118934c073d5a1acc22abf7f3d089b2add3899e083
-  data.tar.gz: 01d4eb164137dcb0e5b3b11703e2b1099efff6749f633466c3b37e1cccec07904e04006ddca1961d30702b5f133e35b88f8c37edce2bfa72ad61a6172dbbbada
+  metadata.gz: ffc4cd085d9a6aa5b45f3ee605a0fa043e20c68d63fdaab5c736acecbb17f160066d4412a76496300614545577d74d5ca4b232d4354adf625913753f8c0e8477
+  data.tar.gz: 190ea601a010b401fdf0c6aa52670a030ce81d39ebcd1af1f81c9fb2399baf6d074f3135b3b05814d1bcfcd24db53321a779b760c8eebe8d5a9ce2e029476754

data/.rubocop.yml

@@ -1,10 +1,6 @@
 AllCops:
   TargetRubyVersion: 3.0
 
-Metrics/BlockLength:
-  Exclude:
-    - spec/**/*
-
 Style/Documentation:
   Enabled: false
 
@@ -33,6 +29,8 @@ Metrics/BlockLength:
   Exclude:
     - lib/json/repairer.rb
     - spec/**/*
+    # restore RuboCop's default exclusion, lost when Exclude is overridden
+    - '*.gemspec'
 
 Metrics/BlockNesting:
   Exclude:

data/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changes
 
+### 2026-06-12 (0.11.1)
+
+* Fix a `TypeError` crash on input ending in a lone backslash inside a
+  string: `"abc\` now repairs to `"abc"` (likewise `"\` → `""`,
+  `["abc\` → `["abc"]`, `{"a": "b\` → `{"a":"b"}`), matching upstream
+  [jsonrepair](https://github.com/josdejong/jsonrepair) v3.14.0. This
+  was a porting bug — JS `charAt` past EOF returns `''` where Ruby
+  `String#[]` returns `nil`, so the invalid-escape repair in
+  `parse_string` crashed on `str << nil` instead of ending the string,
+  violating the contract that `JSONRepairError` is the only error
+  raised. Found by differential fuzzing during the 0.11.0 review.
+
+### 2026-06-12 (0.11.0)
+
+* Repair object string values with unescaped quotes around a colon
+  ("doubled colon"): `{"a": "b": "c"}` → `{"a":"b\": \"c"}` — the
+  value reads as `b": "c`, the unescaped-quotes interpretation. The
+  merge preserves the literal characters between the strings
+  (whitespace, original quote style) and repeats greedily
+  (`{"a": "b": "c": "d"}` → value `b": "c": "d`). Only the
+  string–colon–string shape is repaired: non-string shapes like
+  `{"a": "b": 1}` or `{"a": 1: 2}` still raise `JSONRepairError`
+  rather than silently dropping data (Python `json_repair` drops the
+  `: 1` there). Previously all of these raised "Object key expected".
+  Deliberate divergence from upstream
+  [jsonrepair](https://github.com/josdejong/jsonrepair) (raises as of
+  v3.14.0), matching
+  [Go json-repair](https://github.com/RealAlexandreAI/json-repair)
+  and Python
+  [`json_repair`](https://github.com/mangiucugna/json_repair) on the
+  canonical case.
+
 ### 2026-06-11 (0.10.0)
 
 * Repair Markdown list markers in front of top-level values:

data/CLAUDE.md

@@ -10,18 +10,25 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 - `bundle exec rspec spec/json_spec.rb:42` — run a single example by line number; nearly all behavioral specs live in `spec/json_spec.rb`.
 - `bundle exec rubocop` — lint. Project-specific exclusions in `.rubocop.yml` deliberately disable several `Metrics/*` cops for `lib/json/repairer.rb` and `lib/json/repair/string_utils.rb` because the parser is long by design — don't try to "fix" it by chopping methods up.
 - `bin/console` — IRB with the gem preloaded.
-- `bundle exec rake install` / `bundle exec rake release` — local install / publish to rubygems.org.
+- `bundle exec rake bench` — benchmark-ips regression baseline (`benchmark/run.rb`); run before/after perf-sensitive changes.
+- `bundle exec rake install` / `bundle exec rake release` — local install / publish to rubygems.org (release prompts for a rubygems MFA OTP).
 - Type checking: `Steepfile` checks `lib/` against `sig/`. `bundle exec steep check` (typecheck) and `bundle exec rbs validate` (sig syntax) both run in CI and as part of the default rake task. `steep` and `rbs` are dev dependencies in the `Gemfile`.
 
-Ruby `>= 3.0.0` is required (per gemspec). CI runs against Ruby 3.3.1.
+Ruby `>= 3.0.0` is required (per gemspec). CI runs against all currently maintained Ruby branches (3.3, 3.4, 4.0).
 
 ## Architecture
 
 This gem is a **Ruby port of the [josdejong/jsonrepair](https://github.com/josdejong/jsonrepair) TypeScript library**. The upstream version currently mirrored is tracked in `CHANGELOG.md` (presently v3.14.0). When syncing upstream changes, the goal is parity with the JS implementation, not idiomatic refactoring — keep method names, control flow, and repair heuristics aligned with the JS source so future syncs stay tractable.
 
+A few repair heuristics deliberately go **beyond** upstream (leading-dot numbers like `.5`, Markdown list markers like `- {...}`). Each such site carries a "Divergence from upstream" comment in the code and a CHANGELOG note — keep that convention when adding more, so future upstream syncs can tell ported behavior from local extensions.
+
 ### Entry point
 
-`JSON.repair(str)` in `lib/json/repair.rb` is a thin wrapper that constructs `JSON::Repairer.new(str).repair`. `JSON::JSONRepairError` is the only error raised for unrecoverable inputs.
+`JSON.repair(str)` in `lib/json/repair.rb` first tries stdlib `JSON.parse` (fast path; opt out with `skip_json_loads: true`), and falls back to `JSON::Repairer.new(str).repair` when that raises. Either way the result is re-serialized with `JSON.generate`, so **output is canonical** — whitespace collapsed, numbers normalized, duplicate keys last-write-wins — and both paths agree on it. A `REPAIR_REQUIRED_PATTERN` regex routes inputs containing comments or invalid escapes straight to the Repairer even though the bundled `json` gem would accept them. `return_objects: true` returns the parsed Ruby value instead of a string; `JSON.repair_file(path)` / `JSON.repair_io(io)` are convenience wrappers forwarding both options.
+
+`JSON::JSONRepairError` is the only error raised for unrecoverable inputs; it exposes the failure `#position`. If the Repairer ever emits a string stdlib cannot parse (a Repairer bug), the `JSON::ParserError` is wrapped in `JSONRepairError` rather than leaked.
+
+`exe/json-repair` is a CLI wrapper (`lib/json/repair/cli.rb`) reading stdin or a file, writing stdout, `--output FILE`, or `--overwrite`.
 
 ### The parser (`lib/json/repairer.rb`)
 
@@ -43,7 +50,7 @@ Two patterns recur and are worth knowing before editing:
 - **Backtracking via snapshots.** Methods like `parse_string` capture `i_before = @index` and `o_before = @output.length` before tentatively consuming input. If a later check (e.g. "the end quote turned out not to be a real end quote") fails, they restore both and re-invoke themselves with different flags (e.g. `stop_at_delimiter: true`, `stop_at_index: …`). Preserve this pattern when modifying string/number parsing.
 - **Repair-by-rewriting-tail.** Helpers like `insert_before_last_whitespace(@output, ',')` and `@output = strip_last_occurrence(@output, ',')` patch the already-emitted output to fix things like missing or trailing commas. These run *after* the malformed input has been partially emitted — they are the mechanism for "I now realize that earlier token needed a comma after it."
 
-`repair` (the public method) drives `parse_value` then handles top-level concerns: stripping Markdown fences (` ```json ... ``` `), converting newline-delimited JSON at the root into an array, dropping redundant trailing braces/brackets, and rejecting any non-whitespace trailing garbage.
+`repair` (the public method) drives `parse_value` then handles top-level concerns: stripping Markdown fences (` ```json ... ``` `), skipping Markdown list markers like `- ` / `* ` / `1. ` before the root value and each newline-delimited line (`markdown_list_marker_length` / `skip_markdown_list_marker` — top-level only, never inside nested structures), converting newline-delimited JSON at the root into an array, dropping redundant trailing braces/brackets, and rejecting any non-whitespace trailing garbage.
 
 ### Shared helpers (`lib/json/repair/string_utils.rb`)
 
@@ -62,6 +69,13 @@ RBS signatures mirror the public surface of `JSON.repair`, `JSON::Repairer`, and
 
 ### Test layout
 
-- `spec/json_spec.rb` — the substantive behavioral suite (700+ examples covering every repair heuristic). New behavior — and every sync from upstream — belongs here.
+- `spec/json_spec.rb` — the substantive behavioral suite (130+ examples, hundreds of assertions, covering every repair heuristic). New behavior — and every sync from upstream — belongs here.
+- `spec/json/repair/cli_spec.rb` — the `exe/json-repair` CLI (argument handling, IO errors, exit codes).
+- `spec/json/repair/string_utils_spec.rb` — direct unit coverage for a few `StringUtils` edge cases the behavioral suite can't reach.
 - `spec/json/repair_spec.rb` — sanity check on `JSON::Repair::VERSION` only.
-- `.rspec_status` is committed and tracks per-example pass/fail so `--only-failures` / `--next-failure` work across runs.
+- SimpleCov enforces 100% line and branch coverage on the full run, so a filtered `rspec -e ...` run "fails" at the coverage gate even when all selected examples pass — ignore that exit code during TDD.
+- `.rspec_status` is gitignored (local pass/fail tracking for `--only-failures` / `--next-failure`).
+
+## Local planning notes
+
+`docs/` (the `TODO.md` backlog plus design specs and implementation plans under `docs/superpowers/`) is gitignored local planning material — read it for context, update it as work completes, but never commit anything under `docs/`.

data/README.md

@@ -4,7 +4,7 @@ This is a Ruby gem designed to repair broken JSON strings. Inspired by and based
 
 ## Installation
 
-Add this gem to your application's Gemfield by executing:
+Add this gem to your application's Gemfile by executing:
 
 ```bash
 $ bundle add json-repair
@@ -37,6 +37,12 @@ Markdown markup in LLM output is handled too: fenced code blocks like `` ```json
 JSON.repair("- {\"a\": 1}\n- {\"b\": 2}")  # => '[{"a":1},{"b":2}]'
 ```
 
+Object values containing unescaped quotes around a colon are merged back into a single string value:
+
+```ruby
+JSON.repair('{"a": "b": "c"}')  # => '{"a":"b\": \"c"}' — the value reads as 'b": "c'
+```
+
 Pass `return_objects: true` to get the parsed Ruby value (Hash, Array, or scalar) instead of a string:
 
 ```ruby

data/lib/json/repair/string_utils.rb

@@ -127,6 +127,10 @@ module JSON
         whitespace_except_newline?(char) || special_whitespace?(char)
       end
 
+      def whitespace_or_special?(char)
+        whitespace?(char) || special_whitespace?(char)
+      end
+
       def quote?(char)
         double_quote_like?(char) || single_quote_like?(char)
       end

data/lib/json/repair/version.rb

@@ -2,6 +2,6 @@
 
 module JSON
   module Repair
-    VERSION = '0.10.0'
+    VERSION = '0.11.1'
   end
 end

data/lib/json/repairer.rb

@@ -68,7 +68,7 @@ module JSON
       end
 
       # repair redundant end quotes
-      while @json[@index] == CLOSING_BRACE || @json[@index] == CLOSING_BRACKET
+      while [CLOSING_BRACE, CLOSING_BRACKET].include?(@json[@index])
         @index += 1
         parse_whitespace_and_skip_comments
       end
@@ -236,7 +236,6 @@ module JSON
 
       initial = true
       while @index < @json.length && @json[@index] != CLOSING_BRACE
-        processed_comma = true
         if initial
           initial = false
         else
@@ -294,6 +293,10 @@ module JSON
           end
           # :nocov:
         end
+
+        # repair: an object string value with unescaped quotes around a
+        # colon, like {"a": "b": "c"}
+        repair_doubled_colon if processed_value
       end
 
       if @json[@index] == CLOSING_BRACE
@@ -307,6 +310,59 @@ module JSON
       true
     end
 
+    # Repair an object value with unescaped quotes around a colon, like
+    # {"a": "b": "c"}, by merging it all into one string value: 'b": "c'
+    # (the unescaped-quotes reading of the input). Greedy: keeps merging
+    # while another `: "..."` follows. Only the string-colon-string
+    # shape is repaired; anything else falls through to the regular
+    # error paths. Divergence from upstream (which raises "Object key
+    # expected" as of v3.14.0), matching the Go and Python json-repair
+    # libraries on the canonical case.
+    def repair_doubled_colon
+      loop do
+        colon = @index
+        # :nocov: kept for symmetry with the start_quote scan below; unreachable
+        # because @index never rests on whitespace here. On first entry,
+        # parse_value ends with parse_whitespace_and_skip_comments. On greedy
+        # re-entry, every parse_string exit leaves @index off-whitespace: the
+        # EOF path (nil is not whitespace), the stop_at_index path (a
+        # prev_non_whitespace_index position), and the end-quote path (ends in
+        # parse_concatenated_string, whose leading whitespace skip consumes
+        # newlines too). If a future parse_value/parse_string change breaks
+        # that, this scan becomes live and the :nocov: will hide it.
+        colon += 1 while whitespace_or_special?(@json[colon])
+        # :nocov:
+        return unless @json[colon] == COLON
+
+        # scan past special whitespace too (unlike prev_non_whitespace_index):
+        # parse_whitespace treats NBSP and friends as whitespace, so this
+        # repair should as well. The value's last character (at worst the
+        # object's opening brace) always stops the scan before index 0.
+        end_quote = colon - 1
+        end_quote -= 1 while whitespace_or_special?(@json[end_quote])
+        return unless quote?(@json[end_quote])
+
+        start_quote = colon + 1
+        start_quote += 1 while whitespace_or_special?(@json[start_quote])
+        return unless quote?(@json[start_quote])
+
+        # repair: replace the end quote already emitted (plus any copied
+        # trailing whitespace) with the literal input span from that end
+        # quote through the next start quote, escaped as string content
+        @output = strip_last_occurrence(@output, '"', strip_remaining_text: true)
+        @json[end_quote..start_quote].each_char do |char|
+          @output << (char == DOUBLE_QUOTE ? '\"' : CONTROL_CHARACTERS.fetch(char, char))
+        end
+
+        # let parse_string consume the rest of the merged string, then
+        # drop the start quote it emits (already emitted escaped above)
+        @index = start_quote
+        start = @output.length
+        parse_string
+        @output = remove_at_index(@output, start, 1)
+      end
+    end
+
     def skip_character(char)
       if @json[@index] == char
         @index += 1
@@ -474,7 +530,9 @@ module JSON
           return true
         elsif @json[@index] == BACKSLASH
           # handle escaped content like \n or ★
-          char = @json[@index + 1]
+          # nil at EOF: '' mirrors JS charAt, making the invalid-escape
+          # repair below a no-op that ends the string
+          char = @json[@index + 1] || ''
           escape_char = ESCAPE_CHARACTERS[char]
           if escape_char
             str << @json[@index, 2]

data/sig/json/repair/string_utils.rbs

@@ -141,6 +141,8 @@ module JSON
 
       def same_line_whitespace?: (::String? char) -> bool
 
+      def whitespace_or_special?: (::String? char) -> bool
+
       def quote?: (::String? char) -> bool
 
       def double_quote?: (::String? char) -> bool

data/sig/json/repairer.rbs

@@ -52,6 +52,10 @@ module JSON
     # Parse an object like '{"key": "value"}'
     def parse_object: () -> bool
 
+    # Repair an object value with unescaped quotes around a colon,
+    # like {"a": "b": "c"}, by merging it into one string value.
+    def repair_doubled_colon: () -> void
+
     def skip_character: (::String char) -> bool
 
     # Skip ellipsis like "[1,2,3,...]" or "[1,2,3,...,9]" or "[...,7,8,9]"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: json-repair
 version: !ruby/object:Gem::Version
-  version: 0.10.0
+  version: 0.11.1
 platform: ruby
 authors:
 - Aleksandr Zykov
@@ -9,7 +9,11 @@ bindir: exe
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies: []
-description: This is a simple gem that repairs broken JSON strings.
+description: 'Repairs broken JSON: missing quotes and commas, unclosed brackets, trailing
+  commas, unquoted keys, single quotes, comments, Python constants, NDJSON, Markdown
+  code fences and list markers in LLM output, truncated documents, and more. A Ruby
+  port of the jsonrepair JavaScript library — useful whenever JSON from LLMs, APIs,
+  or logs does not strictly follow the standard.'
 email:
 - alexandrz@gmail.com
 executables:
@@ -44,6 +48,8 @@ metadata:
   homepage_uri: https://github.com/sashazykov/json-repair-rb
   source_code_uri: https://github.com/sashazykov/json-repair-rb
   changelog_uri: https://github.com/sashazykov/json-repair-rb/blob/main/CHANGELOG.md
+  documentation_uri: https://rubydoc.info/gems/json-repair
+  bug_tracker_uri: https://github.com/sashazykov/json-repair-rb/issues
 rdoc_options: []
 require_paths:
 - lib
@@ -60,5 +66,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 3.6.9
 specification_version: 4
-summary: Repairs broken JSON strings.
+summary: Repair invalid or malformed JSON documents, including LLM output.
 test_files: []