json-repair 0.3.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: db2b6fb7849a2e75329405c1f85fa7de836b0fa2f079623032571f42d359514d
-  data.tar.gz: 1c845714c4c443bad3c9277a2ceae6cef8ff346125f52f89473aaa50b9ff2132
+  metadata.gz: 83ede681caac8ebbc294902937c8af11ebc90fe066beed41e15013782fbd0b96
+  data.tar.gz: da309d72a1c564f05dd405c02115985572c38c8b2c28aad85f778b94cc2a68e9
 SHA512:
-  metadata.gz: 53929154af31033e2f380ed89979430f4339c97c94c088b6f85da27ac251d658b98840e44085c4ba9b4972bab75c1bb0f8ad750beddd4bb79e439efb135e0386
-  data.tar.gz: b4b5150aee81c518eaee8847bb2f5d8d8131a15719bb93badce465a2d447ddc361888155b2d33125fdd69d2568424c08440772c61a7f7f5b35922a4d1270adf8
+  metadata.gz: da63df705106d20701700dd55712ed06676cdb4e6615f36d915cae12f3bdf7ac085ea919a553c77e8e0876663e5ad7d0067b46b00fddd3a7cca837eb38e54fc8
+  data.tar.gz: d4471c07b5a3a39505ec890b8afc359ca12fa3042c953143675af2d88bd8a4f56fe6225f00e0d4665eaf66465862f2b273929a2bb7fd54a81f79f1abf39ecfa1

data/CLAUDE.md

@@ -0,0 +1,67 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Commands
+
+- `bin/setup` — install dependencies via Bundler.
+- `bundle exec rake` — default task; runs both RSpec and RuboCop.
+- `bundle exec rspec` — run the test suite.
+- `bundle exec rspec spec/json_spec.rb:42` — run a single example by line number; nearly all behavioral specs live in `spec/json_spec.rb`.
+- `bundle exec rubocop` — lint. Project-specific exclusions in `.rubocop.yml` deliberately disable several `Metrics/*` cops for `lib/json/repairer.rb` and `lib/json/repair/string_utils.rb` because the parser is long by design — don't try to "fix" it by chopping methods up.
+- `bin/console` — IRB with the gem preloaded.
+- `bundle exec rake install` / `bundle exec rake release` — local install / publish to rubygems.org.
+- Type checking: `Steepfile` checks `lib/` against `sig/`. Run `bundle exec steep check` if/when steep is installed (not in `Gemfile` by default).
+
+Ruby `>= 3.0.0` is required (per gemspec). CI runs against Ruby 3.3.1.
+
+## Architecture
+
+This gem is a **Ruby port of the [josdejong/jsonrepair](https://github.com/josdejong/jsonrepair) TypeScript library**. The upstream version currently mirrored is tracked in `CHANGELOG.md` (presently v3.14.0). When syncing upstream changes, the goal is parity with the JS implementation, not idiomatic refactoring — keep method names, control flow, and repair heuristics aligned with the JS source so future syncs stay tractable.
+
+### Entry point
+
+`JSON.repair(str)` in `lib/json/repair.rb` is a thin wrapper that constructs `JSON::Repairer.new(str).repair`. `JSON::JSONRepairError` is the only error raised for unrecoverable inputs.
+
+### The parser (`lib/json/repairer.rb`)
+
+A single-pass, hand-written recursive-descent parser. State is three instance variables:
+
+- `@json` — the input string (read-only after init).
+- `@index` — the current cursor into `@json`.
+- `@output` — a mutable `+''` buffer that accumulates the *repaired* JSON. The parser writes directly to `@output` as it walks; it does not build an AST.
+
+Each `parse_*` method (`parse_value`, `parse_object`, `parse_array`, `parse_string`, `parse_number`, `parse_keywords`, `parse_unquoted_string`, `parse_regex`, `parse_comment`, `parse_markdown_code_block`, …) follows a contract:
+
+1. Returns truthy if it consumed something, falsy otherwise.
+2. On success, advances `@index` past the consumed input and appends the *valid* JSON form to `@output`.
+3. On a recoverable mismatch (missing quote, missing comma, trailing comma, wrong quote style, etc.) it performs an in-place repair on `@output` using helpers like `insert_before_last_whitespace`, `strip_last_occurrence`, or `remove_at_index`.
+4. On an unrecoverable error it calls one of the `throw_*` helpers, which raise `JSON::JSONRepairError`.
+
+Two patterns recur and are worth knowing before editing:
+
+- **Backtracking via snapshots.** Methods like `parse_string` capture `i_before = @index` and `o_before = @output.length` before tentatively consuming input. If a later check (e.g. "the end quote turned out not to be a real end quote") fails, they restore both and re-invoke themselves with different flags (e.g. `stop_at_delimiter: true`, `stop_at_index: …`). Preserve this pattern when modifying string/number parsing.
+- **Repair-by-rewriting-tail.** Helpers like `insert_before_last_whitespace(@output, ',')` and `@output = strip_last_occurrence(@output, ',')` patch the already-emitted output to fix things like missing or trailing commas. These run *after* the malformed input has been partially emitted — they are the mechanism for "I now realize that earlier token needed a comma after it."
+
+`repair` (the public method) drives `parse_value` then handles top-level concerns: stripping Markdown fences (` ```json ... ``` `), converting newline-delimited JSON at the root into an array, dropping redundant trailing braces/brackets, and rejecting any non-whitespace trailing garbage.
+
+### Shared helpers (`lib/json/repair/string_utils.rb`)
+
+`JSON::Repair::StringUtils` is a mixin included into `Repairer`. It holds:
+
+- Character constants (`OPENING_BRACE`, `BACKSLASH`, smart-quote variants, special whitespace code points, etc.) used in lieu of magic literals.
+- Character-class predicates (`digit?`, `hex?`, `quote?`, `single_quote_like?`, `delimiter?`, `whitespace?`, `special_whitespace?`, `start_of_value?`, …).
+- The keyword machinery — `parse_keywords` / `parse_keyword` — which converts Python `True`/`False`/`None` and Ruby `nil` into their JSON equivalents in addition to recognizing `true`/`false`/`null`.
+- Output-buffer surgery helpers: `strip_last_occurrence`, `insert_before_last_whitespace`, `remove_at_index`, `ends_with_comma_or_newline?`.
+
+Because the mixin reads `@json`, `@index`, and `@output` directly (notably inside `parse_keyword`), it is **not standalone** — it is coupled to `Repairer`'s state and should only be mixed into classes that own those ivars.
+
+### Type signatures (`sig/`)
+
+RBS signatures mirror the public surface of `JSON.repair`, `JSON::Repairer`, and `JSON::Repair::StringUtils`. Update them in lockstep with `lib/` changes; the `Steepfile` will surface drift.
+
+### Test layout
+
+- `spec/json_spec.rb` — the substantive behavioral suite (700+ examples covering every repair heuristic). New behavior — and every sync from upstream — belongs here.
+- `spec/json/repair_spec.rb` — sanity check on `JSON::Repair::VERSION` only.
+- `.rspec_status` is committed and tracks per-example pass/fail so `--only-failures` / `--next-failure` work across runs.

data/README.md

@@ -31,6 +31,21 @@ puts repaired_json  # Outputs: {"name": "Alice", "age": 25}
 
 The `repair` method takes a string containing JSON data and returns a corrected version of this string, ensuring it is valid JSON.
 
+## Command line
+
+The gem ships a `json-repair` executable. It reads from stdin or a file and writes to stdout, `--output FILE`, or back over the input file with `--overwrite`.
+
+```bash
+$ echo '{a:1,}' | json-repair
+{"a":1}
+
+$ json-repair broken.json
+$ json-repair broken.json -o fixed.json
+$ json-repair broken.json --overwrite
+```
+
+Run `json-repair --help` for the full list of options.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/exe/json-repair

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'json/repair/cli'
+
+exit JSON::Repair::CLI.call(ARGV)

data/lib/json/repair/cli.rb

@@ -0,0 +1,133 @@
+# frozen_string_literal: true
+
+require 'fileutils'
+require 'optparse'
+require 'tempfile'
+require_relative '../repair'
+
+module JSON
+  module Repair
+    class CLI
+      def self.call(argv, stdin: $stdin, stdout: $stdout, stderr: $stderr)
+        new(stdin: stdin, stdout: stdout, stderr: stderr).call(argv)
+      end
+
+      def initialize(stdin: $stdin, stdout: $stdout, stderr: $stderr)
+        @stdin = stdin
+        @stdout = stdout
+        @stderr = stderr
+      end
+
+      # Reset per-invocation state so a single instance can be safely reused
+      # (e.g. `cli = CLI.new; cli.call(['-v']); cli.call(['x'])`).
+      def call(argv)
+        @output_path = @halt = nil
+        @overwrite = false
+        run(argv)
+      rescue OptionParser::ParseError, JSON::JSONRepairError, SystemCallError, IOError,
+             SystemStackError => e
+        @stderr.puts "json-repair: #{e.message}"
+        1
+      end
+
+      private
+
+      def run(argv)
+        positional = catch(:halt) { parser.parse(argv) }
+        return @halt if @halt
+
+        input_path = positional.first
+        return 1 unless validate(positional, input_path)
+
+        repaired = JSON.repair(read_input(input_path))
+        write_output(repaired, input_path)
+        0
+      end
+
+      def validate(positional, input_path)
+        error = validation_error(positional, input_path)
+        return true unless error
+
+        @stderr.puts "json-repair: #{error}"
+        false
+      end
+
+      def validation_error(positional, input_path)
+        return "unexpected argument: #{positional[1]}" if positional.length > 1
+        return '--overwrite requires a filename' if @overwrite && input_path.nil?
+        return '--overwrite and --output are mutually exclusive' if @overwrite && @output_path
+
+        nil
+      end
+
+      def read_input(input_path)
+        raw = input_path ? File.read(input_path) : @stdin.read
+        raw.force_encoding(Encoding::UTF_8)
+        raise JSON::JSONRepairError, 'input is not valid UTF-8' unless raw.valid_encoding?
+
+        raw
+      end
+
+      def write_output(repaired, input_path)
+        if @overwrite
+          replace_in_place(input_path, repaired)
+        elsif @output_path
+          File.write(@output_path, repaired)
+        else
+          @stdout.write(repaired)
+          @stdout.write("\n") unless repaired.end_with?("\n")
+        end
+      end
+
+      # Write to a uniquely-named tempfile alongside the input, then move it
+      # over the original. Tempfile.create uses O_EXCL + a random suffix, so
+      # the temp path is safe against symlink / clobber races; FileUtils.mv
+      # with force: true handles cross-device renames and Windows, where
+      # File.rename cannot overwrite an existing destination. The original
+      # file's mode is preserved (Tempfile defaults to 0600).
+      #
+      # Symlinks are followed via File.realpath so the underlying file is
+      # rewritten in place and the link is left pointing at it; otherwise
+      # the rename would replace the link itself with a regular file.
+      def replace_in_place(input_path, repaired)
+        real_path = File.realpath(input_path)
+        original_mode = File.stat(real_path).mode
+        Tempfile.create(['json-repair', '.tmp'], File.dirname(real_path)) do |tmp|
+          tmp.write(repaired)
+          tmp.close
+          File.chmod(original_mode, tmp.path)
+          FileUtils.mv(tmp.path, real_path, force: true)
+        end
+      end
+
+      def parser
+        OptionParser.new do |opts|
+          opts.banner = 'Usage: json-repair [filename] [options]'
+          opts.separator ''
+          opts.separator 'Repair a broken JSON document. Reads stdin when no filename is given.'
+          opts.separator ''
+          define_options(opts)
+        end
+      end
+
+      OVERWRITE_DESC = 'Replace the input file in place (requires filename; conflicts with --output)'
+      private_constant :OVERWRITE_DESC
+
+      def define_options(opts)
+        opts.on('-o', '--output FILE', 'Write repaired JSON to FILE') { |f| @output_path = f }
+        opts.on('--overwrite', OVERWRITE_DESC) { @overwrite = true }
+        opts.on('-v', '--version', 'Print version and exit') { halt_with(JSON::Repair::VERSION) }
+        opts.on('-h', '--help', 'Print this help and exit') { halt_with(opts.help) }
+      end
+
+      # Print to stdout and short-circuit `parser.parse` so trailing args
+      # after --version/--help do not raise OptionParser::ParseError and
+      # flip the exit code (the option text promises "...and exit").
+      def halt_with(message)
+        @stdout.puts message
+        @halt = 0
+        throw :halt
+      end
+    end
+  end
+end

data/lib/json/repair/version.rb

@@ -2,6 +2,6 @@
 
 module JSON
   module Repair
-    VERSION = '0.3.0'
+    VERSION = '0.5.0'
   end
 end

data/lib/json/repair.rb

@@ -4,7 +4,14 @@ require_relative 'repair/version'
 require_relative 'repairer'
 
 module JSON
-  class JSONRepairError < StandardError; end
+  class JSONRepairError < StandardError
+    attr_reader :position
+
+    def initialize(message = nil, position = nil)
+      super(message && position ? "#{message} at index #{position}" : message)
+      @position = position
+    end
+  end
 
   def self.repair(json)
     Repairer.new(json).repair

data/lib/json/repairer.rb

@@ -739,28 +739,28 @@ module JSON
     end
 
     def throw_invalid_character(char)
-      raise JSONRepairError, "Invalid character #{char.inspect} at index #{@index}"
+      raise JSONRepairError.new("Invalid character #{char.inspect}", @index)
     end
 
     def throw_unexpected_character
-      raise JSONRepairError, "Unexpected character #{@json[@index].inspect} at index #{@index}"
+      raise JSONRepairError.new("Unexpected character #{@json[@index].inspect}", @index)
     end
 
     def throw_unexpected_end
-      raise JSONRepairError, 'Unexpected end of json string'
+      raise JSONRepairError.new('Unexpected end of json string', @index)
     end
 
     def throw_object_key_expected
-      raise JSONRepairError, 'Object key expected'
+      raise JSONRepairError.new('Object key expected', @index)
     end
 
     def throw_colon_expected
-      raise JSONRepairError, 'Colon expected'
+      raise JSONRepairError.new('Colon expected', @index)
     end
 
     def throw_invalid_unicode_character
       chars = @json[@index, 6]
-      raise JSONRepairError, "Invalid unicode character #{chars.inspect} at index #{@index}"
+      raise JSONRepairError.new("Invalid unicode character #{chars.inspect}", @index)
     end
   end
 end

data/sig/json/repair/cli.rbs

@@ -0,0 +1,16 @@
+module JSON
+  module Repair
+    class CLI
+      # `::IO | ::StringIO` because `::StringIO` is not an `::IO` subclass; the
+      # specs inject `::StringIO` instances and any other duck-typed stream
+      # that implements `#read` / `#write` / `#puts` would work too.
+      type stream = ::IO | ::StringIO
+
+      def self.call: (::Array[::String] argv, ?stdin: stream, ?stdout: stream, ?stderr: stream) -> ::Integer
+
+      def initialize: (?stdin: stream, ?stdout: stream, ?stderr: stream) -> void
+
+      def call: (::Array[::String] argv) -> ::Integer
+    end
+  end
+end

data/sig/json/repair.rbs

@@ -1,5 +1,8 @@
 module JSON
   class JSONRepairError < StandardError
+    attr_reader position: ::Integer?
+
+    def initialize: (?::String? message, ?::Integer? position) -> void
   end
 
   module Repair

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: json-repair
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.5.0
 platform: ruby
 authors:
 - Aleksandr Zykov
@@ -12,23 +12,28 @@ dependencies: []
 description: This is a simple gem that repairs broken JSON strings.
 email:
 - alexandrz@gmail.com
-executables: []
+executables:
+- json-repair
 extensions: []
 extra_rdoc_files: []
 files:
 - ".rspec"
 - ".rubocop.yml"
 - CHANGELOG.md
+- CLAUDE.md
 - CODE_OF_CONDUCT.md
 - LICENSE.txt
 - README.md
 - Rakefile
 - Steepfile
+- exe/json-repair
 - lib/json/repair.rb
+- lib/json/repair/cli.rb
 - lib/json/repair/string_utils.rb
 - lib/json/repair/version.rb
 - lib/json/repairer.rb
 - sig/json/repair.rbs
+- sig/json/repair/cli.rbs
 - sig/json/repair/string_utils.rbs
 - sig/json/repairer.rbs
 homepage: https://github.com/sashazykov/json-repair-rb