menkar 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 19b9fc9d24c830c4eec2a522a1bec3b0b41b5a6756a52d6e2dbd945456e929f1
+  data.tar.gz: 285609052ef5eb84c5dee828fa4d21097279f433417bc0ca47ba2bc4e2368762
+SHA512:
+  metadata.gz: 7502f91d1b5f17218797d38514a4e49f2615b7d74baad30823e75fb41d60d3cd8ffe790435c9d474fd2f789d71e0ad5952853b264a5da42846ff042c7af80e95
+  data.tar.gz: 913bd5a9b617d91c712a71f714041e3dca5330e07372ca6199b7acca6ada96973e3514ea33ef4cfe35a1997c1e7c3fe27fc4f2f42ef02bad530bf7af2c967ec6

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 0.1.0 - 2026-09-16
+
+- Initial release

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Yudai Takada
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,75 @@
+# Menkar
+
+Menkar is a pure Ruby library for identifying text bytes before they enter a
+UTF-8 editor core. It detects BOMs, binary data, encodings, newline forms, and
+indentation, then performs strict and reversible transcoding.
+
+## Installation
+
+```ruby
+gem "menkar"
+```
+
+## Usage
+
+```ruby
+require "menkar"
+
+bytes = File.binread("legacy.txt")
+detection = Menkar.detect(bytes, hint: "Windows-31J")
+raise "binary file" if detection.binary
+
+text = Menkar.decode(bytes, detection) # valid UTF-8; original newlines remain
+updated = text.sub("旧", "新")
+
+unless Menkar.roundtrip?(updated, detection)
+  warn "The edited text cannot be saved in #{detection.encoding.name}"
+end
+File.binwrite("legacy.txt", Menkar.encode(updated, detection))
+```
+
+`encode` restores the detected BOM and otherwise preserves the string's
+newlines byte for byte. Newline conversion is explicit:
+
+```ruby
+unix_text, original = Menkar.normalize_newlines(text, to: :lf)
+# original is :lf, :crlf, :cr, :mixed, or :none
+```
+
+`detect_file` reads at most `sample + 1` bytes; `detect` examines only `sample`
+bytes. The default is 64 KiB and the accepted maximum is 16 MiB. Supported
+legacy encodings are Windows-31J (Shift_JIS), EUC-JP, ISO-2022-JP,
+Windows-1252, GBK, Big5, and EUC-KR. A hint is an encoding name, `Encoding`, or
+`{encoding: ...}` and adjusts a statistical score; a BOM always wins.
+
+Detection is evidence, not a save policy. Callers decide whether to reload or
+convert, warn for mixed newlines, and require confirmation when `roundtrip?`
+is false. Decoding and encoding never replace invalid or undefined characters.
+
+## Corpus and accuracy
+
+The test corpus contains short excerpts from public-domain works: the Iroha,
+Ogura Hyakunin Isshu, *I Am a Cat*, *The Pillow Book*, *The Tale of the Heike*,
+*The Narrow Road to the Deep North*, and Basho's haiku (Japanese); the
+*Analects* and *Tao Te Ching* (Chinese); *Hunminjeongeum* and traditional Korean
+proverbs; and Shakespeare's *Hamlet*. `script/build_corpus` reproducibly
+transcodes those excerpts into the fixture encodings. The source works are in
+the public domain; the generated fixtures are distributed under the
+repository's MIT license.
+
+The suite requires at least 98% correct classification for Japanese fixtures
+and 90% for the remaining legacy encodings.
+
+## Development
+
+```sh
+bundle install
+bundle exec rake test
+bundle exec rbs -I sig validate
+BUDGET=1 bundle exec rake bench
+gem build --strict menkar.gemspec
+```
+
+## License
+
+Menkar is available under the MIT License.

data/docs/adr/000-template.md

@@ -0,0 +1,17 @@
+# ADR NNN: Implementation decision title
+
+- Status: Proposed
+- Date: YYYY-MM-DD
+
+## Context
+
+Describe the concrete implementation question and its compatibility, data,
+runtime, or component constraints.
+
+## Decision
+
+Describe the durable boundary or architecture choice.
+
+## Consequences
+
+Describe the important positive and negative trade-offs and when to revisit it.

data/docs/adr/001-strict-bounded-detection.md

@@ -0,0 +1,25 @@
+# ADR 001: Keep detection bounded and transcoding strict
+
+- Status: Accepted
+- Date: 2026-09-15
+
+## Context
+
+Editors need a useful encoding guess for large files, but a guess must not
+silently corrupt source bytes. File reading, overwrite confirmation, and save
+policy belong to the editor rather than this encoding library.
+
+## Decision
+
+Detection examines a bounded prefix and reports confidence plus structural
+facts. BOM evidence is decisive; otherwise standard Ruby encoders validate
+candidates before small language-specific scores rank them. Decode and encode
+use strict conversion, preserve detected BOMs and newlines, and expose an exact
+round-trip check. The caller owns every overwrite or conversion decision.
+
+## Consequences
+
+Very short or statistically unusual input can remain ambiguous, while hints
+can improve its ranking. Unsupported or damaged text fails explicitly instead
+of inserting replacement characters. Adding an encoding requires corpus
+coverage rather than changing a caller's save path.

data/docs/adr/README.md

@@ -0,0 +1,3 @@
+# Architecture decision records
+
+These records document Menkar's durable boundaries.

data/lib/menkar/detector.rb

@@ -0,0 +1,154 @@
+# frozen_string_literal: true
+
+module Menkar
+  DEFAULT_SAMPLE = 64 * 1024
+  MAX_SAMPLE = 16 * 1024 * 1024
+
+  BOMS = [
+    ["\xFF\xFE\x00\x00".b.freeze, Encoding::UTF_32LE],
+    ["\x00\x00\xFE\xFF".b.freeze, Encoding::UTF_32BE],
+    ["\xEF\xBB\xBF".b.freeze, Encoding::UTF_8],
+    ["\xFF\xFE".b.freeze, Encoding::UTF_16LE],
+    ["\xFE\xFF".b.freeze, Encoding::UTF_16BE]
+  ].freeze
+
+  SUPPORTED_ENCODINGS = ([Encoding::UTF_8, Encoding::UTF_16LE, Encoding::UTF_16BE,
+    Encoding::UTF_32LE, Encoding::UTF_32BE] + EncodingScorer.encodings).freeze
+
+  module_function
+
+  def detect(bytes, hint: nil, sample: DEFAULT_SAMPLE)
+    validate_bytes!(bytes)
+    limit = validate_sample!(sample)
+    hint_encoding = normalize_hint(hint)
+    chunk = (bytes.byteslice(0, limit) || "").b
+    truncated = bytes.bytesize > chunk.bytesize
+    bom, bom_encoding = BOMS.find { |mark, _encoding| chunk.start_with?(mark) }
+
+    if bom_encoding
+      text = EncodingScorer.decode(chunk.byteslice(bom.bytesize..) || "".b, bom_encoding, truncated: truncated)
+      return detection(bom_encoding, 1.0, bom, text, binary: false)
+    end
+
+    if chunk.include?("\0")
+      encoding, text = nul_encoding(chunk, truncated: truncated)
+      return detection(encoding, 0.95, "".b.freeze, text, binary: false) if encoding
+
+      return detection(nil, 1.0, "".b.freeze, nil, binary: true)
+    end
+
+    return detection(nil, 1.0, "".b.freeze, nil, binary: true) if binary_sample?(chunk)
+
+    if chunk.match?(/\e\$(?:@|B)|\e\([BJ]/n)
+      text = EncodingScorer.decode(chunk, Encoding::ISO_2022_JP, truncated: truncated)
+      return detection(Encoding::ISO_2022_JP, 0.99, "".b.freeze, text, binary: false) if text
+    end
+
+    utf8 = EncodingScorer.decode(chunk, Encoding::UTF_8, truncated: truncated)
+    if utf8
+      if utf8.ascii_only? && hint_encoding && hint_encoding != Encoding::UTF_8
+        return detection(hint_encoding, 0.6, "".b.freeze, utf8, binary: false)
+      end
+      return detection(Encoding::UTF_8, utf8.ascii_only? ? 0.5 : 0.9, "".b.freeze, utf8, binary: false)
+    end
+
+    encoding, confidence, text = EncodingScorer.legacy(chunk, hint_encoding, truncated: truncated)
+    return detection(nil, 1.0, "".b.freeze, nil, binary: true) unless encoding
+
+    detection(encoding, confidence, "".b.freeze, text, binary: false)
+  end
+
+  def detect_file(path, sample: DEFAULT_SAMPLE)
+    limit = validate_sample!(sample)
+    unless path.is_a?(String) && path.valid_encoding? && !path.include?("\0")
+      raise Error, "path must be a valid String without NUL bytes"
+    end
+
+    bytes = File.open(path, "rb") { |file| file.read(limit + 1) || "".b }
+    detect(bytes, sample: limit)
+  rescue SystemCallError => error
+    raise Error, "cannot read #{path}: #{error.message}"
+  end
+
+  def binary?(bytes)
+    detect(bytes).binary
+  end
+
+  def detection(encoding, confidence, bom, text, binary:)
+    Detection.new(encoding, confidence, bom, text && newline_kind(text), text && indent(text), binary)
+  end
+  private_class_method :detection
+
+  def indent(text)
+    prefixes = text.each_line.filter_map { |line| line[/\A[ \t]+(?=\S)/] }
+    return nil if prefixes.empty?
+
+    tabs = prefixes.count { |prefix| prefix.start_with?("\t") }
+    spaces = prefixes.filter_map { |prefix| prefix[/\A +/]&.length }
+    return Indent.new(:tab, nil) if tabs > spaces.length
+    return nil if spaces.empty?
+
+    widths = [2, 3, 4, 8]
+    width = widths.max_by { |candidate| [spaces.count { |value| (value % candidate).zero? }.fdiv(spaces.length), candidate] }
+    width = 1 if spaces.count { |value| (value % width).zero? }.fdiv(spaces.length) < 0.6
+    Indent.new(:space, width)
+  end
+  private_class_method :indent
+
+  def nul_encoding(bytes, truncated:)
+    options = []
+    if bytes.bytesize >= 4
+      columns = 4.times.map { |offset| bytes.bytes.each_with_index.count { |byte, index| (index % 4) == offset && byte.zero? } }
+      groups = bytes.bytesize / 4
+      options << Encoding::UTF_32LE if columns[1..].all? { |count| count.fdiv(groups) > 0.6 }
+      options << Encoding::UTF_32BE if columns[0, 3].all? { |count| count.fdiv(groups) > 0.6 }
+    end
+    if bytes.bytesize >= 2
+      pairs = bytes.bytesize / 2
+      even = bytes.bytes.each_with_index.count { |byte, index| index.even? && byte.zero? }.fdiv(pairs)
+      odd = bytes.bytes.each_with_index.count { |byte, index| index.odd? && byte.zero? }.fdiv(pairs)
+      options << Encoding::UTF_16LE if odd > 0.3 && even < 0.2
+      options << Encoding::UTF_16BE if even > 0.3 && odd < 0.2
+    end
+
+    options.uniq.each do |encoding|
+      text = EncodingScorer.decode(bytes, encoding, truncated: truncated)
+      return [encoding, text] if text && EncodingScorer.textual?(text)
+    end
+    [nil, nil]
+  end
+  private_class_method :nul_encoding
+
+  def binary_sample?(bytes)
+    return false if bytes.empty?
+
+    controls = bytes.each_byte.count { |byte| byte < 32 && ![9, 10, 12, 13, 27].include?(byte) }
+    controls.fdiv(bytes.bytesize) > 0.3
+  end
+  private_class_method :binary_sample?
+
+  def normalize_hint(hint)
+    return nil if hint.nil?
+    hint = hint[:encoding] || hint["encoding"] if hint.is_a?(Hash)
+    encoding = hint.is_a?(Encoding) ? hint : Encoding.find(hint.to_s)
+    encoding = Encoding::Windows_31J if encoding == Encoding::Shift_JIS
+    raise Error, "unsupported encoding hint: #{hint}" unless SUPPORTED_ENCODINGS.include?(encoding)
+
+    encoding
+  rescue ArgumentError
+    raise Error, "unknown encoding hint: #{hint}"
+  end
+  private_class_method :normalize_hint
+
+  def validate_bytes!(bytes)
+    raise Error, "bytes must be a String" unless bytes.is_a?(String)
+  end
+  private_class_method :validate_bytes!
+
+  def validate_sample!(sample)
+    raise Error, "sample must be an Integer between 4 and #{MAX_SAMPLE}" unless sample.is_a?(Integer) && sample.between?(4, MAX_SAMPLE)
+
+    sample
+  end
+  private_class_method :validate_sample!
+end

data/lib/menkar/encoding_scorer.rb

@@ -0,0 +1,91 @@
+# frozen_string_literal: true
+
+module Menkar
+  module EncodingScorer
+    CANDIDATES = [
+      [Encoding::Windows_31J, :japanese],
+      [Encoding::EUC_JP, :japanese],
+      [Encoding::ISO_2022_JP, :japanese],
+      [Encoding::GBK, :simplified_chinese],
+      [Encoding::Big5, :traditional_chinese],
+      [Encoding::EUC_KR, :korean],
+      [Encoding::Windows_1252, :western]
+    ].freeze
+
+    COMMON = {
+      japanese: "のにをはがとでてし日本人一日年大中本時行見言生子上来国私",
+      simplified_chinese: "的一是不了在人有我他这中大来上国个们为时会后发里",
+      traditional_chinese: "的一是不了在人有我他這中大來上國個們為時會後發裡",
+      korean: "이다는을를에의가이은한하로있것수나그되사",
+      western: "etaoinshrdlucmETAOINSHRDLUCM"
+    }.transform_values(&:freeze).freeze
+
+    module_function
+
+    def encodings = CANDIDATES.map(&:first)
+
+    def legacy(bytes, hint, truncated:)
+      scored = []
+      CANDIDATES.each do |encoding, language|
+        text = decode(bytes, encoding, truncated: truncated)
+        next unless text && textual?(text)
+
+        score = language_score(text, language)
+        score += 0.35 if encoding == hint
+        return [encoding, confidence(score), text] if language == :japanese && score >= 1.8
+
+        scored << [score, encoding, text]
+      end
+      return [nil, nil, nil] if scored.empty?
+
+      score, encoding, text = scored.max_by(&:first)
+      [encoding, confidence(score), text]
+    end
+
+    def decode(bytes, encoding, truncated:)
+      cuts = truncated ? 0..[4, bytes.bytesize].min : 0..0
+      cuts.each do |cut|
+        source = bytes.byteslice(0, bytes.bytesize - cut).dup.force_encoding(encoding)
+        next unless source.valid_encoding?
+
+        return source.encode(Encoding::UTF_8)
+      rescue EncodingError
+        next
+      end
+      nil
+    end
+
+    def textual?(text)
+      return true if text.empty?
+      return false if text.include?("\0")
+
+      text.scan(/[\x00-\x08\x0B\x0E-\x1F]/).length.fdiv(text.length) < 0.05
+    end
+
+    def confidence(score)
+      [[0.5 + (score * 0.18), 0.99].min, 0.5].max
+    end
+    private_class_method :confidence
+
+    def language_score(text, language)
+      total = text.count(language == :western ? "^ \t\r\n\f\v" : "^\x00-\x7F")
+      return 0.0 if total.zero?
+
+      ratio = ->(characters) { text.count(characters).fdiv(total) }
+      common = text.count(COMMON.fetch(language)).fdiv(total)
+
+      case language
+      when :japanese
+        0.2 + ratio.call("\u3040-\u30FF") * 3.1 + ratio.call("\uFF61-\uFF9F") * 0.6 +
+          ratio.call("\u3400-\u4DBF\u4E00-\u9FFF\uF900-\uFAFF\u{20000}-\u{2FA1F}") * 0.35 + common * 0.8
+      when :simplified_chinese, :traditional_chinese
+        0.2 + ratio.call("\u3400-\u4DBF\u4E00-\u9FFF\uF900-\uFAFF\u{20000}-\u{2FA1F}") * 1.2 + common * 1.1
+      when :korean
+        0.2 + ratio.call("\u1100-\u11FF\u3130-\u318F\uA960-\uA97F\uAC00-\uD7AF\uD7B0-\uD7FF") * 1.5 + common
+      else
+        0.2 + ratio.call("A-Za-z\u00C0-\u024F\u1E00-\u1EFF") * 1.2 + common * 0.6
+      end
+    end
+    private_class_method :language_score
+  end
+end

data/lib/menkar/transcoder.rb

@@ -0,0 +1,74 @@
+# frozen_string_literal: true
+
+module Menkar
+  NEWLINES = {lf: "\n", crlf: "\r\n", cr: "\r"}.freeze
+
+  module_function
+
+  def decode(bytes, detection)
+    validate_detection!(detection)
+    raise Error, "cannot decode binary input" if detection.binary
+    raise Error, "bytes must be a String" unless bytes.is_a?(String)
+
+    source = bytes.b
+    unless detection.bom.empty?
+      raise Error, "input does not start with the detected BOM" unless source.start_with?(detection.bom)
+
+      source = source.byteslice(detection.bom.bytesize..) || "".b
+    end
+    source.force_encoding(detection.encoding)
+    raise Error, "invalid #{detection.encoding.name} input" unless source.valid_encoding?
+
+    source.encode(Encoding::UTF_8)
+  rescue EncodingError => error
+    raise Error, "cannot decode #{detection.encoding.name}: #{error.message}"
+  end
+
+  def encode(string, detection)
+    validate_detection!(detection)
+    raise Error, "cannot encode binary input" if detection.binary
+    raise Error, "string must be valid text" unless string.is_a?(String) && string.valid_encoding?
+
+    utf8 = string.encode(Encoding::UTF_8)
+    detection.bom + utf8.encode(detection.encoding).b
+  rescue EncodingError => error
+    raise Error, "cannot encode #{detection.encoding.name}: #{error.message}"
+  end
+
+  def roundtrip?(value, detection)
+    return false unless value.is_a?(String)
+
+    if value.encoding == Encoding::UTF_8 && value.valid_encoding?
+      decode(encode(value, detection), detection) == value
+    else
+      encode(decode(value, detection), detection) == value.b
+    end
+  rescue Error
+    false
+  end
+
+  def normalize_newlines(string, to: :lf)
+    raise Error, "string must be valid text" unless string.is_a?(String) && string.valid_encoding?
+    replacement = NEWLINES.fetch(to) { raise Error, "newline must be :lf, :crlf, or :cr" }
+    utf8 = string.encode(Encoding::UTF_8)
+    original = newline_kind(utf8)
+    [utf8.gsub(/\r\n|\r|\n/, replacement), original]
+  end
+
+  def validate_detection!(detection)
+    unless detection.is_a?(Detection) && (detection.encoding.nil? || SUPPORTED_ENCODINGS.include?(detection.encoding)) &&
+        detection.confidence.is_a?(Float) && detection.confidence.finite? && detection.confidence.between?(0.0, 1.0) &&
+        detection.bom.is_a?(String) && detection.bom.encoding == Encoding::BINARY &&
+        [:lf, :crlf, :cr, :mixed, :none, nil].include?(detection.newline) &&
+        (detection.indent.nil? || detection.indent.is_a?(Indent)) &&
+        detection.binary == !!detection.binary
+      raise Error, "invalid detection"
+    end
+    raise Error, "text detection requires an encoding" if !detection.binary && detection.encoding.nil?
+    raise Error, "binary detection cannot specify text metadata" if detection.binary && (detection.encoding || !detection.bom.empty?)
+    if !detection.bom.empty? && !BOMS.include?([detection.bom, detection.encoding])
+      raise Error, "BOM does not match the detected encoding"
+    end
+  end
+  private_class_method :validate_detection!
+end

data/lib/menkar/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module Menkar
+  VERSION = "0.1.0"
+end

data/lib/menkar.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+require_relative "menkar/version"
+
+module Menkar
+  class Error < StandardError; end
+
+  module Value
+    module_function
+
+    def define(*members)
+      return Data.define(*members) if defined?(Data)
+
+      Struct.new(*members) do
+        members.each { |member| undef_method("#{member}=") }
+
+        def initialize(*values, **keywords)
+          if keywords.empty?
+            raise ArgumentError, "wrong number of arguments" unless values.length == self.class.members.length
+
+            super(*values)
+          else
+            raise ArgumentError, "cannot mix positional and keyword arguments" unless values.empty?
+
+            missing = self.class.members - keywords.keys
+            unknown = keywords.keys - self.class.members
+            raise ArgumentError, "missing keyword: #{missing.first.inspect}" unless missing.empty?
+            raise ArgumentError, "unknown keyword: #{unknown.first.inspect}" unless unknown.empty?
+
+            super(*self.class.members.map { |member| keywords.fetch(member) })
+          end
+          freeze
+        end
+
+        def with(**changes)
+          return self if changes.empty?
+
+          unknown = changes.keys - self.class.members
+          raise ArgumentError, "unknown keyword: #{unknown.first.inspect}" unless unknown.empty?
+
+          self.class.new(**to_h.merge(changes))
+        end
+      end
+    end
+  end
+
+  Detection = Value.define(:encoding, :confidence, :bom, :newline, :indent, :binary)
+  Indent = Value.define(:style, :width)
+  private_constant :Value
+
+  def self.newline_kind(text)
+    crlf = text.scan(/\r\n/).length
+    rest = text.gsub(/\r\n/, "")
+    kinds = []
+    kinds << :crlf if crlf.positive?
+    kinds << :lf if rest.include?("\n")
+    kinds << :cr if rest.include?("\r")
+    return :none if kinds.empty?
+
+    kinds.one? ? kinds.first : :mixed
+  end
+  private_class_method :newline_kind
+end
+
+require_relative "menkar/encoding_scorer"
+require_relative "menkar/detector"
+require_relative "menkar/transcoder"
+
+module Menkar
+  private_constant :BOMS, :SUPPORTED_ENCODINGS, :NEWLINES, :EncodingScorer
+end

data/sig/menkar.rbs

@@ -0,0 +1,39 @@
+module Menkar
+  VERSION: String
+  DEFAULT_SAMPLE: Integer
+  MAX_SAMPLE: Integer
+
+  class Error < StandardError
+  end
+
+  class Indent
+    attr_reader style: :tab | :space
+    attr_reader width: Integer?
+    def self.new: (:tab | :space style, Integer? width) -> Indent
+                | (style: :tab | :space, width: Integer?) -> Indent
+    def with: (?style: :tab | :space, ?width: Integer?) -> Indent
+  end
+
+  class Detection
+    attr_reader encoding: Encoding?
+    attr_reader confidence: Float
+    attr_reader bom: String
+    attr_reader newline: :lf | :crlf | :cr | :mixed | :none | nil
+    attr_reader indent: Indent?
+    attr_reader binary: bool
+    def self.new: (Encoding? encoding, Float confidence, String bom,
+      (:lf | :crlf | :cr | :mixed | :none | nil) newline, Indent? indent, bool binary) -> Detection
+                | (encoding: Encoding?, confidence: Float, bom: String,
+      newline: (:lf | :crlf | :cr | :mixed | :none | nil), indent: Indent?, binary: bool) -> Detection
+    def with: (?encoding: Encoding?, ?confidence: Float, ?bom: String,
+      ?newline: (:lf | :crlf | :cr | :mixed | :none | nil), ?indent: Indent?, ?binary: bool) -> Detection
+  end
+
+  def self.detect: (String bytes, ?hint: untyped, ?sample: Integer) -> Detection
+  def self.detect_file: (String path, ?sample: Integer) -> Detection
+  def self.binary?: (String bytes) -> bool
+  def self.decode: (String bytes, Detection detection) -> String
+  def self.encode: (String string, Detection detection) -> String
+  def self.roundtrip?: (String value, Detection detection) -> bool
+  def self.normalize_newlines: (String string, ?to: :lf | :crlf | :cr) -> [String, Symbol]
+end

metadata

@@ -0,0 +1,55 @@
+--- !ruby/object:Gem::Specification
+name: menkar
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Yudai Takada
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies: []
+email:
+- t.yudai92@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- docs/adr/000-template.md
+- docs/adr/001-strict-bounded-detection.md
+- docs/adr/README.md
+- lib/menkar.rb
+- lib/menkar/detector.rb
+- lib/menkar/encoding_scorer.rb
+- lib/menkar/transcoder.rb
+- lib/menkar/version.rb
+- sig/menkar.rbs
+homepage: https://github.com/noxdea/menkar
+licenses:
+- MIT
+metadata:
+  source_code_uri: https://github.com/noxdea/menkar
+  changelog_uri: https://github.com/noxdea/menkar/blob/main/CHANGELOG.md
+  allowed_push_host: https://rubygems.org
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.1'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.16
+specification_version: 4
+summary: Pure Ruby text encoding detection and normalization
+test_files: []