charcutter 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: afb17205fc7226a6a055a6f37f4a4680883fc0d8dd7137fd72e1cb5614e73610
+  data.tar.gz: 854784fe64707c2664e8e7be4586747ffb7d5ccdd0c9f05d9fc2f732d28b3c6d
+SHA512:
+  metadata.gz: c76cb80045c2e404c151a75e4d828b946b8a23e9c825e9b6e6ea7d5e0e87fd561be6a477f998f3bfadbc13b6fdee3fad57c6421ef62836efe71e24bf6e4862d5
+  data.tar.gz: 7fe2550f1498174a43c2f2adbddc4a31a74f3721daedbc878ec5ed39ab36c2e70c0e810f8c4f644f1ba6a9b4d4794df2adfffa8c6d02e2aaa7fec30d5cc97362

data/.gitignore

@@ -0,0 +1,2 @@
+Gemfile.lock
+/pkg

data/CHANGELOG.md

@@ -0,0 +1,6 @@
+## CHANGELOG
+
+### 0.1.0
+
+* Initial release; Only supports removing/restricting to ignorables
+

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at opensource@janlelis.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,8 @@
+source 'https://rubygems.org'
+
+gemspec
+
+gem 'minitest'
+gem 'minitest-mock'
+gem 'rake'
+gem 'irb'

data/MIT-LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2026 Jan Lelis, https://janlelis.com
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,31 @@
+# Charcutter [![[version]](https://badge.fury.io/rb/charcutter.svg)](https://badge.fury.io/rb/charcutter)  [![[ci]](https://github.com/janlelis/charcutter/workflows/Test/badge.svg)](https://github.com/janlelis/charcutter/actions?query=workflow%3ATest)
+
+Be in control of what codepoints your string contains. Charcutter provides Unicode-based regexs to restrict or permit certain characters in your user input.
+
+Version 0.0.1: Only ignorable characters are supported.
+
+Unicode version: **18.0.0** (September 2026)
+
+## Gemfile
+
+```ruby
+gem "charcutter"
+```
+
+## Usage
+
+```ruby
+require "charcutter"
+
+string = "A\u{FE00}B" # U+FE00 is VARIATION SELECTOR-1, an ignorable codepoint
+
+# Wrap string in Charcutter[] and call deny with codepoint types to remove
+Charcutter[string].deny(:ignorable) # => "AB"
+```
+
+See [unicode-confusable](https://github.com/janlelis/unicode-confusable) for for an example usage of this library.
+
+## MIT License
+
+- Copyright (C) 2026 Jan Lelis <https://janlelis.com>. Released under the MIT license.
+- Unicode data: https://www.unicode.org/copyright.html#Exhibit1

data/Rakefile

@@ -0,0 +1,50 @@
+# # #
+# Get gemspec info
+
+gemspec_file = Dir['*.gemspec'].first
+gemspec = eval File.read(gemspec_file), binding, gemspec_file
+info = "#{gemspec.name} | #{gemspec.version} | " \
+       "#{gemspec.runtime_dependencies.size} dependencies | " \
+       "#{gemspec.files.size} files"
+
+# # #
+# Gem build and install task
+
+desc info
+task :gem do
+  puts info + "\n\n"
+  print "  "; sh "gem build #{gemspec_file}"
+  FileUtils.mkdir_p 'pkg'
+  FileUtils.mv "#{gemspec.name}-#{gemspec.version}.gem", 'pkg'
+  puts; sh %{gem install --no-document pkg/#{gemspec.name}-#{gemspec.version}.gem}
+end
+
+# # #
+# Start an IRB session with the gem loaded
+
+desc "#{gemspec.name} | IRB"
+task :irb do
+  sh "irb -I ./lib -r #{gemspec.name.gsub '-','/'}"
+end
+
+# # #
+# Run Specs
+
+desc "#{gemspec.name} | Spec"
+task :spec do
+  if RbConfig::CONFIG['host_os'] =~ /mswin|mingw/
+    sh "for %f in (spec/\*.rb) do ruby spec/%f"
+  else
+    sh "for file in spec/*.rb; do ruby $file; done"
+  end
+end
+task default: :spec
+
+# # #
+# Generate regex
+
+desc "#{gemspec.name} | Generates all regex constants and saves them to charcutter/{constructed,native} directories"
+task :update_regexes do
+  load "scripts/update_regexes.rb", true
+end
+

data/charcutter.gemspec

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8 -*-
+
+require File.dirname(__FILE__) + "/lib/charcutter/version"
+
+Gem::Specification.new do |gem|
+  gem.name          = "charcutter"
+  gem.version       = Charcutter::VERSION
+  gem.summary       = "Be in charge of your strings."
+  gem.description   = "[Unicode #{Charcutter::UNICODE_VERSION}] Be in charge of your strings."
+  gem.authors       = ["Jan Lelis"]
+  gem.email         = ["hi@ruby.consulting"]
+  gem.homepage      = "https://github.com/janlelis/charcutter"
+  gem.license       = "MIT"
+
+  gem.files         = Dir["{**/}{.*,*}"].select{ |path| File.file?(path) && path !~ /^pkg|data/ }
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.require_paths = ["lib"]
+  gem.metadata      = { "rubygems_mfa_required" => "true" }
+
+  gem.required_ruby_version = ">= 3.2"
+end

data/lib/charcutter/regex/constructed/ignorable.rb

@@ -0,0 +1,7 @@
+# This file is auto-generated. See scripts/update_regexes.rb for more info.
+
+class Charcutter
+  module Regex
+    IGNORABLE = /[­͏؜ᅟ-ᅠ឴-឵᠋-᠍᠎᠏​-‏‪-‮⁠-⁤⁥⁦-ㅤ︀-️ﾠ￰-￸𛲠-𛲣𝅳-𝅺󠀀󠀁󠀂-󠀟󠀠-󠁿󠂀-󠃿󠄀-󠇯󠇰-󠿿]/
+  end
+end

data/lib/charcutter/regex/constructed/ignorable_negated.rb

@@ -0,0 +1,7 @@
+# This file is auto-generated. See scripts/update_regexes.rb for more info.
+
+class Charcutter
+  module Regex
+    IGNORABLE_NEGATED = /[^­͏؜ᅟ-ᅠ឴-឵᠋-᠍᠎᠏​-‏‪-‮⁠-⁤⁥⁦-ㅤ︀-️ﾠ￰-￸𛲠-𛲣𝅳-𝅺󠀀󠀁󠀂-󠀟󠀠-󠁿󠂀-󠃿󠄀-󠇯󠇰-󠿿]/
+  end
+end

data/lib/charcutter/regex/native/ignorable.rb

@@ -0,0 +1,7 @@
+# This file is auto-generated. See scripts/update_regexes.rb for more info.
+
+class Charcutter
+  module Regex
+    IGNORABLE = /\p{di}/
+  end
+end

data/lib/charcutter/regex/native/ignorable_negated.rb

@@ -0,0 +1,7 @@
+# This file is auto-generated. See scripts/update_regexes.rb for more info.
+
+class Charcutter
+  module Regex
+    IGNORABLE_NEGATED = /\p{^di}/
+  end
+end

data/lib/charcutter/regex.rb

@@ -0,0 +1,12 @@
+require "rbconfig"
+
+class Charcutter
+  module Regex
+    MODE = UNICODE_VERSION == RbConfig::CONFIG["UNICODE_VERSION"] ?
+      "native" :
+      "constructed"
+
+    autoload :IGNORABLE, File.expand_path("regex/#{MODE}/ignorable", __dir__)
+    autoload :IGNORABLE_NEGATED, File.expand_path("regex/#{MODE}/ignorable_negated", __dir__)
+  end
+end

data/lib/charcutter/string_ext.rb

@@ -0,0 +1,12 @@
+require_relative "../charcutter"
+
+# Optional string extension for your convenience
+class String
+  def allow(...)
+    Charcutter[self].allow(...)
+  end
+
+  def deny(...)
+    Charcutter[self].deny(...)
+  end
+end

data/lib/charcutter/version.rb

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+class Charcutter
+  VERSION = "0.1.0"
+  UNICODE_VERSION = "18.0.0"
+end

data/lib/charcutter.rb

@@ -0,0 +1,31 @@
+require_relative "charcutter/version"
+require_relative "charcutter/regex"
+
+class Charcutter
+  EMPTY = ""
+
+  class << self
+    alias [] new
+  end
+
+  def initialize(string)
+    @string = string
+  end
+
+  def allow(*props, **)
+    if props[0] == :ignorable
+      @string.gsub(Regex::IGNORABLE_NEGATED, EMPTY)
+    else
+      raise ArgumentError, "only :ignorable supported as of 0.0.1"
+    end
+  end
+
+  def deny(*props, **)
+    if props[0] == :ignorable
+      @string.gsub(Regex::IGNORABLE, EMPTY)
+    else
+      raise ArgumentError, "only :ignorable supported as of 0.0.1"
+    end
+  end
+end
+

data/scripts/update_regexes.rb

@@ -0,0 +1,107 @@
+# Based on radarek contribution to unicode-emoji
+
+require "zlib"
+
+index_filename = (File.dirname(__FILE__) + "/../data/charcutter.marshal.gz").freeze
+
+File.open(index_filename, "rb") do |file|
+  serialized_data = Zlib::GzipReader.new(file).read
+  serialized_data.force_encoding Encoding::BINARY
+  INDEX = Marshal.load(serialized_data)
+end
+
+def write_regexes(regexes, dirpath)
+  regexes.each do |const_name, regex|
+    write_regex(const_name, regex, dirpath)
+  end
+end
+
+def write_regex(const_name, regex, dirpath)
+  filename = const_name.to_s.downcase
+  filepath = File.join(dirpath, "#{filename}.rb")
+
+  File.write(filepath, <<~CONTENT)
+    # This file is auto-generated. See scripts/update_regexes.rb for more info.
+
+    class Charcutter
+      module Regex
+        #{const_name} = #{regex.inspect}
+      end
+    end
+  CONTENT
+  puts "#{const_name} written to #{filepath}"
+end
+
+# Converts [1, 2, 3, 5, 6, 20, 21, 22, 23, 100] (it does not need to be sorted) to [[1, 2, 3], [5, 6], [20, 21, 22, 23], [100]]
+def groupify(arr)
+  arr = arr.sort
+  prev = nil
+  arr.slice_before do |el|
+    (prev.nil? || el != prev + 1).tap { prev = el }
+  end
+end
+
+# Converts [1, 2, 3, 5, 6, 20, 21, 22, 23, 100] (it does not need to be sorted) to [1..3, 5, 6, 20..23, 100]
+def rangify(arr)
+  groupify(arr).map do |group|
+    group.size < 3 ? group : Range.new(group.first, group.last)
+  end.flatten
+end
+
+def rangify_from_limits(arr)
+  arr.map do |el_or_limits|
+    if el_or_limits.is_a? Array
+      Range.new(*el_or_limits)
+    else
+      el_or_limits
+    end
+  end
+end
+
+def pack(ord)
+  Regexp.escape(Array(ord).pack("U*"))
+end
+
+def join(*strings)
+  "(?:" + strings.join("|") + ")"
+end
+
+def character_class(ords_with_ranges)
+  "[" + ords_with_ranges.map{ |ord_or_range|
+      ord_or_range.is_a?(Range) ?
+        pack(ord_or_range.first) + "-" + pack(ord_or_range.last) :
+        pack(ord_or_range)
+    }.join +
+  "]"
+end
+
+def pack_and_join(ords)
+  if ords.any? { |e| e.is_a?(Array) }
+    join(*ords.map { |ord| pack(ord) })
+  else
+    character_class(rangify(ords))
+  end
+end
+
+def add_negation(regex_source)
+  regex_source.sub(/(?<=\[|\\p{)/,"^")
+end
+
+def compile(ignorable:)
+  regexes = {}
+
+  regexes[:IGNORABLE] = Regexp.compile(ignorable)
+  regexes[:IGNORABLE_NEGATED] = Regexp.compile(add_negation(ignorable))
+
+  regexes
+end
+
+constructed_regexes = compile(
+  ignorable: character_class(rangify_from_limits(INDEX[:IGNORABLE]))
+)
+write_regexes(constructed_regexes, File.expand_path("../lib/charcutter/regex/constructed", __dir__))
+
+native_regexes = compile(
+  ignorable: "\\p{di}",
+)
+write_regexes(native_regexes, File.expand_path("../lib/charcutter/regex/native", __dir__))

data/spec/charcutter_spec.rb

@@ -0,0 +1,24 @@
+require_relative "../lib/charcutter"
+require "minitest/autorun"
+
+describe Charcutter do
+  describe ".[]" do
+    it "creates a new charcutter instance" do
+      assert_instance_of Charcutter, Charcutter["A"]
+    end
+  end
+
+  describe "#allow" do
+    it "will only allow the given character type in the string" do
+      assert_equal "\u{FE00}", Charcutter["A\u{FE00}B"].allow(:ignorable)
+    end
+  end
+
+  describe "#deny" do
+    it "will remove given character type in the string" do
+      assert_equal "AB", Charcutter["A\u{FE00}B"].deny(:ignorable)
+    end
+  end
+
+end
+

metadata

@@ -0,0 +1,59 @@
+--- !ruby/object:Gem::Specification
+name: charcutter
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Jan Lelis
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies: []
+description: "[Unicode 18.0.0] Be in charge of your strings."
+email:
+- hi@ruby.consulting
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- CHANGELOG.md
+- CODE_OF_CONDUCT.md
+- Gemfile
+- MIT-LICENSE.txt
+- README.md
+- Rakefile
+- charcutter.gemspec
+- lib/charcutter.rb
+- lib/charcutter/regex.rb
+- lib/charcutter/regex/constructed/ignorable.rb
+- lib/charcutter/regex/constructed/ignorable_negated.rb
+- lib/charcutter/regex/native/ignorable.rb
+- lib/charcutter/regex/native/ignorable_negated.rb
+- lib/charcutter/string_ext.rb
+- lib/charcutter/version.rb
+- scripts/update_regexes.rb
+- spec/charcutter_spec.rb
+homepage: https://github.com/janlelis/charcutter
+licenses:
+- MIT
+metadata:
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.10
+specification_version: 4
+summary: Be in charge of your strings.
+test_files: []