uniscribe 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 9f51890eee3fe1e008ed635c68a3598d9e9f5467
+  data.tar.gz: d25eb46898d1de180c085960fe858cbbea57dd2f
+SHA512:
+  metadata.gz: 766dedd2e18ef926e5a962882250c232db1aae66e6008cb179fc7bfaa88a913b61157d8114527f6bb9eff2a0111137658b0311015bf5679fbefb5a9af6210b0b
+  data.tar.gz: bfe050bd760fd40afbbc2dbf6f664d3c2de4cbfff2ffc0cd9ab7d3912f7e3349b57a45de44949ba4e07b06704e08f7796e2bf3675410a5e64583a9240cdaa13c

data/.gitignore

@@ -0,0 +1,2 @@
+Gemfile.lock
+/pkg

data/.travis.yml

@@ -0,0 +1,23 @@
+sudo: false
+language: ruby
+
+rvm:
+- ruby-head
+- 2.4.1
+- 2.3.3
+- 2.2
+- 2.1
+- 2.0
+- jruby-head
+- jruby-9.1.8.0
+
+cache:
+- bundler
+
+matrix:
+  allow_failures:
+    - rvm: jruby-head
+    - rvm: jruby-9.1.8.0
+    - rvm: ruby-head
+    - rvm: 2.0
+#   fast_finish: true

data/CHANGELOG.md

@@ -0,0 +1,6 @@
+## CHANGELOG
+
+### 0.1.0
+
+* Initial release
+

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at opensource@janlelis.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,6 @@
+source 'https://rubygems.org'
+
+gemspec
+
+gem 'minitest'
+gem 'rake'

data/MIT-LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2017 Jan Lelis, mail@janlelis.de
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,113 @@
+# uniscribe | Describe the Unicode [![[version]](https://badge.fury.io/rb/uniscribe.svg)](http://badge.fury.io/rb/uniscribe)  [![[travis]](https://travis-ci.org/janlelis/uniscribe.svg)](https://travis-ci.org/janlelis/uniscribe)
+
+Describes Unicode characters with their name and shows compositions.
+
+- Helps you understand how glyphs and codepoints are structered within the data
+- Gives you the names of glyphs and codepoints, which can be used for further research
+- Highlights invalid/special/blank codepoints
+
+Uses a similar color coding like its lower-level companion tool [unibits](https://github.com/janlelis/unibits).
+
+## Setup
+
+Make sure you have Ruby installed and installing gems works properly. Then do:
+
+```
+$ gem install uniscribe
+```
+
+## Usage
+
+Pass the string to debug to uniscribe:
+
+### From CLI
+
+```
+$ uniscribe "test strı̈ng"
+```
+
+### From Ruby
+
+```ruby
+require "uniscribe/kernel_method"
+uniscribe "test strı̈ng"
+```
+
+### Output
+
+```
+
+0074 ├─ t		├─ LATIN SMALL LETTER T
+0065 ├─ e		├─ LATIN SMALL LETTER E
+0073 ├─ s		├─ LATIN SMALL LETTER S
+0074 ├─ t		├─ LATIN SMALL LETTER T
+0020 ├─ ] [		├─ SPACE
+0073 ├─ s		├─ LATIN SMALL LETTER S
+0074 ├─ t		├─ LATIN SMALL LETTER T
+0072 ├─ r		├─ LATIN SMALL LETTER R
+---- ├┬ ı̈		├┬ Composition
+0131 │├─ ı		│├─ LATIN SMALL LETTER DOTLESS I
+0308 │└─ ◌̈		│└─ COMBINING DIAERESIS
+006E ├─ n		├─ LATIN SMALL LETTER N
+0067 ├─ g		├─ LATIN SMALL LETTER G
+
+```
+
+## Examples
+
+### Tamil
+
+`>> uniscribe "நகரத்தில்"`
+
+![Screenshot Tamil](/screenshots/tamil.png?raw=true "Tamil")
+
+### Thai
+
+`>> uniscribe "ม้าลายหกตัว"`
+
+![Screenshot Thai](/screenshots/thai.png?raw=true "Thai")
+
+### Emoji Sequences
+
+`>> uniscribe "3️⃣🤸‍♀"`
+
+![Screenshot Emoji](/screenshots/emoji.png?raw=true "Emoji")
+
+### Lots of Combining Marks
+
+`>> uniscribe "̶̧̨̱̹̭̯ͧ̾ͬC̷̙̲̝͖ͭ̏ͥͮ͟Oͮ͏̮̪̝͍"`
+
+![Screenshot Marks](/screenshots/marks.png?raw=true "Marks")
+
+### Some Strange Unicode Characters
+
+`>> uniscribe "\0A\u{E01D7}\x7F\r\n\u{D0000}\u{81}\u{FFF9}B\u{FFFB}🏴\u{E0061}\u{E007F}\u{10FFFF}"`
+
+![Screenshot Strange](/screenshots/strange.png?raw=true "Strange")
+
+### Some Blanks
+
+`>> uniscribe "­ᅠ 𝅸"`
+
+![Screenshot Blanks](/screenshots/blanks.png?raw=true "Blanks")
+
+## Notes
+
+The proper detetion of compositions / graphemes / combined characters depends on your Ruby version:
+
+Ruby | Unicode Version
+-----|----------------
+2.4  | 9.0.0
+2.3  | 8.0.0
+2.2  | 7.0.0
+2.1  | 6.1.0
+
+Also see
+
+- [unibits](https://github.com/janlelis/unibits) - visualizes Unicode encodings
+- [symbolify](https://github.com/janlelis/symbolify) - used for safely printing individual codepoints
+- [characteristics](https://github.com/janlelis/characteristics) - used for detecting blanks and similar
+- [unicopy](https://github.com/janlelis/unicopy) - copy codepoints to clipboard
+- [Unicode® Standard Annex #29: Unicode Text Segmentation](http://unicode.org/reports/tr29/)
+
+Copyright (C) 2017 Jan Lelis <http://janlelis.com>. Released under the MIT license.

data/Rakefile

@@ -0,0 +1,38 @@
+# # #
+# Get gemspec info
+
+gemspec_file = Dir['*.gemspec'].first
+gemspec = eval File.read(gemspec_file), binding, gemspec_file
+info = "#{gemspec.name} | #{gemspec.version} | " \
+       "#{gemspec.runtime_dependencies.size} dependencies | " \
+       "#{gemspec.files.size} files"
+
+# # #
+# Gem build and install task
+
+desc info
+task :gem do
+  puts info + "\n\n"
+  print "  "; sh "gem build #{gemspec_file}"
+  FileUtils.mkdir_p 'pkg'
+  FileUtils.mv "#{gemspec.name}-#{gemspec.version}.gem", 'pkg'
+  puts; sh %{gem install --no-document pkg/#{gemspec.name}-#{gemspec.version}.gem}
+end
+
+# # #
+# Start an IRB session with the gem loaded
+
+desc "#{gemspec.name} | IRB"
+task :irb do
+  sh "irb -I ./lib -r #{gemspec.name.gsub '-','/'}/kernel_method"
+end
+
+# # #
+# Run specs
+
+desc "#{gemspec.name} | Spec"
+task :spec do
+  sh "for file in spec/*_spec.rb; do ruby $file; done"
+end
+task default: :spec
+

data/bin/uniscribe

@@ -0,0 +1,75 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "rationalist"
+require "uniscribe"
+
+argv = Rationalist.parse(
+  ARGV,
+  string: '_',
+  alias: {
+    e: 'encoding',
+    v: 'version',
+  },
+  boolean: [
+    'help',
+    'version',
+    'wide-ambiguous',
+  ]
+)
+
+if argv[:version]
+  puts "uniscribe #{Uniscribe::VERSION} by #{Paint["J-_-L", :bold]} <https://github.com/janlelis/uniscribe>"
+  puts "Unicode version is #{Uniscribe::UNICODE_VERSION} (glyph detection #{Uniscribe::UNICODE_VERSION_GLYPH_DETECTION || "[not supported]"})"
+  exit(0)
+end
+
+if argv[:help]
+  puts <<-HELP
+
+  #{Paint["DESCRIPTION", :underline]}
+
+  Describes a string of Unicode characters with their name and shows compositions.
+
+  #{Paint["USAGE", :underline]}
+
+  #{Paint["uniscribe", :bold]} [options] data
+
+  --encoding <encoding> | -e | which (Unicode) encoding to use for given data
+  --help                |    | this help page
+  --version             | -v | displays version of uniscribe
+  --wide-ambiguous      |    | ambiguous characters
+
+  #{Paint["COLOR CODING", :underline]}
+
+  #{Paint["blank",      Uniscribe::COLORS[:blank]]}
+  #{Paint["control",    Uniscribe::COLORS[:control]]}
+  #{Paint["format",     Uniscribe::COLORS[:format]]}
+  #{Paint["mark",       Uniscribe::COLORS[:mark]]}
+  #{Paint["unassigned", Uniscribe::COLORS[:unassigned]]}
+  #{Paint["unassigned and ignorable", Uniscribe::COLORS[:ignorable]]}
+
+  random color for other characters and compositions
+
+  #{Paint["MORE INFO", :underline]}
+
+  https://github.com/janlelis/uniscribe
+
+  HELP
+  exit(0)
+end
+
+if argv[:_] && argv[:_][0]
+  data = argv[:_][0]
+elsif !$stdin.tty?
+  data = $stdin.read
+else
+  data = nil
+end
+
+begin
+  Uniscribe.of(data)
+rescue ArgumentError
+  $stderr.puts Paint[$!.message, :red]
+  exit(1)
+end

data/lib/uniscribe.rb

@@ -0,0 +1,206 @@
+# frozen_string_literal: true
+
+require_relative "uniscribe/version"
+
+require "unicode/name"
+require "unicode/sequence_name"
+require "symbolify"
+require "characteristics"
+require "paint"
+require "unicode/display_width"
+require "unicode/emoji"
+
+module Uniscribe
+  SUPPORTED_ENCODINGS = Encoding.name_list.grep(
+    Regexp.union(
+      /^UTF-8$/,
+      /^UTF8-/,
+      /^UTF-...E$/,
+      /^US-ASCII$/,
+      /^ISO-8859-1$/,
+    )
+  ).sort.freeze
+
+  COLORS = {
+    control: "#0000FF",
+    blank: "#33AADD",
+    format: "#FF00FF",
+    mark: "#228822",
+    unassigned: "#FF5500",
+    ignorable: "#FFAA00",
+  }
+
+  def self.of(string, encoding: nil, wide_ambiguous: false)
+    string = convert_to_encoding_or_raise(string, encoding)
+    glyphs = string.encode("UTF-8").scan(/\X/)
+
+    visualize(glyphs, wide_ambiguous: wide_ambiguous)
+  end
+
+  def self.convert_to_encoding_or_raise(string, encoding)
+    raise ArgumentError, "no data given to uniscribe" if !string || string.empty?
+
+    string.force_encoding(encoding) if encoding
+
+    case string.encoding.name
+    when *SUPPORTED_ENCODINGS
+      unless string.valid_encoding?
+        raise ArgumentError, "uniscribe can only describe strings with a valid encoding"
+      end
+
+      string
+    when 'UTF-16', 'UTF-32'
+      raise ArgumentError, "unibits only supports #{string.encoding.name} with specified endianess, please use #{string.encoding.name}LE or #{string.encoding.name}BE"
+    else
+      raise ArgumentError, "uniscribe can only describe Unicode strings (or US-ASCII or ISO-8859-1)"
+    end
+  end
+
+  def self.visualize(glyphs, wide_ambiguous: false)
+    puts
+    ( glyphs[0..-2] || [] ).each{ |glyph|
+      cps = glyph.codepoints
+      if cps.size > 1
+        puts_composition(cps, wide_ambiguous)
+      else
+        puts_codepoint(cps[0], false, false, wide_ambiguous)
+      end
+    }
+
+    cps = glyphs[-1].codepoints
+    if cps.size > 1
+      puts_composition(cps, wide_ambiguous)
+    else
+      puts_codepoint(cps[0], false, true, wide_ambiguous)
+    end
+    puts
+  end
+
+  def self.puts_composition(cps, wide_ambiguous = false)
+    char = cps.pack("U*")
+    if sequence_name = Unicode::SequenceName.of(char)
+      name = "Composition: #{sequence_name}"
+    else
+      name = "Composition"
+    end
+    char_color = random_color
+    cp_hex = "----"
+    symbolified_char = symbolify_composition(char)
+    padding = determine_padding(symbolified_char, false, wide_ambiguous)
+
+    puts "   %s ├┬ %s%s├┬ %s" % [
+      Paint[cp_hex, char_color],
+      Paint[symbolified_char, char_color],
+      padding,
+      Paint[name, char_color],
+    ]
+    ( cps[0..-2] || [] ).each{ |cp|
+      puts_codepoint(cp, true, false, wide_ambiguous)
+    }
+    puts_codepoint(cps[-1], true, true, wide_ambiguous)
+  end
+
+  def self.puts_codepoint(cp, composed = false, last = false, wide_ambiguous = false)
+    char = [cp].pack("U*")
+    char_info = UnicodeCharacteristics.new(char)
+    char_color = determine_codepoint_color(char_info)
+    cp_hex = cp.to_s(16).rjust(4, "0").rjust(6).upcase
+    symbolified_char = Symbolify.unicode(char, char_info)
+    if composed && !last
+      branch = "│├─"
+    elsif composed && last
+      branch = "│└─"
+    else
+      branch = "├─"
+    end
+    name = determine_codepoint_name(char)
+    padding = determine_padding(symbolified_char, composed, wide_ambiguous)
+
+    puts " %s %s %s%s%s %s" % [
+      Paint[cp_hex, char_color],
+      branch,
+      Paint[symbolified_char, char_color],
+      padding,
+      branch,
+      Paint[name, char_color],
+    ]
+  end
+
+  def self.determine_codepoint_color(char_info)
+    if !char_info.assigned?
+      if char_info.ignorable?
+        COLORS[:ignorable]
+      else
+        COLORS[:unassigned]
+      end
+    elsif char_info.blank?
+      COLORS[:blank]
+    elsif char_info.control?
+      COLORS[:control]
+    elsif char_info.format?
+      COLORS[:format]
+    elsif char_info.unicode? && char_info.category[0] == "M"
+      COLORS[:mark]
+    else
+      random_color
+    end
+  end
+
+  def self.random_color
+    "%.2x%.2x%.2x" % [rand(90) + 60, rand(90) + 60, rand(90) + 60]
+  end
+
+  def self.determine_codepoint_name(char)
+    name = Unicode::Name.correct(char)
+    return name if name
+
+    name = Unicode::Name.label(char)
+    as = Unicode::Name.aliases(char)
+    return name if !as
+
+    alias_ = ( as[:control]      && as[:control][0]      ||
+               as[:figment]      && as[:figment][0]      ||
+               as[:alternate]    && as[:alternate][0]    ||
+               as[:abbreviation] && as[:abbreviation][0] )
+    return name if !alias_
+
+    name + " " + alias_
+  end
+
+  def self.determine_padding(char, composed, wide_ambiguous)
+    required_width = Unicode::DisplayWidth.of(char, wide_ambiguous ? 2 : 1, {}, emoji: true)
+    required_width += 1 if composed
+    required_width = 0 if required_width < 0
+
+    case required_width
+    when  0...5
+      "\t\t"
+    when 5...10
+      "\t"
+    else
+      ""
+    end
+  end
+
+  def self.symbolify_composition(char)
+    char_infos = char.chars.map{ |c| UnicodeCharacteristics.new(c) }
+
+    case
+    when char_infos.any?{ |c| !c.assigned? }
+      "n/a"
+    when char_infos.all?{ |c| c.separator? }
+      "⏎"
+    when char_infos.all?{ |c| c.category == "Mn" || c.category == "Me" }
+      if char_infos.any?{ |c| c.category == "Mn" }
+        "◌" + char
+      else
+        " " + char
+      end
+    when char_infos.all?{ |c| c.blank? }
+      "]" + char + "["
+    else
+      char
+    end
+  end
+end
+

data/lib/uniscribe/kernel_method.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+require_relative '../uniscribe'
+
+module Kernel
+  private
+
+  def uniscribe(string, **kwargs)
+    Uniscribe.of(string, **kwargs)
+  end
+end

data/lib/uniscribe/version.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Uniscribe
+  VERSION = "1.0.0".freeze
+  UNICODE_VERSION = "9.0.0".freeze
+
+  RUBY_UNICODE_VERSIONS = {
+    2.4 => "9.0.0".freeze,
+    2.3 => "8.0.0".freeze,
+    2.2 => "7.0.0".freeze,
+    2.1 => "6.1.0".freeze,
+  }.freeze
+
+  UNICODE_VERSION_GLYPH_DETECTION = RUBY_ENGINE == "ruby" && RUBY_UNICODE_VERSIONS[RUBY_VERSION.to_f]
+end
+

data/spec/uniscribe_spec.rb

@@ -0,0 +1,187 @@
+require_relative "../lib/uniscribe/kernel_method"
+require "minitest/autorun"
+
+describe Uniscribe do
+  def check(string_to_test, match_regex)
+    uniscribe(string_to_test)
+    assert_output(match_regex){ uniscribe(string_to_test) }
+  end
+
+  describe "displays codepoints" do
+    it "LATIN CAPITAL LETTER" do
+      check "AB", /0041.*0042/m
+    end
+
+    it "AERIAL TRAMWAY" do
+      check "🚡", /1F6A1/
+    end
+  end
+
+  describe "displays glyph itself" do
+    it "LATIN CAPITAL LETTER" do
+      check "AB", /A.*B/m
+    end
+
+    it "AERIAL TRAMWAY" do
+      check "🚡", /🚡/
+    end
+  end
+
+  describe "displays names" do
+    it "LATIN CAPITAL LETTER" do
+      check "AB", /LATIN CAPITAL LETTER A.*LATIN CAPITAL LETTER B/m
+    end
+
+    it "AERIAL TRAMWAY" do
+      check "🚡", /AERIAL TRAMWAY/
+    end
+  end
+
+  describe "supported encodings" do
+    it "works with UTF-16" do
+      check "🚡".encode("UTF-16LE"), /AERIAL TRAMWAY/
+    end
+
+    it "works with UTF-32" do
+      check "🚡".encode("UTF-32BE"), /AERIAL TRAMWAY/
+    end
+
+    it "works with US-ASCII" do
+      check "AB".force_encoding("US-ASCII"), /LATIN CAPITAL LETTER A.*LATIN CAPITAL LETTER B/m
+    end
+
+    it "works with ISO-8859-1" do
+      check "AB\x81".force_encoding("ISO-8859-1"), /LATIN CAPITAL LETTER A.*LATIN CAPITAL LETTER B.*<control-0081> HIGH OCTET PRESET/m
+    end
+  end
+
+  describe "example compositions" do
+    describe "combining marks" do
+      it "DIAERESIS" do
+        check "g̈", /Composition.*LATIN SMALL LETTER G.*DIAERESIS/m
+      end
+
+      it "RING BELOW" do
+        check "n̥", /Composition.*LATIN SMALL LETTER N.*COMBINING RING BELOW/m
+      end
+
+      it "ARABIC FATHA" do
+        check "دَ", /Composition.*ARABIC LETTER DAL.*ARABIC FATHA/m
+      end
+
+      it "ACUTE ACCENT" do
+        check "ά", /Composition.*GREEK SMALL LETTER ALPHA.*COMBINING ACUTE ACCENT/m
+      end
+
+      it "HEBREW POINT HIRIQ" do
+        check "חִ", /Composition.*HEBREW LETTER HET.*HEBREW POINT HIRIQ/m
+      end
+
+      it "THAI CHARACTER SARA U" do
+        check "จุ", /Composition.*THAI CHARACTER CHO CHAN.*THAI CHARACTER SARA U/m
+      end
+    end
+
+    describe "misc scripts" do
+      if RUBY_VERSION >= "2.4.0"
+        it "HANGUL" do
+          check "ᅘᆇᇈ", /Composition.*HANGUL CHOSEONG SSANGHIEUH.*HANGUL JUNGSEONG YO-O.*HANGUL JONGSEONG NIEUN-PANSIOS/m
+        end
+
+        it "HANGUL 2" do
+          check "각", /Composition.*HANGUL CHOSEONG KIYEOK.*HANGUL JUNGSEONG A.*HANGUL JONGSEONG KIYEOK/m
+        end
+
+        it "HANGUL 3" do
+          check "ᄇᄉᄐ", /Composition.*HANGUL CHOSEONG PIEUP.*HANGUL CHOSEONG SIOS.*HANGUL CHOSEONG THIEUTH/m
+        end
+
+        it "TAMIL" do
+          check "நி", /Composition.*TAMIL SYLLABLE NI.*TAMIL LETTER NA.*TAMIL VOWEL SIGN I/m
+        end
+
+        it "DEVANAGARI" do
+          check "षि", /Composition.*DEVANAGARI LETTER SSA.*DEVANAGARI VOWEL SIGN I/m
+        end
+      end
+    end
+
+    describe "zwj and zwnj" do
+      if RUBY_VERSION >= "2.4.0"
+        it "ZWJ" do
+          check "क्‍", /Composition.*DEVANAGARI LETTER KA.*DEVANAGARI SIGN VIRAMA.*ZERO WIDTH JOINER/m
+        end
+
+        it "ZWNJ" do
+          check "t‌", /Composition.*LATIN SMALL LETTER T.*ZERO WIDTH NON-JOINER/m
+        end
+      end
+    end
+
+    describe "misc variations" do
+      it "TEXT STYLE" do
+        check "‼︎", /Composition.*(text style).*DOUBLE EXCLAMATION MARK.*VARIATION SELECTOR-15/m
+      end
+
+      it "EMOJI STYLE" do
+        check "‼️", /Composition.*(emoji style).*DOUBLE EXCLAMATION MARK.*VARIATION SELECTOR-16/m
+      end
+
+      it "DOTTED FORM" do
+        check "င︀", /Composition.*(dotted form).*MYANMAR LETTER NGA.*VARIATION SELECTOR-1/m
+      end
+
+      it "MONGOLIAN SECOND FORM" do
+        check "ᠠ᠋", /Composition.*(second form).*MONGOLIAN LETTER A.*MONGOLIAN FREE VARIATION SELECTOR ONE/m
+      end
+
+      it "CJK COMPATIBILITY IDEOGRAPH-2F81F" do
+        check "㓟︀", /Composition.*CJK COMPATIBILITY IDEOGRAPH-2F81F.*CJK UNIFIED IDEOGRAPH-34DF.*VARIATION SELECTOR-1/m
+      end
+
+      it "CID+6238" do
+        check "胥󠄀", /Composition.*CID\+6238.*CJK UNIFIED IDEOGRAPH-80E5.*VARIATION SELECTOR-17/m
+      end
+    end
+
+    describe "misc other" do
+      it "KEYCAP" do
+        check "5⃣", /Composition.*DIGIT FIVE.*COMBINING ENCLOSING KEYCAP/m
+      end
+
+      if RUBY_VERSION >= "2.4.0"
+        it "␍ + ␊" do
+          check "\r\n", /Composition.*<control-000D> CARRIAGE RETURN.*<control-000A> LINE FEED/m
+        end
+
+        it "REGIONAL" do
+          check "🇺🇳", /Composition.*UNITED NATIONS.*REGIONAL INDICATOR SYMBOL LETTER U.*REGIONAL INDICATOR SYMBOL LETTER N/m
+        end
+
+        it "TAG SEQUENCE" do
+          check "🏴󠁧󠁢󠁳󠁣󠁴󠁿", /Composition.*SCOTLAND.*WAVING BLACK FLAG.*TAG LATIN SMALL LETTER G.*TAG LATIN SMALL LETTER B.*TAG LATIN SMALL LETTER S.*TAG LATIN SMALL LETTER C.*TAG LATIN SMALL LETTER T.*CANCEL TAG/m
+        end
+
+        it "EMOJI MODIFIER" do
+          check "🙅🏿", /Composition.*PERSON GESTURING NO: DARK SKIN TONE.*FACE WITH NO GOOD GESTURE.*EMOJI MODIFIER FITZPATRICK TYPE-6/m
+        end
+
+        it "EMOJI ZWJ SEQUENCE" do
+          check "👩‍👩‍👦‍👦", /Composition.*FAMILY.*WOMAN.*ZERO WIDTH JOINER.*WOMAN.*ZERO WIDTH JOINER.*BOY.*ZERO WIDTH JOINER.*BOY/m
+        end
+      end
+    end
+  end
+
+  describe "unusual codepoints" do
+    if RUBY_VERSION >= "2.4.0"
+      it "safely prints and highlights unusual codepoints" do
+        check "\0A\u{E01D7}\x7F\r\n\u{D0000}\u{81}\u{FFF9}B\u{FFFB}🏴\u{E0061}\u{E007F}\u{10FFFF}", /<control-0000> NULL.*Composition.*LATIN CAPITAL LETTER A.*VARIATION SELECTOR-232.*<control-007F> DELETE.*Composition.*<control-000D> CARRIAGE RETURN.*<control-000A> LINE FEED.*<reserved-D0000>.*<control-0081> HIGH OCTET PRESET.*INTERLINEAR ANNOTATION ANCHOR.*LATIN CAPITAL LETTER B.*INTERLINEAR ANNOTATION TERMINATOR.*Composition.*WAVING BLACK FLAG.*TAG LATIN SMALL LETTER A.*CANCEL TAG.*<noncharacter-10FFFF>/m
+      end
+    end
+
+    it "safely prints and highlights various blanks" do
+      check "­ᅠ 𝅸", /SOFT HYPHEN.*HANGUL JUNGSEONG FILLER.*EM QUAD.*INHIBIT ARABIC FORM SHAPING.*ZERO WIDTH NO-BREAK SPACE.*MUSICAL SYMBOL END SLUR/m
+    end
+  end
+end

data/uniscribe.gemspec

@@ -0,0 +1,29 @@
+# -*- encoding: utf-8 -*-
+
+require File.dirname(__FILE__) + "/lib/uniscribe/version"
+
+Gem::Specification.new do |gem|
+  gem.name          = "uniscribe"
+  gem.version       = Uniscribe::VERSION
+  gem.summary       = "Describes Unicode characters."
+  gem.description   = "Describes Unicode characters with their name and shows compositions."
+  gem.authors       = ["Jan Lelis"]
+  gem.email         = ["mail@janlelis.de"]
+  gem.homepage      = "https://github.com/janlelis/uniscribe"
+  gem.license       = "MIT"
+
+  gem.files         = Dir["{**/}{.*,*}"].select{ |path| File.file?(path) && path !~ /^(pkg|screenshots)/}
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
+
+  gem.required_ruby_version = "~> 2.1"
+  gem.add_dependency "unicode-name", "~> 1.4", ">= 1.4.2"
+  gem.add_dependency "unicode-sequence_name", "~> 1.0"
+  gem.add_dependency "unicode-display_width", "~> 1.2", ">= 1.2.1"
+  gem.add_dependency "unicode-emoji", ">= 0.9", "< 2.0"
+  gem.add_dependency "symbolify", "~> 1.2"
+  gem.add_dependency "characteristics", ">= 0.7", "< 2.0"
+  gem.add_dependency "paint", ">= 0.9", "< 3.0"
+  gem.add_dependency "rationalist", "~> 2.0"
+end

metadata

@@ -0,0 +1,203 @@
+--- !ruby/object:Gem::Specification
+name: uniscribe
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- Jan Lelis
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2017-04-17 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: unicode-name
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.4'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.4.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.4'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.4.2
+- !ruby/object:Gem::Dependency
+  name: unicode-sequence_name
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: unicode-display_width
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.2.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 1.2.1
+- !ruby/object:Gem::Dependency
+  name: unicode-emoji
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.9'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.9'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: symbolify
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+- !ruby/object:Gem::Dependency
+  name: characteristics
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.7'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.7'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: paint
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.9'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.9'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: rationalist
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+description: Describes Unicode characters with their name and shows compositions.
+email:
+- mail@janlelis.de
+executables:
+- uniscribe
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".travis.yml"
+- CHANGELOG.md
+- CODE_OF_CONDUCT.md
+- Gemfile
+- Gemfile.lock
+- MIT-LICENSE.txt
+- README.md
+- Rakefile
+- bin/uniscribe
+- lib/uniscribe.rb
+- lib/uniscribe/kernel_method.rb
+- lib/uniscribe/version.rb
+- spec/uniscribe_spec.rb
+- uniscribe.gemspec
+homepage: https://github.com/janlelis/uniscribe
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - "~>"
+    - !ruby/object:Gem::Version
+      version: '2.1'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.8
+signing_key: 
+specification_version: 4
+summary: Describes Unicode characters.
+test_files:
+- spec/uniscribe_spec.rb