unibits 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.gitignore +2 -0
- data/.travis.yml +22 -0
- data/CHANGELOG.md +6 -0
- data/CODE_OF_CONDUCT.md +74 -0
- data/Gemfile +7 -0
- data/MIT-LICENSE.txt +20 -0
- data/README.md +108 -0
- data/Rakefile +38 -0
- data/bin/unibits +58 -0
- data/lib/unibits.rb +174 -0
- data/lib/unibits/kernel_method.rb +9 -0
- data/lib/unibits/version.rb +3 -0
- data/screenshots/ascii.png +0 -0
- data/screenshots/binary.png +0 -0
- data/screenshots/utf-16be.png +0 -0
- data/screenshots/utf-16le.png +0 -0
- data/screenshots/utf-32be.png +0 -0
- data/screenshots/utf-32le.png +0 -0
- data/screenshots/utf-8.png +0 -0
- data/spec/unibits_spec.rb +70 -0
- data/unibits.gemspec +25 -0
- metadata +112 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 0405b9099735bcf25bc05f82e91d9c5629071ee1
|
4
|
+
data.tar.gz: 85d9bde5ddc1e90ac70cf288e7753ddc8511fc87
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: a8d944dfcf50869cf1a64a7237e8145ee8b8fcb57a801db9a475684ac0e7bd4865be2b2d372f943ddeebb6d86f632a3eb9655d147cc6e3b79bc6e0c6dc09fa66
|
7
|
+
data.tar.gz: be6685c1bfc9b9ecf073e9899be4ded5d590ccdd90b527dfdfdfeb1ac654016cac4116033905183f988e249cabedcc767cc8fa3a4d04b3681650ad7852035330
|
data/.gitignore
ADDED
data/.travis.yml
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
sudo: false
|
2
|
+
language: ruby
|
3
|
+
|
4
|
+
rvm:
|
5
|
+
- ruby-head
|
6
|
+
- 2.4.0
|
7
|
+
- 2.3.3
|
8
|
+
- 2.2
|
9
|
+
- 2.1
|
10
|
+
- 2.0
|
11
|
+
- jruby-head
|
12
|
+
- jruby-9.1.7.0
|
13
|
+
|
14
|
+
cache:
|
15
|
+
- bundler
|
16
|
+
|
17
|
+
matrix:
|
18
|
+
allow_failures:
|
19
|
+
- rvm: jruby-head
|
20
|
+
- rvm: ruby-head
|
21
|
+
- rvm: 2.0
|
22
|
+
# fast_finish: true
|
data/CHANGELOG.md
ADDED
data/CODE_OF_CONDUCT.md
ADDED
@@ -0,0 +1,74 @@
|
|
1
|
+
# Contributor Covenant Code of Conduct
|
2
|
+
|
3
|
+
## Our Pledge
|
4
|
+
|
5
|
+
In the interest of fostering an open and welcoming environment, we as
|
6
|
+
contributors and maintainers pledge to making participation in our project and
|
7
|
+
our community a harassment-free experience for everyone, regardless of age, body
|
8
|
+
size, disability, ethnicity, gender identity and expression, level of experience,
|
9
|
+
nationality, personal appearance, race, religion, or sexual identity and
|
10
|
+
orientation.
|
11
|
+
|
12
|
+
## Our Standards
|
13
|
+
|
14
|
+
Examples of behavior that contributes to creating a positive environment
|
15
|
+
include:
|
16
|
+
|
17
|
+
* Using welcoming and inclusive language
|
18
|
+
* Being respectful of differing viewpoints and experiences
|
19
|
+
* Gracefully accepting constructive criticism
|
20
|
+
* Focusing on what is best for the community
|
21
|
+
* Showing empathy towards other community members
|
22
|
+
|
23
|
+
Examples of unacceptable behavior by participants include:
|
24
|
+
|
25
|
+
* The use of sexualized language or imagery and unwelcome sexual attention or
|
26
|
+
advances
|
27
|
+
* Trolling, insulting/derogatory comments, and personal or political attacks
|
28
|
+
* Public or private harassment
|
29
|
+
* Publishing others' private information, such as a physical or electronic
|
30
|
+
address, without explicit permission
|
31
|
+
* Other conduct which could reasonably be considered inappropriate in a
|
32
|
+
professional setting
|
33
|
+
|
34
|
+
## Our Responsibilities
|
35
|
+
|
36
|
+
Project maintainers are responsible for clarifying the standards of acceptable
|
37
|
+
behavior and are expected to take appropriate and fair corrective action in
|
38
|
+
response to any instances of unacceptable behavior.
|
39
|
+
|
40
|
+
Project maintainers have the right and responsibility to remove, edit, or
|
41
|
+
reject comments, commits, code, wiki edits, issues, and other contributions
|
42
|
+
that are not aligned to this Code of Conduct, or to ban temporarily or
|
43
|
+
permanently any contributor for other behaviors that they deem inappropriate,
|
44
|
+
threatening, offensive, or harmful.
|
45
|
+
|
46
|
+
## Scope
|
47
|
+
|
48
|
+
This Code of Conduct applies both within project spaces and in public spaces
|
49
|
+
when an individual is representing the project or its community. Examples of
|
50
|
+
representing a project or community include using an official project e-mail
|
51
|
+
address, posting via an official social media account, or acting as an appointed
|
52
|
+
representative at an online or offline event. Representation of a project may be
|
53
|
+
further defined and clarified by project maintainers.
|
54
|
+
|
55
|
+
## Enforcement
|
56
|
+
|
57
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
58
|
+
reported by contacting the project team at opensource@janlelis.com. All
|
59
|
+
complaints will be reviewed and investigated and will result in a response that
|
60
|
+
is deemed necessary and appropriate to the circumstances. The project team is
|
61
|
+
obligated to maintain confidentiality with regard to the reporter of an incident.
|
62
|
+
Further details of specific enforcement policies may be posted separately.
|
63
|
+
|
64
|
+
Project maintainers who do not follow or enforce the Code of Conduct in good
|
65
|
+
faith may face temporary or permanent repercussions as determined by other
|
66
|
+
members of the project's leadership.
|
67
|
+
|
68
|
+
## Attribution
|
69
|
+
|
70
|
+
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
|
71
|
+
available at [http://contributor-covenant.org/version/1/4][version]
|
72
|
+
|
73
|
+
[homepage]: http://contributor-covenant.org
|
74
|
+
[version]: http://contributor-covenant.org/version/1/4/
|
data/Gemfile
ADDED
data/MIT-LICENSE.txt
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2017 Jan Lelis, mail@janlelis.de
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,108 @@
|
|
1
|
+
# unibits [![[version]](https://badge.fury.io/rb/unibits.svg)](http://badge.fury.io/rb/unibits) [![[travis]](https://travis-ci.org/janlelis/unibits.svg)](https://travis-ci.org/janlelis/unibits)
|
2
|
+
|
3
|
+
Ruby library and CLI command that visualizes various Unicode and ASCII encodings in the terminal.
|
4
|
+
|
5
|
+
Helps you debugging and learning about Unicode encodings.
|
6
|
+
|
7
|
+
Supported encodings: **UTF-8**, **UTF-16LE**, **UTF-16BE**, **UTF-32LE**, **UTF-32BE**, **BINARY**, **ASCII**
|
8
|
+
|
9
|
+
## Setup
|
10
|
+
|
11
|
+
```
|
12
|
+
$ gem install unibits
|
13
|
+
```
|
14
|
+
|
15
|
+
## Usage
|
16
|
+
### From CLI
|
17
|
+
|
18
|
+
```
|
19
|
+
$ unibits "🌫 Idiosyncrätic ℜսᖯʏ"
|
20
|
+
```
|
21
|
+
|
22
|
+
### From Ruby
|
23
|
+
|
24
|
+
```ruby
|
25
|
+
require 'unibits/kernel_method'
|
26
|
+
unibits "🌫 Idiosyncrätic ℜսᖯʏ"
|
27
|
+
```
|
28
|
+
|
29
|
+
### Options
|
30
|
+
|
31
|
+
`unibits` takes three optional options:
|
32
|
+
|
33
|
+
- *encoding (e)*: The encoding of the given string (uses your default encoding if none given)
|
34
|
+
- *convert (c)*: An encoding the string should be converted to before visualizing it
|
35
|
+
- *stats*: Whether to show a short stats header (default: `true`), you can deactivate on the CLI with `--no-stats`
|
36
|
+
|
37
|
+
**Please note**: This uses Ruby's built-in encoding support. Currently, only strings with valid encodings are supported.
|
38
|
+
|
39
|
+
## Output for Different Encodings
|
40
|
+
### UTF-8
|
41
|
+
|
42
|
+
CLI: `$ unibits -e utf-8 -c utf-8 "🌫 Idiosyncrätic ℜսᖯʏ"`
|
43
|
+
|
44
|
+
Ruby: `unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-8'`
|
45
|
+
|
46
|
+
![Screenshot UTF-8](/screenshots/utf-8.png?raw=true "UTF-8")
|
47
|
+
|
48
|
+
### UTF-16LE
|
49
|
+
|
50
|
+
CLI: `$ unibits -e utf-8 -c utf-8 "🌫 Idiosyncrätic ℜսᖯʏ"`
|
51
|
+
|
52
|
+
Ruby: `unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-8'`
|
53
|
+
|
54
|
+
![Screenshot UTF-16LE](/screenshots/utf-16le.png?raw=true "UTF-16LE")
|
55
|
+
|
56
|
+
### UTF-16BE
|
57
|
+
|
58
|
+
CLI: `$ unibits -e utf-8 -c utf-8 "🌫 Idiosyncrätic ℜսᖯʏ"`
|
59
|
+
|
60
|
+
Ruby: `unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-8'`
|
61
|
+
|
62
|
+
![Screenshot UTF-16BE](/screenshots/utf-16be.png?raw=true "UTF-16BE")
|
63
|
+
|
64
|
+
### UTF-32LE
|
65
|
+
|
66
|
+
CLI: `$ unibits -e utf-8 -c utf-8 "🌫 Idiosyncrätic ℜսᖯʏ"`
|
67
|
+
|
68
|
+
Ruby: `unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-8'`
|
69
|
+
|
70
|
+
![Screenshot UTF-32LE](/screenshots/utf-32le.png?raw=true "UTF-32LE")
|
71
|
+
|
72
|
+
### UTF-32BE
|
73
|
+
|
74
|
+
CLI: `$ unibits -e utf-8 -c utf-8 "🌫 Idiosyncrätic ℜսᖯʏ"`
|
75
|
+
|
76
|
+
Ruby: `unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'utf-8', convert: 'utf-8'`
|
77
|
+
|
78
|
+
![Screenshot UTF-32BE](/screenshots/utf-32be.png?raw=true "UTF-32BE")
|
79
|
+
|
80
|
+
### BINARY
|
81
|
+
|
82
|
+
CLI: `$ unibits -e binary "🌫 Idiosyncrätic ℜսᖯʏ"`
|
83
|
+
|
84
|
+
Ruby: `unibits "🌫 Idiosyncrätic ℜսᖯʏ", encoding: 'binary'`
|
85
|
+
|
86
|
+
![Screenshot BINARY](/screenshots/binary.png?raw=true "BINARY")
|
87
|
+
|
88
|
+
### ASCII
|
89
|
+
|
90
|
+
CLI: `$ unibits -e utf-8 -c ascii "ascii"`
|
91
|
+
|
92
|
+
Ruby: `unibits "ASCII String", encoding: 'utf-8', convert: 'ascii'`
|
93
|
+
|
94
|
+
![Screenshot ASCII](/screenshots/ascii.png?raw=true "ASCII")
|
95
|
+
|
96
|
+
## Notes
|
97
|
+
|
98
|
+
Also see
|
99
|
+
|
100
|
+
- [UTF-8 (Wikipedia)](https://en.wikipedia.org/wiki/UTF-8#Description)
|
101
|
+
- [UTF-16 (Wikipedia)](https://en.wikipedia.org/wiki/UTF-16#Description)
|
102
|
+
- [UTF-32 (Wikipedia)](https://en.wikipedia.org/wiki/UTF-32)
|
103
|
+
- [Ruby's Encoding class](https://ruby-doc.org/core/Encoding.html)
|
104
|
+
- [Unicode Micro Libraries for Ruby](https://github.com/janlelis/unicode-x)
|
105
|
+
|
106
|
+
Lots of thanks to @damienklinnert for the motivation and inspiration required to build this! 🎆
|
107
|
+
|
108
|
+
Copyright (C) 2017 Jan Lelis <http://janlelis.com>. Released under the MIT license.
|
data/Rakefile
ADDED
@@ -0,0 +1,38 @@
|
|
1
|
+
# # #
|
2
|
+
# Get gemspec info
|
3
|
+
|
4
|
+
gemspec_file = Dir['*.gemspec'].first
|
5
|
+
gemspec = eval File.read(gemspec_file), binding, gemspec_file
|
6
|
+
info = "#{gemspec.name} | #{gemspec.version} | " \
|
7
|
+
"#{gemspec.runtime_dependencies.size} dependencies | " \
|
8
|
+
"#{gemspec.files.size} files"
|
9
|
+
|
10
|
+
# # #
|
11
|
+
# Gem build and install task
|
12
|
+
|
13
|
+
desc info
|
14
|
+
task :gem do
|
15
|
+
puts info + "\n\n"
|
16
|
+
print " "; sh "gem build #{gemspec_file}"
|
17
|
+
FileUtils.mkdir_p 'pkg'
|
18
|
+
FileUtils.mv "#{gemspec.name}-#{gemspec.version}.gem", 'pkg'
|
19
|
+
puts; sh %{gem install --no-document pkg/#{gemspec.name}-#{gemspec.version}.gem}
|
20
|
+
end
|
21
|
+
|
22
|
+
# # #
|
23
|
+
# Start an IRB session with the gem loaded
|
24
|
+
|
25
|
+
desc "#{gemspec.name} | IRB"
|
26
|
+
task :irb do
|
27
|
+
sh "irb -I ./lib -r #{gemspec.name.gsub '-','/'}"
|
28
|
+
end
|
29
|
+
|
30
|
+
# # #
|
31
|
+
# Run specs
|
32
|
+
|
33
|
+
desc "#{gemspec.name} | Spec"
|
34
|
+
task :spec do
|
35
|
+
sh "for file in spec/*_spec.rb; do ruby $file; done"
|
36
|
+
end
|
37
|
+
task default: :spec
|
38
|
+
|
data/bin/unibits
ADDED
@@ -0,0 +1,58 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
require "rationalist"
|
4
|
+
require "unibits"
|
5
|
+
|
6
|
+
argv = Rationalist.parse(
|
7
|
+
ARGV,
|
8
|
+
string: '_',
|
9
|
+
alias: {
|
10
|
+
e: 'encoding',
|
11
|
+
c: 'convert',
|
12
|
+
v: 'version',
|
13
|
+
},
|
14
|
+
boolean: [
|
15
|
+
'help',
|
16
|
+
'version',
|
17
|
+
'stats'
|
18
|
+
],
|
19
|
+
default: {
|
20
|
+
stats: true
|
21
|
+
},
|
22
|
+
)
|
23
|
+
|
24
|
+
if argv[:version]
|
25
|
+
puts Unibits::VERSION
|
26
|
+
exit(0)
|
27
|
+
end
|
28
|
+
|
29
|
+
if argv[:help]
|
30
|
+
puts <<-HELP
|
31
|
+
|
32
|
+
#{Paint["USAGE", :underline]}
|
33
|
+
|
34
|
+
#{Paint["unibits", :bold]} [--encoding <encoding>] [--convert <encoding>] [--no-stats] data
|
35
|
+
|
36
|
+
Supported encodings: #{Unibits::SUPPORTED_ENCODINGS.join(', ')}
|
37
|
+
Explanation of stats: bytes / codepoints / glyphs / expected terminal width
|
38
|
+
|
39
|
+
More info and examples at: https://github.com/janlelis/unibits
|
40
|
+
|
41
|
+
HELP
|
42
|
+
exit(0)
|
43
|
+
end
|
44
|
+
|
45
|
+
if argv[:_] && argv[:_][0]
|
46
|
+
data = argv[:_][0]
|
47
|
+
elsif !$stdin.tty?
|
48
|
+
data = $stdin.read
|
49
|
+
else
|
50
|
+
data = nil
|
51
|
+
end
|
52
|
+
|
53
|
+
begin
|
54
|
+
Unibits.of(data, encoding: argv[:encoding], convert: argv[:convert], stats: argv[:stats])
|
55
|
+
rescue ArgumentError
|
56
|
+
$stderr.puts Paint[$!.message, :red]
|
57
|
+
exit(1)
|
58
|
+
end
|
data/lib/unibits.rb
ADDED
@@ -0,0 +1,174 @@
|
|
1
|
+
require_relative "unibits/version"
|
2
|
+
|
3
|
+
require "io/console"
|
4
|
+
require "paint"
|
5
|
+
require "unicode/display_width"
|
6
|
+
|
7
|
+
module Unibits
|
8
|
+
SUPPORTED_ENCODINGS = [
|
9
|
+
'UTF-8',
|
10
|
+
'UTF-16LE',
|
11
|
+
'UTF-16BE',
|
12
|
+
'UTF-32LE',
|
13
|
+
'UTF-32BE',
|
14
|
+
'ASCII-8BIT',
|
15
|
+
'US-ASCII',
|
16
|
+
].freeze
|
17
|
+
|
18
|
+
def self.of(string, encoding: nil, convert: nil, stats: true)
|
19
|
+
if !string || string.empty?
|
20
|
+
raise ArgumentError, "no data given to unibits"
|
21
|
+
end
|
22
|
+
|
23
|
+
string = string.dup.force_encoding(encoding) if encoding
|
24
|
+
string = string.encode(convert) if convert
|
25
|
+
|
26
|
+
case string.encoding.name
|
27
|
+
when *SUPPORTED_ENCODINGS
|
28
|
+
unless string.valid_encoding?
|
29
|
+
raise ArgumentError, "the data given is not valid #{string.encoding.name}"
|
30
|
+
end
|
31
|
+
|
32
|
+
puts stats(string) if stats
|
33
|
+
puts visualize(string)
|
34
|
+
when 'UTF-16', 'UTF-32'
|
35
|
+
raise ArgumentError, "unibits only supports #{string.encoding.name} with specified endianess, please use #{string.encoding.name}LE or #{string.encoding.name}BE"
|
36
|
+
else
|
37
|
+
raise ArgumentError, "unibits does not support strings of encoding #{string.encoding}"
|
38
|
+
end
|
39
|
+
end
|
40
|
+
|
41
|
+
def self.stats(string)
|
42
|
+
"\n " \
|
43
|
+
"#{Paint[string.encoding.name, :bold]} (" \
|
44
|
+
"#{string.bytesize}/" \
|
45
|
+
"#{string.size}/" \
|
46
|
+
"#{string.scan(Regexp.compile('\X'.encode(string.encoding))).size}/" \
|
47
|
+
"#{Unicode::DisplayWidth.of(string)})"
|
48
|
+
end
|
49
|
+
|
50
|
+
def self.visualize(string)
|
51
|
+
cols = determine_terminal_cols
|
52
|
+
|
53
|
+
cp_buffer = [" "]
|
54
|
+
enc_buffer = [" "]
|
55
|
+
hex_buffer = [" "]
|
56
|
+
bin_buffer = [" "]
|
57
|
+
separator = [" "]
|
58
|
+
|
59
|
+
puts
|
60
|
+
string.each_char{ |char|
|
61
|
+
current_color = random_color
|
62
|
+
|
63
|
+
char.each_byte.with_index{ |byte, index|
|
64
|
+
if Paint.unpaint(hex_buffer[-1]).bytesize > cols - 12
|
65
|
+
cp_buffer << " "
|
66
|
+
enc_buffer << " "
|
67
|
+
hex_buffer << " "
|
68
|
+
bin_buffer << " "
|
69
|
+
separator << " "
|
70
|
+
end
|
71
|
+
|
72
|
+
if index == 0
|
73
|
+
cp_buffer[-1] << Paint[
|
74
|
+
("U+%04X" % char.ord).ljust(10), current_color, :bold
|
75
|
+
]
|
76
|
+
|
77
|
+
symbolified_char = symbolify(char)
|
78
|
+
padding = 10 - Unicode::DisplayWidth.of(symbolified_char)
|
79
|
+
|
80
|
+
enc_buffer[-1] << Paint[
|
81
|
+
symbolified_char, current_color
|
82
|
+
]
|
83
|
+
enc_buffer[-1] << " " * padding if padding > 0
|
84
|
+
else
|
85
|
+
cp_buffer[-1] << " " * 10
|
86
|
+
enc_buffer[-1] << " " * 10
|
87
|
+
end
|
88
|
+
|
89
|
+
hex_buffer[-1] << Paint[
|
90
|
+
("%02X" % byte).ljust(10, " "), current_color
|
91
|
+
]
|
92
|
+
|
93
|
+
bin_byte_complete = byte.to_s(2).rjust(8, "0")
|
94
|
+
|
95
|
+
case string.encoding.name
|
96
|
+
when 'US-ASCII'
|
97
|
+
bin_byte_1 = bin_byte_complete[0...1]
|
98
|
+
bin_byte_2 = bin_byte_complete[1...8]
|
99
|
+
when 'ASCII-8BIT'
|
100
|
+
bin_byte_1 = ""
|
101
|
+
bin_byte_2 = bin_byte_complete
|
102
|
+
when 'UTF-8'
|
103
|
+
if index == 0
|
104
|
+
if bin_byte_complete =~ /^(0|1{2,4}0)([01]+)$/
|
105
|
+
bin_byte_1 = $1
|
106
|
+
bin_byte_2 = $2
|
107
|
+
else
|
108
|
+
bin_byte_1 = ""
|
109
|
+
bin_byte_2 = bin_byte_complete
|
110
|
+
end
|
111
|
+
else
|
112
|
+
bin_byte_1 = bin_byte_complete[0...2]
|
113
|
+
bin_byte_2 = bin_byte_complete[2...8]
|
114
|
+
end
|
115
|
+
when 'UTF-16LE'
|
116
|
+
if char.ord <= 0xFFFF || index == 0 || index == 2
|
117
|
+
bin_byte_1 = ""
|
118
|
+
bin_byte_2 = bin_byte_complete
|
119
|
+
else
|
120
|
+
bin_byte_complete =~ /^(11011[01])([01]+)$/
|
121
|
+
bin_byte_1 = $1
|
122
|
+
bin_byte_2 = $2
|
123
|
+
end
|
124
|
+
when 'UTF-16BE'
|
125
|
+
if char.ord <= 0xFFFF || index == 1 || index == 3
|
126
|
+
bin_byte_1 = ""
|
127
|
+
bin_byte_2 = bin_byte_complete
|
128
|
+
else
|
129
|
+
bin_byte_complete =~ /^(11011[01])([01]+)$/
|
130
|
+
bin_byte_1 = $1
|
131
|
+
bin_byte_2 = $2
|
132
|
+
end
|
133
|
+
when 'UTF-32LE', 'UTF-32BE'
|
134
|
+
bin_byte_1 = ""
|
135
|
+
bin_byte_2 = bin_byte_complete
|
136
|
+
end
|
137
|
+
bin_buffer[-1] << Paint[
|
138
|
+
bin_byte_1, current_color
|
139
|
+
] unless !bin_byte_1 || bin_byte_1.empty?
|
140
|
+
bin_buffer[-1] << Paint[
|
141
|
+
bin_byte_2, current_color, :underline
|
142
|
+
] unless !bin_byte_2 || bin_byte_2.empty?
|
143
|
+
bin_buffer[-1] << " "
|
144
|
+
}
|
145
|
+
}
|
146
|
+
|
147
|
+
if string.encoding.name[0, 3] == "UTF"
|
148
|
+
enc_buffer.zip(cp_buffer, hex_buffer, bin_buffer, separator).flatten.join("\n")
|
149
|
+
else
|
150
|
+
enc_buffer.zip(hex_buffer, bin_buffer, separator).flatten.join("\n")
|
151
|
+
end
|
152
|
+
end
|
153
|
+
|
154
|
+
def self.random_color
|
155
|
+
"%.2x%.2x%.2x" %[rand(90) + 60, rand(90) + 60, rand(90) + 60]
|
156
|
+
end
|
157
|
+
|
158
|
+
def self.symbolify(char)
|
159
|
+
return char.inspect unless char.encoding.name[0, 3] == "UTF"
|
160
|
+
char
|
161
|
+
.tr("\x00-\x1F".encode(char.encoding), "\u{2400}-\u{241F}".encode(char.encoding))
|
162
|
+
.gsub(
|
163
|
+
Regexp.compile('[\p{Space}]'.encode(char.encoding)),
|
164
|
+
']\0['.encode(char.encoding)
|
165
|
+
)
|
166
|
+
.encode('UTF-8')
|
167
|
+
end
|
168
|
+
|
169
|
+
def self.determine_terminal_cols
|
170
|
+
STDIN.winsize[1] || 80
|
171
|
+
rescue Errno::ENOTTY
|
172
|
+
return 80
|
173
|
+
end
|
174
|
+
end
|
Binary file
|
Binary file
|
Binary file
|
Binary file
|
Binary file
|
Binary file
|
Binary file
|
@@ -0,0 +1,70 @@
|
|
1
|
+
require_relative "../lib/unibits/kernel_method"
|
2
|
+
require "minitest/autorun"
|
3
|
+
|
4
|
+
describe Unibits do
|
5
|
+
it "will not work for unsupported encodings" do
|
6
|
+
proc{
|
7
|
+
Unibits.of("string".encode("UTF-32"))
|
8
|
+
}.must_raise ArgumentError
|
9
|
+
end
|
10
|
+
|
11
|
+
describe 'Encodings' do
|
12
|
+
let(:string){ "🌫 Idiosyncrätic ℜսᖯʏ" }
|
13
|
+
|
14
|
+
it "works with UTF-8" do
|
15
|
+
result = Paint.unpaint(Unibits.visualize(string))
|
16
|
+
result.must_match "ℜ"
|
17
|
+
result.must_match "U+211C"
|
18
|
+
result.must_match /E2.*84.*9C/m
|
19
|
+
result.must_match /11100010.*10000100.*10011100/m
|
20
|
+
end
|
21
|
+
|
22
|
+
it "works with UTF-16LE" do
|
23
|
+
result = Paint.unpaint(Unibits.visualize(string.encode('UTF-16LE')))
|
24
|
+
result.must_match "ℜ"
|
25
|
+
result.must_match "U+211C"
|
26
|
+
result.must_match /1C.*21/m
|
27
|
+
result.must_match /00011100.*00100001/m
|
28
|
+
end
|
29
|
+
|
30
|
+
it "works with UTF-16BE" do
|
31
|
+
result = Paint.unpaint(Unibits.visualize(string.encode('UTF-16BE')))
|
32
|
+
result.must_match "ℜ"
|
33
|
+
result.must_match "U+211C"
|
34
|
+
result.must_match /21.*1C/m
|
35
|
+
result.must_match /00100001.*00011100/m
|
36
|
+
end
|
37
|
+
|
38
|
+
it "works with UTF-32LE" do
|
39
|
+
result = Paint.unpaint(Unibits.visualize(string.encode('UTF-32LE')))
|
40
|
+
result.must_match "ℜ"
|
41
|
+
result.must_match "U+211C"
|
42
|
+
result.must_match /1C.*21.*00.*00/m
|
43
|
+
result.must_match /00011100.*00100001.*00000000.*00000000/m
|
44
|
+
end
|
45
|
+
|
46
|
+
it "works with UTF-32BE" do
|
47
|
+
result = Paint.unpaint(Unibits.visualize(string.encode('UTF-32BE')))
|
48
|
+
result.must_match "ℜ"
|
49
|
+
result.must_match "U+211C"
|
50
|
+
result.must_match /00.*00.*21.*1C/m
|
51
|
+
result.must_match /00000000.*00000000.*00100001.*00011100/m
|
52
|
+
end
|
53
|
+
|
54
|
+
it "works with BINARY" do
|
55
|
+
result = Paint.unpaint(Unibits.visualize(string.dup.force_encoding('BINARY')))
|
56
|
+
# testing for the UTF-8 encoded "ℜ"
|
57
|
+
result.must_match "\\xE2"
|
58
|
+
result.must_match "\\x84"
|
59
|
+
result.must_match "\\x9C"
|
60
|
+
result.must_match /11100010.*10000100.*10011100/m
|
61
|
+
end
|
62
|
+
|
63
|
+
it "works with ASCII" do
|
64
|
+
result = Paint.unpaint(Unibits.visualize("ASCII string".force_encoding('ASCII')))
|
65
|
+
result.must_match "C"
|
66
|
+
result.must_match "43"
|
67
|
+
result.must_match "01000011"
|
68
|
+
end
|
69
|
+
end
|
70
|
+
end
|
data/unibits.gemspec
ADDED
@@ -0,0 +1,25 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
|
3
|
+
require File.dirname(__FILE__) + "/lib/unibits/version"
|
4
|
+
|
5
|
+
Gem::Specification.new do |gem|
|
6
|
+
gem.name = "unibits"
|
7
|
+
gem.version = Unibits::VERSION
|
8
|
+
gem.summary = "Visualizes Unicode encodings."
|
9
|
+
gem.description = "Visualizes Unicode encodings in the terminal. Supports UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE, US-ASCII, and ASCII-8BIT. Comes as CLI command and as Ruby Kernel method."
|
10
|
+
gem.authors = ["Jan Lelis"]
|
11
|
+
gem.email = ["mail@janlelis.de"]
|
12
|
+
gem.homepage = "https://github.com/janlelis/unibits"
|
13
|
+
gem.license = "MIT"
|
14
|
+
|
15
|
+
gem.files = Dir["{**/}{.*,*}"].select{ |path| File.file?(path) && path !~ /^pkg/ }
|
16
|
+
gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
|
17
|
+
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
18
|
+
gem.require_paths = ["lib"]
|
19
|
+
|
20
|
+
gem.add_dependency 'paint', '~> 2.0'
|
21
|
+
gem.add_dependency 'unicode-display_width', '~> 1.1'
|
22
|
+
gem.add_dependency 'rationalist', '~> 2.0'
|
23
|
+
|
24
|
+
gem.required_ruby_version = "~> 2.0"
|
25
|
+
end
|
metadata
ADDED
@@ -0,0 +1,112 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: unibits
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.0.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Jan Lelis
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2017-03-05 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: paint
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '2.0'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '2.0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: unicode-display_width
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '1.1'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '1.1'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rationalist
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '2.0'
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '2.0'
|
55
|
+
description: Visualizes Unicode encodings in the terminal. Supports UTF-8, UTF-16LE,
|
56
|
+
UTF-16BE, UTF-32LE, UTF-32BE, US-ASCII, and ASCII-8BIT. Comes as CLI command and
|
57
|
+
as Ruby Kernel method.
|
58
|
+
email:
|
59
|
+
- mail@janlelis.de
|
60
|
+
executables:
|
61
|
+
- unibits
|
62
|
+
extensions: []
|
63
|
+
extra_rdoc_files: []
|
64
|
+
files:
|
65
|
+
- ".gitignore"
|
66
|
+
- ".travis.yml"
|
67
|
+
- CHANGELOG.md
|
68
|
+
- CODE_OF_CONDUCT.md
|
69
|
+
- Gemfile
|
70
|
+
- Gemfile.lock
|
71
|
+
- MIT-LICENSE.txt
|
72
|
+
- README.md
|
73
|
+
- Rakefile
|
74
|
+
- bin/unibits
|
75
|
+
- lib/unibits.rb
|
76
|
+
- lib/unibits/kernel_method.rb
|
77
|
+
- lib/unibits/version.rb
|
78
|
+
- screenshots/ascii.png
|
79
|
+
- screenshots/binary.png
|
80
|
+
- screenshots/utf-16be.png
|
81
|
+
- screenshots/utf-16le.png
|
82
|
+
- screenshots/utf-32be.png
|
83
|
+
- screenshots/utf-32le.png
|
84
|
+
- screenshots/utf-8.png
|
85
|
+
- spec/unibits_spec.rb
|
86
|
+
- unibits.gemspec
|
87
|
+
homepage: https://github.com/janlelis/unibits
|
88
|
+
licenses:
|
89
|
+
- MIT
|
90
|
+
metadata: {}
|
91
|
+
post_install_message:
|
92
|
+
rdoc_options: []
|
93
|
+
require_paths:
|
94
|
+
- lib
|
95
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
96
|
+
requirements:
|
97
|
+
- - "~>"
|
98
|
+
- !ruby/object:Gem::Version
|
99
|
+
version: '2.0'
|
100
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
101
|
+
requirements:
|
102
|
+
- - ">="
|
103
|
+
- !ruby/object:Gem::Version
|
104
|
+
version: '0'
|
105
|
+
requirements: []
|
106
|
+
rubyforge_project:
|
107
|
+
rubygems_version: 2.6.8
|
108
|
+
signing_key:
|
109
|
+
specification_version: 4
|
110
|
+
summary: Visualizes Unicode encodings.
|
111
|
+
test_files:
|
112
|
+
- spec/unibits_spec.rb
|