lz_string 0.2.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 32266dd809c6410d9b821eca33a4f6a690873a8e2911e9f402db6c796345a643
4
- data.tar.gz: f658e1872ab70b804e30d39ec0704f65495fad99da03cb5825f2bccfe193e652
3
+ metadata.gz: 2fbe7383cf742da0cfee289795f4f51e5b686db961520ef3d99fca2212182281
4
+ data.tar.gz: 0be89c68f03f6253f24571b08c153df6ac7767b70f4190f78688d3a2110abe00
5
5
  SHA512:
6
- metadata.gz: 3d2d77da7df4f32ae0d50bd58e38b491950c2e48a4b10f1bff5b2f6a30dc3bff292e821407feaaf5ca964acf1dcc78543fb7a8f7275feb64d058b89a3ae8b928
7
- data.tar.gz: 9e3850db9f8b4557e9fa9b5aa60770926f1f1e71f91d478cdf8c4c04dd5144081ccce636c564c9e5013215988de1cd5b11bb0483a087e7491a1ff9e9922f2647
6
+ metadata.gz: 155134bf111c17d8f874c09450b68d250e4232edb619a89c6fd6b0d14c26d4e73904a5f5f25ad996df170a863d70f8dadc625c08027685560e109cce5aa8e658
7
+ data.tar.gz: 5071d19377af3302458676e5f74b488b0b63c1fb612bdc5fd374210e30c73054002800a1f211a5cd5a7e811838010783ee46164d9d8d6b86c3559b72a13fd8c2
data/CHANGELOG.md ADDED
@@ -0,0 +1,55 @@
1
+ # Changelog
2
+
3
+ ## 1.0.0 (2026-10-16)
4
+
5
+ Now it gives the same results as JavaScript lz-string for every input, emoji too
6
+ ([#3](https://github.com/altivi/lz_string/issues/3)), and it passes the test suite of lz-string itself.
7
+
8
+ ### Fixed
9
+
10
+ - Emoji and all other characters. Like JS, compress and decompress now work on UTF-16 code units.
11
+ Before, emoji gave wrong results, `nil` or a `RangeError`.
12
+ - `decompress` always returns UTF-8. Before, it could return a binary string (`"\xF6"` for `"ö"`).
13
+ - Text with `__proto__` in it compresses like in JS.
14
+ - `compress("")` gives what JS gives (`"Q==="` in Base64), not `""`.
15
+ - `LZString.decompress("")` returns `nil`, like JS.
16
+ - `UriSafe.decompress` no longer changes the string you give it, and works with frozen strings.
17
+ - Faster: on 100 KB of text, Base64 compress takes 0.03 s instead of 0.10 s, and decompress 0.008 s instead of 0.12 s.
18
+
19
+ ### Added
20
+
21
+ - `LZString::Uint8Array`: compress to bytes and back (`compressToUint8Array` in JS).
22
+ - `LZString::Custom`: compress with your own alphabet (`compressToCustom`, new in lz-string 2).
23
+ - The JS function names: `LZString.compress_to_base64`, `LZString.decompress_from_base64` and all the others.
24
+ - `decompress_to_utf16le` for every format: the exact JS result, which can hold a lone surrogate.
25
+ - The `lz_string` command line tool, with the same options as the one of lz-string 2.
26
+
27
+ ### Changed (this can break your code)
28
+
29
+ - `LZString.compress` (raw) returns a UTF-16LE string. Like in JS, its result often has lone surrogates,
30
+ and UTF-8 can't hold them. Before, it returned UTF-8 with `"�"` in it, which could not be decompressed.
31
+ The Base64, URI-safe and UTF-16 results are UTF-8, as before.
32
+ - Data that starts with code 3 (lz-string never writes that) raises `ArgumentError`, like lz-string 2.
33
+ - A binary (ASCII-8BIT) string with bytes above 127 raises `Encoding::UndefinedConversionError`.
34
+ Before, its bytes were read as Latin-1. Use `force_encoding` to say what the bytes are.
35
+ - It needs Ruby 2.7 or newer.
36
+
37
+ ## 0.3.0 (2025-04-14)
38
+
39
+ - Better Unicode handling. Emoji still did not work (fixed in 1.0.0).
40
+
41
+ ## 0.2.0 (2023-06-13)
42
+
43
+ - URI-safe compression: `LZString::UriSafe` ([#2](https://github.com/altivi/lz_string/pull/2)).
44
+
45
+ ## 0.1.2 (2020-01-27)
46
+
47
+ - Updated dependencies.
48
+
49
+ ## 0.1.1 (2019-12-05)
50
+
51
+ - UTF-16 and Base64 compression.
52
+
53
+ ## 0.1.0 (2019-12-05)
54
+
55
+ - First release: raw compression.
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2019 Alexander Tishchenko
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md CHANGED
@@ -1,67 +1,145 @@
1
- # lz-string
2
- Ruby implementation of [LZ-String](https://github.com/pieroxy/lz-string) compression algorithm.
1
+ # lz_string
3
2
 
4
- Supports:
5
- * Raw compression
6
- * UTF-16 compression
7
- * Base64 compression
3
+ [![Gem Version](https://img.shields.io/gem/v/lz_string?logo=rubygems&logoColor=white)](https://rubygems.org/gems/lz_string)
4
+ [![Downloads](https://img.shields.io/gem/dt/lz_string?logo=rubygems&logoColor=white&label=downloads)](https://rubygems.org/gems/lz_string)
5
+ [![Downloads of the latest version](https://img.shields.io/gem/dtv/lz_string?logo=rubygems&logoColor=white&label=downloads%40latest)](https://rubygems.org/gems/lz_string)
6
+ [![Tests](https://github.com/altivi/lz_string/actions/workflows/test.yml/badge.svg)](https://github.com/altivi/lz_string/actions/workflows/test.yml)
7
+ [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
8
8
 
9
- ### Installation
9
+ Ruby port of [lz-string](https://github.com/pieroxy/lz-string), the LZ-based compression for JavaScript.
10
10
 
11
- Install the latest release:
11
+ It gives **the same results as the JavaScript library, for every input**: any text, emoji too, and even broken data.
12
+ So you can compress in the browser and decompress in Ruby, or the other way round.
12
13
 
13
- ```
14
+ - **Every format of lz-string:** raw, Base64, URI-safe, UTF-16, Uint8Array and your own alphabet.
15
+ - **Tested against JavaScript:** it passes [the test suite of lz-string itself](https://github.com/pieroxy/lz-string/tree/master/test)
16
+ (70 of 70 checks), and its specs check it against results of the JavaScript library.
17
+ - **The same command line tool** as lz-string 2, with the same options.
18
+ - No dependencies. Ruby 2.7 or newer.
19
+
20
+ ## Install
21
+
22
+ ```console
14
23
  $ gem install lz_string
15
24
  ```
16
25
 
17
- In Rails, add it to your Gemfile:
26
+ Or add it to your Gemfile:
18
27
 
19
28
  ```ruby
20
- gem 'lz_string'
29
+ gem "lz_string"
21
30
  ```
22
31
 
23
- ### How to use
32
+ ## Use
24
33
 
25
- #### Normal Compression and Decompression:
34
+ ```ruby
35
+ require "lz_string"
26
36
 
27
- ``` ruby
28
- # Compress
29
- compressed = LZString.compress("Hello world!")
30
- => "҅〶惶@✰Ӏ葀"
31
-
32
- # Decompress
33
- LZString.decompress(compressed)
34
- => "Hello world!"
37
+ compressed = LZString.compress_to_base64("Hello, 🎶 world!")
38
+ # => "BIUwNmD2A0AEg8G4Nv3YHdICcwBMCEQ="
39
+ LZString.decompress_from_base64(compressed)
40
+ # => "Hello, 🎶 world!"
35
41
  ```
36
42
 
37
- #### UTF-16 Compression and Decompression:
38
-
39
- ``` ruby
40
- some_string = '{"some": "json", "foo": [{"bar": "؋", "key": "؄"}], "ঞᕠ": "൱ඵቜ"}'
41
-
42
- # Compress
43
- compressed = LZString::UTF16.compress(some_string)
44
- => "ᯡࡓ䈌\u0B80匰ᜠр\u0AF2Ǹ䀺㈦イ\u0530්C¦¼䒨ᨬිnj〩痐࠸С㸢璑Ч䲤U⋴ҕ䈥㛢ĉ႙ "
45
-
46
- # Decompress
47
- LZString::UTF16.decompress(compressed)
48
- => "{\"some\": \"json\", \"foo\": [{\"bar\": \"؋\", \"key\": \"؄\"}], \"ঞᕠ\": \"൱ඵቜ\"}"
43
+ The methods have the same names as in JavaScript:
44
+
45
+ | JavaScript | Ruby | Gives |
46
+ |-----------------------------------------------------------------------|---------------------------------------------------------------------------------------|-------------------------------------|
47
+ | `compressToBase64` / `decompressFromBase64` | `LZString.compress_to_base64` / `LZString.decompress_from_base64` | Base64 |
48
+ | `compressToEncodedURIComponent` / `decompressFromEncodedURIComponent` | `LZString.compress_to_encoded_uri_component` / `.decompress_from_encoded_uri_component` | text that is safe in a URL |
49
+ | `compressToUTF16` / `decompressFromUTF16` | `LZString.compress_to_utf16` / `.decompress_from_utf16` | valid UTF-16 text, for localStorage |
50
+ | `compressToUint8Array` / `decompressFromUint8Array` | `LZString.compress_to_uint8_array` / `.decompress_from_uint8_array` | bytes |
51
+ | `compressToCustom` / `decompressFromCustom` | `LZString.compress_to_custom(text, alphabet)` / `.decompress_from_custom(data, alphabet)` | text from your own alphabet |
52
+ | `compress` / `decompress` | `LZString.compress` / `LZString.decompress` | raw: a UTF-16LE string (see below) |
53
+
54
+ Each format also has its own class, as in older versions of this gem: `LZString::Base64`, `LZString::UriSafe`,
55
+ `LZString::UTF16`, `LZString::Uint8Array` and `LZString::Custom`. Each has `compress` and `decompress`.
56
+
57
+ ```ruby
58
+ LZString::UriSafe.compress('{"page": 2, "search": "café ☕"}')
59
+ # => "N4IgDghg5gpiBcACATAGkSAzjCAnAxgBYIb4QBmAl4oKhkIAvkA"
60
+
61
+ LZString.compress_to_uint8_array("🎶")
62
+ # => "\x8F\x06\xE3o\xDA\x00"
63
+ LZString.decompress_from_uint8_array([143, 6, 227, 111, 218, 0])
64
+ # => "🎶"
65
+
66
+ LZString.compress_to_custom("Hello world!", "0123456789abcdef")
67
+ # => "0485303660f60040ee90273004c08440"
49
68
  ```
50
69
 
51
- #### Base64 Compression and Decompression:
70
+ ### Strings in and out
71
+
72
+ - **In:** any Ruby string. It is read as text in its own encoding (UTF-8, Latin-1, …), like a JavaScript string.
73
+ - **Out of `decompress`:** a UTF-8 string, or `nil` when JavaScript gives `null` (broken data).
74
+ - **Raw `compress`** gives a UTF-16LE string. Like in JavaScript, this result is often not valid UTF-16
75
+ (it has lone surrogates), and UTF-8 can't hold it. To store or send it, use one of the other formats.
76
+ - **Uint8Array** gives a binary string. `decompress_from_uint8_array` takes a binary string or a list of numbers.
77
+ - A JavaScript string can hold half of an emoji (a lone surrogate). UTF-8 can't, so `decompress` writes `"�"` for it.
78
+ If you need the result exactly as JavaScript has it, use `decompress_to_utf16le` (every format has it).
79
+
80
+ ## Command line
81
+
82
+ ```console
83
+ $ lz_string -h
84
+ Usage: lz_string [options] [input-file]
85
+
86
+ Use lz-string to compress or decompress a file
52
87
 
53
- ``` ruby
54
- # Compress
55
- compressed = LZString::Base64.compress("Hello world!")
56
- => "BIUwNmD2AEDukCcwBMCEQ==="
57
-
58
- # Decompress
59
- LZString::Base64.decompress(compressed)
60
- => "Hello world!"
88
+ Arguments:
89
+ input-file file to process, if no file then read from stdin
90
+
91
+ Options:
92
+ -V, --version output the version number
93
+ -d, --decompress if unset then this will compress
94
+ -e, --encoder TYPE character encoding to use (choices: base64, custom, encodeduri, raw, uint8array, utf16, default: raw)
95
+ -c, --custom DICTIONARY dictionary for custom encoder
96
+ -v, --verify verify before returning
97
+ -o, --output OUTPUT_FILE output file, otherwise write to stdout
98
+ -q, --quiet don't print any error messages
99
+ -h, --help display help for command
100
+
101
+ $ lz_string -e base64 photo.jpg -o photo.lz
102
+ $ lz_string -d -e base64 photo.lz -o photo.jpg
61
103
  ```
62
-
63
- ### Tests
64
104
 
65
- ``` bash
105
+ Like the lz-string 2 tool, it works on the bytes of a file (2 bytes are one UTF-16 code unit), not on text.
106
+ So it can compress any file and give it back the same. It writes the same bytes as the JavaScript tool.
107
+
108
+ ### Check it with the tests of lz-string
109
+
110
+ ```console
111
+ $ git clone https://github.com/pieroxy/lz-string && cd lz-string/test
112
+ $ cat > profiles/lz_string.sh <<'EOF'
113
+ optDescription="The lz_string Ruby gem"
114
+ optCommand=lz_string
115
+ optDecompress="-d"
116
+ optEncoder="-e"
117
+ optEncoders="base64,encodeduri,raw,uint8array,utf16"
118
+ optOutput="-o"
119
+ optQuiet="-q"
120
+ EOF
121
+ $ ./test.sh lz_string
122
+ ```
123
+
124
+ ## Upgrade from 0.x
125
+
126
+ Emoji now work, and all results are the same as in JavaScript. Some things changed, see the [changelog](CHANGELOG.md):
127
+
128
+ - `LZString.compress` (raw) returns a UTF-16LE string, not UTF-8.
129
+ - `LZString.decompress("")` returns `nil`, like in JavaScript.
130
+ - Data that starts with code 3 (lz-string never writes that) raises `ArgumentError`, like lz-string 2.
131
+ - A binary string with bytes above 127 raises an error. Use `force_encoding` to say what the bytes are.
132
+
133
+ ## Development
134
+
135
+ ```console
136
+ $ bundle install
66
137
  $ rake
67
138
  ```
139
+
140
+ The specs compare the gem with results of the JavaScript library, kept in `spec/fixtures/js_results.json`.
141
+ To make that file again, see `spec/fixtures/js_results.js`.
142
+
143
+ ## License
144
+
145
+ MIT
data/exe/lz_string ADDED
@@ -0,0 +1,4 @@
1
+ #!/usr/bin/env ruby
2
+ require "lz_string/cli"
3
+
4
+ exit LZString::CLI.run(ARGV)
@@ -1,393 +1,206 @@
1
1
  module LZString
2
2
  # Base compression class.
3
+ #
4
+ # A port of lz-string (JavaScript) that gives the same results as lz-string 2, and as 1.5.0 for
5
+ # all real data. JS strings are made of UTF-16 code units, so the work here is done on code units
6
+ # too: an emoji like "🎶" is two units.
3
7
  class Base
8
+ # JS keeps the dictionary in a plain object, and an object can't store the key "__proto__".
9
+ # So JS never finds that word in its dictionary, and we must not find it either.
10
+ PROTO = "__proto__".encode(Encoding::UTF_16LE).b.freeze
11
+
12
+ # @param string [String]
13
+ # @return [Array<Integer>] the UTF-16 code units, like charCodeAt in JS
14
+ def self.code_units(string)
15
+ string.encode(Encoding::UTF_16LE).unpack("v*")
16
+ end
17
+
18
+ # UTF-8 can't hold a lone surrogate. Like JS TextEncoder, we write "�" for it.
19
+ #
20
+ # @param string [String, nil] UTF-16LE
21
+ # @return [String, nil] UTF-8
22
+ def self.to_utf8(string)
23
+ string && string.encode(Encoding::UTF_8, invalid: :replace)
24
+ end
25
+
4
26
  # @param uncompressed [String]
5
27
  # @param bits_per_char [Integer]
6
- # @param get_char_from_int [Integer]
28
+ # @param get_char_from_int [Proc]
7
29
  def self.compress(uncompressed, bits_per_char, get_char_from_int)
8
30
  return "" if uncompressed.nil?
9
31
 
10
- i, value, ii = nil
11
- context_dictionary = {}
12
- context_dictionary_to_create = {}
13
- context_c = ""
14
- context_wc = ""
15
- context_w = ""
32
+ dictionary = {}
33
+ dictionary_to_create = {}
34
+ w = "".b
16
35
  # Compensate for the first entry which should not count
17
- context_enlarge_in = 2
18
- context_dict_size = 3
19
- context_num_bits = 2
20
- context_data = []
21
- context_data_val = 0
22
- context_data_position = 0
23
-
24
- for ii in 0...uncompressed.length do
25
- context_c = uncompressed[ii]
26
-
27
- if (!context_dictionary.has_key?(context_c))
28
- context_dictionary[context_c] = context_dict_size
29
- context_dictionary_to_create[context_c] = true
30
- context_dict_size += 1
31
- end
32
-
33
- context_wc = context_w + context_c
34
- if (context_dictionary.has_key?(context_wc))
35
- context_w = context_wc
36
- else
37
- if (context_dictionary_to_create.has_key?(context_w))
38
- if (context_w[0].ord < 256)
39
- for i in 0...context_num_bits do
40
- context_data_val = (context_data_val << 1)
41
- if (context_data_position == bits_per_char - 1)
42
- context_data_position = 0
43
- context_data.push(get_char_from_int[context_data_val])
44
- context_data_val = 0
45
- else
46
- context_data_position += 1
47
- end
48
- end
49
-
50
- value = context_w[0].ord
51
- for i in 0...8 do
52
- context_data_val = (context_data_val << 1) | (value & 1)
53
- if (context_data_position == bits_per_char - 1)
54
- context_data_position = 0
55
- context_data.push(get_char_from_int[context_data_val])
56
- context_data_val = 0
57
- else
58
- context_data_position += 1
59
- end
60
- value = value >> 1
61
- end
62
- else
63
- value = 1
64
- for i in 0...context_num_bits do
65
- context_data_val = (context_data_val << 1) | value
66
- if (context_data_position == bits_per_char-1)
67
- context_data_position = 0
68
- context_data.push(get_char_from_int[context_data_val])
69
- context_data_val = 0
70
- else
71
- context_data_position += 1
72
- end
73
- value = 0
74
- end
75
- value = context_w[0].ord
76
- for i in 0...16 do
77
- context_data_val = (context_data_val << 1) | (value & 1)
78
- if (context_data_position == bits_per_char - 1)
79
- context_data_position = 0
80
- context_data.push(get_char_from_int[context_data_val])
81
- context_data_val = 0
82
- else
83
- context_data_position += 1
84
- end
85
- value = value >> 1
86
- end
87
- end
88
- context_enlarge_in -= 1
89
- if (context_enlarge_in == 0)
90
- context_enlarge_in = 2**context_num_bits
91
- context_num_bits += 1
92
- end
93
- context_dictionary_to_create.delete(context_w)
36
+ enlarge_in = 2
37
+ dict_size = 3
38
+ num_bits = 2
39
+ data = []
40
+ data_val = 0
41
+ data_position = 0
42
+
43
+ # Writes the lowest `count` bits of `value`, lowest bit first.
44
+ write_bits = lambda do |value, count|
45
+ count.times do
46
+ data_val = (data_val << 1) | (value & 1)
47
+ if data_position == bits_per_char - 1
48
+ data_position = 0
49
+ data << get_char_from_int.call(data_val)
50
+ data_val = 0
94
51
  else
95
- value = context_dictionary[context_w]
96
- for i in 0...context_num_bits do
97
- context_data_val = (context_data_val << 1) | (value & 1)
98
- if (context_data_position == bits_per_char - 1)
99
- context_data_position = 0
100
- context_data.push(get_char_from_int[context_data_val])
101
- context_data_val = 0
102
- else
103
- context_data_position += 1
104
- end
105
- value = value >> 1
106
- end
107
- end
108
-
109
- context_enlarge_in -= 1
110
-
111
- if (context_enlarge_in == 0)
112
- context_enlarge_in = 2**context_num_bits
113
- context_num_bits += 1
52
+ data_position += 1
114
53
  end
115
-
116
- # Add wc to the dictionary.
117
- context_dictionary[context_wc] = context_dict_size
118
- context_dict_size += 1
119
- context_w = context_c.to_s
54
+ value >>= 1
120
55
  end
121
56
  end
122
57
 
123
- # Output the code for w.
124
- if (context_w != "")
125
- if (context_dictionary_to_create.has_key?(context_w))
126
- if (context_w[0].ord < 256)
127
- for i in 0...context_num_bits do
128
- context_data_val = (context_data_val << 1)
129
- if (context_data_position == bits_per_char-1)
130
- context_data_position = 0
131
- context_data.push(get_char_from_int[context_data_val])
132
- context_data_val = 0
133
- else
134
- context_data_position += 1
135
- end
136
- end
137
- value = context_w[0].ord
138
- for i in 0...8 do
139
- context_data_val = (context_data_val << 1) | (value & 1)
140
- if (context_data_position == bits_per_char-1)
141
- context_data_position = 0
142
- context_data.push(get_char_from_int[context_data_val])
143
- context_data_val = 0
144
- else
145
- context_data_position += 1
146
- end
147
- value = value >> 1
148
- end
58
+ # Writes the code for w: a new character, or its place in the dictionary.
59
+ write_w = lambda do
60
+ if dictionary_to_create.key?(w)
61
+ code_unit = w.unpack1("v")
62
+ if code_unit < 256
63
+ write_bits.call(0, num_bits)
64
+ write_bits.call(code_unit, 8)
149
65
  else
150
- value = 1
151
- for i in 0...context_num_bits do
152
- context_data_val = (context_data_val << 1) | value
153
- if (context_data_position == bits_per_char-1)
154
- context_data_position = 0
155
- context_data.push(get_char_from_int[context_data_val])
156
- context_data_val = 0
157
- else
158
- context_data_position += 1
159
- end
160
- value = 0
161
- end
162
- value = context_w[0].ord
163
- for i in 0...16 do
164
- context_data_val = (context_data_val << 1) | (value & 1)
165
- if (context_data_position == bits_per_char-1)
166
- context_data_position = 0
167
- context_data.push(get_char_from_int[context_data_val])
168
- context_data_val = 0
169
- else
170
- context_data_position += 1
171
- end
172
- value = value >> 1
173
- end
66
+ write_bits.call(1, num_bits)
67
+ write_bits.call(code_unit, 16)
174
68
  end
175
- context_enlarge_in -= 1
176
- if (context_enlarge_in == 0)
177
- context_enlarge_in = 2**context_num_bits
178
- context_num_bits += 1
69
+ enlarge_in -= 1
70
+ if enlarge_in == 0
71
+ enlarge_in = 2**num_bits
72
+ num_bits += 1
179
73
  end
180
- context_dictionary_to_create.delete(context_w)
74
+ dictionary_to_create.delete(w)
181
75
  else
182
- value = context_dictionary[context_w]
183
- for i in 0...context_num_bits
184
- context_data_val = (context_data_val << 1) | (value & 1)
185
- if (context_data_position == bits_per_char-1)
186
- context_data_position = 0
187
- context_data.push(get_char_from_int[context_data_val])
188
- context_data_val = 0
189
- else
190
- context_data_position += 1
191
- end
192
- value = value >> 1
193
- end
76
+ write_bits.call(dictionary[w], num_bits)
194
77
  end
195
-
196
- context_enlarge_in -= 1
197
- if (context_enlarge_in == 0)
198
- context_enlarge_in = 2**context_num_bits
199
- context_num_bits += 1
78
+ enlarge_in -= 1
79
+ if enlarge_in == 0
80
+ enlarge_in = 2**num_bits
81
+ num_bits += 1
200
82
  end
201
83
  end
202
84
 
203
- # Mark the end of the stream
204
- value = 2
205
- for i in 0...context_num_bits do
206
- context_data_val = (context_data_val << 1) | (value & 1)
207
- if (context_data_position == bits_per_char - 1)
208
- context_data_position = 0
209
- context_data.push(get_char_from_int[context_data_val])
210
- context_data_val = 0
85
+ # Each character is one UTF-16 code unit: 2 bytes, little endian.
86
+ input = uncompressed.encode(Encoding::UTF_16LE).b
87
+ 0.step(input.bytesize - 1, 2) do |i|
88
+ c = input.byteslice(i, 2)
89
+ unless dictionary.key?(c)
90
+ dictionary[c] = dict_size
91
+ dict_size += 1
92
+ dictionary_to_create[c] = true
93
+ end
94
+
95
+ wc = w + c
96
+ if dictionary.key?(wc)
97
+ w = wc
211
98
  else
212
- context_data_position += 1
99
+ write_w.call
100
+ # Add wc to the dictionary.
101
+ dictionary[wc] = dict_size unless wc == PROTO
102
+ dict_size += 1
103
+ w = c
213
104
  end
214
- value = value >> 1
215
105
  end
216
106
 
107
+ # Output the code for w.
108
+ write_w.call unless w.empty?
109
+
110
+ # Mark the end of the stream
111
+ write_bits.call(2, num_bits)
112
+
217
113
  # Flush the last char
218
- while (true)
219
- context_data_val = (context_data_val << 1)
220
- if (context_data_position == bits_per_char-1)
221
- context_data.push(get_char_from_int[context_data_val])
114
+ loop do
115
+ data_val <<= 1
116
+ if data_position == bits_per_char - 1
117
+ data << get_char_from_int.call(data_val)
222
118
  break
223
- else
224
- context_data_position += 1
225
119
  end
120
+ data_position += 1
226
121
  end
227
122
 
228
- return context_data.join("")
123
+ data.join
229
124
  end
230
125
 
231
126
  # @param length [Integer]
232
127
  # @param reset_value [Integer]
233
128
  # @param get_next_value [Proc]
234
- def self.decompress(length, reset_value, get_next_value, encoding = "ASCII-8BIT")
129
+ # @return [String, nil] UTF-16LE, exactly as JS has it (it can hold lone surrogates)
130
+ def self.decompress(length, reset_value, get_next_value)
235
131
  dictionary = [0, 1, 2]
236
132
  enlarge_in = 4
237
133
  dict_size = 4
238
134
  num_bits = 3
239
- entry = ""
240
135
  result = []
241
- data = {
242
- val: get_next_value[(0)],
243
- position: reset_value,
244
- index: 1
245
- }
246
- bits = 0
247
- maxpower = 2**2
248
- power = 1
249
- i, w, resb, c = nil
136
+ data_val = get_next_value.call(0)
137
+ data_position = reset_value
138
+ data_index = 1
250
139
 
251
- while (power != maxpower)
252
- resb = data[:val] & data[:position]
253
- data[:position] >>= 1
254
- if (data[:position] == 0)
255
- data[:position] = reset_value
256
- data[:val] = get_next_value[data[:index]]
257
- data[:index] += 1
258
- end
259
- bits |= (resb > 0 ? 1 : 0) * power
260
- power <<= 1
261
- end
262
-
263
- case(n = bits)
264
- when 0
265
- bits = 0
266
- maxpower = 2**8
267
- power = 1
268
- while power != maxpower
269
- resb = data[:val] & data[:position]
270
- data[:position] >>= 1
271
- if (data[:position] == 0)
272
- data[:position] = reset_value
273
- data[:val] = get_next_value[data[:index]]
274
- data[:index] += 1
275
- end
276
- bits |= (resb > 0 ? 1 : 0) * power
277
- power <<= 1
278
- end
279
- c = bits.chr(encoding)
280
- when 1
140
+ # Reads `count` bits, lowest bit first.
141
+ read_bits = lambda do |count|
281
142
  bits = 0
282
- maxpower = 2*16
283
- power = 1
284
- while (power != maxpower)
285
- resb = data[:val] & data[:position]
286
- data[:position] >>= 1
287
- if (data[:position] == 0)
288
- data[:position] = reset_value
289
- data[:val] = get_next_value[data[:index]]
290
- data[:index] += 1
143
+ count.times do |i|
144
+ resb = data_val & data_position
145
+ data_position >>= 1
146
+ if data_position == 0
147
+ data_position = reset_value
148
+ data_val = get_next_value.call(data_index)
149
+ data_index += 1
291
150
  end
292
- bits |= (resb > 0 ? 1 : 0) * power
293
- power <<= 1
151
+ bits |= 1 << i if resb > 0
294
152
  end
295
- c = bits.chr(encoding)
296
- when 2
297
- ""
153
+ bits
298
154
  end
299
155
 
156
+ # Each character is one UTF-16 code unit: 2 bytes, little endian.
157
+ c = case read_bits.call(2)
158
+ when 0 then [read_bits.call(8)].pack("v")
159
+ when 1 then [read_bits.call(16)].pack("v")
160
+ when 2 then return "".encode(Encoding::UTF_16LE)
161
+ # Real data never starts with 3. lz-string 2 throws here, so we raise too
162
+ # (1.5.0 went on with an undefined value and gave junk, null or a TypeError).
163
+ else raise ArgumentError, "No character found"
164
+ end
300
165
  dictionary[3] = c
301
166
  w = c
302
167
  result << c
303
168
 
304
- while(true)
305
- return "" if (data[:index] > length)
306
-
307
- bits = 0
308
- maxpower = 2**num_bits
309
- power = 1
310
- while (power != maxpower)
311
- resb = data[:val] & data[:position]
312
- data[:position] >>= 1
313
- if (data[:position] == 0)
314
- data[:position] = reset_value
315
- data[:val] = get_next_value[data[:index]]
316
- data[:index] += 1
317
- end
318
- bits |= (resb > 0 ? 1 : 0) * power
319
- power <<= 1
320
- end
321
-
322
- case(c = bits)
323
- when 0
324
- bits = 0
325
- maxpower = 2**8
326
- power = 1
327
- while (power != maxpower)
328
- resb = data[:val] & data[:position]
329
- data[:position] >>= 1
330
- if (data[:position] == 0)
331
- data[:position] = reset_value
332
- data[:val] = get_next_value[data[:index]]
333
- data[:index] += 1
334
- end
335
- bits |= (resb > 0 ? 1 : 0) * power
336
- power <<= 1
337
- end
169
+ loop do
170
+ return "".encode(Encoding::UTF_16LE) if data_index > length
338
171
 
339
- dictionary[dict_size] = bits.chr(encoding)
172
+ case (c = read_bits.call(num_bits))
173
+ when 0, 1
174
+ dictionary[dict_size] = [read_bits.call(c == 0 ? 8 : 16)].pack("v")
175
+ c = dict_size
340
176
  dict_size += 1
341
- c = dict_size - 1
342
- enlarge_in -= 1
343
- when 1
344
- bits = 0
345
- maxpower = 2**16
346
- power = 1
347
- while (power != maxpower)
348
- resb = data[:val] & data[:position]
349
- data[:position] >>= 1
350
- if (data[:position] == 0)
351
- data[:position] = reset_value
352
- data[:val] = get_next_value[data[:index]]
353
- data[:index] += 1
354
- end
355
- bits |= (resb > 0 ? 1 : 0) * power
356
- power <<= 1
357
- end
358
- dictionary[dict_size] = bits.chr(encoding)
359
- dict_size += 1
360
- c = dict_size - 1
361
177
  enlarge_in -= 1
362
178
  when 2
363
- return result.join("")
179
+ return result.join.force_encoding(Encoding::UTF_16LE)
364
180
  end
365
181
 
366
- if (enlarge_in == 0)
182
+ if enlarge_in == 0
367
183
  enlarge_in = 2**num_bits
368
184
  num_bits += 1
369
185
  end
370
186
 
371
- if (dictionary[c])
187
+ if dictionary[c]
372
188
  entry = dictionary[c]
189
+ elsif c == dict_size
190
+ entry = w + w.byteslice(0, 2)
373
191
  else
374
- if (c === dict_size)
375
- entry = w + w[0]
376
- else
377
- return nil
378
- end
192
+ return nil
379
193
  end
380
-
381
194
  result << entry
382
195
 
383
196
  # Add w+entry[0] to the dictionary.
384
- dictionary[dict_size] = w + entry[0]
197
+ dictionary[dict_size] = w + entry.byteslice(0, 2)
385
198
  dict_size += 1
386
199
  enlarge_in -= 1
387
200
 
388
201
  w = entry
389
202
 
390
- if (enlarge_in == 0)
203
+ if enlarge_in == 0
391
204
  enlarge_in = 2**num_bits
392
205
  num_bits += 1
393
206
  end
@@ -3,6 +3,8 @@ module LZString
3
3
  class Base64
4
4
  # Base64 alphabet.
5
5
  KEY_STR_BASE64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/="
6
+ # Code unit of each alphabet character => its value.
7
+ VALUES = KEY_STR_BASE64.each_char.with_index.map { |char, i| [char.ord, i] }.to_h
6
8
 
7
9
  # @param input [String]
8
10
  def self.compress(input)
@@ -20,32 +22,20 @@ module LZString
20
22
  end
21
23
 
22
24
  # @param compressed [String]
25
+ # @return [String, nil] UTF-8
23
26
  def self.decompress(compressed)
24
- return "" if compressed.nil?
25
- return nil if compressed == ""
26
-
27
- LZString::Base.decompress(
28
- compressed.length,
29
- 32,
30
- lambda { |index| get_base_value(KEY_STR_BASE64, compressed[index]) }
31
- )
27
+ LZString::Base.to_utf8(decompress_to_utf16le(compressed))
32
28
  end
33
29
 
34
- # @param alphabet [String]
35
- # @param character [String]
36
- def self.get_base_value(alphabet, character)
37
- base_reverse_dic = {}
38
-
39
- if (!base_reverse_dic[alphabet])
40
- base_reverse_dic[alphabet] = {}
41
- for i in 0...alphabet.length do
42
- base_reverse_dic[alphabet][alphabet[i]] = i
43
- end
44
- end
30
+ # @param compressed [String]
31
+ # @return [String, nil] UTF-16LE, see LZString.decompress_to_utf16le
32
+ def self.decompress_to_utf16le(compressed)
33
+ return "" if compressed.nil?
34
+ return nil if compressed.empty?
45
35
 
46
- base_reverse_dic[alphabet][character]
36
+ # Like in JS, a character outside the alphabet counts as 0.
37
+ values = LZString::Base.code_units(compressed).map { |unit| VALUES.fetch(unit, 0) }
38
+ LZString::Base.decompress(values.length, 32, lambda { |index| values[index] || 0 })
47
39
  end
48
-
49
- private_class_method :get_base_value
50
40
  end
51
41
  end
@@ -0,0 +1,107 @@
1
+ require "optparse"
2
+ require "lz_string"
3
+
4
+ module LZString
5
+ # The command line tool. It has the same options as the one of lz-string 2 (JS), so
6
+ # lz-string's own test suite (test/test.sh in its repository) can check this gem too.
7
+ #
8
+ # Like the JS tool it is binary safe: a file is read and written as UTF-16 code units,
9
+ # 2 bytes each, big endian.
10
+ module CLI
11
+ FORMATS = {
12
+ "base64" => LZString::Base64,
13
+ "custom" => LZString::Custom,
14
+ "encodeduri" => LZString::UriSafe,
15
+ "raw" => LZString,
16
+ # The JS tool writes the same bytes for raw and uint8array.
17
+ "uint8array" => LZString,
18
+ "utf16" => LZString::UTF16,
19
+ }.freeze
20
+
21
+ # @param argv [Array<String>]
22
+ # @return [Integer] the exit code
23
+ def self.run(argv, stdin: $stdin, stdout: $stdout, stderr: $stderr)
24
+ options = { encoder: "raw" }
25
+ parser = OptionParser.new do |opts|
26
+ opts.banner = "Usage: lz_string [options] [input-file]\n\nUse lz-string to compress or decompress a file"
27
+ opts.separator ""
28
+ opts.separator "Arguments:"
29
+ opts.separator " input-file file to process, if no file then read from stdin"
30
+ opts.separator ""
31
+ opts.separator "Options:"
32
+ opts.on("-V", "--version", "output the version number") { options[:version] = true }
33
+ opts.on("-d", "--decompress", "if unset then this will compress") { options[:decompress] = true }
34
+ opts.on("-e", "--encoder TYPE", FORMATS.keys,
35
+ "character encoding to use (choices: #{FORMATS.keys.join(", ")}, default: raw)") { |type| options[:encoder] = type }
36
+ opts.on("-c", "--custom DICTIONARY", "dictionary for custom encoder") { |dictionary| options[:dictionary] = dictionary }
37
+ opts.on("-v", "--verify", "verify before returning") { options[:verify] = true }
38
+ opts.on("-o", "--output OUTPUT_FILE", "output file, otherwise write to stdout") { |file| options[:output] = file }
39
+ opts.on("-q", "--quiet", "don't print any error messages") { options[:quiet] = true }
40
+ opts.on("-h", "--help", "display help for command") { options[:help] = true }
41
+ end
42
+
43
+ begin
44
+ file, = parser.parse(argv)
45
+ rescue OptionParser::ParseError => e
46
+ stderr.puts("error: #{e.message}", parser.help)
47
+ return 1
48
+ end
49
+ return stdout.puts(parser.help) || 0 if options[:help]
50
+ return stdout.puts(LZString::VERSION) || 0 if options[:version]
51
+
52
+ error = lambda do |message|
53
+ stderr.puts(message) unless options[:quiet]
54
+ 1
55
+ end
56
+ return error.call("Unable to find #{file}") if file && !File.exist?(file)
57
+ return error.call("Unable to access #{file}") if file && !File.readable?(file)
58
+
59
+ format = FORMATS.fetch(options[:encoder])
60
+ dictionary = []
61
+ if format == LZString::Custom
62
+ return error.call("The custom encoder needs a dictionary (-c)") unless options[:dictionary]
63
+
64
+ dictionary = [options[:dictionary]]
65
+ end
66
+
67
+ input = to_string(file ? File.binread(file) : stdin.binmode.read)
68
+ begin
69
+ result = if options[:decompress]
70
+ format.decompress_to_utf16le(input, *dictionary)
71
+ else
72
+ format.compress(input, *dictionary)
73
+ end
74
+ return error.call("Unable to process #{file || "stdin"}") if result.nil?
75
+
76
+ if options[:verify]
77
+ back = options[:decompress] ? format.compress(result, *dictionary) : format.decompress_to_utf16le(result, *dictionary)
78
+ return error.call("Unable to validate #{file || "stdin"}") unless back && Base.code_units(back) == Base.code_units(input)
79
+ end
80
+ rescue ArgumentError, EncodingError
81
+ return error.call("Unable to process #{file || "stdin"}")
82
+ end
83
+
84
+ if options[:output]
85
+ File.binwrite(options[:output], to_bytes(result))
86
+ else
87
+ stdout.binmode.write(to_bytes(result))
88
+ end
89
+ 0
90
+ end
91
+
92
+ # Bytes to a string of UTF-16 code units, 2 bytes each, big endian. A last odd byte is the
93
+ # high byte of a code unit.
94
+ def self.to_string(bytes)
95
+ units = bytes.unpack("C*").each_slice(2).map { |high, low| (high << 8) | low.to_i }
96
+ units.pack("v*").force_encoding(Encoding::UTF_16LE)
97
+ end
98
+
99
+ # The other way. Like the JS tool, a last 0 byte is left out, so a file with an odd count
100
+ # of bytes comes back the same.
101
+ def self.to_bytes(string)
102
+ units = Base.code_units(string)
103
+ bytes = units.pack("n*")
104
+ !units.empty? && units.last % 256 == 0 ? bytes.byteslice(0, bytes.bytesize - 1) : bytes
105
+ end
106
+ end
107
+ end
@@ -0,0 +1,60 @@
1
+ module LZString
2
+ # Custom compressing algorithm: writes each code unit of the raw result as a number,
3
+ # with a fixed count of digits from your own alphabet (from lz-string 2).
4
+ class Custom
5
+ # @param input [String]
6
+ # @param dictionary [String] the alphabet: at least 2 characters
7
+ # @return [String] UTF-8
8
+ def self.compress(input, dictionary)
9
+ return "" if input.nil?
10
+
11
+ alphabet = LZString::Base.code_units(dictionary)
12
+ # JS never ends with a 1-character alphabet.
13
+ raise ArgumentError, "The dictionary needs at least 2 characters" if alphabet.size < 2
14
+
15
+ digits = digits_per_code_unit(alphabet.size)
16
+ units = LZString.compress(input).unpack("v*").flat_map do |unit|
17
+ (digits - 1).downto(0).map { |j| alphabet[unit / alphabet.size**j % alphabet.size] }
18
+ end
19
+ units.pack("v*").force_encoding(Encoding::UTF_16LE).encode(Encoding::UTF_8)
20
+ end
21
+
22
+ # @param compressed [String]
23
+ # @param dictionary [String]
24
+ # @return [String, nil] UTF-8
25
+ def self.decompress(compressed, dictionary)
26
+ LZString::Base.to_utf8(decompress_to_utf16le(compressed, dictionary))
27
+ end
28
+
29
+ # @param compressed [String]
30
+ # @param dictionary [String]
31
+ # @return [String, nil] UTF-16LE, see LZString.decompress_to_utf16le
32
+ def self.decompress_to_utf16le(compressed, dictionary)
33
+ return "" if compressed.nil?
34
+ return nil if compressed.empty?
35
+
36
+ alphabet = LZString::Base.code_units(dictionary)
37
+ return nil if alphabet.size < 2
38
+
39
+ digits = digits_per_code_unit(alphabet.size)
40
+ units = LZString::Base.code_units(compressed)
41
+ return nil if units.size % digits != 0
42
+
43
+ # The first place of each character, like indexOf in JS.
44
+ places = {}
45
+ alphabet.each_with_index { |unit, place| places[unit] ||= place }
46
+ values = units.each_slice(digits).map do |number|
47
+ # Like JS: a character outside the alphabet counts as -1, and String.fromCharCode keeps 16 bits.
48
+ number.reduce(0) { |value, unit| value * alphabet.size + places.fetch(unit, -1) } & 0xFFFF
49
+ end
50
+ LZString.decompress_to_utf16le(values.pack("v*").force_encoding(Encoding::UTF_16LE))
51
+ end
52
+
53
+ # Digits needed for 65536 values. Same float math as JS (checked for every size up to 70000).
54
+ def self.digits_per_code_unit(base)
55
+ (Math.log(65536) / Math.log(base)).ceil
56
+ end
57
+
58
+ private_class_method :digits_per_code_unit
59
+ end
60
+ end
@@ -0,0 +1,27 @@
1
+ module LZString
2
+ # Uint8Array compressing algorithm: the raw result as bytes, 2 per code unit, big endian.
3
+ class Uint8Array
4
+ # @param input [String]
5
+ # @return [String] binary (ASCII-8BIT)
6
+ def self.compress(input)
7
+ LZString.compress(input).unpack("v*").pack("n*")
8
+ end
9
+
10
+ # @param compressed [String, Array<Integer>] the bytes, as a binary string or a list of numbers
11
+ # @return [String, nil] UTF-8
12
+ def self.decompress(compressed)
13
+ LZString::Base.to_utf8(decompress_to_utf16le(compressed))
14
+ end
15
+
16
+ # @param compressed [String, Array<Integer>]
17
+ # @return [String, nil] UTF-16LE, see LZString.decompress_to_utf16le
18
+ def self.decompress_to_utf16le(compressed)
19
+ return "" if compressed.nil?
20
+
21
+ bytes = compressed.is_a?(String) ? compressed.bytes : compressed
22
+ # Like lz-string 2, a last odd byte is the high byte of the last code unit.
23
+ units = bytes.each_slice(2).map { |high, low| (high << 8) | low.to_i }
24
+ LZString.decompress_to_utf16le(units.pack("v*").force_encoding(Encoding::UTF_16LE))
25
+ end
26
+ end
27
+ end
@@ -2,6 +2,8 @@ module LZString
2
2
  class UriSafe
3
3
  # Base64 alphabet.
4
4
  KEY_STR_URISAFE = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-$"
5
+ # Code unit of each alphabet character => its value.
6
+ VALUES = KEY_STR_URISAFE.each_char.with_index.map { |char, i| [char.ord, i] }.to_h
5
7
 
6
8
  # @param input [String]
7
9
  def self.compress(input)
@@ -11,32 +13,21 @@ module LZString
11
13
  end
12
14
 
13
15
  # @param compressed [String]
16
+ # @return [String, nil] UTF-8
14
17
  def self.decompress(compressed)
15
- return "" if compressed.nil?
16
- return nil if compressed == ""
17
- compressed.gsub!(" ","+")
18
- LZString::Base.decompress(
19
- compressed.length,
20
- 32,
21
- lambda { |index| get_base_value(KEY_STR_URISAFE, compressed[index]) }
22
- )
18
+ LZString::Base.to_utf8(decompress_to_utf16le(compressed))
23
19
  end
24
20
 
25
- # @param alphabet [String]
26
- # @param character [String]
27
- def self.get_base_value(alphabet, character)
28
- base_reverse_dic = {}
29
-
30
- if (!base_reverse_dic[alphabet])
31
- base_reverse_dic[alphabet] = {}
32
- for i in 0...alphabet.length do
33
- base_reverse_dic[alphabet][alphabet[i]] = i
34
- end
35
- end
21
+ # @param compressed [String]
22
+ # @return [String, nil] UTF-16LE, see LZString.decompress_to_utf16le
23
+ def self.decompress_to_utf16le(compressed)
24
+ return "" if compressed.nil?
25
+ return nil if compressed.empty?
36
26
 
37
- base_reverse_dic[alphabet][character]
27
+ # Like in JS, a space is read as "+" (a URL can turn "+" into a space),
28
+ # and any other character outside the alphabet counts as 0.
29
+ values = LZString::Base.code_units(compressed).map { |unit| VALUES.fetch(unit == " ".ord ? "+".ord : unit, 0) }
30
+ LZString::Base.decompress(values.length, 32, lambda { |index| values[index] || 0 })
38
31
  end
39
-
40
- private_class_method :get_base_value
41
32
  end
42
33
  end
@@ -9,16 +9,19 @@ module LZString
9
9
  end
10
10
 
11
11
  # @param compressed [String]
12
+ # @return [String, nil] UTF-8
12
13
  def self.decompress(compressed)
14
+ LZString::Base.to_utf8(decompress_to_utf16le(compressed))
15
+ end
16
+
17
+ # @param compressed [String]
18
+ # @return [String, nil] UTF-16LE, see LZString.decompress_to_utf16le
19
+ def self.decompress_to_utf16le(compressed)
13
20
  return "" if (compressed == nil)
14
- return nil if (compressed == "")
21
+ return nil if compressed.empty?
15
22
 
16
- LZString::Base.decompress(
17
- compressed.length,
18
- 16384,
19
- lambda { |index| compressed[index].ord - 32 },
20
- "UTF-8"
21
- )
23
+ values = LZString::Base.code_units(compressed)
24
+ LZString::Base.decompress(values.length, 16384, lambda { |index| values[index] ? values[index] - 32 : 0 })
22
25
  end
23
26
  end
24
27
  end
@@ -1,4 +1,4 @@
1
1
  module LZString
2
2
  # :nodoc:
3
- VERSION = "0.2.0"
3
+ VERSION = "1.0.0"
4
4
  end
data/lib/lz_string.rb CHANGED
@@ -1,23 +1,82 @@
1
1
  require "lz_string/base"
2
2
  require "lz_string/base64"
3
- require "lz_string/utf16"
3
+ require "lz_string/custom"
4
+ require "lz_string/uint8_array"
4
5
  require "lz_string/urisafe"
6
+ require "lz_string/utf16"
5
7
  require "lz_string/version"
6
8
 
7
9
  # LZ-based compression algorithm.
8
10
  module LZString
11
+ # Returns the same UTF-16 code units as LZString.compress in JS. Like in JS, they are often not
12
+ # valid UTF-16 (a lone surrogate), and UTF-8 can't hold them, so the result is a UTF-16LE string.
13
+ #
9
14
  # @param input [String]
15
+ # @return [String] UTF-16LE
10
16
  def self.compress(input)
11
17
  return "" if input.nil?
12
18
 
13
- LZString::Base.compress(input, 16, lambda { |a| a.chr("UTF-8") })
19
+ LZString::Base.compress(input, 16, lambda { |a| [a].pack("v") }).force_encoding(Encoding::UTF_16LE)
14
20
  end
15
21
 
16
- # @param compressed [String]
22
+ # @param compressed [String] the UTF-16LE string from .compress (other encodings work too)
23
+ # @return [String, nil] UTF-8
17
24
  def self.decompress(compressed)
25
+ LZString::Base.to_utf8(decompress_to_utf16le(compressed))
26
+ end
27
+
28
+ # Like .decompress, but gives the result exactly as JS has it: a UTF-16LE string, which can
29
+ # also hold a lone surrogate (half of an emoji). Every format has this method.
30
+ #
31
+ # @param compressed [String]
32
+ # @return [String, nil] UTF-16LE
33
+ def self.decompress_to_utf16le(compressed)
18
34
  return "" if compressed.nil?
19
- return nil if compressed == ""
35
+ return nil if compressed.empty?
36
+
37
+ values = LZString::Base.code_units(compressed)
38
+ LZString::Base.decompress(values.length, 32768, lambda { |index| values[index] || 0 })
39
+ end
40
+
41
+ # The other functions of JS lz-string, by their JS names.
42
+
43
+ def self.compress_to_base64(input)
44
+ LZString::Base64.compress(input)
45
+ end
46
+
47
+ def self.decompress_from_base64(compressed)
48
+ LZString::Base64.decompress(compressed)
49
+ end
50
+
51
+ def self.compress_to_encoded_uri_component(input)
52
+ LZString::UriSafe.compress(input)
53
+ end
54
+
55
+ def self.decompress_from_encoded_uri_component(compressed)
56
+ LZString::UriSafe.decompress(compressed)
57
+ end
58
+
59
+ def self.compress_to_utf16(input)
60
+ LZString::UTF16.compress(input)
61
+ end
62
+
63
+ def self.decompress_from_utf16(compressed)
64
+ LZString::UTF16.decompress(compressed)
65
+ end
66
+
67
+ def self.compress_to_uint8_array(input)
68
+ LZString::Uint8Array.compress(input)
69
+ end
70
+
71
+ def self.decompress_from_uint8_array(compressed)
72
+ LZString::Uint8Array.decompress(compressed)
73
+ end
74
+
75
+ def self.compress_to_custom(input, dictionary)
76
+ LZString::Custom.compress(input, dictionary)
77
+ end
20
78
 
21
- LZString::Base.decompress(compressed.length, 32768, lambda { |index| compressed[index].ord })
79
+ def self.decompress_from_custom(compressed, dictionary)
80
+ LZString::Custom.decompress(compressed, dictionary)
22
81
  end
23
82
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: lz_string
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0
4
+ version: 1.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Altivi
8
8
  autorequire:
9
- bindir: bin
9
+ bindir: exe
10
10
  cert_chain: []
11
- date: 2023-06-13 00:00:00.000000000 Z
11
+ date: 2026-10-16 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -122,24 +122,36 @@ dependencies:
122
122
  - - ">="
123
123
  - !ruby/object:Gem::Version
124
124
  version: '0'
125
- description: Ruby implementation of LZ-String compression algorithm
125
+ description: 'A full Ruby port of lz-string (JavaScript), with the same results for
126
+ every input, emoji too: raw, Base64, URI-safe, UTF-16, Uint8Array and custom alphabets,
127
+ and the same command line tool. It passes the test suite of lz-string itself.'
126
128
  email:
127
129
  - altivi.prog@gmail.com
128
- executables: []
130
+ executables:
131
+ - lz_string
129
132
  extensions: []
130
133
  extra_rdoc_files: []
131
134
  files:
135
+ - CHANGELOG.md
136
+ - LICENSE
132
137
  - README.md
138
+ - exe/lz_string
133
139
  - lib/lz_string.rb
134
140
  - lib/lz_string/base.rb
135
141
  - lib/lz_string/base64.rb
142
+ - lib/lz_string/cli.rb
143
+ - lib/lz_string/custom.rb
144
+ - lib/lz_string/uint8_array.rb
136
145
  - lib/lz_string/urisafe.rb
137
146
  - lib/lz_string/utf16.rb
138
147
  - lib/lz_string/version.rb
139
- homepage: https://github.com/Altivi/lz-string
148
+ homepage: https://github.com/altivi/lz_string
140
149
  licenses:
141
150
  - MIT
142
- metadata: {}
151
+ metadata:
152
+ source_code_uri: https://github.com/altivi/lz_string
153
+ changelog_uri: https://github.com/altivi/lz_string/blob/master/CHANGELOG.md
154
+ bug_tracker_uri: https://github.com/altivi/lz_string/issues
143
155
  post_install_message:
144
156
  rdoc_options: []
145
157
  require_paths:
@@ -148,14 +160,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
148
160
  requirements:
149
161
  - - ">="
150
162
  - !ruby/object:Gem::Version
151
- version: '0'
163
+ version: '2.7'
152
164
  required_rubygems_version: !ruby/object:Gem::Requirement
153
165
  requirements:
154
166
  - - ">="
155
167
  - !ruby/object:Gem::Version
156
168
  version: '0'
157
169
  requirements: []
158
- rubygems_version: 3.1.6
170
+ rubygems_version: 3.5.22
159
171
  signing_key:
160
172
  specification_version: 4
161
173
  summary: Ruby implementation of LZ-String compression algorithm