smstools 0.0.1 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 4180ff46eb40f9d709cf9895d1dc493aeec4203c
-  data.tar.gz: cba4fd705516f2418dfae93ce3732231e30faba5
+SHA256:
+  metadata.gz: f2cecee4608c47f5abf1cf0a980b3a3a646e358d50a72e6b0f1931f554f86c5f
+  data.tar.gz: 46eae0938780419f4672581f4b1105a8d51cc4fe7f6150b2e44fdc3c00f16c6e
 SHA512:
-  metadata.gz: b3b26b73c00f2acbb9e107ecc24738b97a713a3579c685339226b9ba369c0d2080ff7b40357a472c0dc9ebe41bca0f8ed38e28cd132242ce7ed3e514a78d69c3
-  data.tar.gz: 9b7a0e6a64fdaec4f8f05130dc3b4ac735f2848fddfd1bbd8d274ec2b6e9781fcdb993ef121357e77f6b4c486508a067e6480516a72fd33d7c28007397f0fde5
+  metadata.gz: 6f40d959431dc1185a989b179c91858363978b5200bba7504e499b214ba8b1493c859eebb3308b343ac8000ec747db89ccfbc664692721315bb68c41f96000a0
+  data.tar.gz: 7670ac023de1612cd5e4573ad4879526e5f45e38c871416cef3466e4bbee6d1740d0b2ae760b907d57811f913670862e2907b3ed3c8f268b876085195bed2a61

data/CHANGELOG.md

@@ -1,3 +1,25 @@
-## 0.1.1 (17 January 2014)
+## 0.2.2 (20 Jan 2021)
+
+* #9 Fix the way some complex Unicode characters (like composite emojis) are counted. Thanks to @bryanrite for the neat implementation. Note the fix could be **potentially backwards-incompatible** if you were relying on the incorrect behaviour previously. Technically it's still a bug fix.
+
+## 0.2.1 (18 Aug 2020)
+
+* #7 Introduce `SmsTools.use_ascii_encoding` option (defaults to `true` for backwards-compatibility) that allows disabling the `:ascii` workaround encoding. See #6 and #7 for details. Thanks @kingsley-wang.
+
+## 0.2.0 (2 March 2017)
+
+* The non-breaking space character (0x00A0 in Unicode and "\xC2\xA0" in UTF-8) is no longer regarded as a valid GSM 7-bit symbol. [#4](https://github.com/livebg/smstools/issues/4)
+* GsmEncoding.to_utf8 will now raise errors in case the provided argument is not a valid GSM 7-bit text.
+
+## 0.1.1 (18 April 2016)
+
+* Replaces small c with cedilla to capital one, as per the GSM 03.38 standard (by @skliask)
+
+## 0.1.0 (08 October 2015)
+
+* distinguish between ascii encoding and gsm encoding
+* add option for preventing the use of gsm encoding, that is to use unicode instead
+
+## 0.0.1 (17 January 2014)
 
 * Initial release.

data/README.md

@@ -1,23 +1,80 @@
 # Sms Tools
 
-A small collection of useful Ruby and JavaScript classes implementing often
-needed functionality for dealing with SMS messages.
+A small collection of Ruby and JavaScript classes implementing often needed functionality for
+dealing with SMS messages.
 
-The gem is also a Rails engine and using it in your Rails app will allow you
-to also use the JavaScript classes via the asset pipeline.
+The gem can also be used in a Rails application as an engine. It integrates with the asset pipeline
+and gives you access to some client-side SMS manipulation functionality.
 
 ## Features
 
 The following features are available on both the server side and the client
 side:
 
-- Detection of the most optimal encoding for sending an SMS message (GSM 7-bit
-  or Unicode).
-- Correctly determining the message's length according to the most optimal
-  encoding.
+- Detection of the most optimal encoding for sending an SMS message (GSM 7-bit or Unicode).
+- Correctly determining a message's length in the most optimal encoding.
 - Concatenation detection and concatenated message parts counting.
 
-And more.
+The following can be accomplished only on the server with Ruby:
+
+- Converting a UTF-8 string to a GSM 7-bit encoding and vice versa.
+- Detecting if a UTF-8 string can be safely represented in a GSM 7-bit encoding.
+- Detection of double-byte chars in the GSM 7-bit encoding.
+
+And possibly more.
+
+### Note on the GSM encoding
+
+All references to the "GSM" encoding or the "GSM 7-bit alphabet" in this text actually refer to the
+[GSM 03.38 spec](http://en.wikipedia.org/wiki/GSM_03.38) and [its latest
+version](ftp://ftp.unicode.org/Public/MAPPINGS/ETSI/GSM0338.TXT), as defined by the Unicode
+consortium.
+
+This encoding is the most widely used one when sending SMS messages.
+
+### Note regarding non-ASCII symbols from the GSM encoding
+
+The GSM 03.38 encoding is used by default. This standard defines a set of
+symbols which can be encoded in 7-bits each, thus allowing up to 160 symbols
+per SMS message (each SMS message can contain up to 140 bytes of data).
+
+This standard covers most of the ASCII table, but also includes some non-ASCII
+symbols such as `æ`, `ø` and `å`. If you use these in your messages, you can
+still send them as GSM encoded, having a 160-symbol limit. This is technically
+correct.
+
+In reality, however, some SMS routes have problems delivering messages which
+contain such non-ASCII symbols in the GSM encoding. The special symbols might
+be omitted, or the message might not arrive at all.
+
+Thus, it might be safer to just send messages in Unicode if the message's text
+contains any non-ASCII symbols. This is not the default as it reduces the max
+symbols count to 70 per message, instead of 160, and you might not have any
+issues with GSM-encoded messages. In case you do, however, you can turn off
+support for the GSM encoding and just treat messages as Unicode if they contain
+non-ASCII symbols.
+
+In case you decide to do so, you have to specify it in both the Ruby and the
+JavaScript part of the library, like so:
+
+#### In Ruby
+
+    SmsTools.use_gsm_encoding = false
+
+#### In Javascript
+
+    //= require sms_tools
+    SmsTools.use_gsm_encoding = false;
+
+There is another alternative as well. As explained in this commit – f1ffd948d4b8c – SmsTools will by
+default detect the encoding as `:ascii` if the SMS message contains ASCII-only symbols. The safest
+way to send messages would be to use an ASCII subset of the GSM encodnig.
+
+The `:ascii` encoding is informative only, however. Your SMS sending implementation will have to
+decide how to handle it. You may also find it confusing that the dummy `:ascii` encoding does not
+consider double-byte chars at all when counting the length of the message.
+
+To disable this dummy `:ascii` encoding, set `SmsTools.use_ascii_encoding` to `false`.
 
 ## Installation
 
@@ -33,32 +90,99 @@ Or install it yourself as:
 
     $ gem install smstools
 
+If you're using the gem in Rails, you may also want to add the following to your `application.js`
+manifest file to gain access to the client-side features:
+
+    //= require sms_tools
+
 ## Usage
 
 The gem consists of both server-side (Ruby) and client-side classes. You can
-use either one.
+use either.
 
 ### Server-side code
 
-If you use the gem in Rails or via Bundler, just use the appropriate class,
-such as `SmsTools::EncodingDetection` or `SmsTools::GsmEncoding`.
+First make sure you have installed the gem and have required the appropriate files.
+
+#### Encoding detection
+
+The `SmsTools::EncodingDetection` class provides you with a few simple methods to detect the most
+optimal encoding for sending an SMS message, to correctly caclulate its length in that encoding and
+to see if the text would need to be concatenated or will fit in a single message.
+
+Here is an example with a non-concatenated message which is best encoded in the GSM 7-bit alphabet:
+
+```ruby
+sms_text     = 'Text in GSM 03.38: ÄäøÆ with a double-byte char: ~ '
+sms_encoding = SmsTools::EncodingDetection.new sms_text
+
+sms_encoding.gsm?               # => true
+sms_encoding.unicode?           # => false
+sms_encoding.length             # => 52 (because of the double-byte char)
+sms_encoding.concatenated?      # => false
+sms_encoding.concatenated_parts # => 1
+sms_encoding.encoding           # => :gsm
+```
+
+Here's another example with a concatenated Unicode message:
+
+```ruby
+sms_text     = 'Я' * 90
+sms_encoding = SmsTools::EncodingDetection.new sms_text
+
+sms_encoding.gsm?               # => false
+sms_encoding.unicode?           # => true
+sms_encoding.length             # => 90
+sms_encoding.concatenated?      # => true
+sms_encoding.concatenated_parts # => 2
+sms_encoding.encoding           # => :unicode
+```
+
+You can check the specs for this class for more examples.
 
-#### `EncodingDetection`
-#### `GsmEncoding`
+#### GSM 03.38 encoding conversion
+
+The `SmsTools::GsmEncoding` class can be used to check if a given UTF-8 string can be fully
+represented in the GSM 03.38 encoding as well as to convert from UTF-8 to GSM 03.38 and vice-versa.
+
+The main API this class provides is the following:
+
+```ruby
+SmsTools::GsmEncoding.valid? message_text_in_utf8   # => true or false
+
+SmsTools::GsmEncoding.from_utf8 utf8_encoded_string # => a GSM 03.38 encoded string
+SmsTools::GsmEncoding.to_utf8 gsm_encoded_string    # => an UTF-8 encoded string
+```
+
+Check out the source code of the class to find out more.
 
 ### Client-side code
 
-If you're using the gem in Rails 3.x or newer, you can just add the following
-to your `application.js` file to gain access to the JavaScript classes:
+If you're using the gem in Rails 3.1 or newer, you can gain access to the `SmsTools.Message` class.
+Its interface is similar to the one of `SmsTools::EncodingDetection`. Here is an example in
+CoffeeScript:
 
-    #= require 'sms_tools/all'
+```coffeescript
+message = new SmsTools.Message 'The text of the message: ~'
 
-Or require only the files you need:
+message.encoding               # => 'gsm'
+message.length                 # => 27
+message.concatenatedPartsCount # => 1
+```
 
-    #= require 'sms_tools/message'
+You can also check how long can this message be in the current most optimal encoding, if we want to
+limit the number of concatenated messages we will allow to be sent:
 
-Note that this assumes you're using the asset pipeline. You need to have a
-CoffeeScript preprocessor set up.
+```coffeescript
+maxConcatenatedPartsCount = 2
+message.maxLengthFor(maxConcatenatedPartsCount) # => 306
+```
+
+This allows you to have a dynamic instead of a fixed length limit, for when you use a non-GSM 03.38
+symbol in your text, your message length limit decreases significantly.
+
+Note that to use this client-side code, a Rails application with an active asset pipeline is
+assumed. It might be possible to use it in other setups as well, but you're on your own there.
 
 ## Contributing
 
@@ -69,3 +193,10 @@ CoffeeScript preprocessor set up.
 5. Commit your changes (`git commit -am 'Add some feature'`)
 6. Push to the branch (`git push origin my-new-feature`)
 7. Send a pull request.
+
+## Publishing a new version
+
+1. Pick a version number according to Semantic Versioning.
+2. Update `CHANGELOG.md`, `version.rb` and potentially this readme.
+3. Commit the changes, tag them with `vX.Y.Z` (e.g. `v0.2.1`) and push all with `git push --tags`.
+4. Build and publish the new version of the gem with `gem build smstools.gemspec && gem push *.gem`.

data/Rakefile

@@ -1,11 +1,10 @@
 require 'bundler/gem_tasks'
+require 'rake/testtask'
 
-task :test do
-  test_files = Dir[File.expand_path('../spec/**/*_spec.rb', __FILE__)]
-  command    = "ruby -Ispec #{test_files.join ' '}"
+task default: :test
 
-  puts "Running #{command}"
-  system command
+Rake::TestTask.new do |t|
+  t.libs << 'spec'
+  t.test_files = FileList['spec/**/*_spec.rb']
+  t.verbose = true
 end
-
-task default: :test

data/lib/assets/javascripts/sms_tools/message.js.coffee

@@ -2,6 +2,9 @@ window.SmsTools ?= {}
 
 class SmsTools.Message
   maxLengthForEncoding:
+    ascii:
+      normal: 160
+      concatenated: 153
     gsm:
       normal: 160
       concatenated: 153
@@ -20,6 +23,7 @@ class SmsTools.Message
     '€':  true
     '\\': true
 
+  asciiPattern: /^[\x00-\x7F]*$/
   gsmEncodingPattern: /^[0-9a-zA-Z@Δ¡¿£_!Φ"¥Γ#èΛ¤éΩ%ùΠ&ìΨòΣçΘΞ:Ø;ÄäøÆ,<Ööæ=ÑñÅß>Üüåɧà€~ \$\.\-\+\(\)\*\\\/\?\|\^\}\{\[\]\'\r\n]*$/
 
   constructor: (@text) ->
@@ -33,8 +37,25 @@ class SmsTools.Message
 
     concatenatedPartsCount * @maxLengthForEncoding[@encoding][messageType]
 
+  use_gsm_encoding: ->
+    if SmsTools['use_gsm_encoding'] == undefined
+      true
+    else
+      SmsTools['use_gsm_encoding']
+
+   use_ascii_encoding: ->
+    if SmsTools['use_ascii_encoding'] == undefined
+      true
+    else
+      SmsTools['use_ascii_encoding']
+
   _encoding: ->
-    if @gsmEncodingPattern.test(@text) then 'gsm' else 'unicode'
+    if @asciiPattern.test(@text) and @use_ascii_encoding()
+      'ascii'
+    else if @use_gsm_encoding() and @gsmEncodingPattern.test(@text)
+      'gsm'
+    else
+      'unicode'
 
   _concatenatedPartsCount: ->
     encoding = @encoding
@@ -45,9 +66,9 @@ class SmsTools.Message
     else
       parseInt Math.ceil(length / @maxLengthForEncoding[encoding].concatenated), 10
 
-   # Returns the number of symbols, which the given text will take up in an SMS
-   # message, taking into account any double-space symbols in the GSM 03.38
-   # encoding.
+  # Returns the number of symbols which the given text will eat up in an SMS
+  # message, taking into account any double-space symbols in the GSM 03.38
+  # encoding.
   _length: ->
     length = @text.length
 

data/lib/sms_tools.rb

@@ -1,7 +1,28 @@
 require 'sms_tools/version'
 require 'sms_tools/encoding_detection'
 require 'sms_tools/gsm_encoding'
+require 'sms_tools/unicode_encoding'
 
 if defined?(::Rails) and ::Rails.version >= '3.1'
   require 'sms_tools/rails/engine'
 end
+
+module SmsTools
+  class << self
+    def use_gsm_encoding?
+      @use_gsm_encoding.nil? ? true : @use_gsm_encoding
+    end
+
+    def use_gsm_encoding=(value)
+      @use_gsm_encoding = value
+    end
+
+    def use_ascii_encoding?
+      @use_ascii_encoding.nil? ? true : @use_ascii_encoding
+    end
+
+    def use_ascii_encoding=(value)
+      @use_ascii_encoding = value
+    end
+  end
+end

data/lib/sms_tools/encoding_detection.rb

@@ -3,6 +3,10 @@ require 'sms_tools/gsm_encoding'
 module SmsTools
   class EncodingDetection
     MAX_LENGTH_FOR_ENCODING = {
+      ascii: {
+        normal:       160,
+        concatenated: 153,
+      },
       gsm: {
         normal:       160,
         concatenated: 153,
@@ -20,7 +24,18 @@ module SmsTools
     end
 
     def encoding
-      @encoding ||= GsmEncoding.valid?(text) ? :gsm : :unicode
+      @encoding ||=
+        if text.ascii_only? and SmsTools.use_ascii_encoding?
+          :ascii
+        elsif SmsTools.use_gsm_encoding? and GsmEncoding.valid?(text)
+          :gsm
+        else
+          :unicode
+        end
+    end
+
+    def ascii?
+      encoding == :ascii
     end
 
     def gsm?
@@ -49,12 +64,16 @@ module SmsTools
       concatenated_parts * MAX_LENGTH_FOR_ENCODING[encoding][message_type]
     end
 
-     # Returns the number of symbols, which the given text will take up in an SMS
-     # message, taking into account any double-space symbols in the GSM 03.38
-     # encoding.
+    # Returns the number of symbols which the given text will eat up in an SMS
+    # message, taking into account any double-space symbols in the GSM 03.38
+    # encoding.
     def length
-      length = text.length
-      length += text.chars.count { |char| GsmEncoding.double_byte?(char) } if gsm?
+      if unicode?
+        length = text.chars.sum { |char| UnicodeEncoding.character_count(char) }
+      else
+        length = text.length
+        length += text.chars.count { |char| GsmEncoding.double_byte?(char) } if gsm?
+      end
 
       length
     end

data/lib/sms_tools/gsm_encoding.rb

@@ -4,6 +4,8 @@ module SmsTools
   module GsmEncoding
     extend self
 
+    GSM_EXTENSION_TABLE_ESCAPE_CODE = "\x1B".freeze
+
     UTF8_TO_GSM_BASE_TABLE = {
       0x0040 => "\x00", # COMMERCIAL AT
       0x00A3 => "\x01", # POUND SIGN
@@ -14,7 +16,7 @@ module SmsTools
       0x00F9 => "\x06", # LATIN SMALL LETTER U WITH GRAVE
       0x00EC => "\x07", # LATIN SMALL LETTER I WITH GRAVE
       0x00F2 => "\x08", # LATIN SMALL LETTER O WITH GRAVE
-      0x00E7 => "\x09", # LATIN SMALL LETTER C WITH CEDILLA
+      0x00C7 => "\x09", # LATIN CAPITAL LETTER C WITH CEDILLA
       0x000A => "\x0A", # LINE FEED
       0x00D8 => "\x0B", # LATIN CAPITAL LETTER O WITH STROKE
       0x00F8 => "\x0C", # LATIN SMALL LETTER O WITH STROKE
@@ -32,7 +34,7 @@ module SmsTools
       0x03A3 => "\x18", # GREEK CAPITAL LETTER SIGMA
       0x0398 => "\x19", # GREEK CAPITAL LETTER THETA
       0x039E => "\x1A", # GREEK CAPITAL LETTER XI
-      0x00A0 => "\x1B", # ESCAPE TO EXTENSION TABLE
+      nil    => "\x1B", # ESCAPE TO EXTENSION TABLE or NON-BREAKING SPACE
       0x00C6 => "\x1C", # LATIN CAPITAL LETTER AE
       0x00E6 => "\x1D", # LATIN SMALL LETTER AE
       0x00DF => "\x1E", # LATIN SMALL LETTER SHARP S (German)
@@ -176,20 +178,25 @@ module SmsTools
     def to_utf8(gsm_encoded_string)
       utf8_encoded_string = ''
       escape              = false
-      escape_code         = "\e".freeze
 
       gsm_encoded_string.each_char do |char|
-        if char == escape_code
+        if char == GSM_EXTENSION_TABLE_ESCAPE_CODE
           escape = true
         elsif escape
           escape = false
-          utf8_encoded_string << [GSM_TO_UTF8[escape_code + char]].pack('U')
+          utf8_encoded_string << [fetch_utf8_char(GSM_EXTENSION_TABLE_ESCAPE_CODE + char)].pack('U')
         else
-          utf8_encoded_string << [GSM_TO_UTF8[char]].pack('U')
+          utf8_encoded_string << [fetch_utf8_char(char)].pack('U')
         end
       end
 
       utf8_encoded_string
     end
+
+    private
+
+    def fetch_utf8_char(char)
+      GSM_TO_UTF8.fetch(char) { raise "Unsupported symbol in GSM-7 encoding: #{char}" }
+    end
   end
 end

data/lib/sms_tools/unicode_encoding.rb

@@ -0,0 +1,15 @@
+module SmsTools
+  module UnicodeEncoding
+    extend self
+
+    BASIC_PLANE = 0x0000..0xFFFF
+
+    # UCS-2/UTF-16 is used for unicode text messaging. UCS-2/UTF-16 represents characters in minimum
+    # 2-bytes, any characters in the basic plane are represented with 2-bytes, so each codepoint
+    # within the Basic Plane counts as a single character. Any codepoint outside the Basic Plane is
+    # encoded using 4-bytes and therefore counts as 2 characters in a text message.
+    def character_count(char)
+      char.each_codepoint.sum { |codepoint| BASIC_PLANE.include?(codepoint) ? 1 : 2 }
+    end
+  end
+end

data/lib/sms_tools/version.rb

@@ -1,3 +1,3 @@
 module SmsTools
-  VERSION = '0.0.1'
+  VERSION = '0.2.2'
 end

data/spec/sms_tools/encoding_detection_spec.rb

@@ -1,5 +1,5 @@
 require 'spec_helper'
-require 'sms_tools/encoding_detection'
+require 'sms_tools'
 
 describe SmsTools::EncodingDetection do
   it "exposes the original text as a method" do
@@ -7,27 +7,77 @@ describe SmsTools::EncodingDetection do
   end
 
   describe "encoding" do
-    it "defaults to GSM encoding for empty messages" do
-      detection_for('').encoding.must_equal :gsm
+    it "defaults to ASCII encoding for empty messages" do
+      detection_for('').encoding.must_equal :ascii
     end
 
-    it "returns GSM as encoding for simple ASCII text" do
-      detection_for('foo bar baz').encoding.must_equal :gsm
+    it "returns ASCII as encoding for simple ASCII text" do
+      detection_for('foo bar baz').encoding.must_equal :ascii
     end
 
     it "returns GSM as encoding for special symbols defined in GSM 03.38" do
-      detection_for('09azAZ@Δ¡¿£_!Φ"¥Γ#èΛ¤éΩ%ùΠ&ìΨòΣçΘΞ:Ø;ÄäøÆ,<Ööæ=ÑñÅß>Üüåɧà€~').encoding.must_equal :gsm
+      detection_for('09azAZ@Δ¡¿£_!Φ"¥Γ#èΛ¤éΩ%ùΠ&ìΨòΣCΘΞ:Ø;ÄäøÆ,<Ööæ=ÑñÅß>Üüåɧà€~').encoding.must_equal :gsm
     end
 
-    it "returns GSM as encoding for puntucation and newline symbols" do
-      detection_for('Foo bar {} [baz]! Larodi $5. What else?').encoding.must_equal :gsm
-      detection_for("Spaces and newlines are GSM 03.38, too: \r\n").encoding.must_equal :gsm
+    it "returns ASCII as encoding for puntucation and newline symbols" do
+      detection_for('Foo bar {} [baz]! Larodi $5. What else?').encoding.must_equal :ascii
+      detection_for("Spaces and newlines are GSM 03.38, too: \r\n").encoding.must_equal :ascii
     end
 
     it "returns Unicode when non-GSM Unicode symbols are used" do
       detection_for('Foo bar лароди').encoding.must_equal :unicode
       detection_for('∞').encoding.must_equal :unicode
     end
+
+    it 'considers the non-breaking space character as a non-GSM Unicode symbol' do
+      non_breaking_space = "\xC2\xA0"
+
+      detection_for(non_breaking_space).encoding.must_equal :unicode
+    end
+
+    describe 'with SmsTools.use_gsm_encoding = false' do
+      before do
+        SmsTools.use_gsm_encoding = false
+      end
+
+      after do
+        SmsTools.use_gsm_encoding = true
+      end
+
+      it "returns Unicode as encoding for special symbols defined in GSM 03.38" do
+        detection_for('09azAZ@Δ¡¿£_!Φ"¥Γ#èΛ¤éΩ%ùΠ&ìΨòΣCΘΞ:Ø;ÄäøÆ,<Ööæ=ÑñÅß>Üüåɧà€~').encoding.must_equal :unicode
+      end
+
+      it 'returns ASCII for simple ASCII text' do
+        detection_for('Hello world.').encoding.must_equal :ascii
+      end
+
+      it "defaults to ASCII encoding for empty messages" do
+        detection_for('').encoding.must_equal :ascii
+      end
+    end
+
+    describe 'with SmsTools.use_ascii_encoding = false' do
+      before do
+        SmsTools.use_ascii_encoding = false
+      end
+
+      after do
+        SmsTools.use_ascii_encoding = true
+      end
+
+      it "returns GSM 03.38 as encoding for special symbols defined in GSM 03.38" do
+        detection_for('09azAZ@Δ¡¿£_!Φ"¥Γ#èΛ¤éΩ%ùΠ&ìΨòΣCΘΞ:Ø;ÄäøÆ,<Ööæ=ÑñÅß>Üüåɧà€~').encoding.must_equal :gsm
+      end
+
+      it 'returns GSM 03.38 for simple ASCII text' do
+        detection_for('Hello world.').encoding.must_equal :gsm
+      end
+
+      it "defaults to GSM 03.38 encoding for empty messages" do
+        detection_for('').encoding.must_equal :gsm
+      end
+    end
   end
 
   describe "message length" do
@@ -38,7 +88,7 @@ describe SmsTools::EncodingDetection do
     end
 
     it "computes the length of non-trivial GSM encoded messages correctly" do
-      detection_for('GSM: 09azAZ@Δ¡¿£_!Φ"¥Γ#èΛ¤éΩ%ùΠ&ìΨòΣçΘΞ:Ø;ÄäøÆ,<Ööæ=ÑñÅß>Üüåɧà').length.must_equal 63
+      detection_for('GSM: 09azAZ@Δ¡¿£_!Φ"¥Γ#èΛ¤éΩ%ùΠ&ìΨòΣÇΘΞ:Ø;ÄäøÆ,<Ööæ=ÑñÅß>Üüåɧà').length.must_equal 63
     end
 
     it "correctly counts the length of whitespace-only messages" do
@@ -67,6 +117,34 @@ describe SmsTools::EncodingDetection do
       detection_for('Уникод: ^{}[~]|€\\').length.must_equal 17
       detection_for('Уникод: Σ: €').length.must_equal 12
     end
+
+    it "counts ZWJ unicode characters correctly" do
+      detection_for('😴').length.must_equal 2
+      detection_for('🛌🏽').length.must_equal 4
+      detection_for('🤾🏽‍♀️').length.must_equal 7
+      detection_for('🇵🇵').length.must_equal 4
+      detection_for('👩‍❤️‍👩').length.must_equal 8
+    end
+
+    describe 'with SmsTools.use_gsm_encoding = false' do
+      before do
+        SmsTools.use_gsm_encoding = false
+      end
+
+      it "returns ASCII encoded length for some specific symbols which are also in GSM 03.38" do
+        detection_for('[]').length.must_equal 2
+      end
+    end
+
+    describe 'with SmsTools.use_ascii_encoding = false' do
+      before do
+        SmsTools.use_ascii_encoding = false
+      end
+
+      it "returns GSM 03.38 encoded length for some specific symbols which are also in ASCII" do
+        detection_for('[]').length.must_equal 4
+      end
+    end
   end
 
   describe "concatenated message parts counting" do
@@ -96,11 +174,16 @@ describe SmsTools::EncodingDetection do
       concatenated_parts_for length: 135, encoding: :unicode, must_be: 3
     end
 
-    it "counts parts for actual GSM-encoded and Unicode messages" do
+    it "counts parts for actual GSM-encoded messages" do
       detection_for('').concatenated_parts.must_equal 1
-      detection_for('Я').concatenated_parts.must_equal 1
       detection_for('Σ' * 160).concatenated_parts.must_equal 1
       detection_for('Σ' * 159 + '~').concatenated_parts.must_equal 2
+    end
+
+    it "counts parts for actual Unicode-encoded messages" do
+      detection_for('Я').concatenated_parts.must_equal 1
+      detection_for('Я' * 70).concatenated_parts.must_equal 1
+      detection_for('Я' * 71).concatenated_parts.must_equal 2
       detection_for('Я' * 133 + '~').concatenated_parts.must_equal 2
     end
   end

data/spec/sms_tools/gsm_encoding_spec.rb

@@ -0,0 +1,36 @@
+require 'spec_helper'
+require 'sms_tools'
+
+describe SmsTools::GsmEncoding do
+  describe 'from_utf8' do
+    it 'converts simple UTF-8 text to GSM 03.38' do
+      SmsTools::GsmEncoding.from_utf8('simple').must_equal 'simple'
+    end
+
+    it 'converts UTF-8 text with double-byte chars to GSM 03.38' do
+      SmsTools::GsmEncoding.from_utf8('foo []').must_equal "foo \e<\e>"
+    end
+
+    it 'raises an exception if the UTF-8 text contains chars outside of GSM 03.38' do
+      -> { SmsTools::GsmEncoding.from_utf8('баба') }.must_raise RuntimeError, /Unsupported symbol in GSM-7 encoding/
+    end
+  end
+
+  describe 'to_utf8' do
+    it 'converts simple GSM 03.38 to UTF-8' do
+      SmsTools::GsmEncoding.to_utf8('simple').must_equal 'simple'
+    end
+
+    it 'converts UTF-8 text with double-byte chars to GSM 03.38' do
+      SmsTools::GsmEncoding.to_utf8("GSM \e<\e>").must_equal 'GSM []'
+    end
+
+    it 'raises an exception if the UTF-8 text contains chars outside of GSM 03.38' do
+      -> { SmsTools::GsmEncoding.to_utf8('баба') }.must_raise RuntimeError, /Unsupported symbol in GSM-7 encoding/
+    end
+
+    it 'ignores single occurrences of the GSM-7 extension table escape code' do
+      SmsTools::GsmEncoding.to_utf8("\x1B").must_equal ''
+    end
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: smstools
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.2.2
 platform: ruby
 authors:
 - Dimitar Dimitrov
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2014-01-17 00:00:00.000000000 Z
+date: 2021-01-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -86,16 +86,18 @@ files:
 - lib/sms_tools/encoding_detection.rb
 - lib/sms_tools/gsm_encoding.rb
 - lib/sms_tools/rails/engine.rb
+- lib/sms_tools/unicode_encoding.rb
 - lib/sms_tools/version.rb
 - lib/smstools.rb
 - smstools.gemspec
 - spec/sms_tools/encoding_detection_spec.rb
+- spec/sms_tools/gsm_encoding_spec.rb
 - spec/spec_helper.rb
 homepage: https://github.com/mitio/smstools
 licenses:
 - MIT
 metadata: {}
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -110,11 +112,11 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.2.0
-signing_key: 
+rubygems_version: 3.0.3
+signing_key:
 specification_version: 4
 summary: Small library of classes for common SMS-related functionality.
 test_files:
 - spec/sms_tools/encoding_detection_spec.rb
+- spec/sms_tools/gsm_encoding_spec.rb
 - spec/spec_helper.rb