enco 0.1.0

data/.gitignore

@@ -0,0 +1,13 @@
+*.gem
+.bundle
+Gemfile.lock
+pkg/*
+.idea/
+doc/
+log/*.log
+/tmp/
+tmp/
+db/*.sqlite3
+/coverage
+/coverage.data
+/.yardoc

data/.rspec

@@ -0,0 +1 @@
+--colour --format documentation

data/.travis.yml

@@ -0,0 +1,4 @@
+rvm:
+  - 1.9.2
+  - 1.9.3
+  - ruby-head

data/CONTRIBUTING.md

@@ -0,0 +1,161 @@
+Contributing to Enco
+==========================
+Enco is an open source project. Anyone can use the code, but more importantly, anyone can contribute. This is a
+ group effort and we value your input. Please consider making a contribution to help improve the Enco. This
+ guide covers:
+
+* How to file a ticket when you discover a bug
+* How to contribute fixes and improvements to the core
+* Information on how to improve the documentation
+
+1. Who can Contribute?
+----------------------
+Enco is an open source project and as such contributions are always welcome. Our community is one which
+ encourages involvement from all developers regardless of their ability level. We ask that you be patient with the
+ other members of the community and maintain a respectful attitude towards other people’s work. Open source is a
+ great way to learn a new technology so don’t be afraid to jump right in, even if you are new to Ruby/Rails.
+
+2. Before you Contribute
+------------------------
+Open source projects tend to be a collaborative effort. Since many people are relying upon Enco for their real
+ world applications, changes to the code can have major implications. Before you write a bug fix or code a new
+ feature, you should find out if anybody is interested in your proposed change. You may find that the thing you’re
+ trying to “fix” is actually desired behavior. You might also discover that someone else is working on it. Either
+ way you can save yourself valuable time by announcing your intentions before starting work.
+
+### 2.1. Notification via Ticket
+
+You can also search existing bug reports/issues and file a new one if you do not find an issue relevant to your
+ proposed change. See Filing an Issue for more details.
+
+The important thing is that you communicate your intention in advance of doing a lot of work. Simple bug fixes and
+ non-controversial changes do not require this approach but you can save some time by suggesting an improvement and
+ having it rejected before you write a bunch of the code.
+
+3. Filing an Issue
+-----------------
+If you would like to file a bug report, please create an issue in our Github Issues Tracker. You should do a basic
+ search of the issues database before creating a new issue to ensure that you are not creating a duplicate issue.
+
+Please do not assign labels or create new labels to your issue. We will assign the appropriate labels to ensure your
+ ticket is handled in the appropriate manner.
+
+### 3.1. Providing a Patch
+
+If you are filing and issue and supplying a patch at the same time, please file a Pull Request instead. The pull
+ request will also create an issue at the same time but its superior to just creating an issue because the code and
+ issue can be linked.
+
+If the ticket already exists, however, and you want to supply a patch after the fact,
+ you can simply reference the issue number in your commit message. For example, if your commit fixed issue #123 you
+ could use the following commit message:
+
+    Fixed a problem with Facebook authentication.
+
+    [Fixes #123]
+Github will automatically detect this commit message when you push it and link the issue. Please see the detailed
+ Github Issues blog post for more details.
+
+### 3.2. Feature Requests
+
+We’re interested in hearing your ideas for new features but creating feature requests in the Issue Tracker is not
+ the proper way to ask for a feature. A feature request is any idea you have to improve the software experience that
+ is not strictly related to a bug or error of omission.
+
+Feature requests that are accompanied by source code are always welcome. In this case you should read the next
+ section on Creating a Pull Request.
+
+Feature requests without accompanying code will be closed. We simply cannot respond efficiently to feature requests
+ through our Issue Tracker. If you want to suggest a feature, please use the mailing list or the user voice forum.
+
+### 3.3. How We Prioritize Issues
+
+We try our best to respond to all of the questions and issues our users have. We use the following criteria to
+prioritize issues:
+
+* Does this bug effect the latest stable release?
+* Is there a patch associated with the issue?
+* Is ther a test included in the patch?
+* Has someone else verified the bug?
+* Are there details on how to reproduce the problem?
+* We give highest priority to issues where the answer is “yes” to all of these questions. Next highest priority is for
+ issues that answer “yes” to most of these questions, particularly the first few criteria.
+
+You need to include a brief description of the problem and simple steps needed to reproduce it. If you fail to
+ supply this minimum level of information your issue will likely be ignored.
+
+
+4. Creating a Pull Request
+--------------------------
+
+If you are going to contribute code to the Enco project, the best mechanism for doing this is to create a pull
+ request in Github. If you’re unfamiliar with the general concept of pull requests you may want to read more on pull
+ requests in Github.
+
+If your code is associated with an existing issue then you can provide a patch instead of creating a pull request.
+
+### 4.1. Creating a Fork
+
+The official Enco source code is maintained in Github under the AlexParamonov/enco
+
+You simply need to “fork” the project and then start hacking.
+
+See the Github guide on creating forks for more details.
+
+### 4.2. Topic Branches
+
+Git branches are “cheap.” Creating branches in Git is incredibly easy and its an ideal way to isolate a specific set
+ of changes. You may be fixing several things at one time but by keeping your changes isolated it will help us to
+ find and apply only the changes we’re interested in. You should create a clean branch based on the latest
+ enco/master when doing this. It is important you follow these steps exactly,
+ it will prevent you from accidentally including unrelated changes from your local repository into the branch.
+
+For example, if we were submitting a patch to fix an issue with the CSS in the flash error message you could create
+ a branch as follows:
+
+    $ git remote add upstream git://github.com/AlexParamonov/enco.git
+    $ git fetch upstream
+    $ git checkout -b fix-css-for-error-flash --track upstream/master
+
+The fetch command will grab all of the latest commits from the Enco master branch. Don’t worry,
+ it doesn’t alter your working repository in a harmful way. The track part of the command will tell git that this
+ branch should track with the remote version of the upstream master. This is another way of saying that the branch
+ should be based on a clean copy of the latest official source code (without any of your unrelated local changes.)
+
+You can then do work locally on this topic branch and push it up to your Github fork when you are done. So in our
+ previous example we do something like:
+
+    $ git push origin fix-css-for-error-flash
+
+Of course if you want the fix for yourself to use in your own local code you should probably merge it down to your
+ own personal master branch that you’re using for development
+
+    $ git checkout master
+    $ git merge fix-css-for-error-flash
+
+You should probably also clean up after yourself a little. The branch has been pushed to Github and you’ve merged it
+  locally so you don’t really need a local copy of the branch laying around.
+
+    $ git branch -D fix-css-for-error-flash
+
+### 4.3. Including a Test
+
+Ideally your pull request will also include a test that verifies a bug (or the absence of the new feature) before
+ your fix and also verifies proper functionality when you are finished. Please read the Testing Guide for more
+ information on writing and running your tests.
+
+Pull requests with tests are given top priority. Failure to include a test will likely delay acceptance of your patch.
+
+### 4.4. Creating the Pull Request
+
+Once your code is ready to go and you have pushed your topic branch to Github then you are ready to create the pull
+ request and notify the Enco team that your contribution is ready. You do this by browsing your project in
+ Github and changing to the topic branch you just pushed. Once you are on the topic branch simply create a pull
+ request by pressing the “Pull Request” button.
+
+The Github guide on pull requests describes this in more detail with screenshots if you’re still confused on this
+ part.
+
+5 Contributing to the Documentation
+-----------------------------------
+Improvements to the documentation is encouraged.

data/Gemfile

@@ -0,0 +1,4 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in enco.gemspec
+gemspec

data/HISTORY.md

@@ -0,0 +1,7 @@
+0.1.0
+=====
+Main functionality implemented
+
+0.0.1
+=====
+Initial release. Add README, LICENSE, .gitignore, etc.

data/LICENSE

@@ -0,0 +1,7 @@
+Copyright (c) 2011 Alexander N Paramonov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,61 @@
+Enco
+==========
+Enco will convert any string to utf-8
+
+Installation
+------------
+    gem install enco
+
+Usage
+-----
+It is simple. Just call
+
+    my_utf8_string = Enco.to_utf8 any_string
+
+It will return non string objects back:
+
+    Enco.to_utf8(not_a_string) === not_a_string
+
+works correctly with frozen strings. If you dont want to convert frozen string, pass :ignore_frozen => true and Enco
+will ignore that string:
+
+    Enco.to_utf8 any_string.frozen, :ignore_frozen => true # returns frozen string
+    Enco.to_utf8 any_string.frozen                         # returns dup of any_string, converted to utf-8
+
+### Plugins
+Add
+
+    require 'enco/string_to_utf8'
+
+end you'll get
+
+    "any string".to_utf8
+method that accept same options as
+
+    Enco.to_utf8 "any string"
+
+
+Requirements
+------------
+
+* Ruby 1.9
+* rchardet19
+
+Compatibility
+-------------
+tested with Ruby
+
+* 1.9.2
+* 1.9.3
+* ruby-head
+
+see [build history](http://travis-ci.org/#!/AlexParamonov/enco/builds)
+
+Contributing
+-------------
+see [contributing guide](http://github.com/AlexParamonov/enco/blob/master/CONTRIBUTING.md)
+
+Copyright
+---------
+Copyright © 2011 Alexander N Paramonov.
+Released under the MIT License. See the LICENSE file for further details.

data/Rakefile

@@ -0,0 +1,12 @@
+# encoding: utf-8
+# GEMS
+require "bundler/gem_tasks"
+
+# SPECS
+require 'rspec/core'
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec) do |spec|
+  spec.pattern = FileList['spec/**/*_spec.rb']
+end
+#Rake::Task["spec"].execute
+task :default => :spec

data/enco.gemspec

@@ -0,0 +1,26 @@
+# -*- encoding: utf-8 -*-
+$:.push File.expand_path("../lib", __FILE__)
+require "enco/version"
+
+Gem::Specification.new do |s|
+  s.name        = "enco"
+  s.version     = Enco::VERSION
+  s.authors     = ["Alexander Paramonov"]
+  s.email       = ["alexander.n.paramonov@gmail.com"]
+  s.homepage    = "http://github.com/AlexParamonov/enco"
+  s.summary     = %q{Nice utf8 converter}
+  s.description = %q{Enco will convert any string to utf-8.}
+
+  s.rubyforge_project = "enco"
+
+  s.files         = `git ls-files`.split("\n")
+  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
+  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  s.require_paths = ["lib"]
+
+  # specify any dependencies here; for example:
+  s.add_development_dependency "rspec", ">= 2.6"
+  s.add_development_dependency "rake"
+  s.add_runtime_dependency "rchardet19"
+  #s.add_runtime_dependency "activesupport"
+end

data/lib/enco.rb

@@ -0,0 +1,32 @@
+require 'rchardet19'
+
+# Enco will convert any string to utf-8
+# Just call Enco.to_utf8 any_string
+# It will return non string objects back: Enco.to_utf8(not_a_string) === not_a_string
+module Enco
+  # @param [String] string
+  # @param [Hash] options
+  # @option options [Boolean] :ignore_frozen
+  def self.to_utf8 string, options = {}
+    return string unless string.is_a? String
+    if string.frozen?
+      return string if true == options[:ignore_frozen]
+      string = string.dup
+    end
+
+    cd = ::CharDet.detect(string, :silent => true)
+
+    if cd.confidence > 0.5 then
+      string.encode('utf-8', cd.encoding, :invalid => :replace)
+    else
+      force_to_utf8 string
+    end
+  end
+
+  private
+  def self.force_to_utf8 string
+    require 'iconv'
+    ic = Iconv.new('UTF-8//IGNORE', 'UTF-8')
+    ic.iconv(string + ' ')[0..-2]
+  end
+end

data/lib/enco/string_to_utf8.rb

@@ -0,0 +1,6 @@
+require "enco"
+class String
+  def to_utf8 *args
+    Enco.to_utf8 self, *args
+  end
+end

data/lib/enco/version.rb

@@ -0,0 +1,3 @@
+module Enco
+  VERSION = "0.1.0"
+end

data/spec/enco_spec.rb

@@ -0,0 +1,86 @@
+# -*- encoding: utf-8 -*-
+require "spec_helper"
+
+# TODO add more Test cases
+describe Enco do
+  before(:each) do
+    @simple_string   = %Q{test string\nin }
+    @cyrillic_string = %Q{тестовая строка\nin }
+  end
+  describe "#to_utf8" do
+    describe "argument is" do
+      context "a string in" do
+        specify "UTF-8 encoding" do
+          result = Enco.to_utf8(string_in 'UTF-8')
+          result.should eq @cyrillic_string + "UTF-8\n"
+          result.encoding.to_s.should == "UTF-8"
+        end
+
+        specify "KOI8-R encoding" do
+          result = Enco.to_utf8(string_in 'KOI8-R')
+          result.should eq @cyrillic_string + "KOI8-R\n"
+          result.encoding.to_s.should == "UTF-8"
+        end
+
+        specify "CP866 encoding" do
+          result = Enco.to_utf8(string_in 'CP866')
+          result.should eq @cyrillic_string + "CP866\n"
+          result.encoding.to_s.should == "UTF-8"
+        end
+
+        specify "ISO-8859-15 encoding" do
+          result = Enco.to_utf8(string_in 'ISO-8859-15')
+          result.should eq @simple_string + "ISO-8859-15\n"
+          result.encoding.to_s.should == "UTF-8"
+        end
+
+        specify "WINDOWS-1251 encoding" do
+          result = Enco.to_utf8(string_in 'WINDOWS-1251')
+          result.should eq @cyrillic_string + "WINDOWS-1251\n"
+          result.encoding.to_s.should == "UTF-8"
+        end
+      end
+
+      context "a frozen string" do
+        before(:each) do
+          @test_string = "test string".force_encoding "ISO-8859-15"
+        end
+
+        it "should perform conversion" do
+          result = Enco.to_utf8 @test_string.freeze
+          result.encoding.to_s.should == "UTF-8"
+        end
+
+        it "should not change input string" do
+          Enco.to_utf8 @test_string.freeze
+
+          @test_string.encoding.to_s.should == "ISO-8859-15"
+        end
+
+        context "ignore_frozen flag set to true" do
+          it "should return string itself" do
+            result = Enco.to_utf8 @test_string.freeze, :ignore_frozen => true
+            result.should eq @test_string
+            result.encoding.to_s.should == "ISO-8859-15"
+          end
+        end
+      end
+
+      context "not a string" do
+        describe "should ignore argument and return it back for" do
+          specify "nil" do
+            result = Enco.to_utf8 nil
+            result.should be_nil
+          end
+
+          specify "object" do
+            object = Object.new
+            result = Enco.to_utf8 object
+            result.should eql object
+          end
+        end
+      end
+
+    end
+  end
+end

data/spec/fixtures/CP866

@@ -0,0 +1,2 @@
+��⮢�� ��ப�
+in CP866

data/spec/fixtures/ISO-8859-15

@@ -0,0 +1,2 @@
+test string
+in ISO-8859-15

data/spec/fixtures/KOI8-R

@@ -0,0 +1,2 @@
+�������� ������
+in KOI8-R

data/spec/fixtures/UTF-8

@@ -0,0 +1,2 @@
+тестовая строка
+in UTF-8

data/spec/fixtures/WINDOWS-1251

@@ -0,0 +1,2 @@
+�������� ������
+in WINDOWS-1251

data/spec/spec_helper.rb

@@ -0,0 +1,2 @@
+require "enco"
+Dir[File.dirname(File.expand_path(__FILE__)) + "/support/**/*.rb"].each {|f| require f}

data/spec/string_to_utf8_spec.rb

@@ -0,0 +1,23 @@
+# -*- encoding: utf-8 -*-
+require "spec_helper"
+require "enco/string_to_utf8"
+
+# TODO add more Test cases
+describe String do
+  describe "#to_utf8" do
+    before(:each) do
+      @test_string = "test string"
+    end
+    it "should delegate to Enco#to_utf8" do
+      Enco.should_receive(:to_utf8).with(@test_string).once
+
+      @test_string.to_utf8
+    end
+
+    it "should pass options to Enco#to_utf8" do
+      Enco.should_receive(:to_utf8).with(@test_string, :some_param => true).once
+
+      @test_string.to_utf8 :some_param => true
+    end
+  end
+end

data/spec/support/encoding_helper.rb

@@ -0,0 +1,3 @@
+def string_in(encoding)
+  File.read(File.dirname(File.expand_path(__FILE__)) << "/../fixtures/" << encoding.upcase, :encoding => "BINARY")
+end

metadata

@@ -0,0 +1,100 @@
+--- !ruby/object:Gem::Specification
+name: enco
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+  prerelease: 
+platform: ruby
+authors:
+- Alexander Paramonov
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2011-12-04 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: &16372380 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '2.6'
+  type: :development
+  prerelease: false
+  version_requirements: *16372380
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: &16371400 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: *16371400
+- !ruby/object:Gem::Dependency
+  name: rchardet19
+  requirement: &16370800 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: *16370800
+description: Enco will convert any string to utf-8.
+email:
+- alexander.n.paramonov@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- .rspec
+- .travis.yml
+- CONTRIBUTING.md
+- Gemfile
+- HISTORY.md
+- LICENSE
+- README.md
+- Rakefile
+- enco.gemspec
+- lib/enco.rb
+- lib/enco/string_to_utf8.rb
+- lib/enco/version.rb
+- spec/enco_spec.rb
+- spec/fixtures/CP866
+- spec/fixtures/ISO-8859-15
+- spec/fixtures/KOI8-R
+- spec/fixtures/UTF-8
+- spec/fixtures/WINDOWS-1251
+- spec/spec_helper.rb
+- spec/string_to_utf8_spec.rb
+- spec/support/encoding_helper.rb
+homepage: http://github.com/AlexParamonov/enco
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: enco
+rubygems_version: 1.8.12
+signing_key: 
+specification_version: 3
+summary: Nice utf8 converter
+test_files: []