shortener 0.5.5 → 0.5.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 0b5132743f6f7bf1ccab83e28c3ae18f822b277a
-  data.tar.gz: 58d20decd09055a4a76a21548e66010acac00da5
+  metadata.gz: 0f710d548cc2a095e643ed04c786691a723f1449
+  data.tar.gz: e3fc832851d5e4f72750883844cc6eaee97c5e65
 SHA512:
-  metadata.gz: 1f307fff4f9bdb34387849e33c424c109606f1a0abd8c55b769c4b96da08dedbc1da3380748cb8b5ef2ff4c58603878a2c372ce9054a4052dd25d563c7619326
-  data.tar.gz: 0c682ae7f3f3ef7fd99020e8ee31e69124f4f1d6bb825962c6eca853afa6db11f1c6027778f4a13f555cae91951977718022bfd3aa2156e12d907f68e78e131a
+  metadata.gz: e89866e4b81029b37adcadbe79e42cca296b57a60fb1c6fd4e50860258b2837d9c8250e2e4924f9d56f3026f1ac76b35b54a45ffdb87f43b72c55a44139c371d
+  data.tar.gz: 3b79ef1a1fbf5b9145431bc239f1ab7aac331d37188cb37516f13a92415b48bac1a6601abb7190526cf14b19c86816c44b1eb3d0c9eb6925081f987d2007d8bd

data/README.rdoc

@@ -1,4 +1,4 @@
-{<img src="https://secure.travis-ci.org/jpmcgrath/shortener.png?branch=master" alt="Build Status" />}[http://travis-ci.org/jpmcgrath/shortener]
+{<img src="https://secure.travis-ci.org/jpmcgrath/shortener.png?branch=master" alt="Build Status" />}[http://travis-ci.org/jpmcgrath/shortener]  {<img src="https://codeclimate.com/github/jpmcgrath/shortener/badges/gpa.svg" />}[https://codeclimate.com/github/jpmcgrath/shortener]
 
 = Shortener
 
@@ -52,7 +52,6 @@ migration:
 * There has not been an attempt to remove ambiguous characters (i.e. 1 l and capital i, or 0 and O etc.) from the unique key generated for the link. This means people might copy the link incorrectly if copying the link by hand;
 * The system could pre-generate unique keys in advance, avoiding the database penalty when checking that a newly generated key is unique;
 * The system could store the shortened URL if the url is to be continually rendered;
-* Some implementations might want duplicate links to be generated each time a user request a shortened link.
 
 == Installation
 
@@ -102,7 +101,7 @@ To generate and display a shortened URL in your application use the helper metho
 
 Pass in subdomain, protocol and other options that the UrlHelper url_for accepts:
 
-  short_url("http://example.com", url_options: { subdomain: 'foo', host: 'bar, protocol: 'https' } )
+  short_url("http://example.com", url_options: { subdomain: 'foo', host: 'bar', protocol: 'https' } )
 
 This will generate a shortened URL. store it to the db and return a string representing the shortened URL.
 
@@ -159,12 +158,30 @@ In rails you can put next line into config/initializers/shortener.rb
 
   Shortener.forbidden_keys.concat %w(terms promo)
 
-=== Helper Parameters
+=== Ignoring Robots
+
+By default Shortener will count all visits to a shortened url, including any crawler
+robots like the Google Web Crawler, or Twitter's link unshortening bot. To ignore
+these visits, Shortener makes use of the excellent voight_kampff gem to identify
+web robots. This feature is disabled by default. To enable add the following to
+your shortener configuration:
+
+  Shortener.ignore_robots = true
+
+=== URL Parameters
 
 Parameters are passed though from the shortened url, to the destination URL. If the destination
 URL has the same parameters as the destination URL, the parameters on the shortened url take
 precedence over those on the destination URL.
 
+For example, if we have an orginal URL of:
+> http://destination.com?test=yes&happy=defo (idenfified with token ABCDEF)
+Which is shortened into:
+> http://coolapp.io/s/ABCDEF?test=no&why=not
+Then, the resulting URL will be:
+>http://destination.com?test=no&happy=defo&why=not
+Note how the test parameter takes the value given on the short URL.
+
 === Shorten URLs in generated emails
 
 You can register the included mail interceptor to shorten all links in the emails generated by your Rails app. For example, add to your mailer:
@@ -181,6 +198,26 @@ This will replace all long URLs in the emails generated by MyMailer with shorten
 
 The interceptor supports a few more arguments, see the implementation for details.
 
+=== Logging, stats and other tricks
+
+If you want more things to happen when a user accesses one of your short urls, you can create your own `show` action as follows:
+
+    def show
+      token = ::Shortener::ShortenedUrl.extract_token(params[:id])
+      url   = ::Shortener::ShortenedUrl.fetch_with_token(token: token, additional_params: params)
+      # do some logging, store some stats
+      redirect_to url[:url], status: :moved_permanently
+    end
+
+=== Fetch with Token
+  The `::Shortener::ShortenedUrl.fetch_with_token(token: token, additional_params: params)` does the following:
+  1. finds the ShortenedUrl for the supplied token
+  2. increments the use_count for the retrieved ShortenedURL
+  3. combines the additional parameters with the URL in the retrieved ShortenedURL
+  4. returns a hash of the format `{url: <the original url>, shortened_url: <the retrieved ShortenedURL>}
+
+  *note:* If no shortened URL is found, the url will be `default_redirect` or `/`
+
 == Origins
 
 For a bit of backstory to Shortener see this {blog post}[http://jamespmcgrath.com/a-simple-link-shortener-in-rails/].
@@ -189,7 +226,8 @@ For a bit of backstory to Shortener see this {blog post}[http://jamespmcgrath.co
 
 Shortener is used in a number of production systems, including, but not limited to:
 
-{Doorkeeper - An Event Management Tool}[http://www.doorkeeperhq.com/]
+- {Doorkeeper - An Event Management Tool}[http://www.doorkeeperhq.com/]
+- {1001tweets - Repost your tweets to get more clicks}[http://www.1001tweets.com/]
 
 If you are using Shortener in your project and would like to be added to this list, please get in touch!
 

data/app/controllers/shortener/shortened_urls_controller.rb

@@ -1,41 +1,12 @@
+require 'voight_kampff'
+
 class Shortener::ShortenedUrlsController < ActionController::Base
+  include Shortener
 
-  # find the real link for the shortened link key and redirect
   def show
-    # only use the leading valid characters
-    token = /^([#{Shortener.key_chars.join}]*).*/.match(params[:id])[1]
-
-    # pull the link out of the db
-    sl = ::Shortener::ShortenedUrl.unexpired.where(unique_key: token).first
-
-    if sl
-      # don't want to wait for the increment to happen, make it snappy!
-      # this is the place to enhance the metrics captured
-      # for the system. You could log the request origin
-      # browser type, ip address etc.
-      Thread.new do
-        sl.increment!(:use_count)
-        ActiveRecord::Base.connection.close
-      end
-
-      params.except! *[:id, :action, :controller]
-      url = sl.url
-
-      if params.present?
-        uri = URI.parse(sl.url)
-        existing_params = Rack::Utils.parse_nested_query(uri.query)
-        merged_params   = existing_params.merge(params)
-        uri.query       = merged_params.to_query
-        url             = uri.to_s
-      end
-
-      # do a 301 redirect to the destination url
-      redirect_to url, status: :moved_permanently
-    else
-      # if we don't find the shortened link, redirect to the root
-      # make this configurable in future versions
-      redirect_to Shortener.default_redirect
-    end
+    token = ::Shortener::ShortenedUrl.extract_token(params[:id])
+    track = Shortener.ignore_robots.blank? || request.human?
+    url   = ::Shortener::ShortenedUrl.fetch_with_token(token: token, additional_params: params, track: track)
+    redirect_to url[:url], status: :moved_permanently
   end
-
 end

data/app/helpers/shortener/shortener_helper.rb

@@ -10,7 +10,7 @@ module Shortener::ShortenerHelper
                                                 )
 
     if short_url
-      options = { controller: :"shortener/shortened_urls", action: :show, id: short_url.unique_key, only_path: false }.merge(url_options)
+      options = { controller: :"/shortener/shortened_urls", action: :show, id: short_url.unique_key, only_path: false }.merge(url_options)
       url_for(options)
     else
       url

data/app/models/shortener/shortened_url.rb

@@ -10,6 +10,8 @@ class Shortener::ShortenedUrl < ActiveRecord::Base
   # exclude records in which expiration time is set and expiration time is greater than current time
   scope :unexpired, -> { where(arel_table[:expires_at].eq(nil).or(arel_table[:expires_at].gt(::Time.current.to_s(:db)))) }
 
+  attr_accessor :custom_key
+
   # ensure the url starts with it protocol and is normalized
   def self.clean_url(url)
 
@@ -26,11 +28,11 @@ class Shortener::ShortenedUrl < ActiveRecord::Base
   def self.generate!(destination_url, owner: nil, custom_key: nil, expires_at: nil, fresh: false)
     # if we get a shortened_url object with a different owner, generate
     # new one for the new owner. Otherwise return same object
-    if destination_url.is_a? Shortener::ShortenedUrl
+    result = if destination_url.is_a? Shortener::ShortenedUrl
       if destination_url.owner == owner
-        result = destination_url
+        destination_url
       else
-        result = generate!(destination_url.url,
+        generate!(destination_url.url,
                             owner:      owner,
                             custom_key: custom_key,
                             expires_at: expires_at,
@@ -39,12 +41,8 @@ class Shortener::ShortenedUrl < ActiveRecord::Base
       end
     else
       scope = owner ? owner.shortened_urls : self
-
-      if fresh
-        result = scope.where(url: clean_url(destination_url)).create(unique_key: custom_key, expires_at: expires_at)
-      else
-        result = scope.where(url: clean_url(destination_url)).first_or_create(unique_key: custom_key, expires_at: expires_at)
-      end
+      creation_method = fresh ? 'create' : 'first_or_create'
+      scope.where(url: clean_url(destination_url)).send(creation_method, unique_key: custom_key, custom_key: custom_key, expires_at: expires_at)
     end
 
     result
@@ -60,14 +58,61 @@ class Shortener::ShortenedUrl < ActiveRecord::Base
     end
   end
 
+  def self.extract_token(token_str)
+    # only use the leading valid characters
+    # escape to ensure custom charsets with protected chars do not fail
+    /^([#{Regexp.escape(Shortener.key_chars.join)}]*).*/.match(token_str)[1]
+  end
+
+  def self.fetch_with_token(token: nil, additional_params: {}, track: true)
+    shortened_url = ::Shortener::ShortenedUrl.unexpired.where(unique_key: token).first
+
+    url = if shortened_url
+      shortened_url.increment_usage_count if track
+      merge_params_to_url(url: shortened_url.url, params: additional_params)
+    else
+      Shortener.default_redirect || '/'
+    end
+
+    { url: url, shortened_url: shortened_url }
+  end
+
+  def self.merge_params_to_url(url: nil, params: {})
+    params.try(:except!, *[:id, :action, :controller])
+
+    if params.present?
+      uri = URI.parse(url)
+      existing_params = Rack::Utils.parse_nested_query(uri.query)
+      uri.query       = existing_params.symbolize_keys.merge(params).to_query
+      url = uri.to_s
+    end
+
+    url
+  end
+
+  def increment_usage_count
+    Thread.new do
+      ActiveRecord::Base.connection_pool.with_connection do |conn|
+        increment!(:use_count)
+      end
+    end
+  end
+
+  def to_param
+    unique_key
+  end
+
   private
 
   # the create method changed in rails 4...
   CREATE_METHOD_NAME =
-    if Rails::VERSION::MAJOR >= 4
+    if Rails::VERSION::MAJOR >= 5
+      "_create_record"
+    elsif Rails::VERSION::MAJOR == 4
       # And again in 4.0.6/4.1.2
-      if ((Rails::VERSION::MINOR == 0) && (Rails::VERSION::TINY < 6)) ||
-         ((Rails::VERSION::MINOR == 1) && (Rails::VERSION::TINY < 2))
+      if Rails::VERSION::MAJOR == 4 && (
+          ((Rails::VERSION::MINOR == 0) && (Rails::VERSION::TINY < 6)) ||
+          ((Rails::VERSION::MINOR == 1) && (Rails::VERSION::TINY < 2)))
         "create_record"
       else
         "_create_record"
@@ -81,9 +126,11 @@ class Shortener::ShortenedUrl < ActiveRecord::Base
   define_method CREATE_METHOD_NAME do
     count = 0
     begin
-      self.unique_key = generate_unique_key
+      self.unique_key = custom_key || generate_unique_key
       super()
     rescue ActiveRecord::RecordNotUnique, ActiveRecord::StatementInvalid => err
+      logger.info("Failed to generate ShortenedUrl with unique_key: #{unique_key}")
+      self.unique_key = nil
       if (count +=1) < 5
         logger.info("retrying with different unique key")
         retry

data/lib/shortener.rb

@@ -28,6 +28,11 @@ module Shortener
   mattr_accessor :forbidden_keys
   self.forbidden_keys = []
 
+  # ignore_robots - set to true to not count visits by identified webcrawlers
+  mattr_accessor :ignore_robots
+  self.ignore_robots = false
+
+
   def self.key_chars
     CHARSETS[charset]
   end

data/lib/shortener/version.rb

@@ -1,3 +1,3 @@
 module Shortener
-  VERSION = '0.5.5'
+  VERSION = '0.5.6'
 end

data/shortener.gemspec

@@ -15,6 +15,7 @@ Gem::Specification.new do |s|
   s.rubyforge_project         = "shortener"
   s.required_rubygems_version = "> 1.3.6"
   s.add_dependency "rails", ">= 3.0.7"
+  s.add_dependency "voight_kampff", '~>1.0.2'
   s.add_development_dependency "sqlite3"
   s.add_development_dependency "rspec-rails", '~> 3.3.0'
   s.add_development_dependency "shoulda-matchers", '~> 3'

data/spec/controllers/shortened_urls_controller_spec.rb

@@ -18,6 +18,26 @@ describe Shortener::ShortenedUrlsController, type: :controller do
         it 'redirects to the destination url' do
           expect(response).to redirect_to destination
         end
+
+        context "when request is not from an human" do
+          before do
+            allow_any_instance_of(Rack::Request).to receive(:human?).and_return(false)
+          end
+          context 'Shortener.ignore_robots == true' do
+            it "calls fetch with token with track argument as false" do
+              Shortener.ignore_robots = true
+              expect(Shortener::ShortenedUrl).to receive(:fetch_with_token).with(hash_including(track: false)).and_call_original
+              get :show, { id: key }
+            end
+          end
+          context 'Shortener.ignore_robots == false (default)' do
+            it "calls fetch with token with track argument as true" do
+              Shortener.ignore_robots = false
+              expect(Shortener::ShortenedUrl).to receive(:fetch_with_token).with(hash_including(track: true)).and_call_original
+              get :show, { id: key }
+            end
+          end
+        end
       end
 
       context 'real key with trailing characters' do

data/spec/models/shortened_url_spec.rb

@@ -154,4 +154,114 @@ describe Shortener::ShortenedUrl, type: :model do
       expect(unexpired_scope).not_to include(expired_url)
     end
   end
+
+  describe '#increment_usage_count' do
+    let(:url) { 'https://example.com'}
+    let(:short_url) { Shortener::ShortenedUrl.generate!(url) }
+    it 'increments the use_count on the shortenedLink' do
+      original_count = short_url.use_count
+      short_url.increment_usage_count
+      sleep 0.001
+      expect(short_url.use_count).to eq (original_count + 1)
+    end
+  end
+
+  describe '#merge_params_to_url' do
+    context 'without params on original url' do
+      let(:url) { 'https://example.com'}
+
+      context 'no additional params' do
+        let(:params) { nil }
+
+        it 'returns the original url' do
+          expect(Shortener::ShortenedUrl.merge_params_to_url(url: url, params: params)).to eq 'https://example.com'
+        end
+      end
+
+      context 'additional params' do
+        let(:params) { {foo: 'bar', hello: 'world' } }
+
+        it 'merges the params to create a new url' do
+          expect(Shortener::ShortenedUrl.merge_params_to_url(url: url, params: params)).to eq 'https://example.com?foo=bar&hello=world'
+        end
+      end
+    end
+
+    context 'with params on original url' do
+      let(:url) { 'http://example.com/pathname?different=yes&foo=test'}
+
+      context 'no additional params' do
+        let(:params) { nil }
+
+        it 'returns the original url' do
+          expect(Shortener::ShortenedUrl.merge_params_to_url(url: url, params: params)).to eq 'http://example.com/pathname?different=yes&foo=test'
+        end
+      end
+
+      context 'additional params' do
+        let(:params) { {foo: 'manchoo', hello: 'world' } }
+
+        it 'merges the params to create a new url with the new params taking preceedence' do
+          expect(Shortener::ShortenedUrl.merge_params_to_url(url: url, params: params)).to eq 'http://example.com/pathname?different=yes&foo=manchoo&hello=world'
+        end
+      end
+    end
+  end
+
+  describe '#fetch_with_token' do
+    shared_examples_for 'invalid token supplied' do
+      context 'default_path set' do
+        it 'returns the default_redirect url, but no shortened url' do
+          Shortener.default_redirect = "/default_path"
+          result = Shortener::ShortenedUrl.fetch_with_token(token: nil)
+          expect(result[:url]).to eq '/default_path'
+          expect(result[:shortened_url]).to eq nil
+        end
+      end
+      context 'no default_path set' do
+        it 'returns the root path, but no shortened url' do
+          Shortener.default_redirect = nil
+          result = Shortener::ShortenedUrl.fetch_with_token(token: nil)
+          expect(result[:url]).to eq '/'
+          expect(result[:shortened_url]).to eq nil
+        end
+      end
+    end
+
+    context 'invalid token' do
+      it_should_behave_like 'invalid token supplied' do
+        let(:token) { 'invalid_token'}
+      end
+    end
+
+    context 'valid token' do
+      let(:url) { Faker::Internet.url }
+
+      context 'non expired short url' do
+        let(:short_url) { Shortener::ShortenedUrl.generate!(url) }
+        it 'returns both the url and the shortened url' do
+          result = Shortener::ShortenedUrl.fetch_with_token(token: short_url.unique_key)
+          expect(result[:url]).to eq url
+          expect(result[:shortened_url]).to eq short_url
+        end
+        it "increments use count" do
+          expect_any_instance_of(Shortener::ShortenedUrl).to receive(:increment_usage_count)
+          Shortener::ShortenedUrl.fetch_with_token(token: short_url.unique_key)
+        end
+        context "with track set to false" do
+          it "does not increment use count" do
+            expect_any_instance_of(Shortener::ShortenedUrl).not_to receive(:increment_usage_count)
+            Shortener::ShortenedUrl.fetch_with_token(token: short_url.unique_key, track: false)
+          end
+        end
+      end
+
+      context 'expired short url' do
+        let(:short_url) { Shortener::ShortenedUrl.generate!(url, expires_at: 1.second.ago) }
+        it_should_behave_like 'invalid token supplied' do
+          let(:token) { short_url.unique_key }
+        end
+      end
+    end
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: shortener
 version: !ruby/object:Gem::Version
-  version: 0.5.5
+  version: 0.5.6
 platform: ruby
 authors:
 - James P. McGrath
@@ -9,90 +9,104 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-11-08 00:00:00.000000000 Z
+date: 2016-11-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: 3.0.7
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: 3.0.7
+- !ruby/object:Gem::Dependency
+  name: voight_kampff
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.0.2
 - !ruby/object:Gem::Dependency
   name: sqlite3
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: rspec-rails
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: 3.3.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: 3.3.0
 - !ruby/object:Gem::Dependency
   name: shoulda-matchers
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '3'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '3'
 - !ruby/object:Gem::Dependency
   name: faker
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: byebug
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 description: Shortener is a Rails Engine Gem that makes it easy to create and interpret
@@ -106,8 +120,8 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- .gitignore
-- .travis.yml
+- ".gitignore"
+- ".travis.yml"
 - Gemfile
 - MIT-LICENSE
 - README.rdoc
@@ -167,17 +181,17 @@ require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>'
+  - - ">"
     - !ruby/object:Gem::Version
       version: 1.3.6
 requirements: []
 rubyforge_project: shortener
-rubygems_version: 2.0.2
+rubygems_version: 2.5.1
 signing_key: 
 specification_version: 4
 summary: Shortener is a Rails Engine that makes it easy to create shortened URLs for