i18n-instrument 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: a0463e97ab45fd96ee54d900280a3afa10ee6d95
+  data.tar.gz: 83c48ca90a3e6a82cc40c9a8cf33af38c59a4e20
+SHA512:
+  metadata.gz: 310fca52c96c4119221e0f285a02a81291324aad34daa0e14ceba50c933446699deb04a0ea97849d57a53245a04afca0aa2fb6b1e5ceb351af172b1bab6b1c47
+  data.tar.gz: bf0c3eb4be1e07e60e1aef5aaf45fc7b76c789ce844fa0ca4e3a29ac30c103e4bac23fc2e5621bda42c4d9df58008cc08347b50e6b2b6362ba4e293ff4c85dde

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright 2016 Lumos Labs, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,141 @@
+# i18n-instrument
+Instrument calls to I18n.t in Ruby and JavaScript in your Rails app.
+
+## Installation
+
+Add it to your Gemfile:
+
+```ruby
+gem 'i18n-instrument', require: 'i18n/instrument'
+```
+
+## The Problem
+
+Platforms like iOS and Android make it easy to tell which of your localization strings are currently in use - just do a bit of grepping or run a script and voilà! Once you've identified them, you can remove any unused, crufty strings from your translation files and move on with your life.
+
+Due to Ruby's dynamic nature and the fact that you can never tell what Rails is actually doing, identifying unused strings is much more difficult. The static analysis that worked so well with iOS and Android projects won't work for your Rails app.
+
+Consider this ERB template. It lives in app/views/products/index.html.erb:
+
+```HTML+ERB
+<div class="description">
+  <%= I18n.t('.description') %>: <%= @product.description %>
+</div>
+```
+
+Under the hood, the i18n gem (which provides Rails' localization implementation) fully qualifies `.description` into `products.index.description`. Because this qualification is done at runtime, static analysis (i.e. grepping) won't be able to identify all the strings your app is currently using.
+
+The problem is compounded by the fact that the key you pass to `I18n.t` is just a string, and can therefore be generated in any way the Ruby language allows, for example:
+
+```HTML+ERB
+<div class="fields">
+  <% @product.attributes.each_pair do |attr, value|
+    <div class="<%= attr %>">
+      <%= I18n.t(".#{attr}") %>: <%= value %>
+    </div>
+  <% end %>
+</div>
+```
+First of all, I sincerely hope you never write code like this - it's probably bad practice to loop over *all* the attributes in your model and print them out (security blah blah blah). Hopefully my example illustrates the problem however - namely that the `I18n.t` method can accept any string, including string variables, interpolated strings, and the rest. Unless your static analyzer is very clever, it won't be able to tell you which of your localization strings are currently being used.
+
+## Ok, so what should I do about it?
+
+So glad you asked.
+
+The only foolproof way to collect information about the localization strings your app is using is during runtime. This gem, i18n-instrument, is capable of annotating calls to `I18n.t` in Ruby/Rails and Javascript (provided you're using the capable [i18n-js gem](https://github.com/fnando/i18n-js)). Whenever one of these methods is called, i18n-instrument will fire the `on_record` callback and pass you some useful information. From there, the possibilities are endless. You could log the information to the console, write it down somewhere useful, save it to a database, you name it.
+
+## Callbacks
+
+* **`on_lookup(key : String, value : String)`**: Fired whenever `I18n.t` is called. Yields the localization key (i.e. `es.products.index.description`) and the corresponding translation string.
+
+* **`on_record(params : Hash)`**: Similar to `on_lookup` but comes with a bunch of useful information.
+
+  The `params` hash contains the following keys (all values are strings):
+  
+  * **`controller`**: the controller that served the request.
+  * **`action`**: the action that served the request.
+  * **`trace`**: the filename and line number from the first application stack frame, i.e. the place in your code `I18n.t` was called.
+  * **`key`**: the localization key.
+  * **`source`**: either "ruby" or "javascript".
+  * **`locale`**: the value of `I18n.locale` in Ruby or Javascript.
+
+* **`on_error(e : Exception)`**: Fired whenever an error occurs. The default behavior is to re-raise the exception. You may want to add your own exception handling callback so your app doesn't crash. See the configuration section below for details.
+
+* **`on_check_enabled() : Boolean`**: Fired on every lookup. The return value must be `true` if the lookup should be recorded and `false` otherwise. The default behavior is to look for the existence of a file named config/enable\_i18n\_instrumentation. If the file exists, `I18n.t` calls are recorded. If not, calls are not recorded.
+
+## Configuration
+
+i18n-instrument is a piece of Rack middleware. It sits in between the Internet and your Rails app. You can configure it any time before your app is finished booting. I'd suggest doing it in a Rails initializer, for example config/initializers/i18n\_instrument.rb:
+
+```ruby
+I18n::Instrument.configure do |config|
+  # The first stack trace line that begins with this string will be passed to
+  # `on_record` as the `trace` value. Defaults to `Rails.root`.
+  config.stack_trace_prefix = 'custom/path/to/app'
+  
+  # The URL you want your app to send js instrumentation requests to. Defaults
+  # to '/i18n/instrument.json'.
+  config.js_endpoint = '/my_i18n/foo.json'
+
+  # all of the callback methods are available here
+  
+  config.on_record do |params|
+    # print params to the rails log
+    Rails.logger.info(params.inspect)
+  end
+  
+  config.on_lookup do |key, value|
+    puts "Looked up i18n key #{key} and got value #{value}"
+  end
+  
+  config.on_error do |e|
+    # report errors to our error aggregation service
+    Rollbar.error(e)
+  end
+  
+  config.on_check_enabled do
+    # always enable
+    true
+  end
+end
+```
+Be careful not to call `I18n::Instrument.configure` more than once - doing so will replace any existing configuration you may have already done. Instead, try this:
+
+```ruby
+# replace any existing on_record behavior, but leave all other configuration intact
+I18n::Instrument.config.on_record do |params|
+  ...
+end
+```
+
+## Javascript Land
+
+Using i18n-instrument in Javascript is pretty straightforward - just include it in your application.js file:
+
+```javascript
+//= require i18n
+//= require i18n/instrument
+```
+Make sure to include i18n/instrument ***after*** i18n. This is critical since i18n-instrument overrides i18n's `t` method.
+
+Also, don't forget to enable Javascript instrumentation by adding this line, probably at the bottom of application.js:
+
+```javascript
+I18n.instrumentation_enabled = true
+```
+
+## Requirements
+
+i18n-instrument is designed to work with Rails 4 and up, although it has not been tested under Rails 5 yet.
+
+## Testing
+
+Run the test suite with `bundle exec rspec`.
+
+## Authors
+
+* Cameron C. Dutro ([@camertron](https://github.com/camertron)) on behalf of Lumos Labs, Inc.
+
+## License
+
+Licensed under the MIT license.

data/i18n-instrument.gemspec

@@ -0,0 +1,22 @@
+$:.unshift File.join(File.dirname(__FILE__), 'lib')
+require 'i18n/instrument/version'
+
+Gem::Specification.new do |s|
+  s.name     = 'i18n-instrument'
+  s.version  = ::I18n::Instrument::VERSION
+  s.authors  = ['Cameron Dutro']
+  s.email    = ['camertron@gmail.com']
+  s.homepage = 'https://github.com/lumoslabs/i18n-instrument'
+
+  s.description = s.summary = 'Instrument calls to I18n.t in Ruby and JavaScript.'
+
+  s.platform = Gem::Platform::RUBY
+  s.has_rdoc = true
+
+  s.add_dependency 'i18n-debug', '~> 1.0'
+  s.add_dependency 'railties', '~> 4'
+  s.add_dependency 'request_store', '~> 1.0'
+
+  s.require_path = 'lib'
+  s.files = Dir['{lib,spec}/**/*', 'README.md', 'i18n-instrument.gemspec', 'LICENSE']
+end

data/lib/assets/javascripts/i18n/instrument.js.erb

@@ -0,0 +1,33 @@
+(function() {
+  if (I18n.translate == null) {
+    return;
+  }
+
+  var originalFunction = I18n.translate
+
+  // override original t and translate functions
+  I18n.t = I18n.translate = function(scope, options) {
+    if (I18n.instrumentation_enabled) {
+      // get fully qualified key
+      var opts = I18n.prepareOptions(options);
+      var key = I18n.getFullScope(scope, opts);
+      var data = {
+        key: key,
+        locale: I18n.locale,
+        url: window.location.href
+      };
+
+      // make request to instrumentation middleware
+      $.ajax({
+        url: '<%= I18n::Instrument.config.js_endpoint %>',
+        type: 'post',
+        data: JSON.stringify(data),
+        dataType: 'json',
+        headers: {'Content-Type': 'application/json'}
+      });
+    }
+
+    // use `call` to be able to pass in `I18n` as `this`
+    return originalFunction.call(I18n, scope, options);
+  }
+})();

data/lib/i18n/instrument.rb

@@ -0,0 +1,20 @@
+require 'i18n/instrument/railtie'
+
+module I18n
+  module Instrument
+    autoload :Configurator, 'i18n/instrument/configurator'
+    autoload :Middleware,   'i18n/instrument/middleware'
+
+    class << self
+      attr_reader :config
+
+      def configure
+        @config = Configurator.new
+        yield @config if block_given?
+      end
+    end
+  end
+
+  # set default config
+  Instrument.configure
+end

data/lib/i18n/instrument/configurator.rb

@@ -0,0 +1,43 @@
+module I18n
+  module Instrument
+    class Configurator
+      DEFAULT_ENABLED_FILE = File.join('config', 'enable_i18n_instrumentation')
+      DEFAULT_JS_ENDPOINT = '/i18n/instrument.json'
+
+      attr_accessor :js_endpoint, :stack_trace_prefix
+
+      def initialize
+        @js_endpoint = DEFAULT_JS_ENDPOINT
+        @stack_trace_prefix = Rails.root.to_s
+
+        @on_check_enabled_proc = -> do
+          Rails.root.join(DEFAULT_ENABLED_FILE).exist?
+        end
+
+        @on_error_proc = ->(e) { raise e }
+        @on_record_proc = ->(*) { }
+        @on_lookup_proc = ->(*) { }
+      end
+
+      def on_lookup(&block)
+        block ? @on_lookup_proc = block : @on_lookup_proc
+      end
+
+      def on_record(&block)
+        block ? @on_record_proc = block : @on_record_proc
+      end
+
+      def on_error(&block)
+        block ? @on_error_proc = block : @on_error_proc
+      end
+
+      def on_check_enabled(&block)
+        block ? @on_check_enabled_proc = block : @on_check_enabled_proc
+      end
+
+      def enabled?
+        @on_check_enabled_proc.call
+      end
+    end
+  end
+end

data/lib/i18n/instrument/middleware.rb

@@ -0,0 +1,122 @@
+require 'i18n/debug'
+require 'request_store'
+
+module I18n
+  module Instrument
+    class Middleware
+      # used to store request information in the request store
+      STORE_KEY = :i18n_instrumentation
+
+      # canned response sent to js instrumentation requests
+      JS_RESPONSE = [
+        200, {
+          'Content-Type' => 'application/json',
+          'Content-Length' => 2
+        }, [
+          '{}'
+        ]
+      ]
+
+      def initialize(app)
+        @app = app
+
+        # this will fire on every call to I18n.t
+        I18n::Debug.on_lookup do |key, value|
+          next unless enabled?
+          config.on_lookup.call(key, value)
+
+          begin
+            # find the first application-specific line in the stack trace
+            raw_trace = filter_stack_trace(::Kernel.caller)
+            trace = raw_trace.split(":in `").first if raw_trace
+
+            # grab path params (set in `call` method below)
+            url = store.fetch(STORE_KEY, {}).fetch(:url, nil)
+
+            if url.present? && trace.present?
+              record_translation_lookup(
+                url: url, trace: trace, key: key,
+                locale: I18n.locale.to_s, source: 'ruby'
+              )
+            end
+          rescue => e
+            config.on_error.call(e)
+          end
+        end
+      end
+
+      def call(env)
+        if i18n_js_request?(env)
+          handle_i18n_js_request(env)
+        else
+          handle_regular_request(env)
+        end
+      rescue => e
+        config.on_error.call(e)
+        @app.call(env)
+      end
+
+      private
+
+      def filter_stack_trace(trace)
+        trace.find { |line| line.start_with?(config.stack_trace_prefix) }
+      end
+
+      def record_translation_lookup(url:, trace:, key:, locale:, source:)
+        config.on_record.call({
+          url: url, trace: trace, key: key,
+          source: source, locale: locale
+        })
+      end
+
+      def handle_regular_request(env)
+        return @app.call(env) unless enabled?
+
+        # store params from env in request storage so they can be used in the
+        # I18n on_lookup callback above
+        store[STORE_KEY] = { url: env['REQUEST_URI'] }
+
+        @app.call(env)
+      end
+
+      def handle_i18n_js_request(env)
+        return JS_RESPONSE unless enabled?
+
+        body = JSON.parse(env['rack.input'].read)
+        url = body['url']
+        key = body['key']
+        locale = body['locale']
+
+        if url.present? && key.present?
+          record_translation_lookup(
+            url: url, trace: nil, key: key,
+            locale: locale, source: 'javascript'
+          )
+        end
+
+        JS_RESPONSE
+      end
+
+      def i18n_js_request?(env)
+        env.fetch('REQUEST_URI', '').include?(js_endpoint) &&
+          env.fetch('REQUEST_METHOD', '').upcase == 'POST'
+      end
+
+      def config
+        I18n::Instrument.config
+      end
+
+      def js_endpoint
+        config.js_endpoint
+      end
+
+      def enabled?
+        config.enabled?
+      end
+
+      def store
+        RequestStore.store
+      end
+    end
+  end
+end

data/lib/i18n/instrument/railtie.rb

@@ -0,0 +1,10 @@
+module I18n
+  module Instrument
+    class Railtie < Rails::Railtie
+      initializer 'i18n.instrument.middleware' do |app|
+        app.config.assets.paths << File.expand_path('../../../assets/javascripts', __FILE__)
+        app.middleware.use(I18n::Instrument::Middleware)
+      end
+    end
+  end
+end

data/lib/i18n/instrument/version.rb

@@ -0,0 +1,5 @@
+module I18n
+  module Instrument
+    VERSION = '1.0.0'
+  end
+end

data/spec/Rakefile

@@ -0,0 +1,8 @@
+require 'rails'
+require 'action_controller/railtie'
+require 'action_view/railtie'
+
+require File.expand_path('../config/application', __FILE__)
+
+I18n::Instrument::DummyApplication.initialize!
+I18n::Instrument::DummyApplication.load_tasks