wovnrb 0.2.01 → 0.2.02

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1531f2d2d0c565e28d1473918d96f1c4566e3a8a
-  data.tar.gz: 1de5c991d544fc7578de043f53da32e069d7ae36
+  metadata.gz: d1df053b985e8697f9164898024e44c06f61e249
+  data.tar.gz: 0a516cb7c4712ba7f52c5a70e767721dba64cc42
 SHA512:
-  metadata.gz: 08da2a6e09ca5109e475103ee38d213cfb4a45f6d471c12fa55b4270a7406dadb769b7449c5fb06dee1ca87027f73d6c8ed15a2fe98195516c47c235831c8992
-  data.tar.gz: f70c5edad9d5ae50b8702c4f3c7d55d4160369d0d8faa1cc0bdbdd373161b11d0bdc47b5c9a88430b3cc1f12a14476fa755200e3dd04dfb3139a4edfd417a8bd
+  metadata.gz: 62e2aa38e740242c0ef89c88e076aeaeb9e69d72c9bc0ac2164547165bc531f8a8d39346541530b5ab0e6164213e6ca074e9c3be93da0a3b5a4b39ccb014eac3
+  data.tar.gz: 14ac3a44106d4a114023b91c73aa00740bec2f6c3387861d7002cfee0a8792a73fac3a8f8607620ec2ae62d8915e2d913378ae0890bdfec70e678f796d24dc47

data/.gitignore

@@ -1,14 +1,10 @@
 /.bundle/
-<<<<<<< HEAD
 /vendor
-=======
-/vendor/bundle/
->>>>>>> 78ccf8410c19bdee508e96a2386a5273abfad0ba
-/.yardoc
+.yardoc/
 /Gemfile.lock
 /_yardoc/
 /coverage/
-/doc/
+doc/
 /pkg/
 /spec/reports/
 /tmp/
@@ -19,4 +15,5 @@
 *.swp
 mkmf.log
 .idea/
+*.iml
 .ruby-version

data/README.md

@@ -1,28 +1,132 @@
-# Wovn
+# WOVN.io Ruby Library
 
-TODO: Write a gem description
+The WOVN.io Ruby library is a library that uses WOVN.io in order to provide translations. The WOVN.io Ruby Library is packaged as Rack Middleware.
 
-## Installation
+This document explains the the process of installing WOVN.io Ruby, as well as set up and configuration.
 
-Add this line to your application's Gemfile:
+## 1. Install
+
+### 1.1. Creating a WOVN.io account.
+
+In order to use the WOVN.io Ruby Library, you need a WOVN.io account. If you do not have an account, please first sign up for one at WOVN.io.
+
+### 1.2. Adding a Page
+
+After logging into WOVN.io, add a page you would like translated.
+
+### 1.3. Ruby Application Settings
+
+To use the WOVN.io Ruby Library, insert the following line into your Ruby Application's Gemfile. WOVN.io currently supports version 0.2 and up.
 
 ```ruby
-gem 'wovnrb'
+gem 'wovnrb', '>= 0.2'
+```
+
+After setting the above, execute the following command to install the WOVN.io Ruby Library.
+
+```bash
+bundle install
+```
+
+After installing the library, insert the following into your Ruby Application's settings file.
+
+* If you're using Ruby on Rails
+
+Insert the following into either config/application.rb or config/environments/.
+
+```ruby
+...
+
+config.wovnrb = {
+:user_token => '2Wle3',
+:secret_key => 'secret',
+}
+
+...
+```
+
+* If you're using Sinatra
+
+Insert the following into either the Application File or config.ru.
+
+```ruby
+...
+
+require 'wovnrb'
+
+use Wovnrb::Interceptor, {
+:user_token => '2Wle3',
+:secret_key => 'secret',
+}
+
+...
 ```
 
-And then execute:
+After completing setup, start the Ruby Application, and make sure the WOVN.io library is working correctly.
+
+## 2. Parameter Setting
+
+WOVN.io Ruby Library's valid parameters are as follows.
+
+Parameter Name | Required | Default Setting
+-------------- | -------- | ----------------
+user_token     | yes      | ''
+secret_key     | yes      | ''
+url_pattern    | yes      | 'path'
+query          |          | []
+default_lang   | yes      | 'en'
+
+### 2.1. user_token
+
+Set your WOVN.io Account's user token. This parameter is required.
+
+### 2.2. secret_key
+
+This parameter is in development; it is not currently used. However, it is a required parameter, so ensure to set a value for it.
+
+### 2.3. url_pattern
+
+The Library works in the Ruby Application by adding new URL's to be translated. You can set the type of url with the url_pattern parameter. There are 3 types that can be set.
+
+parameters  | Translated page's URL           | Notes
+----------- | ------------------------------- | -------
+'path'      | https://wovn.io/ja/contact      | Default Value. If no settings have been set, url_pattern defaults to this value.
+'subdomain' | https://ja.wovn.io/contact      | DNS settings must be set.
+'query'     | https://wovn.io/contact?wovn=ja | The least amount of changes to the application required to complete setup.
+
+※ The following is an example of a URL that has been translated by the library using the above URL's.
+
+	https://wovn.io/contact
+
+### 2.4. query
+
+Using the query parameter, you can setup query parameters you wish to be ignored within the URL when translating via WOVN.io. There is no default value, in this case, all query parameters are included within the translated page's URL.
+
+	https://wovn.io/ja/contact?os=mac&keyboard=us
+
+If the defualt_lang is 'en', and the query is set to 'os=mac&keyboard=us', the above URL will be modified into the following URL to search for the page's translation.
+
+	https://wovn.io/contact?os=mac&keyboard=us
+
+If the default_lang is 'en', and the query is set to 'os=mac', the above URL will be modified into the following URL to search for the page's translation.
+
+	https://wovn.io/contact?os=mac
+
+### 2.5. default_lang
+
+This sets the Ruby application's default language. The default value is English ('en').
 
-    $ bundle
+If a requested page, where the default language's parameter is included in the URL, the request is redirected before translating. The default_lang parameter is used for this purpose.
 
-Or install it yourself as:
+If the default_lang is set to 'en', when receiving a request for the following URL,
 
-    $ gem install wovnrb
+	https://wovn.io/en/contact
 
-## Usage
+The library will redirect to the following URL.
 
-TODO: Write usage instructions here
+	https://wovn.io/contact
 
-## Contributing
+## 3. Contributing
 
 1. Fork it ( https://github.com/[my-github-username]/wovnrb/fork )
 2. Create your feature branch (`git checkout -b my-new-feature`)

data/lib/wovnrb/api_data.rb

@@ -0,0 +1,59 @@
+module Wovnrb
+  class ApiData
+    def initialize(access_url, store)
+      @access_url = access_url.gsub(/\/$/, '')
+      @store = store
+    end
+
+    def get_data
+      cache_key = to_key(@access_url)
+      data = get_data_value(cache_key)
+      JSON.parse(data)
+    end
+
+    private
+    def get_data_value(cache_key)
+      cache_value = CacheBase.get_single.get(cache_key)
+      return cache_value if cache_value
+
+      uri = build_api_uri
+      begin
+        response = get_from_api_server(uri)
+      rescue => e
+        response = '{}'
+        WovnLogger.instance.error("API server GET request failed :\nurl: #{uri}\n#{e.message}")
+      end
+
+      # Always cache response, even when error returns to avoid DDOS
+      CacheBase.get_single.put(cache_key, response)
+      response
+    end
+
+    @@cache_prefix = 'api::cache::'
+    def to_key(url)
+      "#{@@cache_prefix}#{url}"
+    end
+
+    def build_api_uri
+      t = CGI::escape(@store.settings['user_token'])
+      u = CGI::escape(@access_url)
+      URI.parse("#{@store.settings['api_url']}?token=#{t}&url=#{u}")
+    end
+
+    def get_from_api_server(uri)
+      http = Net::HTTP.new(uri.host, uri.port)
+      http.use_ssl = true if uri.scheme == 'https'
+      http.open_timeout = @store.settings['api_timeout_seconds']
+      http.read_timeout= @store.settings['api_timeout_seconds']
+      response = http.start {
+        http.get(uri.request_uri)
+      }
+
+      if response.code == '200'
+        response.body
+      else
+        raise "Response Code is not success: #{response.code}"
+      end
+    end
+  end
+end

data/lib/wovnrb/headers.rb

@@ -10,11 +10,18 @@ module Wovnrb
     attr_reader :pathname
     attr_reader :redis_url
 
+    # Generates new instance of Wovnrb::Headers.
+    # Its parameters are set by parsing env variable.
+    #
     def initialize(env, settings)
       @env = env
       @settings = settings
       @protocol = @env['rack.url_scheme']
-      @unmasked_host = @env['HTTP_HOST']
+      if settings['use_proxy'] && @env.has_key?('HTTP_X_FORWARDED_HOST')
+        @unmasked_host = @env['HTTP_X_FORWARDED_HOST']
+      else
+        @unmasked_host = @env['HTTP_HOST']
+      end
       unless @env.has_key?('REQUEST_URI')
         # Add '/' to PATH_INFO as a possible fix for pow server
         @env['REQUEST_URI'] = (@env['PATH_INFO'] =~ /^[^\/]/ ? '/' : '') + @env['PATH_INFO'] + (@env['QUERY_STRING'].size == 0 ? '' : "?#{@env['QUERY_STRING']}")
@@ -27,7 +34,12 @@ module Wovnrb
       @unmasked_pathname = @env['REQUEST_URI'].split('?')[0]
       @unmasked_pathname += '/' unless @unmasked_pathname =~ /\/$/ || @unmasked_pathname =~ /\/[^\/.]+\.[^\/.]+$/
       @unmasked_url = "#{@protocol}://#{@unmasked_host}#{@unmasked_pathname}"
-      @host = settings['url_pattern'] == 'subdomain' ? remove_lang(@env['HTTP_HOST'], self.lang_code) : @env['HTTP_HOST']
+      if settings['use_proxy'] && @env.has_key?('HTTP_X_FORWARDED_HOST')
+        @host = @env['HTTP_X_FORWARDED_HOST']
+      else
+        @host = @env['HTTP_HOST']
+      end
+      @host = settings['url_pattern'] == 'subdomain' ? remove_lang(@host, self.lang_code) : @host
       @pathname, @query = @env['REQUEST_URI'].split('?')
       @pathname = settings['url_pattern'] == 'path' ? remove_lang(@pathname, self.lang_code) : @pathname
       @query = @query || ''
@@ -59,10 +71,19 @@ module Wovnrb
       (self.path_lang && self.path_lang.length > 0) ? self.path_lang : @settings['default_lang']
     end
 
+    # picks up language code from requested URL by using url_pattern_reg setting.
+    # when language code is invalid, this method returns empty string.
+    # if you want examples, please see test/lib/headers_test.rb.
+    #
+    # @return [String] language code in requrested URL.
     def path_lang
       if @path_lang.nil?
         rp = Regexp.new(@settings['url_pattern_reg'])
-        match = "#{@env['SERVER_NAME']}#{@env['REQUEST_URI']}".match(rp)
+        if @settings['use_proxy'] && @env.has_key?('HTTP_X_FORWARDED_HOST')
+          match = "#{@env['HTTP_X_FORWARDED_HOST']}#{@env['REQUEST_URI']}".match(rp)
+        else
+          match = "#{@env['SERVER_NAME']}#{@env['REQUEST_URI']}".match(rp)
+        end
         if match && match[:lang] && Lang.get_lang(match[:lang])
           @path_lang = Lang.get_code(match[:lang])
         else
@@ -130,8 +151,12 @@ module Wovnrb
         @env['QUERY_STRING'] = remove_lang(@env['QUERY_STRING']) if @env.has_key?('QUERY_STRING')
         @env['ORIGINAL_FULLPATH'] = remove_lang(@env['ORIGINAL_FULLPATH']) if @env.has_key?('ORIGINAL_FULLPATH')
       when 'subdomain'
-        @env["HTTP_HOST"] = remove_lang(@env["HTTP_HOST"])
-        @env["SERVER_NAME"] = remove_lang(@env["SERVER_NAME"])
+        if @settings['use_proxy'] && @env.has_key?('HTTP_X_FORWARDED_HOST')
+          @env['HTTP_X_FORWARDED_HOST'] = remove_lang(@env['HTTP_X_FORWARDED_HOST'])
+        else
+          @env["HTTP_HOST"] = remove_lang(@env["HTTP_HOST"])
+          @env["SERVER_NAME"] = remove_lang(@env["SERVER_NAME"])
+        end
         if @env.has_key?('HTTP_REFERER')
           @env["HTTP_REFERER"] = remove_lang(@env["HTTP_REFERER"])
         end

data/lib/wovnrb/html_replacers/image_replacer.rb

@@ -0,0 +1,44 @@
+module Wovnrb
+  class ImageReplacer < ReplacerBase
+    def initialize(url, text_index, src_index, img_src_prefix)
+      @url = url
+      @text_index = text_index
+      @src_index = src_index
+      @img_src_prefix = img_src_prefix
+    end
+
+    def replace(dom, lang)
+      dom.xpath('//img').each do |node|
+        next if wovn_ignore?(node)
+
+        # use regular expressions to support case insensitivity (right?)
+        if node.to_html =~ /src=['"][^'"]*['"]/i
+          src = node.to_html.match(/src=['"]([^'"]*)['"]/i)[1]
+          # THIS SRC CORRECTION DOES NOT HANDLE ONE IMPORTANT CASE
+          # 1) "../path/with/ellipse"
+          # if this is not an absolute src
+          if src !~ /:\/\//
+            # if this is a path with a leading slash
+            if src =~ /^\//
+              src = "#{@url[:protocol]}://#{@url[:host]}#{src}"
+            else
+              src = "#{@url[:protocol]}://#{@url[:host]}#{@url[:path]}#{src}"
+            end
+          end
+
+          # shouldn't need size check, but for now...
+          if @src_index[src] && @src_index[src][lang.lang_code] && @src_index[src][lang.lang_code].size > 0
+            node.attribute('src').value = "#{@img_src_prefix}#{@src_index[src][lang.lang_code][0]['data']}"
+          end
+        end
+
+        if node.get_attribute('alt')
+          alt = node.get_attribute('alt').strip
+          if @text_index[alt] && @text_index[alt][lang.lang_code] && @text_index[alt][lang.lang_code].size > 0
+            node.attribute('alt').value = alt.gsub(/^(\s*)[\S\s]*(\s*)$/, '\1' + @text_index[alt][lang.lang_code][0]['data'] + '\2')
+          end
+        end
+      end
+    end
+  end
+end

data/lib/wovnrb/html_replacers/link_replacer.rb

@@ -0,0 +1,18 @@
+module Wovnrb
+  class LinkReplacer < ReplacerBase
+    def initialize(pattern, headers)
+      @pattern = pattern
+      @headers = headers
+    end
+
+    def replace(dom, lang)
+      dom.xpath('//a').each do |node|
+        next if wovn_ignore?(node)
+
+        href = node.get_attribute('href')
+        new_href = lang.add_lang_code(href, @pattern, @headers)
+        node.set_attribute('href', new_href)
+      end
+    end
+  end
+end

data/lib/wovnrb/html_replacers/meta_replacer.rb

@@ -0,0 +1,20 @@
+module Wovnrb
+  class MetaReplacer < ReplacerBase
+    def initialize(text_index)
+      @text_index = text_index
+    end
+
+    def replace(dom, lang)
+      dom.xpath('//meta').select { |node|
+        next if wovn_ignore?(node)
+        (node.get_attribute('name') || node.get_attribute('property') || '') =~ /^(description|title|og:title|og:description|twitter:title|twitter:description)$/
+      }.each do |node|
+        node_content = node.get_attribute('content').strip
+        # shouldn't need size check, but for now...
+        if @text_index[node_content] && @text_index[node_content][lang.lang_code] && @text_index[node_content][lang.lang_code].size > 0
+          node.set_attribute('content', node.get_attribute('content').gsub(/^(\s*)[\S\s]*?(\s*)$/, '\1' + @text_index[node_content][lang.lang_code][0]['data'] + '\2'))
+        end
+      end
+    end
+  end
+end

data/lib/wovnrb/html_replacers/replacer_base.rb

@@ -0,0 +1,17 @@
+module Wovnrb
+  class ReplacerBase
+    def replace(dom, lang)
+      raise NotImplementedError.new('replace is not defined')
+    end
+
+    protected
+    def wovn_ignore?(node)
+      if !node.get_attribute('wovn-ignore').nil?
+        return true
+      elsif node.name === 'html'
+        return false
+      end
+      wovn_ignore?(node.parent)
+    end
+  end
+end

data/lib/wovnrb/html_replacers/script_replacer.rb

@@ -0,0 +1,39 @@
+module Wovnrb
+  class ScriptReplacer < ReplacerBase
+    def initialize(store)
+      @store = store
+    end
+
+    def replace(dom, lang)
+      remove_embed_wovn_script(dom)
+      add_wovn_script(dom, lang)
+    end
+
+    private
+    def remove_embed_wovn_script(dom)
+      dom.xpath('//script').each do |script_node|
+        if script_node['src'] && script_node['src'] =~ /^\/\/j.(dev-)?wovn.io(:3000)?\//
+          script_node.remove
+        end
+      end
+    end
+
+    def add_wovn_script(dom, lang)
+      parent_node = dom.at_css('head') || dom.at_css('body') || dom.at_css('html')
+
+      # INSERT BACKEND WIDGET
+      insert_node = Nokogiri::XML::Node.new('script', dom)
+      insert_node['src'] = '//j.wovn.io/1'
+      insert_node['async'] = true
+      version = defined?(VERSION) ? VERSION : ''
+      insert_node['data-wovnio'] = "key=#{@store.settings['user_token']}&backend=true&currentLang=#{lang.lang_code}&defaultLang=#{@store.settings['default_lang']}&urlPattern=#{@store.settings['url_pattern']}&version=#{version}"
+      # do this so that there will be a closing tag (better compatibility with browsers)
+      insert_node.content = ' '
+      if parent_node.children.size > 0
+        parent_node.children.first.add_previous_sibling(insert_node)
+      else
+        parent_node.add_child(insert_node)
+      end
+    end
+  end
+end

data/lib/wovnrb/html_replacers/text_replacer.rb

@@ -0,0 +1,19 @@
+module Wovnrb
+  class TextReplacer < ReplacerBase
+    def initialize(text_index)
+      @text_index = text_index
+    end
+
+    def replace(dom, lang)
+      dom.xpath('//text()').each do |node|
+        next if wovn_ignore?(node)
+
+        node_text = node.content.strip
+        # shouldn't need size check, but for now...
+        if @text_index[node_text] && @text_index[node_text][lang.lang_code] && @text_index[node_text][lang.lang_code].size > 0
+          node.content = node.content.gsub(/^(\s*)[\S\s]*?(\s*)$/, '\1' + @text_index[node_text][lang.lang_code][0]['data'] + '\2')
+        end
+      end
+    end
+  end
+end

data/lib/wovnrb/lang.rb

@@ -30,29 +30,6 @@ module Wovnrb
       'tr' => {name: 'Türkçe',            code: 'tr',     en: 'Turkish'},
       'uk' => {name: 'Українська',        code: 'uk',     en: 'Ukrainian'},
       'vi' => {name: 'Tiếng Việt',        code: 'vi',     en: 'Vietnamese'},
-
-=begin
-      * denotes no Google support
-      *{name: 'Urdu', code: 'ur', en: 'Urdu'},
-      {name: 'Català', code: 'ca'},
-      {name: 'Čeština', code: 'cs'},
-      {name: 'Български', code: 'bg'},
-      {name: 'Estonian', code: 'et'},
-      *{name: 'Haitian Creoloe', code: 'ht'},
-      *{name: 'Hmong Daw', code: 'mww'},
-      {name: 'Hungarian', code: 'hu'},
-      *{name: 'Klingon', code: 'tlh'},
-      *{name: 'Klingon (plqaD)', code: 'tlh-Qaak'},
-      {name: 'Latvian', code: 'lv'},
-      {name: 'Lithuanian', code: 'lt'},
-      {name: 'Maltese', code: 'mt'},
-      {name: 'Persian', code: 'fa'},
-      {name: 'Romanian', code: 'ro'},
-      {name: 'Slovak', code: 'sk'},
-      {name: 'Slovenian', code: 'sl'},
-      {name: 'Ukranian', code: 'uk'},
-      {name: 'Welsh', code: 'cy'},
-=end
     }
 
     def self.get_code(lang_name)
@@ -71,5 +48,138 @@ module Wovnrb
       return LANG[lang_code]
     end
 
+    def initialize(lang_name)
+      @lang_code = Lang.get_code(lang_name)
+    end
+
+    def lang_code
+      @lang_code
+    end
+
+    # Adds language code to URL in "href" variable by "pattern" variable and own @lang_code.
+    #  When @lang_code is 'ja', add_lang_code('https://wovn.io', 'path', headers) returns 'https://wovn.io/ja/'.
+    # If you want to know more examples, see also test/lib/lang_test.rb.
+    #
+    # @param  [String] href            original URL.
+    # @param  [String] pattern         url_pattern of the settings. ('path', 'subdomain' or 'query')
+    # @param  [Wovnrb::Header] headers instance of Wovn::Header. It generates new env variable for original request.
+    # @return [String]                 URL added langauge code.
+    def add_lang_code(href, pattern, headers)
+      return href if href =~ /^(#.*)?$/
+      # absolute links
+      new_href = href
+      if href && href =~ /^(https?:)?\/\//i
+        # in the future, perhaps validate url rather than using begin rescue
+        # "#{url =~ /\// ? 'http:' : ''}#{url}" =~ URI::regexp
+        begin
+          uri = URI(href)
+        rescue
+          return new_href
+        end
+        # only add lang if it's an internal link
+        # DNS names are case insensitive
+        if uri.host.downcase === headers.host.downcase
+          case pattern
+            when 'subdomain'
+              sub_d = href.match(/\/\/([^\.]*)\./)[1]
+              sub_code = Lang.get_code(sub_d)
+              if sub_code && sub_code.downcase == @lang_code.downcase
+                new_href = href.sub(Regexp.new(@lang_code, 'i'), @lang_code.downcase)
+              else
+                new_href = href.sub(/(\/\/)([^\.]*)/, '\1' + @lang_code.downcase + '.' + '\2')
+              end
+            when 'query'
+              new_href = href =~ /\?/ ? href + '&wovn=' + @lang_code : href + '?wovn=' + @lang_code
+            else # path
+              new_href = href.sub(/([^\.]*\.[^\/]*)(\/|$)/, '\1/' + @lang_code + '/')
+          end
+        end
+      elsif href
+        case pattern
+          when 'subdomain'
+            lang_url = headers.protocol + '://' + @lang_code.downcase + '.' + headers.host
+            current_dir = headers.pathname.sub(/[^\/]*\.[^\.]{2,6}$/, '')
+            if href =~ /^\.\..*$/
+              # ../path
+              new_href = lang_url + '/' + href.gsub(/^\.\.\//, '')
+            elsif href =~ /^\..*$/
+              # ./path
+              new_href = lang_url + current_dir + '/' + href.gsub(/^\.\//, '')
+            elsif href =~ /^\/.*$/
+              # /path
+              new_href = lang_url + href
+            else
+              # path
+              new_href = lang_url + current_dir + '/' + href
+            end
+          when 'query'
+            new_href = href =~ /\?/ ? href + '&wovn=' + @lang_code : href + '?wovn=' + @lang_code
+          else # path
+            if href =~ /^\//
+              new_href = '/' + @lang_code + href
+            else
+              current_dir = headers.pathname.sub(/[^\/]*\.[^\.]{2,6}$/, '')
+              current_dir = '/' if current_dir == ''
+              new_href = '/' + @lang_code + current_dir + href
+            end
+        end
+      end
+      new_href
+    end
+
+    def switch_dom_lang(dom, store, values, url, headers)
+      replace_dom_values(dom, values, store, url, headers)
+
+      # INSERT LANGUAGE METALINKS
+      parent_node = dom.at_css('head') || dom.at_css('body') || dom.at_css('html')
+      published_langs = get_langs(values)
+      published_langs.each do |l|
+        insert_node = Nokogiri::XML::Node.new('link', dom)
+        insert_node['rel'] = 'alternate'
+        insert_node['hreflang'] = l
+        insert_node['href'] = headers.redirect_location(l)
+        parent_node.add_child(insert_node)
+      end
+
+      # set lang property on HTML tag
+      if dom.at_css('html') || dom.at_css('HTML')
+        (dom.at_css('html') || dom.at_css('HTML')).set_attribute('lang', @lang_code)
+      end
+
+      dom.to_html.gsub(/href="([^"]*)"/) { |m| "href=\"#{URI.decode($1)}\"" }
+    end
+
+    private
+    def replace_dom_values(dom, values, store, url, headers)
+      text_index = values['text_vals'] || {}
+      src_index = values['img_vals'] || {}
+      img_src_prefix = values['img_src_prefix'] || ''
+
+      replacers = []
+      # add lang code to anchors href if not default lang
+      if @lang_code != store.settings['default_lang']
+        pattern = store.settings['url_pattern']
+        replacers << LinkReplacer.new(pattern, headers)
+      end
+
+      replacers << TextReplacer.new(text_index)
+      replacers << MetaReplacer.new(text_index)
+      replacers << ImageReplacer.new(url, text_index, src_index, img_src_prefix)
+      replacers << ScriptReplacer.new(store)
+
+      replacers.each do |replacer|
+        replacer.replace(dom, self)
+      end
+    end
+
+    def get_langs(values)
+      langs = Set.new
+      (values['text_vals'] || {}).merge(values['img_vals'] || {}).each do |key, index|
+        index.each do |l, val|
+          langs.add(l)
+        end
+      end
+      langs
+    end
   end
 end