i18nliner 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/LICENSE.txt ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2013 Jon Jensen
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,320 @@
1
+ # I18nliner
2
+
3
+ yay readme-driven development!
4
+
5
+ ## TODO
6
+
7
+ * inferred placeholders (instance vars and methods)
8
+ * ERB pre-processor
9
+ * wrapper inference
10
+ * helper/placeholder extraction
11
+ * rake tasks
12
+ * dump
13
+ * diff
14
+ * import
15
+
16
+ ====
17
+
18
+ I18nliner is I18n made simple.
19
+
20
+ No .yml files. Inline defaults. Optional keys. Inferred interpolation values.
21
+ Wrappers and blocks, so your templates look template-y and your translations
22
+ HTML-free.
23
+
24
+ ## TL;DR
25
+
26
+ I18nliner lets you do stuff like this:
27
+
28
+ t "Ohai %{@user.name}, my default translation is right here in the code. " +
29
+ "Inferred keys and placeholder values, oh my!"
30
+
31
+ and even this:
32
+
33
+ <%= t do %>
34
+ Hey <%= amigo %>!
35
+ Although I am <%= link_to "linking to something", random_path %> and
36
+ have some <strong>bold text</strong>, the translators will see
37
+ <strong><em>absolutely no markup</em></strong> and will only have a
38
+ single string to translate :o
39
+ <% end %>
40
+
41
+ ## Installation
42
+
43
+ Add the following to your Gemfile:
44
+
45
+ gem 'i18nliner'
46
+
47
+ ## Features
48
+
49
+ ### No more en.yml
50
+
51
+ Instead of maintaining .yml files and doing stuff like this:
52
+
53
+ I18n.t :account_page_title
54
+
55
+ Forget the .yml and just do:
56
+
57
+ I18n.t :account_page_title, "My Account"
58
+
59
+ Regular I18n options follow the (optional) default translation, so you can do
60
+ the usual stuff (placeholders, etc.).
61
+
62
+ #### Okay, but don't the translators need en.yml?
63
+
64
+ Sure, but *you* don't need to write it. Just run:
65
+
66
+ rake i18nliner:dump
67
+
68
+ This extracts all default translations from your codebase, merges them with any
69
+ other ones (from rails or pre-existing .yml files), and outputs them to
70
+ `config/locales/generated/en.yml` (or rather, `"#{I18n.default_locale}.yml"`).
71
+
72
+ ### It's okay to lose your keys
73
+
74
+ Why waste time coming up with keys that are less descriptive than the default
75
+ translation? I18nliner makes keys optional, so you can just do this:
76
+
77
+ I18n.t "My Account"
78
+
79
+ I18nliner will create a [unique key](CONFIG.md) based on the translation (e.g.
80
+ `:my_account`), so you don't have to.
81
+
82
+ This can actually be a **good thing**, because when the `en` changes, the key
83
+ changes, which means you know you need to get it retranslated (instead of
84
+ letting a now-inaccurate translation hang out indefinitely). Whether you want
85
+ to show "[ missing translation ]" or the `en` value in the meantime is up to
86
+ you.
87
+
88
+ ### Inferred Interpolation Values
89
+
90
+ Interpolation values may be inferred by I18nliner if not provided. So long as
91
+ it's an instance variable or method (or chain), you don't need to specify its
92
+ value. So this:
93
+
94
+ <p>
95
+ <%= t "Hello, %{user}. This request was a %{request_method}.",
96
+ :user => @user.name,
97
+ :request_method => request.method
98
+ %>
99
+ </p>
100
+
101
+ Can just be this:
102
+
103
+ <p>
104
+ <%= t "Hello, %{@user.name}. This request was a %{request.method}." %>
105
+ </p>
106
+
107
+ Note that local variables cannot be inferred.
108
+
109
+ ### Wrappers and Blocks
110
+
111
+ #### The Problem
112
+
113
+ Suppose you have something like this in your ERB:
114
+
115
+ <p>
116
+ You can <%= link_to "lead", new_discussion_path %> a new discussion or
117
+ <%= link_to "join", discussion_search_path %> an existing one.
118
+ </p>
119
+
120
+ You might try something like this:
121
+
122
+ <p>
123
+ <%= t "You can %{lead} a new discussion or %{join} an existing one.",
124
+ :lead => link_to(t("lead"), new_discussion_path),
125
+ :join => link_to(t("join"), discussion_search_path)
126
+ %>
127
+ </p>
128
+
129
+ This is not great, because:
130
+
131
+ 1. There are three strings to translate.
132
+ 2. When translating the verbs, the translator has no context for where it's
133
+ being used... Is "lead" a verb or a noun?
134
+ 3. Translators have their hands somewhat tied as far as what is inside the
135
+ links and what is not.
136
+
137
+ So you might try this instead:
138
+
139
+ <p>
140
+ <%= t :discussion_html,
141
+ "You can <a href="%{lead_url}">lead</a> a new discussion or " +
142
+ "<a href="%{join_url}">join</a> an existing one.",
143
+ :lead_url => new_discussion_path,
144
+ :join_url => discussion_search_path
145
+ %>
146
+ </p>
147
+
148
+ This isn't much better, because now you have HTML in your translations. If you
149
+ want to add a class to the link, you have to go update all the translations.
150
+ A translator could accidentally break your page (or worse, cross-site script
151
+ it).
152
+
153
+ So what do you do?
154
+
155
+ #### Wrappers
156
+
157
+ I18nliner lets you specify wrappers, so you can keep HTML out the translations,
158
+ while still just having a single string needing translation:
159
+
160
+ <p>
161
+ <%= t "You can *lead* a new discussion or **join** an existing one.",
162
+ :wrappers => [
163
+ link_to('\1', new_discussion_path),
164
+ link_to('\1', discussion_search_path)
165
+ ]
166
+ %>
167
+ </p>
168
+
169
+ Default delimiters are increasing numbers of asterisks, but you can specify
170
+ any string as a delimiter by using a hash rather than an array.
171
+
172
+ #### Blocks
173
+
174
+ But wait, there's more!
175
+
176
+ Perhaps you want your templates to look like, well, templates. Try this:
177
+
178
+ <p>
179
+ <%= t do %>
180
+ Welcome to the internets, <%= user.name %>
181
+ <% end %>
182
+ </p>
183
+
184
+ Or even this:
185
+
186
+ <p>
187
+ <%= t do %>
188
+ <b>Ohai <%= user.name %>,</b>
189
+ you can <%= link_to "lead", new_discussion_path %> a new discussion or
190
+ <%= link_to "join", discussion_search_path %> an existing one.
191
+ <% end %>
192
+ </p>
193
+
194
+ In case you're curious about the man behind the curtain, I18nliner adds an ERB
195
+ pre-processor that turns the second example into something like this right
196
+ before it hits ERB:
197
+
198
+ <p>
199
+ <%= t :some_unique_key,
200
+ "*Ohai %{user_name}*, you can **lead** a new discussion or ***join*** an existing one.",
201
+ :user_name => @user.name,
202
+ :wrappers => [
203
+ '<b>\1</b>',
204
+ link_to('\1', new_discussion_path),
205
+ link_to('\1', discussion_search_path)
206
+ ]
207
+ %>
208
+ </p>
209
+
210
+ In other words, it will infer wrappers from your (balanced) markup and
211
+ [`link_to` calls](INFERRED_WRAPPERS.md), and will create placeholders for any
212
+ other ERB expressions. ERB statements (e.g.
213
+ `<% if some_condition %>...`) are *not* supported inside block translations, with
214
+ the notable exception of nested translations, e.g.
215
+
216
+ <%= t do %>
217
+ Be sure to
218
+ <a href="/account/" title="<%= t do %>Account Settings<% end %>">
219
+ set up your account
220
+ </a>.
221
+ <% end %>
222
+
223
+ #### HTML Safety
224
+
225
+ I18nliner ensures translations, interpolated values, and wrappers all play
226
+ nicely (and safely) when it comes to HTML escaping. If any translation,
227
+ interpolated value, or wrapper is HTML-safe, everything else will be HTML-
228
+ escaped.
229
+
230
+ ### Inline Pluralization Support
231
+
232
+ Pluralization can be tricky, but [I18n gives you some flexibility](http://guides.rubyonrails.org/i18n.html#pluralization).
233
+ I18nliner brings this inline with a default translation hash, e.g.
234
+
235
+ t({:one => "There is one light!", :other => "There are %{count} lights!"},
236
+ :count => picard.visible_lights.count)
237
+
238
+ Note that the :count interpolation value needs to be explicitly set when doing
239
+ pluralization. On the ERB side, I18nliner enhances pluralize to take a block,
240
+ so you can do this:
241
+
242
+ <%= pluralize picard.visible_lights.count do %>
243
+ <% one do %>There is one light!<% end %>
244
+ <% other do %>There are <%= count %> lights!<% end %>
245
+ <% end %>
246
+
247
+ If you just want to pluralize a single word, there's a shortcut:
248
+
249
+ t "person", :count => users.count
250
+
251
+ This is equivalent to:
252
+
253
+ t({:one => "1 person", :other => "%{count} people"},
254
+ :count => users.count)
255
+
256
+ I18nliner uses the pluralize helper to determine the default one/other values,
257
+ so if your `I18n.default_locale` is something other than English, you may need
258
+ to [add some inflections](https://gist.github.com/838188).
259
+
260
+ ## Rake Tasks
261
+
262
+ ### i18nliner:check
263
+
264
+ Ensures that there are no problems with your translate calls (e.g. missing
265
+ interpolation values, reusing a key for a different translation, etc.). **Go
266
+ add this to your Jenkins/Travis tasks.**
267
+
268
+ ### i18nliner:dump
269
+
270
+ Does an i18nliner:check, and then extracts all default translations from your
271
+ codebase, merges them with any other ones (from rails or pre-existing .yml
272
+ files), and outputs them to `config/locales/generated/en.yml`.
273
+
274
+ ### i18nliner:diff
275
+
276
+ Does an i18nliner:dump and creates a diff from a previous one (path or git
277
+ commit hash). This is useful if you only want to see what has changed since a
278
+ previous release of your app.
279
+
280
+ ### i18nliner:import
281
+
282
+ Imports a translated yml file. Ensures that all placeholders and wrappers are
283
+ present.
284
+
285
+ #### .i18nignore and more
286
+
287
+ By default, the check and dump tasks will look for inline translations in any
288
+ .rb or .erb files. You can tell it to always skip certain
289
+ files/directories/patterns by creating a .i18nignore file. The syntax is the
290
+ same as [.gitignore](http://www.kernel.org/pub/software/scm/git/docs/gitignore.html),
291
+ though it supports
292
+ [a few extra things](https://github.com/jenseng/globby#compatibility-notes).
293
+
294
+ If you only want to check a particular file/directory/pattern, you can set the
295
+ environment variable `ONLY` when you run the command, e.g.
296
+
297
+ rake i18nliner:check ONLY=/app/**/user*
298
+
299
+ ## Compatibility
300
+
301
+ I18nliner is backwards compatible with I18n, so you can add it to an
302
+ established (and already internationalized) Rails app. Your existing
303
+ translation calls, keys and yml files will still just work without
304
+ modification.
305
+
306
+ I18nliner requires at least Ruby 1.9.3 and Rails 3.
307
+
308
+ ## What about JavaScript/Handlebars?
309
+
310
+ Coming soon. I18nliner was inspired by some [I18n extensions](https://github.com/instructure/canvas-lms/tree/master/lib/i18n_extraction)
311
+ I did in [Canvas-LMS](https://github.com/instructure/canvas-lms). While
312
+ it also has the JavaScript/Handlebars equivalents, they are tightly
313
+ coupled to Canvas-LMS and are written in Ruby. So basically we're talking
314
+ a full reimplementation in JavaScript using
315
+ [esprima](http://esprima.org/) instead of the [shameful, brittle, regexy hack](http://cdn.memegenerator.net/instances/400x/24091937.jpg)
316
+ that is js_extractor.
317
+
318
+ ## License
319
+
320
+ Copyright (c) 2013 Jon Jensen, released under the MIT license
data/Rakefile ADDED
@@ -0,0 +1,9 @@
1
+ require 'rake'
2
+
3
+ require 'rspec/core'
4
+ require 'rspec/core/rake_task'
5
+ RSpec::Core::RakeTask.new(:spec) do |spec|
6
+ spec.pattern = FileList['spec/**/*_spec.rb']
7
+ end
8
+
9
+ task :default => :spec
@@ -0,0 +1,69 @@
1
+ require 'iconv'
2
+ require 'zlib'
3
+
4
+ module I18nliner
5
+ module CallHelpers
6
+ ALLOWED_PLURALIZATION_KEYS = [:zero, :one, :few, :many, :other]
7
+ REQUIRED_PLURALIZATION_KEYS = [:one, :other]
8
+
9
+ def normalize_key(key, scope, receiver)
10
+ key = key.to_s
11
+ scope.normalize_key(key)
12
+ end
13
+
14
+ def normalize_default(default, translate_options = {})
15
+ default = infer_pluralization_hash(default, translate_options)
16
+ default.strip! if default.is_a?(String)
17
+ default
18
+ end
19
+
20
+ def infer_pluralization_hash(default, translate_options)
21
+ return default unless default.is_a?(String) &&
22
+ default =~ /\A[\w\-]+\z/ &&
23
+ translate_options.include?(:count)
24
+ {:one => "1 #{default}", :other => "%{count} #{default.pluralize}"}
25
+ end
26
+
27
+ def infer_key(default, translate_options = {})
28
+ default = default[:other].to_s if default.is_a?(Hash)
29
+ keyify(normalize_default(default, translate_options))
30
+ end
31
+
32
+ def keyify_underscored(string)
33
+ Iconv.iconv('ascii//translit//ignore', 'utf-8', string).
34
+ to_s.
35
+ downcase.
36
+ gsub(/[^a-z0-9_\.]+/, '_').
37
+ gsub(/\A_|_\z/, '')[0..50]
38
+ end
39
+
40
+ def keyify_underscored_crc32(string)
41
+ checksum = Zlib.crc32(string.size.to_s + ":" + string).to_s(16)
42
+ keyify_underscored(string) + "_#{checksum}"
43
+ end
44
+
45
+ def keyify(string)
46
+ case I18nliner.inferred_key_format
47
+ when :underscored then keyify_underscored(string)
48
+ when :underscored_crc32 then keyify_underscored_crc32(string)
49
+ else string
50
+ end
51
+ end
52
+
53
+ # Possible translate signatures:
54
+ #
55
+ # key [, options]
56
+ # key, default_string [, options]
57
+ # key, default_hash, options
58
+ # default_string [, options]
59
+ # default_hash, options
60
+ def key_provided?(scope, receiver, key_or_default = nil, default_or_options = nil, maybe_options = nil, *others)
61
+ return false if key_or_default.is_a?(Hash)
62
+ return true if key_or_default.is_a?(Symbol)
63
+ return true if default_or_options.is_a?(String)
64
+ return true if maybe_options
65
+ return true if I18nliner.look_up(normalize_key(key_or_default, scope, receiver))
66
+ false
67
+ end
68
+ end
69
+ end
@@ -0,0 +1,13 @@
1
+ module I18nliner
2
+ module Commands
3
+ module BasicFormatter
4
+ def red(text)
5
+ text
6
+ end
7
+
8
+ def green(text)
9
+ text
10
+ end
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,59 @@
1
+ module I18nliner
2
+ module Commands
3
+ class Check < GenericCommand
4
+ attr_reader :translations
5
+
6
+ def initialize(options)
7
+ super
8
+ @errors = []
9
+ @translations = TranslationHash.new(I18nliner.manual_translations)
10
+ end
11
+
12
+ def processors
13
+ @processors ||= I18nliner::Processors.all.map do |klass|
14
+ klass.new :only => @options[:only],
15
+ :translations => @translations,
16
+ :checker => method(:check_file)
17
+ end
18
+ end
19
+
20
+ def check_files
21
+ processors.each &:check_files
22
+ end
23
+
24
+ def check_file(file)
25
+ print green(".") if yield file
26
+ rescue SyntaxError, StandardError, ExtractionError
27
+ @errors << "#{$!}\n#{file}"
28
+ print red("F")
29
+ end
30
+
31
+ def failure
32
+ @errors.size > 0
33
+ end
34
+
35
+ def print_summary
36
+ translation_count = processors.sum(&:translation_count)
37
+ file_count = processors.sum(&:file_count)
38
+
39
+ print "\n\n"
40
+
41
+ @errors.each_with_index do |error, i|
42
+ puts "#{i+1})"
43
+ puts red(error)
44
+ print "\n"
45
+ end
46
+
47
+ print "Finished in #{Time.now - @start} seconds\n\n"
48
+ summary = "#{file_count} files, #{translation_count} strings, #{@errors.size} failures"
49
+ puts failure ? red(summary) : green(summary)
50
+ end
51
+
52
+ def run
53
+ check_files
54
+ print_summary
55
+ raise "check command encountered errors" if failure
56
+ end
57
+ end
58
+ end
59
+ end
@@ -0,0 +1,13 @@
1
+ module I18nliner
2
+ module Commands
3
+ module ColorFormatter
4
+ def red(text)
5
+ "\e[31m#{text}\e[0m"
6
+ end
7
+
8
+ def green(text)
9
+ "\e[32m#{text}\e[0m"
10
+ end
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,8 @@
1
+ module I18nliner
2
+ module Commands
3
+ class Dump < GenericCommand
4
+ def run
5
+ end
6
+ end
7
+ end
8
+ end
@@ -0,0 +1,17 @@
1
+ module I18nliner
2
+ module Commands
3
+ class GenericCommand
4
+ include BasicFormatter
5
+
6
+ def initialize(options)
7
+ @options = options
8
+ @start = Time.now
9
+ extend ColorFormatter if $stdout.tty?
10
+ end
11
+
12
+ def self.run(options)
13
+ new(options).run
14
+ end
15
+ end
16
+ end
17
+ end
@@ -0,0 +1,29 @@
1
+ module I18nliner
2
+ class ExtractionError < StandardError
3
+ def initialize(line, detail = nil)
4
+ @line = line
5
+ @detail = detail
6
+ end
7
+
8
+ def to_s
9
+ error = self.class.name.humanize.sub(/ error\z/, '')
10
+ error = "#{error} on line #{@line}"
11
+ @detail ?
12
+ error + " (got #{@detail.inspect})" :
13
+ error
14
+ end
15
+ end
16
+
17
+ class InvalidSignatureError < ExtractionError; end
18
+ class MissingDefaultError < ExtractionError; end
19
+ class AmbiguousKeyError < ExtractionError; end
20
+ class InvalidPluralizationKeyError < ExtractionError; end
21
+ class MissingPluralizationKeyError < ExtractionError; end
22
+ class InvalidPluralizationDefaultError < ExtractionError; end
23
+ class MissingCountValueError < ExtractionError; end
24
+ class MissingInterpolationValueError < ExtractionError; end
25
+ class InvalidOptionsError < ExtractionError; end
26
+ class InvalidOptionKeyError < ExtractionError; end
27
+ class KeyAsScopeError < ExtractionError; end
28
+ class KeyInUseError < ExtractionError; end
29
+ end
@@ -0,0 +1,33 @@
1
+ module I18nliner
2
+ module Extractors
3
+ module AbstractExtractor
4
+ def initialize(options = {})
5
+ @scope = options[:scope] || ''
6
+ @translations = TranslationHash.new(options[:translations] || {})
7
+ @total = 0
8
+ super()
9
+ end
10
+
11
+ def look_up(key)
12
+ @translations[key]
13
+ end
14
+
15
+ def add_translation(full_key, default)
16
+ @total += 1
17
+ @translations[full_key] = default
18
+ end
19
+
20
+ def total_unique
21
+ @translations.total_size
22
+ end
23
+
24
+ def self.included(base)
25
+ base.instance_eval do
26
+ attr_reader :total
27
+ attr_accessor :translations, :scope
28
+ end
29
+ end
30
+ end
31
+ end
32
+ end
33
+
@@ -0,0 +1,102 @@
1
+ module I18nliner
2
+ module Extractors
3
+ class RubyExtractor < SexpProcessor
4
+ TRANSLATE_CALLS = [:t, :translate]
5
+ attr_reader :current_line
6
+
7
+ def initialize(sexps, scope)
8
+ @sexps = sexps
9
+ @scope = scope
10
+ super()
11
+ end
12
+
13
+ def each_translation(&block)
14
+ @block = block
15
+ process(@sexps)
16
+ end
17
+
18
+ def process_call(exp)
19
+ exp.shift
20
+ receiver = process(exp.shift)
21
+ receiver = receiver.last if receiver
22
+ method = exp.shift
23
+
24
+ if extractable_call?(receiver, method)
25
+ @current_line = exp.line
26
+
27
+ # convert s-exps into literals where possible
28
+ args = process_arguments(exp)
29
+
30
+ process_translate_call(receiver, method, args)
31
+ else
32
+ # even if this isn't a translate call, its arguments might contain
33
+ # one
34
+ process exp.shift until exp.empty?
35
+ end
36
+
37
+ s
38
+ end
39
+
40
+ protected
41
+
42
+ def extractable_call?(receiver, method)
43
+ TRANSLATE_CALLS.include?(method) && (receiver.nil? || receiver == :I18n)
44
+ end
45
+
46
+ def process_translate_call(receiver, method, args)
47
+ call = TranslateCall.new(@scope, @current_line, receiver, method, args)
48
+ call.translations.each &@block
49
+ end
50
+
51
+ private
52
+
53
+ def process_arguments(args)
54
+ values = []
55
+ while arg = args.shift
56
+ values << evaluate_expression(arg)
57
+ end
58
+ values
59
+ end
60
+
61
+ def evaluate_expression(exp)
62
+ if exp.sexp_type == :lit || exp.sexp_type == :str
63
+ exp.shift
64
+ return exp.shift
65
+ end
66
+ return string_from(exp) if string_concatenation?(exp)
67
+ return hash_from(exp) if exp.sexp_type == :hash
68
+ process(exp)
69
+ UnsupportedExpression
70
+ end
71
+
72
+ def string_concatenation?(exp)
73
+ exp.sexp_type == :call &&
74
+ exp[2] == :+ &&
75
+ exp.last &&
76
+ exp.last.sexp_type == :str
77
+ end
78
+
79
+ def string_from(exp)
80
+ exp.shift
81
+ lhs = exp.shift
82
+ exp.shift
83
+ rhs = exp.shift
84
+ if lhs.sexp_type == :str
85
+ lhs.last + rhs.last
86
+ elsif string_concatenation?(lhs)
87
+ string_from(lhs) + rhs.last
88
+ else
89
+ return UnsupportedExpression
90
+ end
91
+ end
92
+
93
+ def hash_from(exp)
94
+ exp.shift
95
+ values = exp.map{ |e| evaluate_expression(e) }
96
+ Hash[*values]
97
+ end
98
+ end
99
+
100
+ class UnsupportedExpression; end
101
+ end
102
+ end