magic_cloud 0.0.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: b7ed426c54152c66311804b851d214ff6973a37d
+  data.tar.gz: 45354e6f6d66aa759bb14ddf19c273b3f03a9687
+SHA512:
+  metadata.gz: 85a631a786268da87479796fa4b61e40e0ed043da562662717f92c120484321a362d37baf231eece4790b629cf5a0c76ed1aebc7dd4825d7ecf675353dea0d21
+  data.tar.gz: 3e497cb23c8068ab2ac1761e6298c6e6ffa649b0776df1fe113224c682e841e7c1653800f49cf3a9dfaf130fe07e6921212d2fa1285acfe0111d88e73a50e786

data/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2014-15 Victor 'Zverok' Shepelev
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,142 @@
+MagicCloud - simple pretty word cloud for Ruby
+==============================================
+
+**MagicCloud** is simple, pure-ruby library for making pretty
+[Wordle](http://www.wordle.net/)-like clouds. It uses RMagick as graphic
+backend.
+
+Usage
+-----
+
+```ruby
+words = [
+  [test, 50],
+  [me, 40],
+  [tenderly, 30],
+  # ....
+]
+cloud = MagicCloud::Cloud.new(words, rotate: :free, scale: :log)
+```
+
+Or from command-line:
+
+```
+./bin/magic_cloud --textfile samples/cat-in-the-hat.txt -f test.png --rotate free --scale log
+```
+
+Resulting in:
+
+<img src="https://raw.github.com/zverok/magic_cloud/master/samples/cat.png" alt="Sample word cloud"/>
+
+Installation
+------------
+
+```
+gem install magic_cloud
+```
+
+rmagick is requirement, and it's need compilation, so you may expect
+problems in non-compiler-friendly environment (Windows).
+
+Origins
+-------
+
+At first, it was straightforward port of [d3.layout.cloud.js](https://github.com/jasondavies/d3-cloud)
+by Jason Davies, which, I assume, is an implementation of Wordle algorithm.
+
+Then there was major refatoring, to make code correspond to Ruby
+standards (and be understandable to poor dumb me).
+
+Then collision algorithm was rewritten from scratch.
+
+And now we are here.
+
+References:
+* https://github.com/jasondavies/d3-cloud
+* http://stackoverflow.com/questions/342687/algorithm-to-implement-a-word-cloud-like-wordle
+* http://static.mrfeinberg.com/bv_ch03.pdf
+
+Performance
+-----------
+
+It's reasonable for me. On my small Thinkpad E330, some 50-words cloud 
+image, size 700×500, are typically generated in <3sec. It's not that cool,
+yet not too long for you to fell asleep. The time of cloud making depends
+on words count, size of image (it's faster to find place for all words
+on larger image) and used rotation algorithm (vertical+horizontal words
+only is significantly faster - and, on my opinion, better looking - than
+"cool" free-rotated-words cloud).
+
+Major performance eater is perfect collision detection, which Wordle-like
+cloud needs. MagicCloud for now uses really dumb algortihm with some
+not-so-dumb optimizations. You can look into 
+`lib/magic_cloud/collision_board.rb` - everything can be optimized is 
+there; especially in `CollisionBoard#collides?` method.
+
+I assume, for example, that naive rewriting of code in there as a C
+extension can help significantly.
+
+Another possible way is adding some smart tricks, which eliminate as much
+of pixel-by-pixel comparisons as possible (some of already made are
+criss-cross intersection check, and memoizing of last crossed sprite).
+
+Memory effectiviness
+--------------------
+
+Basically: it's not. 
+
+Plain Ruby arrays are used to represent collision bitmasks (each array 
+member stand for 1 bit), so, for example, 700×500 pixel cloud will requre 
+collision board size `700*500` (i.e. 350k array items only for board, and
+slightly less for all sprites).
+
+It should be wise to use some packing (considering each Ruby Fixmnum can
+represent not one, but whole 32 bits). Unfortunately, all bit array 
+libraries I've tried are causing major slowdown of cloud computation. 
+With, say, 50 words we'll have literally millions of operation 
+`bitmask#[]` and `bitmask#[]=`, so, even methods 
+like `Fixnum#&` and `Fixnum#|` (typically used for bit array representation)
+are causing significant overload.
+
+Configuration
+-------------
+
+```ruby
+cloud = MagicCloud.new(words, palette: palette, rotate: rotate)
+```
+
+* `:palette` (default is `:color20`):
+  * `:category10`, `:category20`, ... - from (d3)[https://github.com/mbostock/d3/wiki/Ordinal-Scales#categorical-colors]
+  * `[array, of, colors]` - each color should be hex color, or any other RMagick color string (See "Color names at http://www.imagemagick.org/RMagick/doc/imusage.html)
+  * any lambda, accepting `(word, index)` and returning color string
+  * any object, responding to `color(word, index)` - so, you can make color 
+    depend on tag text, not only on its number in tags list
+* `:rotate` - rotation algorithm:
+  * `:square` (only horizontal and vertical words) - it's default
+  * `:none` - all words are horizontal (looks boooring)
+  * `:free` - any word rotation angle, looks cool, but not very readable
+    and slower to layout
+  * `[array, of, angles]` - each of possible angle should be number 0..360
+  * any lambda, accepting `(word, index)` and returning 0..360
+  * any object, responding to `rotate(word, index)` and returning 0..360
+* `:scale` - how word sizes would be scaled to fit into (FONT_MIN..FONT_MAX) range:
+  * `:no` - no scaling, all word sizes are treated as is;
+  * `:linear` - linear scaling (default);
+  * `:log` - logarithmic scaling;
+  * `:sqrt` - square root scaling;
+* `:font_family` (Impact is default).
+
+Current state
+-------------
+
+This library is extracted from real-life project. It should be
+pretty stable (apart from bugs introduced during extraction and gemification).
+
+What it really lacks for now, is thorough (or any) testing, and
+some more configuration options.
+
+Also, while core algorithms (collision_board.rb, layouter.rb) are pretty
+accurately written and documented, "wrapping code" (options parsing and
+so on) are a bit more chaotic - it's subject to refactor and cleanup.
+
+All feedback, usage examples, bug reports and feature requests are appreciated!

data/bin/magic_cloud

@@ -0,0 +1,86 @@
+#!/usr/bin/env ruby
+#encoding: utf-8
+require 'rubygems'
+require 'slop'
+
+$:.unshift 'lib'
+require 'magic_cloud'
+
+opts = Slop.parse(help: true, strict: true) do
+  on :width=, "Cloud width, pixels (default 960)", as: Integer, default: 960
+  on :height=, "Cloud heigh, pixels (default 600)", as: Integer, default: 600
+
+  on :words=,
+    "List of words with weights, comma-separated, weight after colon, like 'cat:40,dog:30,eat:20'",
+    as: Array
+  on :textfile=, "Path to file with some text, magic_cloud will calculate frequencies of words in this text and draw the cloud of them"
+  on :maxwords=, "Max words to make a cloud", as: Integer, default: 100
+
+  on :palette=, "Palette name, see README for details"
+  on :rotate=, "Rotation algo, see README for details"
+  on :scale=, "Scaling algo, see README for details"
+  on :font_family=, "Font family (Impact by default)"
+
+  on :f, :file=, "Output file path", required: true
+
+  on :stats, "Output debug log and stats (different operations count while looking for places for words)"
+  on :profile, "Run profiler"
+end
+
+include MagicCloud
+
+Debug.logger.level = Logger::INFO if opts.stats?
+
+words = case
+when opts[:words]
+  opts[:words].
+    map{|w| w.split(':')}.
+    map{|w,c| [w, c.to_i]}.
+    each{|w,c| c.zero? and fail(ArgumentError, "Count for word #{w} not defined")}
+when opts[:textfile]
+  WORD_SEPARATORS = /[\s\u3031-\u3035\u309b\u309c\u30a0\u30fc\uff70]+/
+  STOPWORDS = /^(i|me|my|myself|we|us|our|ours|ourselves|you|your|yours|yourself|yourselves|he|him|his|himself|she|her|hers|herself|it|its|itself|they|them|their|theirs|themselves|what|which|who|whom|whose|this|that|these|those|am|is|are|was|were|be|been|being|have|has|had|having|do|does|did|doing|will|would|should|can|could|ought|i'm|you're|he's|she's|it's|we're|they're|i've|you've|we've|they've|i'd|you'd|he'd|she'd|we'd|they'd|i'll|you'll|he'll|she'll|we'll|they'll|isn't|aren't|wasn't|weren't|hasn't|haven't|hadn't|doesn't|don't|didn't|won't|wouldn't|shan't|shouldn't|can't|cannot|couldn't|mustn't|let's|that's|who's|what's|here's|there's|when's|where's|why's|how's|a|an|the|and|but|if|or|because|as|until|while|of|at|by|for|with|about|against|between|into|through|during|before|after|above|below|to|from|up|upon|down|in|out|on|off|over|under|again|further|then|once|here|there|when|where|why|how|all|any|both|each|few|more|most|other|some|such|no|nor|not|only|own|same|so|than|too|very|say|says|said|shall)$/
+
+  File.read(opts[:textfile]).
+    split(WORD_SEPARATORS).
+    map{|word| word.gsub(/[[:punct:]]/, '')}.
+    map(&:downcase).
+    reject{|word| word =~ STOPWORDS}.
+    group_by{|w| w}.
+    map{|word, group| [word, group.size]}
+else
+  fail ArgumentError, "You should provide either --words or --textfile option, where can I take words for cloud?.."
+end
+
+words = words.sort_by(&:last).reverse.first(opts[:maxwords])
+
+options = {
+  palette: opts[:palette] && opts[:palette].to_sym,
+  rotate: opts[:rotate] && opts[:rotate].to_sym,
+  scale: opts[:scale] && opts[:scale].to_sym,
+  font_family: opts[:font_family]
+}.reject{|k, v| v.nil?}
+
+if opts.profile?
+  require 'ruby-prof'
+  RubyProf.start
+end
+
+start = Time.now
+
+cloud = Cloud.new(words, options)
+img = cloud.draw(opts[:width], opts[:height])
+
+if opts.profile?
+  result = RubyProf.stop
+  require 'fileutils'
+  FileUtils.mkdir_p 'profile'
+  RubyProf::GraphHtmlPrinter.new(result).
+    print(File.open('profile/result.html', 'w'))
+end
+
+img.write(opts[:file])
+
+p Debug.stats if opts.stats?
+
+puts "Ready in %.2f seconds" % (Time.now - start)

data/lib/magic_cloud.rb

@@ -0,0 +1,7 @@
+# encoding: utf-8
+
+# Wordle-like word cloud main module
+module MagicCloud
+end
+
+require_relative 'magic_cloud/cloud'

data/lib/magic_cloud/bit_matrix.rb

@@ -0,0 +1,33 @@
+# encoding: utf-8
+module MagicCloud
+  # Dead simple 2-dimensional "bit matrix", storing 1s and 0s.
+  # Not memory effectife at all, but the fastest pure-Ruby solution
+  # I've tried.
+  class BitMatrix
+    def initialize(width, height)
+      @width, @height = width, height
+      @bits = [0] * height*width
+    end
+
+    attr_reader :bits, :width, :height
+
+    def put(x, y, px = 1)
+      x < width or fail("#{x} outside matrix: #{width}")
+      y < height or fail("#{y} outside matrix: #{height}")
+
+      bits[y*@width + x] = 1 unless px == 0 # It's faster with unless
+    end
+
+    # returns true/false
+    # FIXME: maybe #put should also accept true/false
+    def at(x, y)
+      bits[y*@width + x] != 0 # faster than .zero?
+    end
+
+    def dump
+      (0...height).map{|y|
+        (0...width).map{|x| at(x, y) ? ' ' : 'x'}.join
+      }.join("\n")
+    end
+  end
+end

data/lib/magic_cloud/canvas.rb

@@ -0,0 +1,93 @@
+# encoding: utf-8
+require 'RMagick'
+
+module MagicCloud
+  # Thin wrapper around RMagick, incapsulating ALL the real drawing.
+  # As it's only class that "knows" about underlying graphics library,
+  # it should be possible to replace it with another canvas with same
+  # interface, not using RMagick.
+  class Canvas
+    def initialize(w, h, back = 'transparent')
+      @width, @height = w, h
+      @internal = Magick::Image.new(w, h){|i| i.background_color =  back}
+    end
+
+    attr_reader :internal, :width, :height
+
+    RADIANS = Math::PI / 180
+
+    def draw_text(text, options = {})
+      draw = Magick::Draw.new # FIXME: is it necessary every time?
+      
+      x = options.fetch(:x, 0)
+      y = options.fetch(:y, 0)
+      rotate = options.fetch(:rotate, 0)
+
+      set_text_options(draw, options)
+
+      rect = _measure_text(draw, text, rotate)
+
+      draw.
+        translate(x + rect.width/2, y + rect.height/2).
+        rotate(rotate).
+        translate(0, rect.height/8). # RMagick text_align seems really weird
+        text(0, 0, text).
+        draw(@internal)
+
+      rect
+    end
+
+    def measure_text(text, options)
+      draw = Magick::Draw.new
+      set_text_options(draw, options)
+      _measure_text(draw, text, options.fetch(:rotate, 0))
+    end
+
+    def pixels(x, y, w, h)
+      @internal.export_pixels(x, y, w, h, 'RGBA')
+    end
+
+    # rubocop:disable TrivialAccessors
+    def render
+      @internal
+    end
+    # rubocop:enable TrivialAccessors
+
+    private
+
+    def set_text_options(draw, options)
+      draw.font_family = options[:font_family]
+      draw.font_weight = Magick::NormalWeight
+      draw.font_style = Magick::NormalStyle
+
+      draw.pointsize = options[:font_size]
+      draw.fill_color(options[:color])
+      draw.gravity(Magick::CenterGravity)
+      draw.text_align(Magick::CenterAlign)
+    end
+
+    def _measure_text(draw, text, rotate)
+      metrics = draw.get_type_metrics('"' + text + 'm"')
+      w, h = rotated_metrics(metrics.width, metrics.height, rotate)
+
+      Rect.new(0, 0, w, h)
+    end
+
+    def rotated_metrics(w, h, degrees)
+      radians = degrees * Math::PI / 180
+
+      # FIXME: not too clear, just straightforward from d3.cloud
+      sr = Math.sin(radians)
+      cr = Math.cos(radians)
+      wcr = w * cr
+      wsr = w * sr
+      hcr = h * cr
+      hsr = h * sr
+
+      w = [(wcr + hsr).abs, (wcr - hsr).abs].max.to_i
+      h = [(wsr + hcr).abs, (wsr - hcr).abs].max.to_i
+
+      [w, h]
+    end
+  end
+end

data/lib/magic_cloud/cloud.rb

@@ -0,0 +1,137 @@
+# encoding: utf-8
+require_relative './rect'
+require_relative './canvas'
+require_relative './palettes'
+
+require_relative './word'
+
+require_relative './layouter'
+require_relative './spriter'
+
+require_relative './debug'
+
+module MagicCloud
+  # Main word-cloud class. Takes words with sizes, returns image
+  class Cloud
+    def initialize(words, options = {})
+      @words = words.sort_by(&:last).reverse
+      @options = options
+      @scaler = make_scaler(words, options[:scale] || :log)
+      @rotator = make_rotator(options[:rotate] || :square)
+      @palette = make_palette(options[:palette] || :default)
+    end
+
+    DEFAULT_FAMILY = 'Impact'
+
+    def draw(width, height)
+      # FIXME: do it in init, for specs would be happy
+      shapes = @words.each_with_index.map{|(word, size), i|
+        Word.new(
+          word,
+          font_family: @options[:font_family] || DEFAULT_FAMILY,
+          font_size: scaler.call(word, size, i),
+          color: palette.call(word, i),
+          rotate: rotator.call(word, i)
+        )
+      }
+
+      Debug.reset!
+
+      spriter = Spriter.new
+      spriter.make_sprites!(shapes)
+
+      layouter = Layouter.new(width, height)
+      visible = layouter.layout!(shapes)
+
+      canvas = Canvas.new(width, height, 'white')
+      visible.each{|sh| sh.draw(canvas)}
+
+      canvas.render
+    end
+
+    private
+
+    attr_reader :palette, :rotator, :scaler
+
+    # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity,Metrics/AbcSize
+    def make_palette(source)
+      case source
+      when :default
+        make_const_palette(:category20)
+      when Symbol
+        make_const_palette(source)
+      when Array
+        ->(_, index){source[index % source.size]}
+      when Proc
+        source
+      when ->(s){s.respond_to?(:color)}
+        ->(word, index){source.color(word, index)}
+      else
+        fail ArgumentError, "Unknown palette: #{source.inspect}"
+      end
+    end
+
+    def make_const_palette(sym)
+      palette = PALETTES[sym] or
+        fail(ArgumentError, "Unknown palette: #{sym.inspect}")
+
+      ->(_, index){palette[index % palette.size]}
+    end
+
+    def make_rotator(source)
+      case source
+      when :none
+        ->(*){0}
+      when :square
+        ->(*){
+          (rand * 2).to_i * 90
+        }
+      when :free
+        ->(*){
+          (((rand * 6) - 3) * 30).round
+        }
+      when Array
+        ->(*){
+          source.sample
+        }
+      when Proc
+        source
+      when ->(s){s.respond_to?(:rotate)}
+        ->(word, index){source.rotate(word, index)}
+      else
+        fail ArgumentError, "Unknown rotation algo: #{source.inspect}"
+      end
+    end
+
+    # FIXME: should be options too
+    FONT_MIN = 10
+    FONT_MAX = 100
+
+    def make_scaler(words, algo)
+      norm =
+        case algo
+        when :no
+          # no normalization, treat tag weights as font size
+          return ->(_word, size, _index){size}
+        when :linear
+          ->(x){x}
+        when :log
+          ->(x){Math.log(x) / Math.log(10)}
+        when :sqrt
+          ->(x){Math.sqrt(x)}
+        else
+          fail ArgumentError, "Unknown scaling algo: #{algo.inspect}"
+        end
+
+      smin = norm.call(words.map(&:last).min)
+      smax = norm.call(words.map(&:last).max)
+      koeff = (FONT_MAX - FONT_MIN).to_f / (smax - smin)
+
+      ->(_word, size, _index){
+        ssize = norm.call(size)
+        ((ssize - smin).to_f * koeff + FONT_MIN).to_i
+      }
+    end
+    # rubocop:enable Metrics/MethodLength, Metrics/CyclomaticComplexity,Metrics/AbcSize
+  end
+end