wurfl 1.0.2

data/.gitignore

@@ -0,0 +1,2 @@
+*.swp
+pkg/*

data/LICENSE

@@ -0,0 +1,35 @@
+Copyright (c) 2009, mobalean (http://www.mobalean.com/)
+
+Copyright (c) 2003, Ubiquitous Business Technology (http://ubit.com)
+All rights reserved.
+
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+
+   * Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+
+   * Redistributions in binary form must reproduce the above
+     copyright notice, this list of conditions and the following
+     disclaimer in the documentation and/or other materials provided
+     with the distribution.
+
+   * Neither the name of the WURFL nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.rdoc

@@ -0,0 +1,144 @@
+= WURFL
+
+WURFL is a collection of libraries and command line tools for using and manipulating the WURFL.  To start using it, simply install the gem:
+
+  sudo gem install wurfl
+
+== Tools
+
+=== wurflloader.rb
+
+Is used to parse and load a WURFL XML file into memory or save a
+PStore database that is used by most of the other tools.  This
+application creates WURFL PStore databases that are essential for use
+with the other Ruby applications.
+
+=== wurflinspector.rb
+
+Is a tool that will let you do various searches and queries on the
+WURFL.  This is a very simple, yet powerful program for finding
+information about the handsets in the WURFL.  See the wurflinspector
+examples section for usage.
+
+==== Usage
+
+The command below will search through all handsets and return the ids
+of handsets that match the passed Ruby boolean evaluation
+*) This command will return all handsets that have more than 2 colors.
+"wurflinspector.rb -d pstorehandsets.db -s '{ |hand| hand["colors"].to_i > 2 }'"
+The Ruby query must go in between the single quote marks and needs to
+declare the WurflHandset instance variable name.
+
+*) This command shows you how you can cheat with the current design of
+the wurflinspector and print more user friendly results.  This example
+assumes you have the command line programs sort and uniq, but that is
+only to make the output look better. This example does the same as the
+above, except that it prints out the brand name and model name of the
+matching handsets instead of the WURFL id.
+
+Note: this should all go into one command line call
+"wurflinspector.rb -d pstorehandsets.db -s '{|hand| puts
+"#{hand["brand_name"]} #{hand["model_name"]}" if hand["colors"].to_i >
+2}' | sort | uniq"
+
+
+The following individual handset query commands will tell the value of
+capabilities and from where it obtained the setting.
+
+*) A command to query the handset with the id sonyericsson_t300_ver1
+for all of its' capabilities:
+"wurflinspector.rb -d pstorehandsets.db -i sonyericsson_t300_ver1"
+
+*) A command to query the handset with the id sonyericsson_t300_ver1 for
+backlight capability:
+"wurflinspector.rb -d pstorehandsets.db -i sonyericsson_t300_ver1 -q
+backlight"
+
+=== wurflsanitycheck.rb
+
+Is a partial WURFL validating program.  It does a few simple
+checks to make sure the XML structure is parse-able by the wurflloader.
+If you receive loading errors by the wurflloader, then you can run the
+wurflsanitycheck program to find the lines in the XML file that might
+be causing the problem.
+
+=== wurflcomparator.rb
+
+Is a another simple program that will find the differences from two
+WURFL Ruby PStore databases. This is another way of finding changes
+from the different versions of the WURFL without running a diff on the
+XML files.
+
+=== uaproftowurfl.rb
+
+Is a program that takes UAProfiles and creates an equivalent WURFL
+entry.  It holds all of the mappings used to convert a UAProfile to a
+WURFL entry.  This program alone makes using the Ruby tools worth it.
+
+The mappings are not fully complete and can certainly use your input
+in improving them.
+
+About the source code:
+The main piece of code to read are the methods under the 
+"UAProfile Mappings" comment.
+
+For now you can ignore the details above this comment.
+Basically each method is the name of a UAProfifle component.
+When you parse a UAProfile file it will call each UAProfile
+component's method name.  
+
+So you can simply look at UAProfile component's method to see 
+how it maps to the WURFL.
+If a component is not found as a method it will be logged to 
+Standard Error.  For this log one can then add the method to
+the UAProfToWurfl class later.
+
+==== Usage
+
+A simple usage is:
+
+  ruby uaproftowurfl.rb sampleprofile.xml
+
+Example use from a bash shell with many profiles:
+
+  for i in `ls profiles`; do 
+    ruby uaproftowurfl.rb profiles/$i >output/$i.wurfl 
+    2> output/$i.parser.err; 
+  done
+
+This assumes that you have a profiles directory with all of the UAProf
+file you wish to parse and a output directory to place the results and
+errors.
+
+=== uaprofwurflcomparator.rb
+
+Is a program that compares UAProfiles against the equivalent WURFL
+entries. It takes a file that contains an UAProfile URL and an
+User-Agent per line, in addition to a Ruby Wurfl PStore database. It
+then compares the UAProfile against the Wurfl that matches the same
+User Agent.  The output is a list of the differences and a WURFL
+formatted entry showing the differences.
+
+==== Usage
+
+You pass the program a directory to save the UAProfile files taken
+from the given URL, a file that holds the UAProfil URL and User-Agent
+mappings, and the Ruby PStore database that holds the WURFL. 
+
+The following is a simple example of execution
+uaprofwurflcomparator.rb -d ./profiles -f all-profile.2003-08.log -c -w wurfl.db
+
+== Authors
+
+Zev Blut (Original Author)
+Paul McMahon (gem installer, refactorings)
+
+== Copyright
+
+Copyright (c) 2009, mobalean (http://www.mobalean.com/)
+
+Copyright (c) 2003, Ubiquitous Business Technology (http://ubit.com)
+All rights reserved.
+
+Send enquiries to: mailto:info@mobalean.com
+

data/Rakefile

@@ -0,0 +1,60 @@
+require 'rake/rdoctask'
+require 'rake/testtask'
+require 'rubygems'
+
+task :default => ['test']
+
+Rake::TestTask.new(:test) do |t|
+  t.test_files = FileList['test/*_test.rb']
+  t.ruby_opts = ['-rubygems'] if defined? Gem
+end
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |s|
+    s.name = "wurfl"
+    s.summary = "Library and tools for manipulating the WURFL"
+    s.description = "Library and tools for manipulating the WURFL"
+    s.email = "info@mobalean.com"
+    s.homepage = "http://github.com/pwim/wurfl"
+    s.description = "TODO"
+    s.authors = ["Paul McMahon", "Zev Blut"]
+    s.rubyforge_project = 'wurfl'
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end
+
+Rake::RDocTask.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'WURFL'
+  rdoc.main     = "README.rdoc"
+  rdoc.rdoc_files.include("README.rdoc", "lib/**/*.rb")
+end
+
+begin
+  require 'rake/contrib/sshpublisher'
+  namespace :rubyforge do
+
+    desc "Release gem and RDoc documentation to RubyForge"
+    task :release => ["rubyforge:release:gem", "rubyforge:release:docs"]
+
+    namespace :release do
+      desc "Publish RDoc to RubyForge."
+      task :docs => [:rdoc] do
+        config = YAML.load(
+            File.read(File.expand_path('~/.rubyforge/user-config.yml'))
+        )
+
+        host = "#{config['username']}@rubyforge.org"
+        remote_dir = "/var/www/gforge-projects/wurfl/"
+        local_dir = 'rdoc'
+
+        Rake::SshDirPublisher.new(host, remote_dir, local_dir).upload
+      end
+    end
+  end
+rescue LoadError
+  puts "Rake SshDirPublisher is unavailable or your rubyforge environment is not configured."
+end
+

data/VERSION

@@ -0,0 +1 @@
+1.0.2

data/bin/uaproftowurfl.rb

@@ -0,0 +1,28 @@
+#!/usr/bin/env ruby
+
+$LOAD_PATH << File.join(File.dirname(__FILE__), '..',  'lib')
+require "wurfl/uaproftowurfl"
+
+# The code below is called if this file is executed from the command line.
+
+def usage    
+  puts "Usage: usaprofparser.rb uaprof_files"
+  puts "No files passed to parse."
+  exit 1
+end
+
+if ARGV.size == 0
+  usage
+end
+
+uaprof = Wurfl::UAProfToWURLF.new
+
+# Parse all the files and merge them into one UAProf.
+# Following profs take precedence of previous ones
+ARGV.each do |file|
+  uaprof.parse_UAProf(file)
+end
+
+# Now output the mapped WURFL to standard out
+uaprof.output_WURFL
+  

data/bin/uaprofwurflcomparator.rb

@@ -0,0 +1,223 @@
+#!/usr/bin/env ruby
+
+$LOAD_PATH << File.join(File.dirname(__FILE__), '..',  'lib')
+
+require "getoptlong"
+require "net/http"
+
+require "wurfl/uaproftowurfl"
+require "wurfl/handset"
+require "wurfl/utils"
+include Wurfl::Utils
+
+# An addition to the UAProf to Wurfl to generate a WurflHandset from the UAProf.      
+class Wurfl::UAProfToWURLF
+  def make_wurfl_handset
+    hand = Wurfl::Handset.new("UAProf",@wurfl["user_agent"])
+    @wurflgroups.each do |group|
+      @wurfl[group].sort.each do |key,value|
+	hand[key] =  value
+      end
+    end
+    return hand
+  end
+end
+
+
+def parse_mapping_file(file)
+  if !File.exist?(file)
+    $stderr.puts "Mapping File does not exist.  File passed was #{file}."
+    return Array.new
+  end
+  mappings = Array.new
+  f = File.new(file)
+  f.each do |line|
+    if m = /^"(.*)" "(.*)"$/.match(line.strip)
+      uaprof = m[1]
+      useragent = m[2]
+      mappings<< [uaprof,useragent]
+    else
+      $stderr.puts "Irregular format for line: #{line}" if line.strip != ""
+    end
+  end
+  f.close
+
+  return mappings
+end
+
+def get_uaprofile(uaprof,profiledir,check=false)
+  file = strip_uaprof(uaprof)
+  if File.exists?("#{profiledir}/#{file}") && check
+    return file
+  end
+  
+  get_and_save_uaprof_file(uaprof,profiledir)
+  return file
+end
+
+def strip_uaprof(uaprof)
+  uaprof_file = nil
+  if m = /([^\/]*)$/.match(uaprof)
+    uaprof_file = m[1]
+  else
+    $stderr.puts "Cannot find the base UAProf file in URI: #{uaprof}"
+  end
+  return uaprof_file
+end
+
+def load_pstore(pstorefile)
+  hands = Hash.new
+  begin
+    handsid, = load_wurfl_pstore(pstorefile)
+    handsid.each { |id,val| hands[val.user_agent] = val }
+  rescue => err
+    $stderr.puts "Error: Cannot load PStore file. #{pstorefile}"
+    $stderr.puts err.message
+    exit 1
+  end
+  return hands
+end
+
+def get_and_save_uaprof_file(uaprof_url,savedirectory,limit=0)
+  base,path,port = parse_url(uaprof_url)
+  
+  raise "Too many redirects from original url" if limit > 3
+  raise "Unparseable URL: #{url}" if base.nil?
+
+  port = 80 if port.nil?
+  http = Net::HTTP.new(base,port)
+  begin
+    resp, data = http.get(path)
+    if resp.code == "301"
+      # get location and call self again
+      http.finish
+      limit += 1
+      get_and_save_uaprof_file(resp['location'],savedirectory,limit)
+      return
+    elsif resp.code != "200"
+      raise "Unexpected HTTP Response code:#{resp.code} for #{uaprof_url}"
+    end
+  rescue => err
+    raise
+  end
+
+  f = File.new("#{savedirectory}/#{strip_uaprof(path)}","w")
+  f.write(data)  
+  f.close
+  
+end
+
+def parse_url(url)
+  m = /(http:\/\/)?(.*?)(:(\d*))?\/(.*)/i.match(url.strip)
+  
+  return nil if m.nil?
+  return m[2],"/#{m[5]}",m[4]
+end
+
+def usage
+  puts "Usage: uaprofwurflcomparator.rb  -d profiledirectory -f mappingfile [-w wurfldb] [-c] [-h | --help]"
+  puts "Examples:"
+  puts "uaprofwurflcomparator.rb -d ./profiles -f all-profile.2003-08.log -c -w wurfl.db"
+  exit 1
+end
+
+def help
+  puts "-d --directory : The directory to store the UA Profiles found in the log file."
+  puts "-f --file : The log file that has a UAProfile to User-Agent mapping per line."
+  puts "-c --check : A flag that will make sure to check if the profile is already in the directory or not.  If it is not then it will download it."
+  puts "-w --wurfldb : A Ruby PStore Database of the WURFL, that is used to compare against the UAProfiles."
+  puts "-h --help : This message."
+  exit 1
+end
+
+profiledirectory = mappingfile = pstorefile = nil
+existancecheck = false
+begin
+  opt = GetoptLong.new(
+                       ["-d","--directory", GetoptLong::REQUIRED_ARGUMENT],
+                       ["-f","--file", GetoptLong::REQUIRED_ARGUMENT],
+                       ["-c","--check", GetoptLong::NO_ARGUMENT],
+                       ["-h","--help", GetoptLong::NO_ARGUMENT],
+                       ["-w","--wurfldb", GetoptLong::REQUIRED_ARGUMENT]
+                       )
+  
+  opt.each { |arg,val|
+    case arg
+    when "-d"
+      profiledirectory = val.strip
+    when "-f"
+      mappingfile = val.strip
+    when "-c"
+      existancecheck = true
+    when "-h"
+      help
+    when "-w"
+      pstorefile = val.strip
+    else
+      usage
+    end
+  }
+  usage if mappingfile.nil? || profiledirectory.nil?
+rescue => err
+  usage
+end
+
+profiles = Hash.new
+duplicates = Hash.new
+mappings = parse_mapping_file(mappingfile)
+mappings.each do |uaprof,useragent|
+  begin
+    prof_file = get_uaprofile(uaprof,profiledirectory,existancecheck)
+    uaprof_mapper = UAProfToWURLF.new
+    if profiles.key?(useragent)
+      duplicates[useragent] = Array.new if !duplicates.key?(useragent)
+      duplicates[useragent]<<uaprof
+      next
+    end
+    uaprof_mapper.parse_UAProf("#{profiledirectory}/#{prof_file}")
+    profiles[useragent] = uaprof_mapper
+  rescue => err
+    $stderr.puts "Error: File #{prof_file}; User-Agent:#{useragent}"
+    $stderr.puts "Error:#{err.message}"      
+  end  
+end
+
+duplicates.each do |key,profs|
+  $stderr.puts "Duplicates exist for #{key}"
+  profs.each {|prof| $stderr.puts "-- #{prof}" }
+end
+
+exit 0 if !pstorefile
+
+wurflhandsets = load_pstore(pstorefile)
+
+puts "Comparing WURFL Handsets"
+profiles.each do |key,val|
+  puts "",""
+  
+  if !wurflhandsets.key?(key)
+    puts "UAProf has a new Handset: #{key}"
+    puts "--------------------------------"
+    val.output_WURFL
+    puts "--------------------------------"
+  else
+    uahand = val.make_wurfl_handset                   
+    res = uahand.compare(wurflhandsets[key])
+    if res.size > 0
+      puts "#{key} : For UAProf and WURFL differ"
+      res.each do |dkey,dval,did|
+        next if did == "generic"
+        #Key UAPROF Value WURFL Value WURFL source id
+        puts "  Key:#{dkey}; UVAL:#{uahand[dkey]}; WVAL:#{dval}; WSRCID:#{did}"
+      end
+      #val["user_agent"] = key
+      puts ""
+      puts "WURFL Changes are:"
+      puts ""	  
+      val.output_WURFL(res.map {|entry| entry[0]})
+    else
+      puts "#{key} : For UAProf and WURFL match"
+    end
+  end
+end
+