hutch-xamplr 1.0.0

data/LICENSE

@@ -0,0 +1,3 @@
+Copyright (c) 2009 Bob Hutchison
+
+see COPYING

data/README.rdoc

@@ -0,0 +1,118 @@
+= xamplr
+
+xamplr is a tool for developing Ruby programs, the ruby implementation
+of xampl. It facilitates the 'M' part of an MVC architecture. It
+is meant to be very easy to use, supportive of idiomatic Ruby usage,
+and mostly invisible.
+
+With it's current lack of documentation it will be hard to get going
+with xampl. I am working on documentation that will be appearing
+over the next little while, in the meantime if you are willing to
+wade through a blog, you might try looking at:
+
+http://recursive.ca/hutch/category/software/tools-i-use/xampl/
+
+Features:
+  - pure Ruby, no libraries required except those shipped with Ruby 1.8.2
+  - uses XML to declaratively describe a class structure
+    - uses a collection of example XML documents, no schema are required
+    - supports arbitrary object graphs, not just XML-like trees
+    - Ruby idiomatic facilities to navigate the object structures (e.g. named
+      arrays, hash, iterator support, uses blocks, etc.)
+    - support empty, simple (text content only), data (no or limited text
+      content, but with arbitrary element content), and mixed content (where
+      text and element content can be intermingled)
+    - maintains 'document order' of objects
+    - keeps track of parents
+    - can compare two clusters for equality (limited, but will be improved)
+    - maps XML namespaces to Ruby modules (with handy defaults) and you can
+      suggest preferred namespace prefixes for XML output
+    - converts XML element and attributes names to valid (and predictable)
+      Ruby names
+    - includes an XML pull parser (very fast)
+    - code generation is template driven, so the adventurous can change what
+      is generated
+    - includes a simple but powerful and fast templating engine
+    - includes a simple-minded but effective hash-like class that maintains
+      insertion order (fast insertion, lookup, and iteration, but at the
+      expense of delete)
+    - naming conventions are used when generating code (but these conventions
+      can be changed)
+    - additions to the XML will very rarely break existing code, and so
+      incremental development of functionality is supported
+    - generated code is readable and debuggable
+    - a command line tool to generate the code, or you can generate at runtime
+  - XML binding to the classes (i.e. parsing an XML file will produce an
+    object graph)
+  - can augment existing classes
+  - programmer adds functionality by opening the class or module, no worries
+    about the code generator overwriting your changes
+  - object structures can generate XML output (and there is an XML pretty
+    printer used for emphasising human readable XML output)
+  - easy to use persistence mechanism
+    - persistence is in terms of clusters of objects, the cluster is updated
+      all at once
+    - lazy loading of persisted clusters
+    - LRU cache (of clusters)
+    - the changed clusters are available for inspection
+    - XML, Ruby, and YAML persistence mechanism (purposely no Marshal support)
+    - persist to memory, or hard drive (optional persistence to an FSDB)
+    - sync and rollback (even with memory persistence); this is a weak form
+      of transaction support
+    - 'automatic' and 'manual' modes of persistence ('automatic' means that
+      any xampl cluster created while a persister is active will be
+      automatically managed by that persister, manual means that you have
+      to introduce the xampl cluster to the persister)
+    - fast alternative to persistence that does not support rollback
+    - xampl objects can hold arbitrary attribute values through the
+      persistence 'cycle' if using Ruby or YAML persistence (even xampl
+      object); with XML you won't loose anything but all attributes will
+      be strings after restoration.
+    - multiple persisters, possibly with differing persistence mechanisms,
+      are supported
+    - dead easy single user applications (e.g. desktop apps, CGI), almost as
+      easy for multiple user applications and webapps
+    - you can add your own persister types (and formats for that matter)
+  - general purpose Visitor pattern facility, with pre and post visit methods,
+    cycle and revisit detection and handling
+  - lots of hooks
+  - many unit tests, all execute without warning when run with the -w Ruby flag
+  - there are some examples, and the beginnings of a tutorial in the form of
+    a series of examples that lead you through the core parts of xampl
+
+Features that are coming soon:
+  - many more hooks
+  - more sophisticated augmentation of existing classes
+  - indexing and querying of the persisted object structures
+  - more persisters (e.g. sqlite)
+  - real transactions
+  - a better comparison function (that deals with cyclic structures at all, and
+    better with non-tree acyclic structures)
+
+Known Bugs or Potential Problems or Suspected Problems
+  - generated attribute names may conflict with Ruby keywords and this is not
+    a nice thing to witness (this is easy to fix, but I don't want to do this
+    until the next release at the earliest)
+  - I don't know if this will work on Windows -- I tried, but I don't have a
+    windows machine to test this with -- there might be some pathname stuff
+    assuming unix (but I hope not)
+  - some of the code is unnecessarily ugly (e.g. some of the visitor internals)
+  - there are no Ruby docs (but I promise... really)
+  - comparison of cyclic structures will crash
+
+License: AGPLv3 (see COPYING). An alternative license may be negotiated, contact me.
+
+While the Ruby version of xampl is brand new it is based on a Java
+tool that I've been developing since 1998 and in its fifth generation.
+There are many features in the Java version that are not available in
+this library, but that is either just a matter of time, or they will
+never be needed because Ruby programs don't need the help.
+
+The Java version of the tool has been used to write a number of large
+webapps, and a few smaller desktop applications, and even an applet.
+
+There is a Common Lisp/CLOS version coming.
+
+== Copyright
+
+Copyright (c) 2009 Bob Hutchison. See LICENSE and COPYING for details.

data/Rakefile

@@ -0,0 +1,58 @@
+require 'rubygems'
+require 'rake'
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "xamplr"
+    gem.summary = %Q{xamplr is the ruby version of xampl}
+    gem.email = "hutch@recursive.ca"
+    gem.homepage = "http://github.com/hutch/xamplr"
+    gem.authors = ["Bob Hutchison"]
+
+    gem.add_dependency('hutch-xamplr-pp')
+
+    # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: sudo gem install technicalpickles-jeweler -s http://gems.github.com"
+end
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |test|
+  test.libs << 'lib' << 'test'
+  test.pattern = 'test/**/*_test.rb'
+  test.verbose = true
+end
+
+begin
+  require 'rcov/rcovtask'
+  Rcov::RcovTask.new do |test|
+    test.libs << 'test'
+    test.pattern = 'test/**/*_test.rb'
+    test.verbose = true
+  end
+rescue LoadError
+  task :rcov do
+    abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
+  end
+end
+
+
+task :default => :test
+
+require 'rake/rdoctask'
+Rake::RDocTask.new do |rdoc|
+  if File.exist?('VERSION.yml')
+    config = YAML.load(File.read('VERSION.yml'))
+    version = "#{config[:major]}.#{config[:minor]}.#{config[:patch]}"
+  else
+    version = ""
+  end
+
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = "xamplr #{version}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+

data/VERSION.yml

@@ -0,0 +1,4 @@
+--- 
+:patch: 0
+:major: 1
+:minor: 0

data/examples/random-people/Makefile

@@ -0,0 +1,16 @@
+
+nope:
+
+raw:
+	rm -rf repo/raw
+	time ruby rawtc.rb
+	#time ruby rawtc.rb
+	#time ruby rawtc.rb
+	#time ruby rawtc.rb
+	#time ruby rawtc.rb
+	#time ruby rawtc.rb
+
+batch:
+	rm -rf repo/random-people
+	time ruby batch-load-users.rb
+

data/examples/random-people/batch-load-users.rb

@@ -0,0 +1,64 @@
+$LOAD_PATH.unshift("xampl_generated_code")
+
+require 'fastercsv'
+require 'RandomPeople'
+require 'people'
+
+Xampl.set_default_persister_kind(:tokyo_cabinet)
+#Xampl.set_default_persister_format(:xml_format)
+Xampl.set_default_persister_format(:ruby_format)
+
+module RandomPeople
+
+  start_at = Time.now
+
+  arr_of_arrs = FasterCSV.read("random-names.csv")
+
+  parsed_at = Time.now
+
+  #GivenName,Surname,StreetAddress,City,State,ZipCode,EmailAddress,TelephoneNumber
+  #<people pid='' xmlns="http://xampl.com/people">
+  #    <person pid=''
+  #            given-name=''
+  #            surname=''
+  #            street-address=''
+  #            city=''
+  #            state=''
+  #            postal-code=''
+  #            email=''
+  #            phone=''/>
+  #</people>
+
+  base = 0
+
+  10.times do | iter |
+    inner_start = Time.now
+    Xampl.transaction("random-people") do
+
+      people = People.new('people')
+
+      base += arr_of_arrs.size
+
+      arr_of_arrs.each_with_index do | row, i |
+#        person = people.new_person("person-#{ base + i }")
+        person = Person.new("person-#{ base + i }")
+
+        person.given_name = row[0]
+        person.surname = row[1]
+        person.street_address = row[2]
+        person.city = row[3]
+        person.state = row[4]
+        person.postal_code = row[5]
+        person.email = row[6]
+        person.phone = row[7]
+      end
+      puts "transaction ending..."
+    end
+    puts "iter: #{ iter } in #{ Time.now - inner_start }"
+  end
+
+  processed_at = Time.now
+
+  p "parsed in #{ parsed_at - start_at }, processed in: #{ processed_at - parsed_at }"
+
+end

data/examples/random-people/optimise.rb

@@ -0,0 +1,19 @@
+$LOAD_PATH.unshift("xampl_generated_code")
+
+require 'RandomPeople'
+require 'people'
+
+Xampl.set_default_persister_kind(:tokyo_cabinet)
+#Xampl.set_default_persister_format(:xml_format)
+Xampl.set_default_persister_format(:ruby_format)
+
+module RandomPeople
+
+  start_opt = Time.now
+  Xampl.transaction("random-people") do
+    Xampl.optimise(:indexes_only => true)
+#    Xampl.optimise
+  end
+  puts "optimised in #{ Time.now - start_opt}"
+
+end

data/examples/random-people/people.rb

@@ -0,0 +1,25 @@
+$LOAD_PATH.unshift("xampl_generated_code")
+
+require 'RandomPeople'
+
+Xampl.set_default_persister_kind(:tokyo_cabinet)
+Xampl.set_default_persister_format(:xml_format)
+
+module RandomPeople
+
+  class Person
+
+    Xampl::TokyoCabinetPersister.add_lexical_indexs(%w{ surname city state email })
+
+    def describe_yourself
+      {
+              'surname' => self.surname,
+              'city' => self.city,
+              'state' => self.state,
+              'email' => self.email
+      }
+    end
+  end
+
+end
+

data/examples/random-people/query.rb

@@ -0,0 +1,72 @@
+$LOAD_PATH.unshift("xampl_generated_code")
+
+require 'RandomPeople'
+require 'people'
+
+Xampl.set_default_persister_kind(:tokyo_cabinet)
+#Xampl.set_default_persister_format(:xml_format)
+Xampl.set_default_persister_format(:ruby_format)
+
+module RandomPeople
+
+  #GivenName,Surname,StreetAddress,City,State,ZipCode,EmailAddress,TelephoneNumber
+  #<people pid='' xmlns="http://xampl.com/people">
+  #    <person pid=''
+  #            given-name=''
+  #            surname=''
+  #            street-address=''
+  #            city=''
+  #            state=''
+  #            postal-code=''
+  #            email=''
+  #            phone=''/>
+  #</people>
+
+  5.times do
+    found1 = []
+    found2 = []
+    found3 = []
+
+    start_query_at = Time.now
+#    found1 = Xampl.transaction("random-people") do
+#      Xampl.query do | q |
+#        q.add_condition('city', :equals, 'TORONTO')
+#        q.add_condition('email', :ends_with, 'dodgit.com')
+#
+#        q.order_by('surname', :strasc)
+#      end
+#    end
+
+    start_find_at = Time.now
+    found2 = Xampl.transaction("random-people") do
+      Xampl.find_xampl do | q |
+        q.add_condition('city', :equals, 'TORONTO')
+        q.add_condition('email', :ends_with, 'dodgit.com')
+
+#        q.setlimit(10, 10)
+
+        q.order_by('surname', :strasc)
+      end
+    end
+
+    start_find_pids_at = Time.now
+
+    found3 = Xampl.transaction("random-people") do
+      Xampl.find_pids do | q |
+        q.add_condition('city', :equals, 'TORONTO')
+#        q.add_condition('email', :ends_with, 'dodgit.com')
+
+#        q.setlimit(10, 10)
+
+        q.order_by('surname', :strasc)
+      end
+    end
+    done = Time.now
+
+    #  found1.each do | person_meta |
+    #    puts "surname: #{ person_meta['surname']}, given: #{ person_meta['xampl'].given_name }"
+    #  end
+    puts "found:: query #{ found1.size }, find_xampl: #{ found2.size }, find_pids: #{ found3.size }"
+    puts "query in: #{ start_find_at - start_query_at }, find_xampl in: #{ start_find_pids_at - start_find_at }, pids in: #{ done - start_find_pids_at}\n"
+  end
+end

data/examples/random-people/query2.rb

@@ -0,0 +1,76 @@
+$LOAD_PATH.unshift("xampl_generated_code")
+
+require 'RandomPeople'
+require 'people'
+
+Xampl.set_default_persister_kind(:tokyo_cabinet)
+#Xampl.set_default_persister_format(:xml_format)
+Xampl.set_default_persister_format(:ruby_format)
+
+module RandomPeople
+
+  #GivenName,Surname,StreetAddress,City,State,ZipCode,EmailAddress,TelephoneNumber
+  #<people pid='' xmlns="http://xampl.com/people">
+  #    <person pid=''
+  #            given-name=''
+  #            surname=''
+  #            street-address=''
+  #            city=''
+  #            state=''
+  #            postal-code=''
+  #            email=''
+  #            phone=''/>
+  #</people>
+
+  1.times do
+    found1 = []
+    found2 = []
+    found3 = []
+
+    start = Time.now
+
+    Xampl.transaction("random-people") do
+      found1 = Xampl.find_pids do | q |
+        q.add_condition('city', :equals, 'TORONTO')
+        q.add_condition('surname', :equals, 'Smith')
+      end
+
+      found2 = Xampl.find_pids do | q |
+        q.add_condition('city', :equals, 'LONDON')
+        q.add_condition('surname', :equals, 'Smith')
+      end
+
+      found3 = Xampl.find_pids do | q |
+        q.add_condition('city', :stror, 'TORONTO,LONDON')
+        q.add_condition('surname', :equals, 'Smith')
+      end
+    end
+
+    done = Time.now
+
+    puts "INDEXED found(#{ done - start }):: TORONTO #{ found1.size }, LONDON: #{ found2.size }, TORONTO,LONDON: #{ found3.size }/#{ found1.size + found2.size }"
+
+    start = Time.now
+
+    Xampl.transaction("random-people") do
+      found1 = Xampl.find_pids do | q |
+        q.add_condition('city', :equals, 'TORONTO', true, true)
+        q.add_condition('surname', :equals, 'Smith', true, true)
+      end
+
+      found2 = Xampl.find_pids do | q |
+        q.add_condition('city', :equals, 'LONDON', true, true)
+        q.add_condition('surname', :equals, 'Smith', true, true)
+      end
+
+      found3 = Xampl.find_pids do | q |
+        q.add_condition('city', :stror, 'TORONTO,LONDON', true, true)
+        q.add_condition('surname', :equals, 'Smith', true, true)
+      end
+    end
+
+    done = Time.now
+
+    puts "NOT INDEXED found(#{ done - start }):: TORONTO #{ found1.size }, LONDON: #{ found2.size }, TORONTO,LONDON: #{ found3.size }/#{ found1.size + found2.size }"
+  end
+end