persistent_memoize 0.0.1

data/.gitignore

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in persistent_memoize.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,23 @@
+Copyright (c) 2013 Neil Kandalgaonkar
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject
+to the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR
+ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
+CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+

data/README.md

@@ -0,0 +1,133 @@
+# PersistentMemoize
+
+## Description
+Speed up methods at the cost of disk space. Keep caches on disk to speed
+up later invocations of your program.
+
+## Installation
+Add this line to your application's Gemfile:
+
+    gem 'persistent_memoize'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install persistent_memoize
+
+## Synopsis
+
+Let's imagine we have a script, _fib.rb_:
+
+    def fib(n)
+      return n if n < 2
+      fib(n-1) + fib(n-2)
+    end
+
+    puts fib(40) 
+
+Executing it is slow - on my machine, this is 20 seconds!
+    
+    $ time ruby fib.rb
+    102334155
+
+    real  0m20.107s
+    user  0m20.067s
+    sys 0m0.034s
+
+So let's add memoization.
+
+    require 'persistent_memoize'
+    include PersistentMemoize
+
+    def fib(n)
+      return n if n < 2
+      fib(n-1) + fib(n-2)
+    end
+
+    memoize(:fib, './fib-cache')
+    puts fib(40) 
+
+And executing it now takes about a tenth of a second.
+
+    $ time ruby ./fib.rb
+    102334155
+
+    real  0m0.102s
+    user  0m0.062s
+    sys 0m0.036s
+
+What's more, it stays fast - subsequent invocations after the first one are even faster.
+
+    $ time ruby ./fib.rb
+    102334155
+
+    real  0m0.092s
+    user  0m0.060s
+    sys 0m0.029s
+
+## Motivation
+
+This library is most useful when you have a program that runs repeatedly,
+but which has to incorporate calculations or API results that rarely 
+change. 
+
+I use it to rebuild a static blog, which is based on my postings 
+on other sites, fetched via API. If I tweak the look of the blog, I don't want to have to
+fetch all those postings all over again. So it's a quick modification
+to the library that looks up those results, to cache them locally, via
+memoization. If I know that those blog postings have changed, then I 
+delete the cache, manually, and then it regenerates from scratch. 
+
+## Caveats
+
+As with all memoization, the method memoized must be _pure_, that
+is, the result depends solely on its arguments. Memoizing a function which
+returns the current date will give you the wrong answer tomorrow. 
+
+This can't magically make every method faster; it's all trade-offs. The
+thing-to-be-memoized should be more expensive than computing a hash, and 
+deserializing data from disk. Otherwise memoizing will make it slower.
+
+This depends on arguments having a unique serialization via the Marshal
+library. Certain Ruby constructs cannot be serialized in this way and 
+may raise TypeErrors. See the 
+[Marshal documentation](http://www.ruby-doc.org/core-2.0/Marshal.html) for details. 
+
+
+## Constants
+    PersistentMemoize::PERSISTENT_MEMOIZE_VERSION
+
+Returns the version of this package as a String.
+
+## Methods
+    PersistentMemoize#memoize(method, path)
+Takes a _method_ (symbol) and caches the results of _method_, for 
+particular arguments, on disk, in files under _path_. 
+
+If you call _method_ again with the same arguments, _memoize_ gives
+you the value from disk instead of letting the method compute the
+value again.
+
+## Acknowledgements
+
+Based on memoize by Daniel Berg (https://github.com/djberg96/memoize)
+
+Daniel's library has a file-oriented cache but it creates a single cache file per
+method, containing all the cached results. For my workloads, this cache file 
+gets very large and rewriting it is slow. So persistent_memoize stores each result
+in its own file.
+
+Daniel also included this note in his code, so I might as well acknowledge 
+these people too:
+
+    Code borrowed from Nobu Nakada (ruby-talk:155159).
+    Code borrowed from Ara Howard (ruby-talk:173428).
+    Code borrowed from Andrew Johnson (http://tinyurl.com/8ymx8)
+    Ideas taken from Brian Buckley and Sean O'Halpin.
+    Tiny URL provided for Andrew Johnson because I could not find the ruby-talk
+    reference. The gateway may have been broken at the time.
+
+

data/Rakefile

@@ -0,0 +1,8 @@
+require "bundler/gem_tasks"
+require 'rake/testtask'
+
+Rake::TestTask.new do |t|
+  t.libs << 'test'
+  t.verbose = true
+  t.warning = true
+end

data/examples/example_fibonacci.rb

@@ -0,0 +1,41 @@
+###################################################################
+# fibonacci.rb
+#
+# Demonstrates, via Benchmark, the difference between a memoized
+# version of the fibonnaci method versus a non-memoized version.
+#
+# You can run this via the 'example_fib' rake task.
+###################################################################
+require 'benchmark'
+require 'fileutils'
+require 'persistent_memoize'
+include PersistentMemoize
+
+# Our fibonacci function
+def fib(n)
+   return n if n < 2
+   fib(n-1) + fib(n-2)
+end
+
+file = File.join((ENV['HOME'] || ENV['USERPROFILE']), 'fib.cache')
+
+max_iter = ARGV[0].to_i
+max_fib  = ARGV[1].to_i
+
+max_iter = 100 if max_iter == 0
+max_fib  = 25 if max_fib == 0
+
+print "\nBenchmarking against version: " + PERSISTENT_MEMOIZE_VERSION + "\n\n"
+
+Benchmark.bm(35) do |x|
+   x.report("Not memoized:"){
+      max_iter.times{ fib(max_fib) }
+   }
+
+   x.report("Memoized to file:"){
+      memoize(:fib, file)
+      max_iter.times{ fib(max_fib) }
+   }
+end
+
+FileUtils.remove_dir(file) if File.exists?(file)

data/lib/persistent_memoize.rb

@@ -0,0 +1,26 @@
+require 'digest/md5'
+require 'fileutils'
+require "persistent_memoize/version"
+
+module PersistentMemoize
+  # Memoize the method 'name', with results stored in files under 'path'
+  def memoize(name, path)
+    unless File.exists?(path)
+      FileUtils.mkdir_p(path)
+    end
+
+    (class<<self; self; end).send(:define_method, name) do |*args|
+      key = Digest::MD5.hexdigest(Marshal.dump(args))
+      cacheFile = File.join(path, key)
+      if File.exists?(cacheFile)
+        results = File.open(cacheFile, 'rb'){ |f| Marshal.load(f) }
+      else
+        results = super(*args)
+        File.open(cacheFile, 'wb'){ |f| Marshal.dump(results, f) }
+      end
+      results
+    end
+
+  end
+
+end

data/lib/persistent_memoize/version.rb

@@ -0,0 +1,3 @@
+module PersistentMemoize
+  VERSION = "0.0.1"
+end

data/persistent_memoize.gemspec

@@ -0,0 +1,35 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'persistent_memoize/version'
+
+# Build the gem with the 'rake gem:build' command.
+
+Gem::Specification.new do |spec|
+  spec.name      = 'persistent_memoize'
+  spec.version   = PersistentMemoize::VERSION
+  spec.version   = '0.0.1'
+  spec.authors   = ['Neil Kandalgaonkar']
+  spec.license   = 'MIT'
+  spec.email     = ['neilk@brevity.org']
+  spec.homepage  = 'https://github.com/neilk/persistent_memoize'
+  spec.platform  = Gem::Platform::RUBY
+  spec.summary   = 'Speeds up methods with caches, which persistent on disk'
+  spec.has_rdoc  = true
+  spec.files     = `git ls-files`.split($/)
+  spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
+  spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+  
+  spec.extra_rdoc_files  = ['README.md']
+   
+  spec.add_development_dependency('test-unit', '>= 2.0.2')
+  spec.add_development_dependency "bundler", "~> 1.3"
+  spec.add_development_dependency "rake"
+
+  spec.description = <<-EOF
+    This allows you to cache method calls for faster execution, at the cost of 
+    storage space and IO operations. The caches are kept on disk,
+    persistent between executions of the program. 
+  EOF
+end

data/test/test_memoize.rb

@@ -0,0 +1,54 @@
+###############################################
+# test_memoize.rb
+#
+# Test suite for the memoize library.
+###############################################
+require 'rubygems'
+gem 'test-unit'
+
+require 'test/unit'
+require 'fileutils'
+require 'persistent_memoize'
+
+class TC_PersistentMemoize < Test::Unit::TestCase
+   include PersistentMemoize
+
+   def setup
+      @path   = File.join((ENV['HOME'] || ENV['USERPROFILE']), 'test.cache')
+   end
+
+   def add(x, y)
+     return x + y
+   end
+
+   def fib(n)
+      return n if n < 2
+      fib(n-1) + fib(n-2)
+   end
+
+   def factorial(n)
+     f = 1
+     n.downto(2) { |x| f *= x }
+     f
+   end
+
+   def test_memoize
+      assert_nothing_raised{ fib(5) }
+      assert_nothing_raised{ memoize(:fib, @path) }
+      assert_nothing_raised{ fib(50) }
+      assert_equal(55, fib(10))
+   end
+
+   def test_memoize_directory_properties
+      assert(!File.exists?(@path), "path did not exist")
+      assert_nothing_raised{ memoize(:add, @path) }
+      assert(File.exists?(@path), "path does exist")
+      assert(Dir.entries(@path).select{ |f| f != '.' and f != '..' }.length == 0, "path is empty of files")
+      assert_nothing_raised{ add(10, 20) }
+      assert(Dir.entries(@path).select{ |f| f != '.' and f != '..' }.length == 1, "path has exactly one file")
+   end
+   
+   def teardown
+      FileUtils.remove_dir(@path) if File.exists?(@path)
+   end
+end

metadata

@@ -0,0 +1,108 @@
+--- !ruby/object:Gem::Specification
+name: persistent_memoize
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+  prerelease: 
+platform: ruby
+authors:
+- Neil Kandalgaonkar
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-04-01 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: test-unit
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.2
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 2.0.2
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: ! "    This allows you to cache method calls for faster execution, at
+  the cost of \n    storage space and IO operations. The caches are kept on disk,\n
+  \   persistent between executions of the program. \n"
+email:
+- neilk@brevity.org
+executables: []
+extensions: []
+extra_rdoc_files:
+- README.md
+files:
+- .gitignore
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- examples/example_fibonacci.rb
+- lib/persistent_memoize.rb
+- lib/persistent_memoize/version.rb
+- persistent_memoize.gemspec
+- test/test_memoize.rb
+homepage: https://github.com/neilk/persistent_memoize
+licenses:
+- MIT
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.23
+signing_key: 
+specification_version: 3
+summary: Speeds up methods with caches, which persistent on disk
+test_files:
+- test/test_memoize.rb