RubyGems - frequent-algorithm - Versions diffs - 0.0.3 → 0.0.4 - Mend

frequent-algorithm 0.0.3 → 0.0.4

Files changed (9) hide show

checksums.yaml +4 -4
data/.yardopts +7 -7
data/CHANGELOG +23 -9
data/LICENSE +22 -22
data/README.md +159 -149
data/lib/frequent-algorithm.rb +28 -28
data/lib/frequent/algorithm.rb +201 -182
data/lib/frequent/version.rb +38 -38
metadata +17 -15

checksums.yaml CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c581bac868de994e83f9fee63f34d8987ab2eda9
-  data.tar.gz: a9c068fa92383857a5d0526c9aa55ce4746651d5
+  metadata.gz: 0ab3af8c9299159321aa18f30c5022ff7e8b3042
+  data.tar.gz: 01250f9d283874ae66fb0660415ac74476724b43
 SHA512:
-  metadata.gz: 42571a866be7b6ee00748a10afb5ee6d48d8025198a68877a18401bedc0e372da09b1af3a7ef69ac94c78ed2fb798e52899b30eaddea2c423c0982413a2f9f3e
-  data.tar.gz: 0fd80b41cbc31fec785ea52596ab0e9094e54172d8a95e419f80339cdbcd7e9b275dd4a7a0826380ff43a3f81e8f6519d87fe02773ef4319c4a264211cee6f95
+  metadata.gz: c38e275789110b1abac68a91f60f1a952aac05ddba011d22e213c38e33132b6261d92fc05b994b3310422c4081e6b6e1018c1854cf4f9562c3aea9e40d680a12
+  data.tar.gz: 8a84db9ba16c809a7700973cd62738f8cbd4b19461bff56f6c6f9192000e2e0cc4affc136576682cdcf29fb18a37b0653197404773426ab40629e0a1c493ced3

data/.yardopts CHANGED Viewed

@@ -1,7 +1,7 @@
---no-private
---readme README.md
---markup markdown
---markup-provider rdiscount
--
-LICENSE
-CHANGELOG
+--no-private
+--readme README.md
+--markup markdown
+--markup-provider rdiscount
+-
+LICENSE
+CHANGELOG

data/CHANGELOG CHANGED Viewed

@@ -1,9 +1,23 @@
-## CHANGELOG
-- __2015/03/19 0.0.2 release.
-    - First-stage implementation.
-    - API documentation added.
-    - Fleshing out unit tests.
-- __2015/03/11__: 0.0.1 release.
-    - Initial release.
+## CHANGELOG
+- __2015/05/06__ 0.0.4 release.
+    - Issue 17: Enhance Algorithm: new method to return top-k statistics
+    - Issue 42: Enhance algorithm: Accept one element as parameter in the process method
+    - Issue 45: FIX markdown in CHANGELOG
+    - Issue 53: Get this project to run/test against Rubinius
+    - Initial work to make the code thread-safe
+- __2015/03/23__ 0.0.3 release.
+    - Further refinements to process.
+    - Enhanced strategy for calculating kth largest element in list.
+    - Resolved Issue 24: Refactor - Consistent internal data structure.
+    - Wontfix Issue 28: Add new test cases using String as input.
+    - On-going refinements for unit tests.
+- __2015/03/19__ 0.0.2 release.
+    - First-stage implementation.
+    - API documentation added.
+    - Fleshing out unit tests.
+- __2015/03/11__: 0.0.1 release.
+    - Initial release.

data/LICENSE CHANGED Viewed

@@ -1,22 +1,22 @@
-The MIT License (MIT)
-Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+The MIT License (MIT)
+Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md CHANGED Viewed

@@ -1,149 +1,159 @@
-# frequent-algorithm [![Gem Version](https://badge.fury.io/rb/frequent-algorithm.svg)](http://badge.fury.io/rb/frequent-algorithm) [![Build Status](https://travis-ci.org/buruzaemon/frequent-algorithm.svg)](https://travis-ci.org/buruzaemon/frequent-algorithm)
-Web site usage, social network behavior and Internet traffic are examples
-of systems that appear to follow the [Power law](http://en.wikipedia.org/wiki/Power_law),
-where most of the events are due to the actions of a very small few.
-Knowing at any given point in time which items are trending is valuable
-in understanding the system.
-`frequent-algorithm` is a Ruby implementation of the FREQUENT algorithm
-for identifying frequent items in a data stream in sliding windows.
-Please refer to [Identifying Frequent Items in Sliding Windows over On-Line
-Packet Streams](http://erikdemaine.org/papers/SlidingWindow_IMC2003/), by
-Golab, DeHaan, Demaine, L&#243;pez-Ortiz and Munro (2003).
-## Introduction
-### Challenges
-Challenges for Real-time processing of data streams for _frequent item queries_
-include:
-* data may be of unknown and possibly unbound length
-* data may be arriving a very fast rate
-* it might not be possible to go back and re-read the data
-* too large a window of observation may include stale data
-Therefore, a solution should have the following characteristics:
-* uses limited memory
-* can process events in the stream in &#927;(1) constant time
-* requires only a single-pass over the data
-### The algorithm
-> LOOP<br/>
-> 1. For each element e in the next b elements:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;If a local counter exists for the type of element e:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Increment the local counter.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new local counter for this element type<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to 1.<br/>
-> 2. Add a summary S containing identities and counts of the k most frequent items to the back of queue Q.<br/>
-> 3. Delete all local counters<br/>
-> 4. For each type named in S:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;If a global counter exists for this type:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add to it the count recorded in S.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new global counter for this element type<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to the count recorded in S.<br/>
-> 5. Add the count of the kth largest type in S to δ.<br/>
-> 6. If sizeOf(Q) > N/b:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(a) Remove the summary S' from the front of Q and subtract the count of the kth largest type in S' from δ.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(b) For all element types named in S':<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Subtract from their global counters the counts<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;recorded in S'<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If a counter is decremented to zero:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete it.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(c) Output the identity and value of each global counter > δ.
->
-> &mdash; <cite>Golab, DeHaan, Demaine, López-Ortiz and Munro. Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003</cite>
-## Usage
-    require 'frequent-algorithm'
-    # data is pi to 1000 digits
-    pi = File.read('test/frequent/test_data_pi').strip
-    data = pi.scan(/./).each_slice(b)
-    N = 100  # size of main window
-    b =  20  # size of basic window
-    k =   3  # we are interested in top-3 numerals in pi
-    alg = Frequent::Algorithm.new(N, b, k)
-    # read in and process the 1st basic window
-    alg.process(data.next)
-    # and the top-3 numerals are?
-    top3 = alg.statistics.report
-    puts top3
-    # lather, rinse and repeat
-    alg.process(data.next)
-## Development
-The development of this gem requires the following:
-* [Ruby 1.9.3 or greater](https://www.ruby-lang.org/en/)
-* [rubygems](https://rubygems.org/pages/download)
-* [`bundler`](https://github.com/bundler/bundler)
-* [`rake`](https://github.com/ruby/rake)
-* [`minitest`](https://rubygems.org/gems/minitest) (unit testing)
-* [`yard`](https://rubygems.org/gems/yard) (documentation)
-* [`rdiscount`](https://rubygems.org/gems/rdiscount) (Markdown)
-Building, testing and release of this rubygem uses the following
-`rake` commands:
-    rake clean    # Remove any temporary products
-    rake clobber  # Remove any generated file
-    rake test     # Execute unit tests
-    rake build    # Build frequent-algorithm-n.n.n.gem into the pkg directory
-    rake install  # Build and install frequent-algorithm-n.n.n.gem into system gems
-    rake release  # Create tag vn.n.n and build and push
-                  # frequent-algorithm-n.n.n.gem to Rubygems
-### Documentation
-`frequent-algorithm` uses [`yard`](https://rubygems.org/gems/yard) and
-[`rdiscount`](https://rubygems.org/gems/rdiscount) for Markdown documentation.
-Check out [Getting Started with
-Yard](http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md).
-### Unit Testing
-`frequent-algorithm` uses
-[`MiniTest::Unit`](https://github.com/seattlerb/minitest) for
-unit testing.
-### Releasing
-Please refer to Publishing To Rubygems.org in the
-[Rubygems Guide](http://guides.rubygems.org/make-your-own-gem/).
-### Contributing
-1. Fork it
-2. Begin work on `dev-branch` (`git fetch && git checkout dev-branch`)
-3. Create your feature branch (`git branch my-new-feature && git checkout
-   my-new-feature`)
-4. Commit your changes (`git commit -am 'Add some feature'`)
-5. Push to the branch (`git push origin my-new-feature:dev-branch`)
-6. Create new Pull Request
-You may wish to read the [Git book online](http://git-scm.com/book/en/v2).
-## License
-frequent-algorithm is provided under the terms of the MIT license.
-Copyright &copy; 2015, Willie Tong &amp; Brooke M. Fujita. All rights reserved.
+# frequent-algorithm
+Web site usage, social network behavior and Internet traffic are examples
+of systems that appear to follow the [Power law](http://en.wikipedia.org/wiki/Power_law),
+where most of the events are due to the actions of a very small few.
+Knowing at any given point in time which items are trending is valuable
+in understanding the system.
+`frequent-algorithm` is a Ruby implementation of the FREQUENT algorithm
+for identifying frequent items in a data stream in sliding windows.
+Please refer to [Identifying Frequent Items in Sliding Windows over On-Line
+Packet Streams](http://erikdemaine.org/papers/SlidingWindow_IMC2003/), by
+Golab, DeHaan, Demaine, L&#243;pez-Ortiz and Munro (2003).
+[![License](https://img.shields.io/badge/license-MIT-blue.svg)]() [![Build Status](https://travis-ci.org/buruzaemon/frequent-algorithm.svg?branch=master)](https://travis-ci.org/buruzaemon/frequent-algorithm) [![Gem Version](https://badge.fury.io/rb/frequent-algorithm.svg)](https://rubygems.org/gems/frequent-algorithm)
+## Introduction
+### Challenges
+Challenges for Real-time processing of data streams for _frequent item queries_
+include:
+* data may be of unknown and possibly unbound length
+* data may be arriving a very fast rate
+* it might not be possible to go back and re-read the data
+* too large a window of observation may include stale data
+Therefore, a solution should have the following characteristics:
+* uses limited memory
+* can process events in the stream in &#927;(1) constant time
+* requires only a single-pass over the data
+### The algorithm
+> LOOP<br/>
+> 1. For each element e in the next b elements:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;If a local counter exists for the type of element e:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Increment the local counter.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new local counter for this element type<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to 1.<br/>
+> 2. Add a summary S containing identities and counts of the k most frequent items to the back of queue Q.<br/>
+> 3. Delete all local counters<br/>
+> 4. For each type named in S:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;If a global counter exists for this type:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add to it the count recorded in S.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new global counter for this element type<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to the count recorded in S.<br/>
+> 5. Add the count of the kth largest type in S to δ.<br/>
+> 6. If sizeOf(Q) > N/b:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(a) Remove the summary S' from the front of Q and subtract the count of the kth largest type in S' from δ.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(b) For all element types named in S':<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Subtract from their global counters the counts<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;recorded in S'<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If a counter is decremented to zero:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete it.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(c) Output the identity and value of each global counter > δ.
+>
+> &mdash; <cite>Golab, DeHaan, Demaine, López-Ortiz and Munro. Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003</cite>
+## Usage
+    require 'frequent-algorithm'
+    # data is pi to 1000 digits
+    pi = File.read('test/frequent/test_data_pi').strip
+    data = pi.scan(/./).each_slice(b)
+    N = 100  # size of main window
+    b =  20  # size of basic window
+    k =   3  # we are interested in top-3 numerals in pi
+    alg = Frequent::Algorithm.new(N, b, k)
+    # read in and process the 1st basic window
+    alg.process(data.next)
+    # and the top-3 numerals are?
+    top3 = alg.statistics.report
+    puts top3
+    # lather, rinse and repeat
+    alg.process(data.next)
+## Development
+The development of this gem requires the following:
+* [Ruby 1.9.3 or greater](https://www.ruby-lang.org/en/)
+* [rubygems](https://rubygems.org/pages/download)
+* [`bundler`](https://github.com/bundler/bundler)
+* [`rake`](https://github.com/ruby/rake)
+* [`minitest`](https://rubygems.org/gems/minitest) (unit testing)
+* [`yard`](https://rubygems.org/gems/yard) (documentation)
+* [`rdiscount`](https://rubygems.org/gems/rdiscount) (Markdown)
+Building, testing and release of this rubygem uses the following
+`rake` commands:
+    rake clean    # Remove any temporary products
+    rake clobber  # Remove any generated file
+    rake test     # Execute unit tests
+    rake build    # Build frequent-algorithm-n.n.n.gem into the pkg directory
+    rake install  # Build and install frequent-algorithm-n.n.n.gem into system gems
+    rake release  # Create tag vn.n.n and build and push
+                  # frequent-algorithm-n.n.n.gem to Rubygems
+### Documentation
+`frequent-algorithm` uses [`yard`](https://rubygems.org/gems/yard) and
+[`rdiscount`](https://rubygems.org/gems/rdiscount) for Markdown documentation.
+Check out [Getting Started with
+Yard](http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md).
+### Unit Testing
+`frequent-algorithm` uses
+[`MiniTest::Unit`](https://github.com/seattlerb/minitest) for
+unit testing.
+### Releasing
+Please refer to Publishing To Rubygems.org in the
+[Rubygems Guide](http://guides.rubygems.org/make-your-own-gem/).
+### Contributing
+1. Fork it
+2. Begin work on `dev-branch` (`git fetch && git checkout dev-branch`)
+3. Create your feature branch (`git branch my-new-feature && git checkout
+   my-new-feature`)
+4. Commit your changes (`git commit -am 'Add some feature'`)
+5. Push to the branch (`git push origin my-new-feature:dev-branch`)
+6. Create new Pull Request
+You may wish to read the [Git book online](http://git-scm.com/book/en/v2).
+## Changelog
+Please see the {file:CHANGELOG} for this gem's release history.
+## License
+frequent-algorithm is provided under the terms of the MIT license.
+Copyright &copy; 2015, Willie Tong &amp; Brooke M. Fujita. All rights reserved. Please see the {file:LICENSE} file for further details.

data/lib/frequent-algorithm.rb CHANGED Viewed

@@ -1,28 +1,28 @@
-# coding: utf-8
-require 'frequent/algorithm'
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+require 'frequent/algorithm'
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

data/lib/frequent/algorithm.rb CHANGED Viewed

@@ -1,182 +1,201 @@
-# coding: utf-8
-require 'frequent/version'
-module Frequent
-  ERR_BADLIST = "List cannot be nil or empty".freeze
-  ERR_BADK = "k must be between 1 and %s".freeze
-  # `Frequent::Algorithm` is the Ruby implementation of the
-  # Demaine et al. FREQUENT algorithm for calculating
-  # top-k items in a stream.
-  #
-  # The aims of this algorithm are:
-  # * use limited memory
-  # * require constant processing time per item
-  # * require a single-pass only
-  #
-  class Algorithm
-    # @return [Integer] the number of items in the main window
-    attr_reader :n
-    # @return [Integer] the number of items in a basic window
-    attr_reader :b
-    # @return [Integer] the number of top item categories to track
-    attr_reader :k
-    # @return [Array<Hash<Object,Integer>>] global queue for basic window summaries
-    attr_reader :queue
-    # @return [Hash<Object,Integer>] global mapping of items and counts
-    attr_reader :statistics
-    # @return [Integer] minimum threshold for membership in top-k items
-    attr_reader :delta
-    # Initializes this top-k frequency-calculating instance.
-    #
-    # @param [Integer] n number of items in the main window
-    # @param [Integer] b number of items in a basic window
-    # @param [Integer] k number of top item categories to track
-    # @raise [ArgumentError] if n is not greater than 0
-    # @raise [ArgumentError] if b is not greater than 0
-    # @raise [ArgumentError] if k is not greater than 0
-    # @raise [ArgumentError] if n/b is not greater than 1
-    def initialize(n, b, k=1)
-      if n <= 0
-        raise ArgumentError.new('n must be greater than 0')
-      end
-      if b <= 0
-        raise ArgumentError.new('b must be greater than 0')
-      end
-      if k <= 0
-        raise ArgumentError.new('k must be greater than 0')
-      end
-      if n/b < 1
-        raise ArgumentError.new('n/b must be greater than 1')
-      end
-      @n = n
-      @b = b
-      @k = k
-      @queue = []
-      @statistics = {}
-      @delta = 0
-    end
-    # Processes a single basic window of b items, by first adding
-    # a summary of this basic window in the internal global queue;
-    # and then updating the global statistics accordingly.
-    #
-    # @param [Array] an array of objects representing a basic window
-    def process(elements)
-      # Do we need this?
-      return if elements.length != @b
-      # Step 1
-      summary = {}
-      elements.each do |e|
-        if summary.key? e
-          summary[e] += 1
-        else
-          summary[e] = 1
-        end
-      end
-      # Step 2
-      @queue << summary
-      # Step 3
-      # Done, implicitly
-      # Step 4
-      summary.each do |k,v|
-        if @statistics.key? k
-          @statistics[k] += v
-        else
-          @statistics[k] = v
-        end
-      end
-      # Step 5
-      @delta += kth_largest(summary.values, @k)
-      # Step 6 - sizeOf(Q) > N/b
-      if @queue.length > @n/@b
-        # a
-        summary_p = @queue.shift
-        @delta -= kth_largest(summary_p.values, @k)
-        # b
-        summary_p.each { |k,v| @statistics[k] -= v }
-        @statistics.delete_if { |k,v| v <= 0 }
-        #c
-        @statistics.select { |k,v| v > @delta }
-      else
-        {}
-      end
-    end
-    # Returns the version for this gem.
-    #
-    # @return [String] the version for this gem.
-    def version
-      Frequent::VERSION
-    end
-    private
-    # Given a list of numbers and a number k which should be
-    # between 1 and the length of the given list, return the
-    # element x in the list that is larger than exactly k-1
-    # other elements in the list.
-    #
-    # @param [Array] list of integers
-    # @return [Integer] the kth largest element in list
-    def kth_largest(list, k)
-      raise ArgumentError.new(ERR_BADLIST) if list.nil? or list.empty?
-      raise ArgumentError.new(ERR_BADK) if k < 1
-      ulist = list.uniq
-      k = ulist.size if k > ulist.size
-      def quickselect(aset, k)
-        p = rand(aset.size)
-        lower = aset.select { |e| e < aset[p] }
-        upper = aset.select { |e| e > aset[p] }
-        if k <= lower.size
-          quickselect(lower, k)
-        elsif k > aset.size - upper.size
-          quickselect(upper, k - (aset.size - upper.size))
-        else
-          aset[p]
-        end
-      end
-      quickselect(ulist, ulist.size+1-k)
-    end
-  end
-end
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+require 'frequent/version'
+require 'thread'
+module Frequent
+  ERR_BADLIST = "List cannot be nil or empty".freeze
+  ERR_BADK = "k must be between 1 and %s".freeze
+  # `Frequent::Algorithm` is the Ruby implementation of the
+  # Demaine et al. FREQUENT algorithm for calculating
+  # top-k items in a stream.
+  #
+  # The aims of this algorithm are:
+  # * use limited memory
+  # * require constant processing time per item
+  # * require a single-pass only
+  #
+  class Algorithm
+    # @return [Integer] the number of items in the main window
+    attr_reader :n
+    # @return [Integer] the number of items in a basic window
+    attr_reader :b
+    # @return [Integer] the number of top item categories to track
+    attr_reader :k
+    # @return [Array<Hash<Object,Integer>>] global queue for basic window summaries
+    attr_reader :queue
+    # @return [Hash<Object,Integer>] global mapping of items and counts
+    attr_reader :statistics
+    # @return [Integer] minimum threshold for membership in top-k items
+    attr_reader :delta
+    # @return [Hash<Object,Integer>] latest top k elements and their counts
+    attr_reader :topk
+    # @return [Array[Object]] the window of elements of size b
+    attr_reader :window
+    # Initializes this top-k frequency-calculating instance.
+    #
+    # @param [Integer] n number of items in the main window
+    # @param [Integer] b number of items in a basic window
+    # @param [Integer] k number of top item categories to track
+    # @raise [ArgumentError] if n is not greater than 0
+    # @raise [ArgumentError] if b is not greater than 0
+    # @raise [ArgumentError] if k is not greater than 0
+    # @raise [ArgumentError] if n/b is not greater than 1
+    def initialize(n, b, k=1)
+      @lock = Mutex.new
+      if n <= 0
+        raise ArgumentError.new('n must be greater than 0')
+      end
+      if b <= 0
+        raise ArgumentError.new('b must be greater than 0')
+      end
+      if k <= 0
+        raise ArgumentError.new('k must be greater than 0')
+      end
+      if n/b < 1
+        raise ArgumentError.new('n/b must be greater than 1')
+      end
+      @n = n
+      @b = b
+      @k = k
+      @queue = []
+      @statistics = {}
+      @delta = 0
+      @topk = {}
+      @window = []
+    end
+    # Processes a single basic window of b items, by first adding
+    # a summary of this basic window in the internal global queue;
+    # and then updating the global statistics accordingly.
+    #
+    # @param [Object] an object from a data stream
+    def process(element)
+      @lock.synchronize do
+        @window << element
+        if @window.length == @b
+          # Step 1
+          summary = {}
+          @window.each do |e|
+            if summary.key? e
+              summary[e] += 1
+            else
+              summary[e] = 1
+            end
+          end
+          @window.clear   #current window cleared
+          # Step 2
+          @queue << summary
+          # Step 3
+          # Done, implicitly
+          # Step 4
+          summary.each do |k,v|
+            if @statistics.key? k
+              @statistics[k] += v
+            else
+              @statistics[k] = v
+            end
+          end
+          # Step 5
+          @delta += kth_largest(summary.values, @k)
+          # Step 6 - sizeOf(Q) > N/b
+          if @queue.length > @n/@b
+            # a
+            summary_p = @queue.shift
+            @delta -= kth_largest(summary_p.values, @k)
+            # b
+            summary_p.each { |k,v| @statistics[k] -= v }
+            @statistics.delete_if { |k,v| v <= 0 }
+            #c
+            @topk = @statistics.select { |k,v| v > @delta }
+          end
+        end
+      end
+    end
+    # Return the latest Tok K elements
+    #
+    # @return [Hash<Object,Integer>] a hash which contains the current top K elements and their counts
+    def report
+      @topk
+    end
+    # Returns the version for this gem.
+    #
+    # @return [String] the version for this gem.
+    def version
+      Frequent::VERSION
+    end
+    private
+    # Given a list of numbers and a number k which should be
+    # between 1 and the length of the given list, return the
+    # element x in the list that is larger than exactly k-1
+    # other elements in the list.
+    #
+    # @param [Array] list of integers
+    # @return [Integer] the kth largest element in list
+    def kth_largest(list, k)
+      raise ArgumentError.new(ERR_BADLIST) if list.nil? or list.empty?
+      raise ArgumentError.new(ERR_BADK) if k < 1
+      ulist = list.uniq
+      k = ulist.size if k > ulist.size
+      def quickselect(aset, k)
+        p = rand(aset.size)
+        lower = aset.select { |e| e < aset[p] }
+        upper = aset.select { |e| e > aset[p] }
+        if k <= lower.size
+          quickselect(lower, k)
+        elsif k > aset.size - upper.size
+          quickselect(upper, k - (aset.size - upper.size))
+        else
+          aset[p]
+        end
+      end
+      quickselect(ulist, ulist.size+1-k)
+    end
+  end
+end
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

data/lib/frequent/version.rb CHANGED Viewed

@@ -1,38 +1,38 @@
-# coding: utf-8
-# `Frequent` is the namespace for objects implementing
-# the Demaine et al. FREQUENT algorithm for finding
-# the most frequently-appearing items (top-k) in a
-# data stream in sliding windows.
-#
-# `Frequent::Algorithm` is the implementation class.
-module Frequent
-  # Version string for this Rubygem.
-  VERSION = '0.0.3'
-end
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+# `Frequent` is the namespace for objects implementing
+# the Demaine et al. FREQUENT algorithm for finding
+# the most frequently-appearing items (top-k) in a
+# data stream in sliding windows.
+#
+# `Frequent::Algorithm` is the implementation class.
+module Frequent
+  # Version string for this Rubygem.
+  VERSION = '0.0.4'
+end
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

metadata CHANGED Viewed

@@ -1,44 +1,44 @@
 --- !ruby/object:Gem::Specification
 name: frequent-algorithm
 version: !ruby/object:Gem::Version
-  version: 0.0.3
+  version: 0.0.4
 platform: ruby
 authors:
 - Willie Tong
 - Brooke M. Fujita
-autorequire:
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2015-03-23 00:00:00.000000000 Z
+date: 2015-05-06 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
-  version_requirements: !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  requirement: !ruby/object:Gem::Requirement
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  prerelease: false
-  type: :development
 - !ruby/object:Gem::Dependency
   name: minitest
-  version_requirements: !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  requirement: !ruby/object:Gem::Requirement
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  prerelease: false
-  type: :development
 description: |
   frequent-algorithm is a Ruby implementation of the Demaine et al FREQUENT algorithm for identifying frequent items in a data stream in sliding windows (c.f Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003).
 email:
@@ -59,7 +59,7 @@ homepage: https://github.com/buruzaemon/frequent-algorithm
 licenses:
 - MIT
 metadata: {}
-post_install_message:
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -74,9 +74,11 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
+rubyforge_project:
 rubygems_version: 2.4.5
-signing_key:
+signing_key:
 specification_version: 4
-summary: Identifies frequent items in a data stream in sliding windows using the Demaine et al FREQUENT algorithm.
+summary: Identifies frequent items in a data stream in sliding windows using the Demaine
+  et al FREQUENT algorithm.
 test_files: []
+has_rdoc: