RubyGems - frequent-algorithm - Versions diffs - 0.0.3 → 0.0.4 - Mend

frequent-algorithm 0.0.3 → 0.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (9) hide show

checksums.yaml +4 -4
data/.yardopts +7 -7
data/CHANGELOG +23 -9
data/LICENSE +22 -22
data/README.md +159 -149
data/lib/frequent-algorithm.rb +28 -28
data/lib/frequent/algorithm.rb +201 -182
data/lib/frequent/version.rb +38 -38
metadata +17 -15

checksums.yaml CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c581bac868de994e83f9fee63f34d8987ab2eda9
-  data.tar.gz: a9c068fa92383857a5d0526c9aa55ce4746651d5
+  metadata.gz: 0ab3af8c9299159321aa18f30c5022ff7e8b3042
+  data.tar.gz: 01250f9d283874ae66fb0660415ac74476724b43
 SHA512:
-  metadata.gz: 42571a866be7b6ee00748a10afb5ee6d48d8025198a68877a18401bedc0e372da09b1af3a7ef69ac94c78ed2fb798e52899b30eaddea2c423c0982413a2f9f3e
-  data.tar.gz: 0fd80b41cbc31fec785ea52596ab0e9094e54172d8a95e419f80339cdbcd7e9b275dd4a7a0826380ff43a3f81e8f6519d87fe02773ef4319c4a264211cee6f95
+  metadata.gz: c38e275789110b1abac68a91f60f1a952aac05ddba011d22e213c38e33132b6261d92fc05b994b3310422c4081e6b6e1018c1854cf4f9562c3aea9e40d680a12
+  data.tar.gz: 8a84db9ba16c809a7700973cd62738f8cbd4b19461bff56f6c6f9192000e2e0cc4affc136576682cdcf29fb18a37b0653197404773426ab40629e0a1c493ced3

data/.yardopts CHANGED Viewed

@@ -1,7 +1,7 @@
---no-private
---readme README.md
---markup markdown
---markup-provider rdiscount
--
-LICENSE
-CHANGELOG
+--no-private
+--readme README.md
+--markup markdown
+--markup-provider rdiscount
+-
+LICENSE
+CHANGELOG

data/CHANGELOG CHANGED Viewed

@@ -1,9 +1,23 @@
-## CHANGELOG
-- __2015/03/19 0.0.2 release.
-    - First-stage implementation.
-    - API documentation added.
-    - Fleshing out unit tests.
-- __2015/03/11__: 0.0.1 release.
-    - Initial release.
+## CHANGELOG
+- __2015/05/06__ 0.0.4 release.
+    - Issue 17: Enhance Algorithm: new method to return top-k statistics
+    - Issue 42: Enhance algorithm: Accept one element as parameter in the process method
+    - Issue 45: FIX markdown in CHANGELOG
+    - Issue 53: Get this project to run/test against Rubinius
+    - Initial work to make the code thread-safe
+- __2015/03/23__ 0.0.3 release.
+    - Further refinements to process.
+    - Enhanced strategy for calculating kth largest element in list.
+    - Resolved Issue 24: Refactor - Consistent internal data structure.
+    - Wontfix Issue 28: Add new test cases using String as input.
+    - On-going refinements for unit tests.
+- __2015/03/19__ 0.0.2 release.
+    - First-stage implementation.
+    - API documentation added.
+    - Fleshing out unit tests.
+- __2015/03/11__: 0.0.1 release.
+    - Initial release.

data/LICENSE CHANGED Viewed

@@ -1,22 +1,22 @@
-The MIT License (MIT)
-Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+The MIT License (MIT)
+Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md CHANGED Viewed

@@ -1,149 +1,159 @@
-# frequent-algorithm [![Gem Version](https://badge.fury.io/rb/frequent-algorithm.svg)](http://badge.fury.io/rb/frequent-algorithm) [![Build Status](https://travis-ci.org/buruzaemon/frequent-algorithm.svg)](https://travis-ci.org/buruzaemon/frequent-algorithm)
-Web site usage, social network behavior and Internet traffic are examples
-of systems that appear to follow the [Power law](http://en.wikipedia.org/wiki/Power_law),
-where most of the events are due to the actions of a very small few.
-Knowing at any given point in time which items are trending is valuable
-in understanding the system.
-`frequent-algorithm` is a Ruby implementation of the FREQUENT algorithm
-for identifying frequent items in a data stream in sliding windows.
-Please refer to [Identifying Frequent Items in Sliding Windows over On-Line
-Packet Streams](http://erikdemaine.org/papers/SlidingWindow_IMC2003/), by
-Golab, DeHaan, Demaine, L&#243;pez-Ortiz and Munro (2003).
-## Introduction
-### Challenges
-Challenges for Real-time processing of data streams for _frequent item queries_
-include:
-* data may be of unknown and possibly unbound length
-* data may be arriving a very fast rate
-* it might not be possible to go back and re-read the data
-* too large a window of observation may include stale data
-Therefore, a solution should have the following characteristics:
-* uses limited memory
-* can process events in the stream in &#927;(1) constant time
-* requires only a single-pass over the data
-### The algorithm
-> LOOP<br/>
-> 1. For each element e in the next b elements:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;If a local counter exists for the type of element e:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Increment the local counter.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new local counter for this element type<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to 1.<br/>
-> 2. Add a summary S containing identities and counts of the k most frequent items to the back of queue Q.<br/>
-> 3. Delete all local counters<br/>
-> 4. For each type named in S:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;If a global counter exists for this type:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add to it the count recorded in S.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new global counter for this element type<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to the count recorded in S.<br/>
-> 5. Add the count of the kth largest type in S to δ.<br/>
-> 6. If sizeOf(Q) > N/b:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(a) Remove the summary S' from the front of Q and subtract the count of the kth largest type in S' from δ.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(b) For all element types named in S':<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Subtract from their global counters the counts<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;recorded in S'<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If a counter is decremented to zero:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete it.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(c) Output the identity and value of each global counter > δ.
->
-> &mdash; <cite>Golab, DeHaan, Demaine, López-Ortiz and Munro. Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003</cite>
-## Usage
-    require 'frequent-algorithm'
-    # data is pi to 1000 digits
-    pi = File.read('test/frequent/test_data_pi').strip
-    data = pi.scan(/./).each_slice(b)
-    N = 100  # size of main window
-    b =  20  # size of basic window
-    k =   3  # we are interested in top-3 numerals in pi
-    alg = Frequent::Algorithm.new(N, b, k)
-    # read in and process the 1st basic window
-    alg.process(data.next)
-    # and the top-3 numerals are?
-    top3 = alg.statistics.report
-    puts top3
-    # lather, rinse and repeat
-    alg.process(data.next)
-## Development
-The development of this gem requires the following:
-* [Ruby 1.9.3 or greater](https://www.ruby-lang.org/en/)
-* [rubygems](https://rubygems.org/pages/download)
-* [`bundler`](https://github.com/bundler/bundler)
-* [`rake`](https://github.com/ruby/rake)
-* [`minitest`](https://rubygems.org/gems/minitest) (unit testing)
-* [`yard`](https://rubygems.org/gems/yard) (documentation)
-* [`rdiscount`](https://rubygems.org/gems/rdiscount) (Markdown)
-Building, testing and release of this rubygem uses the following
-`rake` commands:
-    rake clean    # Remove any temporary products
-    rake clobber  # Remove any generated file
-    rake test     # Execute unit tests
-    rake build    # Build frequent-algorithm-n.n.n.gem into the pkg directory
-    rake install  # Build and install frequent-algorithm-n.n.n.gem into system gems
-    rake release  # Create tag vn.n.n and build and push
-                  # frequent-algorithm-n.n.n.gem to Rubygems
-### Documentation
-`frequent-algorithm` uses [`yard`](https://rubygems.org/gems/yard) and
-[`rdiscount`](https://rubygems.org/gems/rdiscount) for Markdown documentation.
-Check out [Getting Started with
-Yard](http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md).
-### Unit Testing
-`frequent-algorithm` uses
-[`MiniTest::Unit`](https://github.com/seattlerb/minitest) for
-unit testing.
-### Releasing
-Please refer to Publishing To Rubygems.org in the
-[Rubygems Guide](http://guides.rubygems.org/make-your-own-gem/).
-### Contributing
-1. Fork it
-2. Begin work on `dev-branch` (`git fetch && git checkout dev-branch`)
-3. Create your feature branch (`git branch my-new-feature && git checkout
-   my-new-feature`)
-4. Commit your changes (`git commit -am 'Add some feature'`)
-5. Push to the branch (`git push origin my-new-feature:dev-branch`)
-6. Create new Pull Request
-You may wish to read the [Git book online](http://git-scm.com/book/en/v2).
-## License
-frequent-algorithm is provided under the terms of the MIT license.
-Copyright &copy; 2015, Willie Tong &amp; Brooke M. Fujita. All rights reserved.
+# frequent-algorithm
+Web site usage, social network behavior and Internet traffic are examples
+of systems that appear to follow the [Power law](http://en.wikipedia.org/wiki/Power_law),
+where most of the events are due to the actions of a very small few.
+Knowing at any given point in time which items are trending is valuable
+in understanding the system.
+`frequent-algorithm` is a Ruby implementation of the FREQUENT algorithm
+for identifying frequent items in a data stream in sliding windows.
+Please refer to [Identifying Frequent Items in Sliding Windows over On-Line
+Packet Streams](http://erikdemaine.org/papers/SlidingWindow_IMC2003/), by
+Golab, DeHaan, Demaine, L&#243;pez-Ortiz and Munro (2003).
+[![License](https://img.shields.io/badge/license-MIT-blue.svg)]() [![Build Status](https://travis-ci.org/buruzaemon/frequent-algorithm.svg?branch=master)](https://travis-ci.org/buruzaemon/frequent-algorithm) [![Gem Version](https://badge.fury.io/rb/frequent-algorithm.svg)](https://rubygems.org/gems/frequent-algorithm)
+## Introduction
+### Challenges
+Challenges for Real-time processing of data streams for _frequent item queries_
+include:
+* data may be of unknown and possibly unbound length
+* data may be arriving a very fast rate
+* it might not be possible to go back and re-read the data
+* too large a window of observation may include stale data
+Therefore, a solution should have the following characteristics:
+* uses limited memory
+* can process events in the stream in &#927;(1) constant time
+* requires only a single-pass over the data
+### The algorithm
+> LOOP<br/>
+> 1. For each element e in the next b elements:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;If a local counter exists for the type of element e:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Increment the local counter.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new local counter for this element type<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to 1.<br/>
+> 2. Add a summary S containing identities and counts of the k most frequent items to the back of queue Q.<br/>
+> 3. Delete all local counters<br/>
+> 4. For each type named in S:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;If a global counter exists for this type:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add to it the count recorded in S.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new global counter for this element type<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to the count recorded in S.<br/>
+> 5. Add the count of the kth largest type in S to δ.<br/>
+> 6. If sizeOf(Q) > N/b:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(a) Remove the summary S' from the front of Q and subtract the count of the kth largest type in S' from δ.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(b) For all element types named in S':<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Subtract from their global counters the counts<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;recorded in S'<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If a counter is decremented to zero:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete it.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(c) Output the identity and value of each global counter > δ.
+>
+> &mdash; <cite>Golab, DeHaan, Demaine, López-Ortiz and Munro. Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003</cite>
+## Usage
+    require 'frequent-algorithm'
+    # data is pi to 1000 digits
+    pi = File.read('test/frequent/test_data_pi').strip
+    data = pi.scan(/./).each_slice(b)
+    N = 100  # size of main window
+    b =  20  # size of basic window
+    k =   3  # we are interested in top-3 numerals in pi
+    alg = Frequent::Algorithm.new(N, b, k)
+    # read in and process the 1st basic window
+    alg.process(data.next)
+    # and the top-3 numerals are?
+    top3 = alg.statistics.report
+    puts top3
+    # lather, rinse and repeat
+    alg.process(data.next)
+## Development
+The development of this gem requires the following:
+* [Ruby 1.9.3 or greater](https://www.ruby-lang.org/en/)
+* [rubygems](https://rubygems.org/pages/download)
+* [`bundler`](https://github.com/bundler/bundler)
+* [`rake`](https://github.com/ruby/rake)
+* [`minitest`](https://rubygems.org/gems/minitest) (unit testing)
+* [`yard`](https://rubygems.org/gems/yard) (documentation)
+* [`rdiscount`](https://rubygems.org/gems/rdiscount) (Markdown)
+Building, testing and release of this rubygem uses the following
+`rake` commands:
+    rake clean    # Remove any temporary products
+    rake clobber  # Remove any generated file
+    rake test     # Execute unit tests
+    rake build    # Build frequent-algorithm-n.n.n.gem into the pkg directory
+    rake install  # Build and install frequent-algorithm-n.n.n.gem into system gems
+    rake release  # Create tag vn.n.n and build and push
+                  # frequent-algorithm-n.n.n.gem to Rubygems
+### Documentation
+`frequent-algorithm` uses [`yard`](https://rubygems.org/gems/yard) and
+[`rdiscount`](https://rubygems.org/gems/rdiscount) for Markdown documentation.
+Check out [Getting Started with
+Yard](http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md).
+### Unit Testing
+`frequent-algorithm` uses
+[`MiniTest::Unit`](https://github.com/seattlerb/minitest) for
+unit testing.
+### Releasing
+Please refer to Publishing To Rubygems.org in the
+[Rubygems Guide](http://guides.rubygems.org/make-your-own-gem/).
+### Contributing
+1. Fork it
+2. Begin work on `dev-branch` (`git fetch && git checkout dev-branch`)
+3. Create your feature branch (`git branch my-new-feature && git checkout
+   my-new-feature`)
+4. Commit your changes (`git commit -am 'Add some feature'`)
+5. Push to the branch (`git push origin my-new-feature:dev-branch`)
+6. Create new Pull Request
+You may wish to read the [Git book online](http://git-scm.com/book/en/v2).
+## Changelog
+Please see the {file:CHANGELOG} for this gem's release history.
+## License
+frequent-algorithm is provided under the terms of the MIT license.
+Copyright &copy; 2015, Willie Tong &amp; Brooke M. Fujita. All rights reserved. Please see the {file:LICENSE} file for further details.

data/lib/frequent-algorithm.rb CHANGED Viewed

@@ -1,28 +1,28 @@
-# coding: utf-8
-require 'frequent/algorithm'
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+require 'frequent/algorithm'
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

data/lib/frequent/algorithm.rb CHANGED Viewed

@@ -1,182 +1,201 @@
-# coding: utf-8
-require 'frequent/version'
-module Frequent
-  ERR_BADLIST = "List cannot be nil or empty".freeze
-  ERR_BADK = "k must be between 1 and %s".freeze
-  # `Frequent::Algorithm` is the Ruby implementation of the
-  # Demaine et al. FREQUENT algorithm for calculating
-  # top-k items in a stream.
-  #
-  # The aims of this algorithm are:
-  # * use limited memory
-  # * require constant processing time per item
-  # * require a single-pass only
-  #
-  class Algorithm
-    # @return [Integer] the number of items in the main window
-    attr_reader :n
-    # @return [Integer] the number of items in a basic window
-    attr_reader :b
-    # @return [Integer] the number of top item categories to track
-    attr_reader :k
-    # @return [Array<Hash<Object,Integer>>] global queue for basic window summaries
-    attr_reader :queue
-    # @return [Hash<Object,Integer>] global mapping of items and counts
-    attr_reader :statistics
-    # @return [Integer] minimum threshold for membership in top-k items
-    attr_reader :delta
-    # Initializes this top-k frequency-calculating instance.
-    #
-    # @param [Integer] n number of items in the main window
-    # @param [Integer] b number of items in a basic window
-    # @param [Integer] k number of top item categories to track
-    # @raise [ArgumentError] if n is not greater than 0
-    # @raise [ArgumentError] if b is not greater than 0
-    # @raise [ArgumentError] if k is not greater than 0
-    # @raise [ArgumentError] if n/b is not greater than 1
-    def initialize(n, b, k=1)
-      if n <= 0
-        raise ArgumentError.new('n must be greater than 0')
-      end
-      if b <= 0
-        raise ArgumentError.new('b must be greater than 0')
-      end
-      if k <= 0
-        raise ArgumentError.new('k must be greater than 0')
-      end
-      if n/b < 1
-        raise ArgumentError.new('n/b must be greater than 1')
-      end
-      @n = n
-      @b = b
-      @k = k
-      @queue = []
-      @statistics = {}
-      @delta = 0
-    end
-    # Processes a single basic window of b items, by first adding
-    # a summary of this basic window in the internal global queue;
-    # and then updating the global statistics accordingly.
-    #
-    # @param [Array] an array of objects representing a basic window
-    def process(elements)
-      # Do we need this?
-      return if elements.length != @b
-      # Step 1
-      summary = {}
-      elements.each do |e|
-        if summary.key? e
-          summary[e] += 1
-        else
-          summary[e] = 1
-        end
-      end
-      # Step 2
-      @queue << summary
-      # Step 3
-      # Done, implicitly
-      # Step 4
-      summary.each do |k,v|
-        if @statistics.key? k
-          @statistics[k] += v
-        else
-          @statistics[k] = v
-        end
-      end
-      # Step 5
-      @delta += kth_largest(summary.values, @k)
-      # Step 6 - sizeOf(Q) > N/b
-      if @queue.length > @n/@b
-        # a
-        summary_p = @queue.shift
-        @delta -= kth_largest(summary_p.values, @k)
-        # b
-        summary_p.each { |k,v| @statistics[k] -= v }
-        @statistics.delete_if { |k,v| v <= 0 }
-        #c
-        @statistics.select { |k,v| v > @delta }
-      else
-        {}
-      end
-    end
-    # Returns the version for this gem.
-    #
-    # @return [String] the version for this gem.
-    def version
-      Frequent::VERSION
-    end
-    private
-    # Given a list of numbers and a number k which should be
-    # between 1 and the length of the given list, return the
-    # element x in the list that is larger than exactly k-1
-    # other elements in the list.
-    #
-    # @param [Array] list of integers
-    # @return [Integer] the kth largest element in list
-    def kth_largest(list, k)
-      raise ArgumentError.new(ERR_BADLIST) if list.nil? or list.empty?
-      raise ArgumentError.new(ERR_BADK) if k < 1
-      ulist = list.uniq
-      k = ulist.size if k > ulist.size
-      def quickselect(aset, k)
-        p = rand(aset.size)
-        lower = aset.select { |e| e < aset[p] }
-        upper = aset.select { |e| e > aset[p] }
-        if k <= lower.size
-          quickselect(lower, k)
-        elsif k > aset.size - upper.size
-          quickselect(upper, k - (aset.size - upper.size))
-        else
-          aset[p]
-        end
-      end
-      quickselect(ulist, ulist.size+1-k)
-    end
-  end
-end
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+require 'frequent/version'
+require 'thread'
+module Frequent
+  ERR_BADLIST = "List cannot be nil or empty".freeze
+  ERR_BADK = "k must be between 1 and %s".freeze
+  # `Frequent::Algorithm` is the Ruby implementation of the
+  # Demaine et al. FREQUENT algorithm for calculating
+  # top-k items in a stream.
+  #
+  # The aims of this algorithm are:
+  # * use limited memory
+  # * require constant processing time per item
+  # * require a single-pass only
+  #
+  class Algorithm
+    # @return [Integer] the number of items in the main window
+    attr_reader :n
+    # @return [Integer] the number of items in a basic window
+    attr_reader :b
+    # @return [Integer] the number of top item categories to track
+    attr_reader :k
+    # @return [Array<Hash<Object,Integer>>] global queue for basic window summaries
+    attr_reader :queue
+    # @return [Hash<Object,Integer>] global mapping of items and counts
+    attr_reader :statistics
+    # @return [Integer] minimum threshold for membership in top-k items
+    attr_reader :delta
+    # @return [Hash<Object,Integer>] latest top k elements and their counts
+    attr_reader :topk
+    # @return [Array[Object]] the window of elements of size b
+    attr_reader :window
+    # Initializes this top-k frequency-calculating instance.
+    #
+    # @param [Integer] n number of items in the main window
+    # @param [Integer] b number of items in a basic window
+    # @param [Integer] k number of top item categories to track
+    # @raise [ArgumentError] if n is not greater than 0
+    # @raise [ArgumentError] if b is not greater than 0
+    # @raise [ArgumentError] if k is not greater than 0
+    # @raise [ArgumentError] if n/b is not greater than 1
+    def initialize(n, b, k=1)
+      @lock = Mutex.new
+      if n <= 0
+        raise ArgumentError.new('n must be greater than 0')
+      end
+      if b <= 0
+        raise ArgumentError.new('b must be greater than 0')
+      end
+      if k <= 0
+        raise ArgumentError.new('k must be greater than 0')
+      end
+      if n/b < 1
+        raise ArgumentError.new('n/b must be greater than 1')
+      end
+      @n = n
+      @b = b
+      @k = k
+      @queue = []
+      @statistics = {}
+      @delta = 0
+      @topk = {}
+      @window = []
+    end
+    # Processes a single basic window of b items, by first adding
+    # a summary of this basic window in the internal global queue;
+    # and then updating the global statistics accordingly.
+    #
+    # @param [Object] an object from a data stream
+    def process(element)
+      @lock.synchronize do
+        @window << element
+        if @window.length == @b
+          # Step 1
+          summary = {}
+          @window.each do |e|
+            if summary.key? e
+              summary[e] += 1
+            else
+              summary[e] = 1
+            end
+          end
+          @window.clear   #current window cleared
+          # Step 2
+          @queue << summary
+          # Step 3
+          # Done, implicitly
+          # Step 4
+          summary.each do |k,v|
+            if @statistics.key? k
+              @statistics[k] += v
+            else
+              @statistics[k] = v
+            end
+          end
+          # Step 5
+          @delta += kth_largest(summary.values, @k)
+          # Step 6 - sizeOf(Q) > N/b
+          if @queue.length > @n/@b
+            # a
+            summary_p = @queue.shift
+            @delta -= kth_largest(summary_p.values, @k)
+            # b
+            summary_p.each { |k,v| @statistics[k] -= v }
+            @statistics.delete_if { |k,v| v <= 0 }
+            #c
+            @topk = @statistics.select { |k,v| v > @delta }
+          end
+        end
+      end
+    end
+    # Return the latest Tok K elements
+    #
+    # @return [Hash<Object,Integer>] a hash which contains the current top K elements and their counts
+    def report
+      @topk
+    end
+    # Returns the version for this gem.
+    #
+    # @return [String] the version for this gem.
+    def version
+      Frequent::VERSION
+    end
+    private
+    # Given a list of numbers and a number k which should be
+    # between 1 and the length of the given list, return the
+    # element x in the list that is larger than exactly k-1
+    # other elements in the list.
+    #
+    # @param [Array] list of integers
+    # @return [Integer] the kth largest element in list
+    def kth_largest(list, k)
+      raise ArgumentError.new(ERR_BADLIST) if list.nil? or list.empty?
+      raise ArgumentError.new(ERR_BADK) if k < 1
+      ulist = list.uniq
+      k = ulist.size if k > ulist.size
+      def quickselect(aset, k)
+        p = rand(aset.size)
+        lower = aset.select { |e| e < aset[p] }
+        upper = aset.select { |e| e > aset[p] }
+        if k <= lower.size
+          quickselect(lower, k)
+        elsif k > aset.size - upper.size
+          quickselect(upper, k - (aset.size - upper.size))
+        else
+          aset[p]
+        end
+      end
+      quickselect(ulist, ulist.size+1-k)
+    end
+  end
+end
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

data/lib/frequent/version.rb CHANGED Viewed

@@ -1,38 +1,38 @@
-# coding: utf-8
-# `Frequent` is the namespace for objects implementing
-# the Demaine et al. FREQUENT algorithm for finding
-# the most frequently-appearing items (top-k) in a
-# data stream in sliding windows.
-#
-# `Frequent::Algorithm` is the implementation class.
-module Frequent
-  # Version string for this Rubygem.
-  VERSION = '0.0.3'
-end
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+# `Frequent` is the namespace for objects implementing
+# the Demaine et al. FREQUENT algorithm for finding
+# the most frequently-appearing items (top-k) in a
+# data stream in sliding windows.
+#
+# `Frequent::Algorithm` is the implementation class.
+module Frequent
+  # Version string for this Rubygem.
+  VERSION = '0.0.4'
+end
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

metadata CHANGED Viewed

@@ -1,44 +1,44 @@
 --- !ruby/object:Gem::Specification
 name: frequent-algorithm
 version: !ruby/object:Gem::Version
-  version: 0.0.3
+  version: 0.0.4
 platform: ruby
 authors:
 - Willie Tong
 - Brooke M. Fujita
-autorequire:
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2015-03-23 00:00:00.000000000 Z
+date: 2015-05-06 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
-  version_requirements: !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  requirement: !ruby/object:Gem::Requirement
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  prerelease: false
-  type: :development
 - !ruby/object:Gem::Dependency
   name: minitest
-  version_requirements: !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  requirement: !ruby/object:Gem::Requirement
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  prerelease: false
-  type: :development
 description: |
   frequent-algorithm is a Ruby implementation of the Demaine et al FREQUENT algorithm for identifying frequent items in a data stream in sliding windows (c.f Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003).
 email:
@@ -59,7 +59,7 @@ homepage: https://github.com/buruzaemon/frequent-algorithm
 licenses:
 - MIT
 metadata: {}
-post_install_message:
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -74,9 +74,11 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
+rubyforge_project:
 rubygems_version: 2.4.5
-signing_key:
+signing_key:
 specification_version: 4
-summary: Identifies frequent items in a data stream in sliding windows using the Demaine et al FREQUENT algorithm.
+summary: Identifies frequent items in a data stream in sliding windows using the Demaine
+  et al FREQUENT algorithm.
 test_files: []
+has_rdoc: