RubyGems - frequent-algorithm - Versions diffs - 0.0.2 → 0.0.3 - Mend

frequent-algorithm 0.0.2 → 0.0.3

Files changed (9) hide show

checksums.yaml +4 -4
data/.yardopts +7 -7
data/CHANGELOG +9 -9
data/LICENSE +22 -22
data/README.md +149 -149
data/lib/frequent-algorithm.rb +28 -28
data/lib/frequent/algorithm.rb +182 -166
data/lib/frequent/version.rb +38 -38
metadata +26 -27

checksums.yaml CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: f799561d8b1543e23483918e587dffcbd0522e78
-  data.tar.gz: 39a5a9ceee0dee33bc889111b2745907ffb49018
+  metadata.gz: c581bac868de994e83f9fee63f34d8987ab2eda9
+  data.tar.gz: a9c068fa92383857a5d0526c9aa55ce4746651d5
 SHA512:
-  metadata.gz: c68087e23dc0ff299f81797a1152c6e1fc6f1f10b7ec46ca39fff35cb8d91836be27eb34fe434c484fc1b28fde939c2f436d42b36f958ddca0afbe9f5107194b
-  data.tar.gz: 6f0da59941492900cc4da2f1266ed0e8398a852cbb55e064fd0ed290f8209a396da2e051c5293a45a21c7564828a099ea03b607dfba53608d824f834a1083bc1
+  metadata.gz: 42571a866be7b6ee00748a10afb5ee6d48d8025198a68877a18401bedc0e372da09b1af3a7ef69ac94c78ed2fb798e52899b30eaddea2c423c0982413a2f9f3e
+  data.tar.gz: 0fd80b41cbc31fec785ea52596ab0e9094e54172d8a95e419f80339cdbcd7e9b275dd4a7a0826380ff43a3f81e8f6519d87fe02773ef4319c4a264211cee6f95

data/.yardopts CHANGED Viewed

@@ -1,7 +1,7 @@
---no-private
---readme README.md
---markup markdown
---markup-provider rdiscount
--
-LICENSE
-CHANGELOG
+--no-private
+--readme README.md
+--markup markdown
+--markup-provider rdiscount
+-
+LICENSE
+CHANGELOG

data/CHANGELOG CHANGED Viewed

@@ -1,9 +1,9 @@
-## CHANGELOG
-- __2015/03/19 0.0.2 release.
-    - First-stage implementation.
-    - API documentation added.
-    - Fleshing out unit tests.
-- __2015/03/11__: 0.0.1 release.
-    - Initial release.
+## CHANGELOG
+- __2015/03/19 0.0.2 release.
+    - First-stage implementation.
+    - API documentation added.
+    - Fleshing out unit tests.
+- __2015/03/11__: 0.0.1 release.
+    - Initial release.

data/LICENSE CHANGED Viewed

@@ -1,22 +1,22 @@
-The MIT License (MIT)
-Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+The MIT License (MIT)
+Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md CHANGED Viewed

@@ -1,149 +1,149 @@
-# frequent-algorithm [![Gem Version](https://badge.fury.io/rb/frequent-algorithm.svg)](http://badge.fury.io/rb/frequent-algorithm) [![Build Status](https://travis-ci.org/buruzaemon/frequent-algorithm.svg)](https://travis-ci.org/buruzaemon/frequent-algorithm)
-Web site usage, social network behavior and Internet traffic are examples
-of systems that appear to follow the [power law](http://en.wikipedia.org/wiki/Power_law),
-where most of the events are due to the actions of a very small few.
-Knowing at any given point in time which items are trending is valuable
-in understanding the system.
-`frequent-algorithm` is a Ruby implementation of the FREQUENT algorithm
-for identifying frequent items in a data stream in sliding windows.
-Please refer to [Identifying Frequent Items in Sliding Windows over On-Line
-Packet Streams](http://erikdemaine.org/papers/SlidingWindow_IMC2003/), by
-Golab, DeHaan, Demaine, L&#243;pez-Ortiz and Munro (2003).
-## Introduction
-### Challenges
-Challenges for Real-time processing of data streams for _frequent item queries_
-include:
-* data may be of unknown and possibly unbound length
-* data may be arriving a very fast rate
-* it might not be possible to go back and re-read the data
-* too large a window of observation may include stale data
-Therefore, a solution should have the following characteristics:
-* uses limited memory
-* can process events in the stream in &#927;(1) constant time
-* requires only a single-pass over the data
-### The algorithm
-> LOOP<br/>
-> 1. For each element e in the next b elements:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;If a local counter exists for the type of element e:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Increment the local counter.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new local counter for this element type<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to 1.<br/>
-> 2. Add a summary S containing identities and counts of the k most frequent items to the back of queue Q.<br/>
-> 3. Delete all local counters<br/>
-> 4. For each type named in S:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;If a global counter exists for this type:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add to it the count recorded in S.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new global counter for this element type<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to the count recorded in S.<br/>
-> 5. Add the count of the kth largest type in S to δ.<br/>
-> 6. If sizeOf(Q) > N/b:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(a) Remove the summary S' from the front of Q and subtract the count of the kth largest type in S' from δ.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(b) For all element types named in S':<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Subtract from their global counters the counts<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;recorded in S'<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If a counter is decremented to zero:<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete it.<br/>
-> &nbsp;&nbsp;&nbsp;&nbsp;(c) Output the identity and value of each global counter > δ.
->
-> &mdash; <cite>Golab, DeHaan, Demaine, López-Ortiz and Munro. Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003</cite>
-## Usage
-    require 'frequent-algorithm'
-    # data is pi to 1000 digits
-    pi = File.read('test/frequent/test_data_pi').strip
-    data = pi.scan(/./).each_slice(b)
-    N = 100  # size of main window
-    b =  20  # size of basic window
-    k =   3  # we are interested in top-3 numerals in pi
-    alg = Frequent::Algorithm.new(N, b, k)
-    # read in and process the 1st basic window
-    alg.process(data.next)
-    # and the top-3 numerals are?
-    top3 = alg.statistics.report
-    puts top3
-    # lather, rinse and repeat
-    alg.process(data.next)
-## Development
-The development of this gem requires the following:
-* [Ruby 1.9.3 or greater](https://www.ruby-lang.org/en/)
-* [rubygems](https://rubygems.org/pages/download)
-* [`bundler`](https://github.com/bundler/bundler)
-* [`rake`](https://github.com/ruby/rake)
-* [`minitest`](https://rubygems.org/gems/minitest) (unit testing)
-* [`yard`](https://rubygems.org/gems/yard) (documentation)
-* [`rdiscount`](https://rubygems.org/gems/rdiscount) (Markdown)
-Building, testing and release of this rubygem uses the following
-`rake` commands:
-    rake build    # Build frequent-algorithm-n.n.n.gem into the pkg directory
-    rake clean    # Remove any temporary products
-    rake clobber  # Remove any generated file
-    rake install  # Build and install frequent-algorithm-n.n.n.gem into system gems
-    rake release  # Create tag vn.n.n and build and push
-                  # frequent-algorithm-n.n.n.gem to Rubygems
-    rake test     # Execute unit tests
-### Documentation
-`frequent-algorithm` uses [`yard`](https://rubygems.org/gems/yard) and
-[`rdiscount`](https://rubygems.org/gems/rdiscount) for Markdown documentation.
-Check out [Getting Started with
-Yard](http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md).
-### Unit Testing
-`frequent-algorithm` uses
-[`MiniTest::Unit`](https://github.com/seattlerb/minitest) for
-unit testing.
-### Releasing
-Please refer to Publishing To Rubygems.org in the
-[Rubygems Guide](http://guides.rubygems.org/make-your-own-gem/).
-### Contributing
-1. Fork it
-2. Begin work on `dev-branch` (`git fetch && git checkout dev-branch`)
-3. Create your feature branch (`git branch my-new-feature && git checkout
-   my-new-feature`)
-4. Commit your changes (`git commit -am 'Add some feature'`)
-5. Push to the branch (`git push origin my-new-feature:dev-branch`)
-6. Create new Pull Request
-You may wish to read the [Git book online](http://git-scm.com/book/en/v2).
-## License
-frequent-algorithm is provided under the terms of the MIT license.
-Copyright &copy; 2015, Willie Tong &amp; Brooke M. Fujita. All rights reserved.
+# frequent-algorithm [![Gem Version](https://badge.fury.io/rb/frequent-algorithm.svg)](http://badge.fury.io/rb/frequent-algorithm) [![Build Status](https://travis-ci.org/buruzaemon/frequent-algorithm.svg)](https://travis-ci.org/buruzaemon/frequent-algorithm)
+Web site usage, social network behavior and Internet traffic are examples
+of systems that appear to follow the [Power law](http://en.wikipedia.org/wiki/Power_law),
+where most of the events are due to the actions of a very small few.
+Knowing at any given point in time which items are trending is valuable
+in understanding the system.
+`frequent-algorithm` is a Ruby implementation of the FREQUENT algorithm
+for identifying frequent items in a data stream in sliding windows.
+Please refer to [Identifying Frequent Items in Sliding Windows over On-Line
+Packet Streams](http://erikdemaine.org/papers/SlidingWindow_IMC2003/), by
+Golab, DeHaan, Demaine, L&#243;pez-Ortiz and Munro (2003).
+## Introduction
+### Challenges
+Challenges for Real-time processing of data streams for _frequent item queries_
+include:
+* data may be of unknown and possibly unbound length
+* data may be arriving a very fast rate
+* it might not be possible to go back and re-read the data
+* too large a window of observation may include stale data
+Therefore, a solution should have the following characteristics:
+* uses limited memory
+* can process events in the stream in &#927;(1) constant time
+* requires only a single-pass over the data
+### The algorithm
+> LOOP<br/>
+> 1. For each element e in the next b elements:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;If a local counter exists for the type of element e:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Increment the local counter.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new local counter for this element type<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to 1.<br/>
+> 2. Add a summary S containing identities and counts of the k most frequent items to the back of queue Q.<br/>
+> 3. Delete all local counters<br/>
+> 4. For each type named in S:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;If a global counter exists for this type:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Add to it the count recorded in S.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;Otherwise:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a new global counter for this element type<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;and set it equal to the count recorded in S.<br/>
+> 5. Add the count of the kth largest type in S to δ.<br/>
+> 6. If sizeOf(Q) > N/b:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(a) Remove the summary S' from the front of Q and subtract the count of the kth largest type in S' from δ.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(b) For all element types named in S':<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Subtract from their global counters the counts<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;recorded in S'<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If a counter is decremented to zero:<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete it.<br/>
+> &nbsp;&nbsp;&nbsp;&nbsp;(c) Output the identity and value of each global counter > δ.
+>
+> &mdash; <cite>Golab, DeHaan, Demaine, López-Ortiz and Munro. Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003</cite>
+## Usage
+    require 'frequent-algorithm'
+    # data is pi to 1000 digits
+    pi = File.read('test/frequent/test_data_pi').strip
+    data = pi.scan(/./).each_slice(b)
+    N = 100  # size of main window
+    b =  20  # size of basic window
+    k =   3  # we are interested in top-3 numerals in pi
+    alg = Frequent::Algorithm.new(N, b, k)
+    # read in and process the 1st basic window
+    alg.process(data.next)
+    # and the top-3 numerals are?
+    top3 = alg.statistics.report
+    puts top3
+    # lather, rinse and repeat
+    alg.process(data.next)
+## Development
+The development of this gem requires the following:
+* [Ruby 1.9.3 or greater](https://www.ruby-lang.org/en/)
+* [rubygems](https://rubygems.org/pages/download)
+* [`bundler`](https://github.com/bundler/bundler)
+* [`rake`](https://github.com/ruby/rake)
+* [`minitest`](https://rubygems.org/gems/minitest) (unit testing)
+* [`yard`](https://rubygems.org/gems/yard) (documentation)
+* [`rdiscount`](https://rubygems.org/gems/rdiscount) (Markdown)
+Building, testing and release of this rubygem uses the following
+`rake` commands:
+    rake clean    # Remove any temporary products
+    rake clobber  # Remove any generated file
+    rake test     # Execute unit tests
+    rake build    # Build frequent-algorithm-n.n.n.gem into the pkg directory
+    rake install  # Build and install frequent-algorithm-n.n.n.gem into system gems
+    rake release  # Create tag vn.n.n and build and push
+                  # frequent-algorithm-n.n.n.gem to Rubygems
+### Documentation
+`frequent-algorithm` uses [`yard`](https://rubygems.org/gems/yard) and
+[`rdiscount`](https://rubygems.org/gems/rdiscount) for Markdown documentation.
+Check out [Getting Started with
+Yard](http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md).
+### Unit Testing
+`frequent-algorithm` uses
+[`MiniTest::Unit`](https://github.com/seattlerb/minitest) for
+unit testing.
+### Releasing
+Please refer to Publishing To Rubygems.org in the
+[Rubygems Guide](http://guides.rubygems.org/make-your-own-gem/).
+### Contributing
+1. Fork it
+2. Begin work on `dev-branch` (`git fetch && git checkout dev-branch`)
+3. Create your feature branch (`git branch my-new-feature && git checkout
+   my-new-feature`)
+4. Commit your changes (`git commit -am 'Add some feature'`)
+5. Push to the branch (`git push origin my-new-feature:dev-branch`)
+6. Create new Pull Request
+You may wish to read the [Git book online](http://git-scm.com/book/en/v2).
+## License
+frequent-algorithm is provided under the terms of the MIT license.
+Copyright &copy; 2015, Willie Tong &amp; Brooke M. Fujita. All rights reserved.

data/lib/frequent-algorithm.rb CHANGED Viewed

@@ -1,28 +1,28 @@
-# coding: utf-8
-require 'frequent/algorithm'
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+require 'frequent/algorithm'
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

data/lib/frequent/algorithm.rb CHANGED Viewed

@@ -1,166 +1,182 @@
-# coding: utf-8
-require 'frequent/version'
-module Frequent
-  # `Frequent::Algorithm` is the Ruby implementation of the
-  # Demaine et al. FREQUENT algorithm for calculating
-  # top-k items in a stream.
-  #
-  # The aims of this algorithm are:
-  # * use limited memory
-  # * require constant processing time per item
-  # * require a single-pass only
-  #
-  class Algorithm
-    # @return [Integer] the number of items in the main window
-    attr_reader :n
-    # @return [Integer] the number of items in a basic window
-    attr_reader :b
-    # @return [Integer] the number of top item categories to track
-    attr_reader :k
-    # @return [Array<Hash<Object,Integer>>] global queue for basic window summaries
-    attr_reader :queue
-    # @return [Hash<Object,Integer>] global mapping of items and counts
-    attr_reader :statistics
-    # @return [Integer] minimum threshold for membership in top-k items
-    attr_reader :delta
-    # Initializes this top-k frequency-calculating instance.
-    #
-    # @param [Integer] n number of items in the main window
-    # @param [Integer] b number of items in a basic window
-    # @param [Integer] k number of top item categories to track
-    # @raise [ArgumentError] if n is not greater than 0
-    # @raise [ArgumentError] if b is not greater than 0
-    # @raise [ArgumentError] if k is not greater than 0
-    # @raise [ArgumentError] if n/b is not greater than 1
-    def initialize(n, b, k=1)
-      if n <= 0
-        raise ArgumentError.new('n must be greater than 0')
-      end
-      if b <= 0
-        raise ArgumentError.new('b must be greater than 0')
-      end
-      if k <= 0
-        raise ArgumentError.new('k must be greater than 0')
-      end
-      if n/b < 1
-        raise ArgumentError.new('n/b must be greater than 1')
-      end
-      @n = n
-      @b = b
-      @k = k
-      @queue = []
-      @statistics = {}
-      @delta = 0
-    end
-    # Processes a single basic window of b items, by first adding
-    # a summary of this basic window in the internal global queue;
-    # and then updating the global statistics accordingly.
-    #
-    # @param [Array] an array of objects representing a basic window
-    def process(elements)
-      # Do we need this?
-      return if elements.length != @b
-      # Step 1
-      summary = {}
-      elements.each do |e|
-        if summary.key? e
-          summary[e] += 1
-        else
-          summary[e] = 1
-        end
-      end
-      # index of the k-th item
-      kth_index = find_kth_largest(summary)
-      # Step 2 & 3
-      # summary is [[item,count],[item,count],[item,count]....]
-      # sorted by descending order of the item count
-      summary = summary.sort { |a,b| b[1]<=>a[1] }[0..kth_index]
-      @queue << summary
-      # Step 4
-      summary.each do |t|
-        if @statistics.key? t[0]
-          @statistics[t[0]] += t[1]
-        else
-          @statistics[t[0]] = t[1]
-        end
-      end
-      # Step 5
-      @delta += summary[kth_index][1]
-      # Step 6
-      if should_pop_oldest_summary
-        # a
-        summary_p = @queue.shift
-        @delta -= summary_p[find_kth_largest(summary_p)][1]
-        # b
-        summary_p.each { |t| @statistics[t[0]] -= t[1] }
-        @statistics.delete_if { |k,v| v <= 0 }
-        #c
-        @statistics.select { |k,v| v > @delta }
-      else
-        {}
-      end
-    end
-    # Returns the version for this gem.
-    #
-    # @return [String] the version for this gem.
-    def version
-      Frequent::VERSION
-    end
-    private
-      # Return true when it is ready to pop oldest summary from queue
-      #
-      # @return [Boolean] whether it is ready to pop oldest summary from queue
-      def should_pop_oldest_summary
-        @queue.length > @n/@b
-      end
-      # Return the k-th index of a summary object
-      #
-      # @param [Object] a summary object
-      # @return [Integer] the k-th index
-      def find_kth_largest(summary)
-        [summary.length, @k].min - 1
-      end
-  end
-end
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+require 'frequent/version'
+module Frequent
+  ERR_BADLIST = "List cannot be nil or empty".freeze
+  ERR_BADK = "k must be between 1 and %s".freeze
+  # `Frequent::Algorithm` is the Ruby implementation of the
+  # Demaine et al. FREQUENT algorithm for calculating
+  # top-k items in a stream.
+  #
+  # The aims of this algorithm are:
+  # * use limited memory
+  # * require constant processing time per item
+  # * require a single-pass only
+  #
+  class Algorithm
+    # @return [Integer] the number of items in the main window
+    attr_reader :n
+    # @return [Integer] the number of items in a basic window
+    attr_reader :b
+    # @return [Integer] the number of top item categories to track
+    attr_reader :k
+    # @return [Array<Hash<Object,Integer>>] global queue for basic window summaries
+    attr_reader :queue
+    # @return [Hash<Object,Integer>] global mapping of items and counts
+    attr_reader :statistics
+    # @return [Integer] minimum threshold for membership in top-k items
+    attr_reader :delta
+    # Initializes this top-k frequency-calculating instance.
+    #
+    # @param [Integer] n number of items in the main window
+    # @param [Integer] b number of items in a basic window
+    # @param [Integer] k number of top item categories to track
+    # @raise [ArgumentError] if n is not greater than 0
+    # @raise [ArgumentError] if b is not greater than 0
+    # @raise [ArgumentError] if k is not greater than 0
+    # @raise [ArgumentError] if n/b is not greater than 1
+    def initialize(n, b, k=1)
+      if n <= 0
+        raise ArgumentError.new('n must be greater than 0')
+      end
+      if b <= 0
+        raise ArgumentError.new('b must be greater than 0')
+      end
+      if k <= 0
+        raise ArgumentError.new('k must be greater than 0')
+      end
+      if n/b < 1
+        raise ArgumentError.new('n/b must be greater than 1')
+      end
+      @n = n
+      @b = b
+      @k = k
+      @queue = []
+      @statistics = {}
+      @delta = 0
+    end
+    # Processes a single basic window of b items, by first adding
+    # a summary of this basic window in the internal global queue;
+    # and then updating the global statistics accordingly.
+    #
+    # @param [Array] an array of objects representing a basic window
+    def process(elements)
+      # Do we need this?
+      return if elements.length != @b
+      # Step 1
+      summary = {}
+      elements.each do |e|
+        if summary.key? e
+          summary[e] += 1
+        else
+          summary[e] = 1
+        end
+      end
+      # Step 2
+      @queue << summary
+      # Step 3
+      # Done, implicitly
+      # Step 4
+      summary.each do |k,v|
+        if @statistics.key? k
+          @statistics[k] += v
+        else
+          @statistics[k] = v
+        end
+      end
+      # Step 5
+      @delta += kth_largest(summary.values, @k)
+      # Step 6 - sizeOf(Q) > N/b
+      if @queue.length > @n/@b
+        # a
+        summary_p = @queue.shift
+        @delta -= kth_largest(summary_p.values, @k)
+        # b
+        summary_p.each { |k,v| @statistics[k] -= v }
+        @statistics.delete_if { |k,v| v <= 0 }
+        #c
+        @statistics.select { |k,v| v > @delta }
+      else
+        {}
+      end
+    end
+    # Returns the version for this gem.
+    #
+    # @return [String] the version for this gem.
+    def version
+      Frequent::VERSION
+    end
+    private
+    # Given a list of numbers and a number k which should be
+    # between 1 and the length of the given list, return the
+    # element x in the list that is larger than exactly k-1
+    # other elements in the list.
+    #
+    # @param [Array] list of integers
+    # @return [Integer] the kth largest element in list
+    def kth_largest(list, k)
+      raise ArgumentError.new(ERR_BADLIST) if list.nil? or list.empty?
+      raise ArgumentError.new(ERR_BADK) if k < 1
+      ulist = list.uniq
+      k = ulist.size if k > ulist.size
+      def quickselect(aset, k)
+        p = rand(aset.size)
+        lower = aset.select { |e| e < aset[p] }
+        upper = aset.select { |e| e > aset[p] }
+        if k <= lower.size
+          quickselect(lower, k)
+        elsif k > aset.size - upper.size
+          quickselect(upper, k - (aset.size - upper.size))
+        else
+          aset[p]
+        end
+      end
+      quickselect(ulist, ulist.size+1-k)
+    end
+  end
+end
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

data/lib/frequent/version.rb CHANGED Viewed

@@ -1,38 +1,38 @@
-# coding: utf-8
-# `Frequent` is the namespace for objects implementing
-# the Demaine et al. FREQUENT algorithm for finding
-# the most frequently-appearing items (top-k) in a
-# data stream in sliding windows.
-#
-# `Frequent::Algorithm` is the implementation class.
-module Frequent
-  # Version string for this Rubygem.
-  VERSION = '0.0.2'
-end
-=begin
-  The MIT License (MIT)
-  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
-  Permission is hereby granted, free of charge, to any person obtaining a
-  copy of this software and associated documentation files (the "Software"),
-  to deal in the Software without restriction, including without limitation
-  the rights to use, copy, modify, merge, publish, distribute, sublicense,
-  and/or sell copies of the Software, and to permit persons to whom the
-  Software is furnished to do so, subject to the following conditions:
-  The above copyright notice and this permission notice shall be included
-  in all copies or substantial portions of the Software.
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-  IN THE SOFTWARE.
-=end
+# coding: utf-8
+# `Frequent` is the namespace for objects implementing
+# the Demaine et al. FREQUENT algorithm for finding
+# the most frequently-appearing items (top-k) in a
+# data stream in sliding windows.
+#
+# `Frequent::Algorithm` is the implementation class.
+module Frequent
+  # Version string for this Rubygem.
+  VERSION = '0.0.3'
+end
+=begin
+  The MIT License (MIT)
+  Copyright (c) 2015 Willie Tong, Brooke M. Fujita
+  Permission is hereby granted, free of charge, to any person obtaining a
+  copy of this software and associated documentation files (the "Software"),
+  to deal in the Software without restriction, including without limitation
+  the rights to use, copy, modify, merge, publish, distribute, sublicense,
+  and/or sell copies of the Software, and to permit persons to whom the
+  Software is furnished to do so, subject to the following conditions:
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+  THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+  IN THE SOFTWARE.
+=end

metadata CHANGED Viewed

@@ -1,44 +1,44 @@
 --- !ruby/object:Gem::Specification
 name: frequent-algorithm
 version: !ruby/object:Gem::Version
-  version: 0.0.2
+  version: 0.0.3
 platform: ruby
 authors:
 - Willie Tong
 - Brooke M. Fujita
-autorequire:
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2015-03-19 00:00:00.000000000 Z
+date: 2015-03-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
-  requirement: !ruby/object:Gem::Requirement
+  version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+  prerelease: false
+  type: :development
 - !ruby/object:Gem::Dependency
   name: minitest
-  requirement: !ruby/object:Gem::Requirement
+  version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
+  requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+  prerelease: false
+  type: :development
 description: |
   frequent-algorithm is a Ruby implementation of the Demaine et al FREQUENT algorithm for identifying frequent items in a data stream in sliding windows (c.f Identifying Frequent Items in Sliding Windows over On-Line Packet Streams, 2003).
 email:
@@ -48,36 +48,35 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- ".yardopts"
+- CHANGELOG
+- LICENSE
+- README.md
 - lib/frequent-algorithm.rb
 - lib/frequent/algorithm.rb
 - lib/frequent/version.rb
-- README.md
-- LICENSE
-- CHANGELOG
-- .yardopts
 homepage: https://github.com/buruzaemon/frequent-algorithm
 licenses:
 - MIT
 metadata: {}
-post_install_message:
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '2.0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
-rubygems_version: 2.0.14
-signing_key:
+rubyforge_project:
+rubygems_version: 2.4.5
+signing_key:
 specification_version: 4
-summary: Identifies frequent items in a data stream in sliding windows using the Demaine
-  et al FREQUENT algorithm.
+summary: Identifies frequent items in a data stream in sliding windows using the Demaine et al FREQUENT algorithm.
 test_files: []