simple-feed 1.0.4 → 2.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 37f4c35a87e3c3ed5b0a3cbdc88b6a88edc795c3
-  data.tar.gz: c88031baedfb60e6376d561621d2547b2db37978
+  metadata.gz: de6f18bbf9e002e7f1ee3fa49a7675ea66320bc4
+  data.tar.gz: 1325dfc4fed1fe485aef9d2fcd0a75442022caee
 SHA512:
-  metadata.gz: eb887ba71541e037975ae5800842b2f5a824901c5134ad6e38cd91489917fb62971edefac865fa5d2f7e21452417d08c77d164caa84d84eb16c0f681bb12a239
-  data.tar.gz: db3fad51130186c6199e6874b54e4659eedada7cf4e995781a5103e720e6beb712e0a610d30f5a2eed9abc9f8b23336f33007539e3825dc2ee3dad5901acce79
+  metadata.gz: d90c16eaca736fe56ae19a4c2df0688a9267f00863f18a713ac678ec9150210e94553945636450651563af35706f4ec68dcb113fa451a7109c7376bcb5704363
+  data.tar.gz: 758243acc5437bc4f022dfbeaa5b93d3af01525ec972760fe33cca1756de220e4b2b344c5e2f7e771578f8fdb24aef662e4abad8a9ba09ace74776c0512e37d8

data/.gitignore

@@ -14,3 +14,4 @@
 ._*
 .Spotlight-V100
 .Trashes
+/lib/simplefeed/providers/serialization/key.rb

data/README.md

@@ -6,6 +6,7 @@
 [![Code Climate](https://codeclimate.com/repos/58339a5b3d9faa74ac006b36/badges/8b899f6df4fc1ed93759/gpa.svg)](https://codeclimate.com/repos/58339a5b3d9faa74ac006b36/feed)
 [![Test Coverage](https://codeclimate.com/repos/58339a5b3d9faa74ac006b36/badges/8b899f6df4fc1ed93759/coverage.svg)](https://codeclimate.com/repos/58339a5b3d9faa74ac006b36/coverage)
 [![Issue Count](https://codeclimate.com/repos/58339a5b3d9faa74ac006b36/badges/8b899f6df4fc1ed93759/issue_count.svg)](https://codeclimate.com/repos/58339a5b3d9faa74ac006b36/feed)
+[![Inline docs](http://inch-ci.org/github/kigster/simple-feed.svg?branch=master)](http://inch-ci.org/github/kigster/simple-feed)
 
 This is a fast, pure-ruby implementation of an activity feed concept commonly used in social networking applications. The implementation is optimized for **read-time performance** and high concurrency (lots of users), and can be extended with custom backend providers. Two providers come bundled: the production-ready Redis provider, and a naive pure Hash-based provider.
 
@@ -35,55 +36,50 @@ What you publish into your feed — i.e. _stories_ or _events_, will depend enti
 
 ## Challenges 
 
-Building a personalized activity feeds tend to be a challenging task, due to the diversity of event types that it often includes, the personalization requirement, and the need for it to often scale to very large numbers of concurrent users.  Therefore common implementations tend to focus on either:
+Building a personalized activity feed tends to be a challenging task, due to the diversity of event types that it often includes, the personalization requirement, and the need for it to often scale to very large numbers of concurrent users.  Therefore common implementations tend to focus on either:
 
  * optimizing the read time performance by pre-computing the feed for each user ahead of time
  * OR optimizing the various ranking algorithms by computing the feed at read time, with complex forms of caching addressing the performance requirements.
  
 The first type of feed is much simpler to implement on a large scale (up to a point), and it scales well if the data is stored in a light-weight in-memory storage such as Redis. This is exactly the approach this library takes.
 
-For more information about various types of feed, and the typical architectures that power them — please read ["How would you go about building an activity feed like Facebook?"](https://hashnode.com/post/architecture-how-would-you-go-about-building-an-activity-feed-like-facebook-cioe6ea7q017aru53phul68t1/answer/ciol0lbaa02q52s530vfqea0t) by [Lee Byron](https://hashnode.com/@leebyron). 
+For more information about various types of feed, and the typical architectures that power them — please read:
+
+ - ["How would you go about building an activity feed like Facebook?"](https://hashnode.com/post/architecture-how-would-you-go-about-building-an-activity-feed-like-facebook-cioe6ea7q017aru53phul68t1/answer/ciol0lbaa02q52s530vfqea0t) by [Lee Byron](https://hashnode.com/@leebyron).
+ - ["Feeding Frenzy: Selectively Materializing Users’ Event Feeds"](http://jeffterrace.com/docs/feeding-frenzy-sigmod10-web.pdf) (Yahoo! Research paper).
 
 ## Overview
 
 The feed library aims to address the following goals:
 
-* To define a minimalistic API for a typical event-based simple feed,
-  without tying it to any concrete provider implementation
-* To make it easy to implement and plug in a new type of provider,
-  eg.using Couchbase or MongoDB
+* To define a minimalistic API for a typical event-based simple feed, without tying it to any concrete provider implementation
+* To make it easy to implement and plug in a new type of provider, eg. using *Couchbase* or *MongoDB*
 * To provide a scalable default provider implementation using Redis, which can support millions of users via sharding
 * To support multiple simple feeds within the same application, but used for different purposes, eg. simple feed of my followers, versus simple feed of my own actions.
 
 ## Usage
 
-First you need to configure the Feed with a valid provider
-implementation and a name.
+First you need to configure the Feed with a valid provider implementation and a name.
 
 ### Configuration
 
-Below we configure a feed called `:newsfeed`, which presumably
-will be populated with the events coming from the followers.
+Below we configure a feed called `:newsfeed`, which presumably will be populated with the events coming from the followers.
 
 ```ruby
 require 'simplefeed'
-require 'simplefeed/providers/redis'
 
 # Let's define a Redis-based feed, and wrap Redis in a in a ConnectionPool. 
 SimpleFeed.define(:newsfeed) do |f|
   f.provider   = SimpleFeed.provider(:redis, 
                                       redis: -> { ::Redis.new },
                                       pool_size: 10)
-  f.per_page   = 50
-  f.batch_size = 10 # default batch size
-  f.namespace  = 'nf'
+  f.per_page   = 50     # default page size
+  f.batch_size = 10     # default batch size
+  f.namespace  = 'nf'   # only needed if you use the same redis for more than one feed
 end
 ```
 
-After the feed is defined, the gem creates a similarly named method
-under the `SimpleFeed` namespace to access the feed. For example, given
-a name such as `:newsfeed` the following are all valid ways of
-accessing the feed:
+After the feed is defined, the gem creates a similarly named method under the `SimpleFeed` namespace to access the feed. For example, given a name such as `:newsfeed` the following are all valid ways of accessing the feed:
 
  * `SimpleFeed.newsfeed`
  * `SimpleFeed.get(:newsfeed)`
@@ -92,21 +88,17 @@ You can also get a full list of currently defined feeds with `SimpleFeed.feed_na
 
 ### Reading from and writing to the feed
 
-For the impatient here is a quick way to get started with the
-`SimpleFeed`.
+For the impatient here is a quick way to get started with the `SimpleFeed`.
 
 ```ruby
-activity = SimpleFeed.get(:followers).activity(@current_user.id)
-
+# This assumes we have previously defined a feed named :newsfeed (see above)
+activity = SimpleFeed.newsfeed.activity(@current_user.id)
 # Store directly the value and the optional time stamp
 activity.store(value: 'hello')
 # => true
 
 # or equivalent:
-@event = SimpleFeed::Event.new(value: 'hello', at: Time.now)
-# or even simpler:
 @event = SimpleFeed::Event.new('hello', Time.now)
-# and then:
 activity.store(event: @event)
 # => false # false indicates that the same event is already in the feed.
 ```
@@ -124,25 +116,25 @@ activity.paginate(page: 1)
 
 ### The Two Forms of the API
 
-The feed API is offered in a single-user and a batch (multi-user) forms.
+The feed API is offered in two forms:
+
+ 1. single-user form, and 
+ 2. a batch (multi-user) form.
 
-The main and only difference is in what the methods return. In the
-single user case, the return of, say, `#total_count` is an `Integer`
-value representing the total count for this user.
+The method names and signatures are the same. The only difference is in what the methods return:
 
-In the multi-user case, the return is a `SimpleFeed::Response` instance,
-that can be thought of as a `Hash`, that has the user IDs as the keys,
-and return results for each user as a value.
+ 1. In the single user case, the return of, say, `#total_count` is an `Integer` value representing the total count for this user.
+ 2. In the multi-user case, the return is a `SimpleFeed::Response` instance, that can be thought of as a `Hash`, that has the user IDs as the keys, and return results for each user as a value. 
 
-Please see further below the details about the [Batch API](#bach-api).
+Please see further below the details about the [Batch API](#batch-api).
 
 <a name="single-user-api"/>
 
 ##### Single-User API 
 
-In the examples below we show responses based on a single-user usage. As previously mentioned, the multi-user case is the same, except for the response values, and is discussed further below.
+In the examples below we show responses based on a single-user usage. As previously mentioned, the multi-user usage is the same, except what the response values are, and is discussed further down below.
 
-Below is a user session that demonstrates simple return values from the feed operations for a given user:
+Let's take a look at a ruby session, which demonstrates return values of the feed operations for a single user:
 
 ```ruby
 require 'simplefeed'
@@ -156,7 +148,7 @@ SimpleFeed.define(:followers) do |f|
 end
 
 # Let's get the Activity instance that wraps this user_id
-activity = SimpleFeed.get(:followers).activity(user_id)   # => [... complex object removed for brevity ]
+activity = SimpleFeed.followers.activity(user_id)   # => [... complex object removed for brevity ]
 # let's clear out this feed to ensure it's empty
 activity.wipe                                             # => true
 # Let's verify that the counts for this feed are at zero
@@ -164,23 +156,31 @@ activity.total_count                                      # => 0
 activity.unread_count                                     # => 0
 # Store some events
 activity.store(value: 'hello')                            # => true
-activity.store(value: 'goodbye')                          # => true
+activity.store(value: 'goodbye', at: Time.now - 20)       # => true
 activity.unread_count                                     # => 2
 # Now we can paginate the events, which by default resets "last_read" timestamp the user
 activity.paginate(page: 1)
 # [
-#     [0] #<SimpleFeed::Event#70138821650220 {"value":"goodbye","at":1480475294.0579991}>,
-#     [1] #<SimpleFeed::Event#70138821649420 {"value":"hello","at":1480475294.057138}>
+#     [0] #<SimpleFeed::Event: value=good bye, at=1480475294.0579991>,
+#     [1] #<SimpleFeed::Event: value=hello, at=1480475294.057138>
 # ]
 # Now the unread_count should return 0 since the user just "viewed" the feed.
 activity.unread_count                                     # => 0
 activity.delete(value: 'hello')                           # => true
-activity.total_count                                      # => 1
+# the next method yields to a passed in block for each event in the user's feed, and deletes
+# all events for which the block returns true. The return of this call is the
+# array of all events that have been deleted for this user.
+activity.delete_if do |event, user_id|
+  event.value =~ /good/
+end                                                       
+# => [
+#     [0] #<SimpleFeed::Event: value=good bye, at=1480475294.0579991>
+# ]   
+activity.total_count                                      # => 0
 ```
 
-You can fetch all items in the feed using `#fetch`, and you can
-`#paginate` without resetting the `last_read` timestamp by passing the
-`peek: true` as a parameter.
+You can fetch all items (optionally filtered by time) in the feed using `#fetch`, and you can
+`#paginate` and reset the `last_read` timestamp by passing the `reset_last_read: true` as a parameter.
 
 <a name="batch-api"/>
 
@@ -214,13 +214,13 @@ user.
 end
 ```
 
-##### DSL 
+##### Activity Feed DSL (Domain-Specific Language) 
 
 The library offers a convenient DSL for adding feed functionality into
 your current scope.
 
 To use the module, just include `SimpleFeed::DSL` where needed, which
-exports just one primary method `with_activity'. You call this method
+exports just one primary method `#with_activity`. You call this method
 and pass an activity object created for a set of users (or a single
 user), like so:
 
@@ -252,6 +252,17 @@ with_activity(activity, countries: data_to_store) do
 end
 ```
 
+The DSL context has access to two additional methods: 
+
+ * `#event(value, at)` returns a fully constructed `SimpleFeed::Event` instance
+ * `#color_dump` prints to STDOUT the ASCII text dump of the current user's activities (events), as well as the counts and the `last_read` shown visually on the time line.
+ 
+##### `#color_dump`
+
+Below is an example output of `color_dump` method, which is intended for the debugging purposes.
+
+[<img src="https://raw.githubusercontent.com/kigster/simple-feed/master/man/sf-color-dump.png" width="450" alt="color_dump output" style="width: 300px; max-width:100%;">](https://raw.githubusercontent.com/kigster/simple-feed/master/man/sf-color-dump.png)
+
 <a name="api"/>
 
 ## Complete API
@@ -275,28 +286,30 @@ responses for each user, accessible via `response[user_id]` method.
 @multi.delete(event:)
 # => [Response] { user_id => [Boolean], ... } true if the value was removed, false if it didn't exist
 
-@multi.delete_if do |user_id, event|
-  # if the block returns true, the event is deleted
+@multi.delete_if do |event, user_id|
+  # if the block returns true, the event is deleted and returned 
 end
+# => [Response] { user_id => [deleted_event1, deleted_event2, ...], ... }
 
 # Wipe the feed for a given user(s)
 @multi.wipe
 # => [Response] { user_id => [Boolean], ... } true if user activity was found and deleted, false otherwise
 
 # Return a paginated list of all items, optionally with the total count of items
-@multi.paginate(page:, per_page:, peek: false, with_total: false)
+@multi.paginate(page:, per_page:, with_total: false, reset_last_read: false)
 # => [Response] { user_id => [Array]<Event>, ... }
 # Options:
-#   peek: true — does not reset last_read, otherwise it does.
+#   reset_last_read: false — reset last read to Time.now (true), or the provided timestamp
 #   with_total: true — returns a hash for each user_id:
 #        => [Response] { user_id => { events: Array<Event>, total_count: 3 }, ... } 
 
 # Return un-paginated list of all items, optionally filtered
-@multi.fetch(since: nil)
+@multi.fetch(since: nil, reset_last_read: false)
 # => [Response] { user_id => [Array]<Event>, ... }
 # Options:
+#   reset_last_read: false — reset last read to Time.now (true), or the provided timestamp
 #   since: <timestamp> — if provided, returns all items posted since then
-#   since: :unread — if provided, returns all unread items and resets +last_read+
+#   since: :last_read — if provided, returns all unread items and resets +last_read+
 
 @multi.reset_last_read
 # => [Response] { user_id => [Time] last_read, ... }

data/lib/simplefeed.rb

@@ -7,35 +7,51 @@ require 'simplefeed/providers/redis'
 require 'simplefeed/providers/hash'
 require 'simplefeed/dsl'
 
+# Main namespace module for the SimpleFeed gem. It provides several shortcuts and entry
+# points into the library, such as ability to define and fetch new feeds via +define+,
+# and so on.
 module SimpleFeed
   @registry = {}
 
-  def self.registry
-    @registry
-  end
+  class << self
+    # @return [Hash<Symbol, Feed>] the registry of the defined feeds
+    def registry
+      @registry
+    end
 
-  def self.define(name, **options, &block)
-    name = name.to_sym unless name.is_a?(Symbol)
-    feed = registry[name] ? registry[name] : SimpleFeed::Feed.new(name)
-    feed.configure(options) do
-      block.call(feed) if block
+    # @param name [Symbol] feed name
+    # @param options [Hash] any key-value pairs to set on the feed
+    #
+    # @return [Feed] the feed with the given name, and defined via options and a block
+    def define(name, **options, &block)
+      name = name.to_sym unless name.is_a?(Symbol)
+      feed = registry[name] ? registry[name] : SimpleFeed::Feed.new(name)
+      feed.configure(options) do
+        block.call(feed) if block
+      end
+      registry[name] = feed
+      feed
     end
-    registry[name] = feed
-    feed
-  end
 
-  def self.get(name)
-    registry[name.to_sym]
-  end
+    # @return [Feed] the pre-defined feed with the given name
+    def get(name)
+      registry[name.to_sym]
+    end
 
-  def self.provider(provider_name, *args, **opts, &block)
-    provider_class = SimpleFeed::Providers.registry[provider_name]
-    raise ArgumentError, "No provider named #{provider_name} was found, #{SimpleFeed::Providers.registry.inspect}" unless provider_class
-    provider_class.new(*args, **opts, &block)
-  end
+    # A factory method that constructs an instance of a provider based on the provider name and arguments.
+    #
+    # @param provider_name [Symbol] short name of the provider, eg, :redis, :hash, etc.
+    # @params args [Array] constructor array arguments of the provider
+    # @params opts [Hash] constructor hash arguments of the provider
+    #
+    # @return [Provider]
+    def provider(provider_name, *args, **opts, &block)
+      provider_class = SimpleFeed::Providers.registry[provider_name]
+      raise ArgumentError, "No provider named #{provider_name} was found, #{SimpleFeed::Providers.registry.inspect}" unless provider_class
+      provider_class.new(*args, **opts, &block)
+    end
 
-  class << self
-    # Forward all other method calls to Provider
+    # Forward all other method calls to the Provider
     def method_missing(name, *args, &block)
       registry[name] || super
     end
@@ -47,6 +63,7 @@ module SimpleFeed
     klass.instance_methods.grep(%r{[^=!]=$}).map { |m| m.to_s.gsub(/=/, '').to_sym }
   end
 
+  # Shortcut method to symbolize hash keys, using Hashie::Extensions
   def self.symbolize!(hash)
     Hashie::Extensions::SymbolizeKeys.symbolize_keys!(hash)
   end

data/lib/simplefeed/activity/multi_user.rb

@@ -27,24 +27,26 @@ module SimpleFeed
       # @multi.delete(event:)
       # # => [Response] { user_id => [Boolean], ... } true if the value was removed, false if it didn't exist
       #
-      # @multi.delete_if do |user_id, event|
-      #   # if the block returns true, the event is deleted
+      # @multi.delete_if do |event, user_id|
+      #   # if the block returns true, the event is deleted and returned
       # end
+      # # => [Response] { user_id => [Array]<Event>, ... }
       #
       # @multi.wipe
       # # => [Response] { user_id => [Boolean], ... } true if user activity was found and deleted, false otherwise
       #
-      # @multi.paginate(page:, per_page:, peek: false, with_total: false)
+      # @multi.paginate(page:, per_page:, reset_last_read: [Bool | Time], with_total: false)
       # # => [Response] { user_id => [Array]<Event>, ... }
       # # Options:
-      # #   peek: true — does not reset last_read, otherwise it does.
+      # #   reset_last_read: false — reset last read to Time.now (true), or provided timestamp
       # #   with_total: true — returns a hash for each user_id:
       # #        => [Response] { user_id => { events: Array<Event>, total_count: 3 }, ... }
       #
       # # Return un-paginated list of all items, optionally filtered
-      # @multi.fetch(since: nil)
+      # @multi.fetch(since: nil, reset_last_read: [Bool | Time] )
       # # => [Response] { user_id => [Array]<Event>, ... }
       # # Options:
+      # #   reset_last_read: false — reset last read to Time.now (true), or provided timestamp
       # #   since: <timestamp> — if provided, returns all items posted since then
       # #   since: :unread — if provided, returns all unread items
       #

data/lib/simplefeed/activity/single_user.rb

@@ -34,17 +34,18 @@ module SimpleFeed
       #  @activity.wipe
       #  # => [Boolean] true if user activity was found and deleted, false otherwise
       #
-      #  @activity.paginate(page:, per_page:, peek: false, with_total: false)
+      #  @activity.paginate(page:, per_page:, reset_last_read: false, with_total: false)
       #  # => [Array]<Event>
       #  # Options:
-      #  #   peek: true — does not reset last_read, otherwise it does.
+      #  #   reset_last_read: false — reset last read to Time.now (true), or the provided timestamp
       #  #   with_total: true — returns a hash for each user_id:
       #  #        => { events: Array<Event>, total_count: 3 }
       #
       #  # Return un-paginated list of all items, optionally filtered
-      #  @activity.fetch(since: nil)
+      #  @activity.fetch(since: nil, reset_last_read: false)
       #  # => [Array]<Event>
       #  # Options:
+      #  #   reset_last_read: false — reset last read to Time.now (true), or the provided timestamp
       #  #   since: <timestamp> — if provided, returns all items posted since then
       #  #   since: :unread — if provided, returns all unread items
       #

data/lib/simplefeed/feed.rb

@@ -1,5 +1,7 @@
 require_relative 'providers'
 require_relative 'activity/base'
+require 'simplefeed/key/template'
+
 module SimpleFeed
   class Feed
 
@@ -11,15 +13,19 @@ module SimpleFeed
     end
 
     def initialize(name)
-      @name      = name
-      @name      = name.underscore.to_sym unless name.is_a?(Symbol)
+      @name         = name
+      @name         = name.underscore.to_sym unless name.is_a?(Symbol)
       # set the defaults if not passed in
-      @meta      = {}
-      @namespace = nil
-      @per_page  ||= 50
-      @max_size  ||= 1000
-      @batch_size||= 10
-      @proxy     = nil
+      @meta         = {}
+      @namespace    = nil
+      @per_page     ||= 50
+      @max_size     ||= 1000
+      @batch_size   ||= 10
+      @proxy        = nil
+    end
+
+    def key_template
+      SimpleFeed::Key::Template.new(namespace)
     end
 
     def provider=(definition)
@@ -32,6 +38,10 @@ module SimpleFeed
       @proxy
     end
 
+    def key(user_id)
+      SimpleFeed::Providers::Key.new(user_id, key_template)
+    end
+
     def provider_type
       SimpleFeed::Providers::Base::Provider.class_to_registry(@proxy.provider.class)
     end

data/lib/simplefeed/key/template.rb

@@ -0,0 +1,52 @@
+require 'base62-rb'
+require 'hashie/mash'
+
+module SimpleFeed
+  module Key
+
+    class TextTemplate < Struct.new(:text)
+      def render(params = {})
+        output = self.text.dup
+        params.each_pair do |key, value|
+          output.gsub!(%r[{{\s*#{key}\s*}}], value.to_s)
+        end
+        output
+      end
+    end
+
+    DEFAULT_TEXT_TEMPLATE = TextTemplate.new('{{ namespace }}u.{{ base62_user_id }}.{{ key_marker }}')
+
+    class Template
+      attr_accessor :namespace, :key_types, :text_template
+
+      def initialize(namespace,
+                     key_types = DEFAULT_TYPES,
+                     text_template = DEFAULT_TEXT_TEMPLATE
+      )
+
+        self.namespace     = namespace
+        self.key_types     = key_types
+        self.text_template = text_template
+
+        self.key_types.each do |type|
+          type.template ||= text_template if text_template
+        end
+      end
+
+      def render_options
+        h = {}
+        h.merge!({ 'namespace' => namespace ? "#{namespace}|" : '' })
+        h
+      end
+
+      # Returns array of key names, such as [:meta, :data]
+      def key_names
+        key_types.map(&:name).map(&:to_s).sort
+      end
+
+      private
+
+    end
+  end
+end
+

data/lib/simplefeed/key/type.rb

@@ -0,0 +1,26 @@
+require_relative 'template'
+
+module SimpleFeed
+  module Key
+    class Type < Struct.new(:name, :marker)
+      attr_accessor :template
+
+      def initialize(name, marker, template = nil)
+        super(name, marker)
+        self.template = template
+      end
+
+      def render(opts = {})
+        self.template.render(opts.merge({ 'key_type' => name, 'key_marker' => marker }))
+      end
+    end
+
+    DEFAULT_TYPES = [
+      { name: :data, marker: 'd' },
+      { name: :meta, marker: 'm' }
+    ].map do |type|
+      Type.new(type[:name], type[:marker], DEFAULT_TEXT_TEMPLATE)
+    end
+
+  end
+end

data/lib/simplefeed/providers.rb

@@ -1,4 +1,4 @@
-require_relative 'providers/serialization/key'
+require_relative 'providers/key'
 require_relative 'providers/proxy'
 
 module SimpleFeed
@@ -13,10 +13,6 @@ module SimpleFeed
       self.registry[provider_name] = provider_class
     end
     
-    def self.key(*args)
-      SimpleFeed::Providers::Serialization::Key.new(*args)
-    end
-
     # These methods must be implemented by each Provider, and operation on a given
     # set of users passed via the user_ids: parameter.
     ACTIVITY_METHODS = %i(store delete delete_if wipe reset_last_read last_read paginate fetch total_count unread_count)

data/lib/simplefeed/providers/base/provider.rb

@@ -1,4 +1,4 @@
-require 'simplefeed/providers/serialization/key'
+require 'simplefeed/providers/key'
 
 module SimpleFeed
   module Providers
@@ -28,13 +28,25 @@ module SimpleFeed
 
         protected
 
+        def reset_last_read_value(user_ids:, at: nil)
+          at = [Time, DateTime, Date].include?(at.class) ? at : Time.now
+          at = at.to_time if at.respond_to?(:to_time)
+          at = at.to_f if at.respond_to?(:to_f)
+
+          if self.respond_to?(:reset_last_read)
+            reset_last_read(user_ids: user_ids, at: at)
+          else
+            raise ArgumentError, "Class #{self.class} does not implement #reset_last_read method"
+          end
+        end
+
         def tap(value)
           yield
           value
         end
 
         def key(user_id)
-          ::SimpleFeed::Providers.key(user_id, feed.namespace)
+          feed.key(user_id)
         end
 
         def to_array(user_ids)
@@ -73,11 +85,6 @@ module SimpleFeed
           response
         end
 
-        def with_result
-          result = yield
-          result = transform_response(nil, result) if self.respond_to?(:transform_response)
-          result
-        end
       end
     end
   end

data/lib/simplefeed/providers/hash/provider.rb

@@ -9,7 +9,7 @@ end
 
 require 'simplefeed/event'
 require_relative 'paginator'
-require_relative '../serialization/key'
+require_relative '../key'
 require_relative '../base/provider'
 
 module SimpleFeed
@@ -47,9 +47,14 @@ module SimpleFeed
 
         def delete_if(user_ids:, &block)
           with_response_batched(user_ids) do |key|
-            activity(key).each do |event|
-              __delete(key, event) if yield(key.user_id, event)
-            end
+            activity(key).map do |event|
+              if yield(event, key.user_id)
+                __delete(key, event)
+                event
+              else
+                nil
+              end
+            end.compact
           end
         end
 
@@ -61,27 +66,34 @@ module SimpleFeed
           end
         end
 
-        def fetch(user_ids:, since: nil)
+        def paginate(user_ids:,
+                     page:,
+                     per_page: feed.per_page,
+                     with_total: false,
+                     reset_last_read: false)
+
+          reset_last_read_value(user_ids: user_ids, at: reset_last_read) if reset_last_read
+
           with_response_batched(user_ids) do |key|
+            activity = activity(key)
+            result = (page && page > 0) ? activity[((page - 1) * per_page)...(page * per_page)] : activity
+            with_total ? { events: result, total_count: activity.length } : result
+          end
+        end
+
+        def fetch(user_ids:, since: nil, reset_last_read: false)
+          response = with_response_batched(user_ids) do |key|
             if since == :unread
-              result = activity(key).reject { |event| event.at < user_record(key).last_read.to_f }
-              reset_last_read(user_ids: user_ids)
-              result
+              activity(key).reject { |event| event.at < user_record(key).last_read.to_f }
             elsif since
               activity(key).reject { |event| event.at < since.to_f }
             else
               activity(key)
             end
           end
-        end
+          reset_last_read_value(user_ids: user_ids, at: reset_last_read) if reset_last_read
 
-        def paginate(user_ids:, page:, per_page: feed.per_page, with_total: false, peek: false)
-          reset_last_read(user_ids: user_ids) unless peek
-          with_response_batched(user_ids) do |key|
-            activity = activity(key)
-            result = (page && page > 0) ? activity[((page - 1) * per_page)...(page * per_page)] : activity
-            with_total ? { events: result, total_count: activity.length } : result
-          end
+          response
         end
 
         def reset_last_read(user_ids:, at: Time.now)

data/lib/simplefeed/providers/key.rb

@@ -0,0 +1,72 @@
+require 'base62-rb'
+require 'hashie/mash'
+require 'simplefeed/key/template'
+require 'simplefeed/key/type'
+
+require 'forwardable'
+
+module SimpleFeed
+  module Providers
+    # Here is a meta key for a given user ID:
+    #
+    #            user   'm' for meta
+    #              ↓        ↓
+    #          "ff|u.f23098.m"
+    #           ↑         ↑
+    #         namespace user_id(base62)
+    #
+    class Key
+      attr_accessor :user_id, :key_template
+
+      extend Forwardable
+      def_delegators :@key_template, :key_names, :key_types
+
+      def initialize(user_id, key_template)
+        self.user_id  = user_id
+        self.key_template = key_template
+
+        define_key_methods
+      end
+
+      def define_key_methods
+        key_template.key_types.each do |type|
+          key_name = type.name
+          unless self.respond_to?(key_name)
+            self.class.send(:define_method, key_name) do
+              instance_variable_get("@#{key_name}") ||
+                instance_variable_set("@#{key_name}", type.render(render_options))
+            end
+          end
+
+        end
+      end
+
+      def base62_user_id
+        @base62_user_id ||= ::Base62.encode(user_id)
+      end
+
+      def keys
+        key_names.map { |name| self.send(name) }
+      end
+
+      def render_options
+        key_template.render_options.merge!({
+                                             'user_id'        => user_id,
+                                             'base62_user_id' => base62_user_id
+                                           })
+      end
+
+      def to_s
+        super + { user_id: user_id, base62_user_id: base62_user_id, keys: keys }.to_s
+      end
+
+      def inspect
+        render_options.inspect
+      end
+
+      private
+
+    end
+  end
+end
+

data/lib/simplefeed/providers/redis/provider.rb

@@ -4,7 +4,7 @@ require 'forwardable'
 require 'redis/pipeline' # defines Redis::Future
 
 require 'simplefeed/providers/base/provider'
-require 'simplefeed/providers/serialization/key'
+require 'simplefeed/providers/key'
 
 require_relative 'driver'
 require_relative 'stats'
@@ -51,23 +51,29 @@ module SimpleFeed
         def delete_if(user_ids:)
           raise ArgumentError, '#delete_if must be called with a block that receives (user_id, event) as arguments.' unless block_given?
           with_response_batched(user_ids) do |key|
-            fetch(user_ids: [key.user_id])[key.user_id].each do |event|
+            fetch(user_ids: [key.user_id])[key.user_id].map do |event|
               with_redis do |redis|
-                yield(key.user_id, event) ? redis.zrem(key.data, event.value) : false
+                if yield(event, key.user_id)
+                  redis.zrem(key.data, event.value) ? event : nil
+                end
               end
-            end
+            end.compact
           end
         end
 
         def wipe(user_ids:)
           with_response_pipelined(user_ids) do |redis, key|
-            should_wipe = block_given? ? yield(key.user_id) : true
-            key.keys.all? { |redis_key| redis.del(redis_key) if should_wipe }
+            key.keys.all? { |redis_key| redis.del(redis_key) }
           end
         end
 
-        def paginate(user_ids:, page:, per_page: feed.per_page, peek: false, with_total: false)
-          reset_last_read(user_ids: user_ids) unless peek
+        def paginate(user_ids:, page:,
+                     per_page: feed.per_page,
+                     with_total: false,
+                     reset_last_read: false)
+
+          reset_last_read_value(user_ids: user_ids, at: reset_last_read) if reset_last_read
+
           with_response_pipelined(user_ids) do |redis, key|
             events = paginated_events(page, per_page, redis, key)
             with_total ? { events:      events,
@@ -75,14 +81,14 @@ module SimpleFeed
           end
         end
 
-        def fetch(user_ids:, since: nil)
+        def fetch(user_ids:, since: nil, reset_last_read: false)
           if since == :unread
             last_read_response = with_response_pipelined(user_ids) do |redis, key|
               get_users_last_read(redis, key)
             end
-            reset_last_read(user_ids: user_ids)
           end
-          with_response_pipelined(user_ids) do |redis, key|
+
+          response = with_response_pipelined(user_ids) do |redis, key|
             if since == :unread
               redis.zrevrangebyscore(key.data, '+inf', (last_read_response.delete(key.user_id) || 0).to_f, withscores: true)
             elsif since
@@ -91,6 +97,10 @@ module SimpleFeed
               redis.zrevrange(key.data, 0, -1, withscores: true)
             end
           end
+
+          reset_last_read_value(user_ids: user_ids, at: reset_last_read) if reset_last_read
+
+          response
         end
 
         def reset_last_read(user_ids:, at: Time.now)

data/lib/simplefeed/response.rb

@@ -25,7 +25,7 @@ module SimpleFeed
     end
 
     def for(key_or_user_id, result = nil)
-      user_id = key_or_user_id.is_a?(SimpleFeed::Providers::Serialization::Key) ?
+      user_id = key_or_user_id.is_a?(SimpleFeed::Providers::Key) ?
         key_or_user_id.user_id :
         key_or_user_id
 

data/lib/simplefeed/version.rb

@@ -1,3 +1,3 @@
 module SimpleFeed
-  VERSION = '1.0.4'
+  VERSION = '2.0.1'
 end

data/simple-feed.gemspec

@@ -23,8 +23,8 @@ Gem::Specification.new do |spec|
   spec.require_paths = ['lib']
 
   spec.add_dependency 'base62-rb'
-  spec.add_dependency 'hiredis', '~> 0.6.0'
-  spec.add_dependency 'redis', '>= 3.2'
+  spec.add_dependency 'hiredis'
+  spec.add_dependency 'redis'
   spec.add_dependency 'hashie'
   spec.add_dependency 'connection_pool', '~> 2'
   spec.add_dependency 'activesupport'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: simple-feed
 version: !ruby/object:Gem::Version
-  version: 1.0.4
+  version: 2.0.1
 platform: ruby
 authors:
 - Konstantin Gredeskoul
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-12-13 00:00:00.000000000 Z
+date: 2016-12-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: base62-rb
@@ -28,30 +28,30 @@ dependencies:
   name: hiredis
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 0.6.0
+        version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 0.6.0
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: redis
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '3.2'
+        version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '3.2'
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: hashie
   requirement: !ruby/object:Gem::Requirement
@@ -273,23 +273,26 @@ files:
 - lib/simplefeed/dsl/formatter.rb
 - lib/simplefeed/event.rb
 - lib/simplefeed/feed.rb
+- lib/simplefeed/key/template.rb
+- lib/simplefeed/key/type.rb
 - lib/simplefeed/providers.rb
 - lib/simplefeed/providers/base/provider.rb
 - lib/simplefeed/providers/hash.rb
 - lib/simplefeed/providers/hash/paginator.rb
 - lib/simplefeed/providers/hash/provider.rb
+- lib/simplefeed/providers/key.rb
 - lib/simplefeed/providers/proxy.rb
 - lib/simplefeed/providers/redis.rb
 - lib/simplefeed/providers/redis/boot_info.yml
 - lib/simplefeed/providers/redis/driver.rb
 - lib/simplefeed/providers/redis/provider.rb
 - lib/simplefeed/providers/redis/stats.rb
-- lib/simplefeed/providers/serialization/key.rb
 - lib/simplefeed/response.rb
 - lib/simplefeed/version.rb
 - man/activity-feed-action.png
 - man/running-example-redis-debug.png
 - man/running-example.png
+- man/sf-color-dump.png
 - simple-feed.gemspec
 homepage: https://github.com/kigster/simple-feed
 licenses:

data/lib/simplefeed/providers/serialization/key.rb

@@ -1,82 +0,0 @@
-require 'base62-rb'
-require 'hashie/mash'
-module SimpleFeed
-  module Providers
-    module Serialization
-      class Key
-        attr_accessor :user_id, :short_id, :prefix, :config
-
-        # This hash defines the paramters for the strategy we use
-        # to create a compact key based on the user id, feed's namespace,
-        # and the functionality, such as 'm' meta — as a hash for storing
-        # arbitrary values (in particular, +last_read+). and 'd' data —
-        # as a sorted set in for storing the actual events.
-        #
-        # ### Examples
-        #
-        # Here is a meta key for a given user ID:
-        #
-        #            user   'm' for meta
-        #              ↓        ↓
-        #          "ff|u.f23098.m"
-        #           ↑         ↑
-        #         namespace user_id(base62)
-        #
-        KEY_CONFIG = Hashie::Mash.new({
-                                        separator:         '.',
-                                        prefix:            '',
-                                        namespace_divider: '|',
-                                        namespace:         nil,
-                                        primary:           ->(user_id) { ::Base62.encode(user_id) },
-                                        primary_marker:    'u',
-                                        secondary_markers: {
-                                          data: 'd',
-                                          meta: 'm'
-                                        }
-                                      })
-
-        def initialize(user_id, namespace = nil, optional_key_config = {})
-          optional_key_config.merge!(namespace: namespace) if namespace
-          self.config = KEY_CONFIG.dup.merge!(optional_key_config)
-
-          self.user_id  = user_id
-          self.short_id = config.primary[user_id]
-
-          self.prefix = configure_prefix
-
-          config.secondary_markers.each_pair do |type, character|
-            self.class.send(:define_method, type) do
-              instance_variable_get("@#{type}") || instance_variable_set("@#{type}", "#{prefix}#{config.separator}#{character}")
-            end
-          end
-        end
-
-        def keys
-          config.secondary_markers.map { |k, v| [k, self.send(k)] }
-        end
-
-        def for(type)
-          "#{prefix}#{config.separator}#{type.to_s}"
-        end
-
-        def to_s
-          super.gsub(/SimpleFeed::Providers::Serialization/, '...*') + { user_id: user_id, short_id: short_id, keys: keys }.to_json
-        end
-
-        def inspect
-
-        end
-
-        private
-
-        # eg. ff|u.123498
-        def configure_prefix
-          namespace = config.namespace ? "#{config.namespace}#{config.namespace_divider}" : ''
-          "#{namespace}#{config.primary_marker}#{config.separator}#{short_id}"
-        end
-
-      end
-    end
-  end
-end
-