spinnaker 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 10e502553f469597a2bcf82321040b8d467323acd0406e79df00e1e53b451535
-  data.tar.gz: cf251bff4735e9468139d96af86d0b8bfb4fab3184910ea0816bdfb1d6836934
+  metadata.gz: 29b7a91a929c2b0a3fce2b6b97059043b4c0522698d29da535e8b76189ccd4b5
+  data.tar.gz: 3d66013517a6a029006b5ca6ffc9cb6bac2ed685fd77a64b7b577eec97ddb1a3
 SHA512:
-  metadata.gz: 7e07368552e7944d464268d8667592b9ccb599177cd40c7e8e2b4f58b426ee36335bc29d4ea0b6680d175b8c23c6ec3b94076d99363c65cb88180260d7075294
-  data.tar.gz: e61f70fbbcf6ec1cee84f1d8ed17ec6ff2cd6b5dc06e23fa2b6b1d8815441f4f7903fe78490cc70fe7e69f5c32d1a8d8670bbdb18e25ee726e2e7c2c97a0a760
+  metadata.gz: 217c7cfcb4403d8e055d1a5802566f0a14e3e427d24a740ffa383b93450366dddb8b62605cac851b1178662827bf8f8c9515311a3fa007baa2c84de90e119fbe
+  data.tar.gz: 5161e790d2e11ebf9dac1e60b33346da5f95f3bca185e613fe7b2a7313394cea249efba02f30f209af97a80b554b0152073f5bff6c4fa823827e5df888ab2beb

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.rubocop.yml

@@ -1,6 +1,21 @@
 AllCops:
   TargetRubyVersion: 2.6
 
+Metrics/AbcSize:
+  Enabled: true
+  Exclude:
+    - lib/spinnaker/api.rb
+
+Metrics/MethodLength:
+  Enabled: true
+  Exclude:
+    - lib/spinnaker/api.rb
+
+Style/ClassVars:
+  Enabled: true
+  Exclude:
+    - lib/spinnaker.rb
+
 Style/StringLiterals:
   Enabled: true
   EnforcedStyle: double_quotes

data/CHANGELOG.md

@@ -0,0 +1,19 @@
+## [0.1.0] - 2023-10-23
+
+This is the initial release.
+
+### Added
+
+- Ability to track page visits
+- Tracks the following info:
+    - Path
+    - Title
+    - Meta info (is it the index, a feed, etc.)
+    - Exists (whether or not the path is valid according to the site)
+    - Visits (the number of visits, as well as the time of each visit)
+- Serves page data over an API
+- Query the API over a specified time period:
+    - Default is to get page info for the last 24 hours
+    - /today: page data for the current day
+    - /week: page data for the current week
+    - /month: page data for the current month

data/README.md

@@ -1,34 +1,70 @@
 # Spinnaker
 
-TODO: Delete this and the text below, and describe your gem
-
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/spinnaker`. To experiment with that code, run `bin/console` for an interactive prompt.
+Spinnaker is a drop-in tool for collecting and serving website page metrics. It's designed for use with [Jekyll](https://github.com/jekyll/jekyll) (and [Lanyon](https://github.com/stomar/lanyon)) in mind, but since it's just middleware for [Rack](https://github.com/rack/rack), it will work for any compatible frameworks. Spinnaker is quite private out of the box, as it only tracks what pages are being requested, and at what times those requests occur.
 
 ## Installation
 
-TODO: Replace `UPDATE_WITH_YOUR_GEM_NAME_PRIOR_TO_RELEASE_TO_RUBYGEMS_ORG` with your gem name right after releasing it to RubyGems.org. Please do not do it earlier due to security reasons. Alternatively, replace this section with instructions to install your gem from git if you don't plan to release to RubyGems.org.
+To add spinnaker to your project, run:
+```sh
+bundle add spinnaker
+```
+
+You could also install it manuall with `gem install spinnaker`.
+
+Then `use` it in your config.ru file.
+```ruby
+# config.ru
+
+require "lanyon"
+require "spinnaker"
 
-Install the gem and add to the application's Gemfile by executing:
+use Spinnaker
+run Lanyon.application
+```
 
-    $ bundle add UPDATE_WITH_YOUR_GEM_NAME_PRIOR_TO_RELEASE_TO_RUBYGEMS_ORG
+If needed you can configure some settings before `use`ing it.
+```ruby
+# config.ru
 
-If bundler is not being used to manage dependencies, install the gem by executing:
+require "lanyon"
+require "spinnaker"
 
-    $ gem install UPDATE_WITH_YOUR_GEM_NAME_PRIOR_TO_RELEASE_TO_RUBYGEMS_ORG
+Spinnaker.new.tap do |s|
+    # Set the endpoint where spinnaker will serve metrics.
+    # The default value is '/spinnaker'.
+    s.endoint = "stats"
+
+    # Set the address (or address range) to listen on the metrics endpoint.
+    # Accepts CIDR range syntax (i.e. 192.168.1.0/24).
+    # Accepts an array of addresses, or simply a single string.
+    # The default value is ['127.0.0.1'].
+    s.listen = ["127.0.0.1", "192.168.1.0/24"]
+end
+
+use Spinnaker
+run Lanyon.application
+```
 
 ## Usage
 
-TODO: Write usage instructions here
+Right now, you can query data over a specified time period on the API. Asumming the endpoint is '/spinnaker':
+| Request Type | Path | Data |
+| - | - | - |
+| GET | /spinnaker{/latest} | Get page data for the last 24 hours |
+| GET | /spinnaker/today | Get page data for the current day |
+| GET | /spinnaker/week | Get page data for the current week |
+| GET | /spinnaker/month | Get page data for the current month |
+
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/spinnaker.
+Bug reports and pull requests are welcome on GitHub at https://github.com/pinecat/spinnaker.
 
 ## License
 

data/Rakefile

@@ -1,16 +1,12 @@
 # frozen_string_literal: true
 
 require "bundler/gem_tasks"
-require "rake/testtask"
+require "rspec/core/rake_task"
 
-Rake::TestTask.new(:test) do |t|
-  t.libs << "test"
-  t.libs << "lib"
-  t.test_files = FileList["test/**/test_*.rb"]
-end
+RSpec::Core::RakeTask.new(:spec)
 
 require "rubocop/rake_task"
 
 RuboCop::RakeTask.new
 
-task default: %i[test rubocop]
+task default: %i[spec rubocop]

data/config.ru

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require_relative "lib/spinnaker"
+
+use Spinnaker
+run do |_env|
+  [200, {}, ["<head><title>The Expanse</title></head><body><p>Welcome to my blog!</p></body>"]]
+end

data/lib/spinnaker/api.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+class Spinnaker
+  # API endpoints for querying the page data.
+  class API
+    # Initializer to setup the the endpoints hash and define routes.
+    def initialize
+      @endpoints = {}
+      routes
+    end
+
+    # Called with a paht to retrieve data from one of the routes.
+    def handle(endpoint)
+      blk = @endpoints[endpoint]
+      return [404, {}, ["Not found"]] if blk.nil?
+
+      [200, {}, [blk.call]]
+    end
+
+    private
+
+    # Simple DSL to define an API route.
+    def route(*eps, &block)
+      eps.each do |ep|
+        @endpoints[ep] = block
+      end
+    end
+
+    # Define all API routes here (TODO: Need a better solution for this)
+    # TODO: These endpoints could be auto-generated with a list of timeframes/periods
+    def routes
+      # /latest, /
+      # Default API endpoint, serves metrics for the last 24 hours.
+      route "", "latest" do
+        period = :the_last_24_hours
+        {
+          visits: Visit.where("timestamp > ?", Helper.period(period)).count,
+          pages: Helper.latest(period)
+        }.to_json
+      end
+
+      # /today
+      # Serves metrics for the current day.
+      route "today" do
+        period = :today
+        {
+          visits: Visit.where("timestamp > ?", Helper.period(period)).count,
+          pages: Helper.latest(period).as_json(period)
+        }.to_json
+      end
+
+      # /week
+      # Serves metrics for the current day.
+      route "week" do
+        period = :this_week
+        {
+          visits: Visit.where("timestamp > ?", Helper.period(period)).count,
+          pages: Helper.latest(period).as_json(period)
+        }.to_json
+      end
+
+      # /month
+      # Serves metrics for the current day.
+      route "week" do
+        period = :this_month
+        {
+          visits: Visit.where("timestamp > ?", Helper.period(period)).count,
+          pages: Helper.latest(period).as_json(period)
+        }.to_json
+      end
+    end
+  end
+end

data/lib/spinnaker/migrate.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+class Spinnaker # :nodoc:
+  # return [String] The name of the pages table.
+  PAGES_TABLE = :spinnaker_pages
+
+  # return [String] The name of the visits table.
+  VISITS_TABLE = :spinnaker_visits
+
+  # Migrations for the Spinnaker table, which keeps track of page hits.
+  class SpinnakerMigrations < ActiveRecord::Migration[7.1]
+    def create_pages
+      create_table PAGES_TABLE do |t|
+        t.string   :path, default: "", null: false
+        t.string   :title, default: nil
+        t.string   :meta, default: nil
+        t.boolean  :exists, default: false
+      end
+
+      add_index PAGES_TABLE, :path
+    end
+
+    def create_visits
+      create_table VISITS_TABLE do |t|
+        t.datetime :timestamp, null: false
+      end
+
+      add_reference VISITS_TABLE, :page, null: false, foreign_key: { to_table: PAGES_TABLE }
+    end
+  end
+end

data/lib/spinnaker/models/helper.rb

@@ -0,0 +1,23 @@
+class Spinnaker # :nodoc:
+  # Helper methods for the models
+  module Helper
+    def self.latest(timeframe)
+      vs = Visit.where("timestamp > ?", period(timeframe))
+      pages = []
+      vs.each do |v|
+        pages << Page.find(v.page_id)
+      end
+
+      pages.uniq
+    end
+
+    def self.period(timeframe)
+      case timeframe
+      when :today then Date.today.beginning_of_day
+      when :this_week then Date.today.beginning_of_week - 1.day
+      when :this_month then Date.today.beginning_of_month
+      else (Time.now - 1.day).beginning_of_hour
+      end
+    end
+  end
+end

data/lib/spinnaker/models/page.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+class Spinnaker # :nodoc:
+  # Model to access data from the 'spinnaker_pages' table.
+  class Page < ActiveRecord::Base
+    #
+    # Database info]
+    #
+    self.table_name = PAGES_TABLE
+    self.primary_key = "id"
+
+    #
+    # Relationships
+    #
+
+    # Visits track the specific time the page was requested.
+    has_many :visits
+
+    #
+    # Aliases
+    #
+
+    # Needed for the relation with visits to work properly.
+    alias_attribute :page_id, :id
+
+    #
+    # Functions
+    #
+
+    # Get when the last visit to the page was.
+    def last_visit
+      visits.last.timestamp.localtime.to_s
+    end
+
+    # Get visits for a specified timeframe:
+    #   :the_last_24_hours
+    #   :today
+    #   :this_week
+    #   :this_month
+    #   :this_year
+    def visits_for(timeframe)
+      visits.where("timestamp > ?", Helper.period(timeframe)).count
+    end
+
+    # Override the default as_json method.
+    def as_json(*options)
+      {
+        path: path,
+        title: title,
+        meta: meta,
+        exists: exists,
+        visits: visits_for(options[0]),
+        last: last_visit
+      }
+    end
+  end
+end

data/lib/spinnaker/models/visit.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+class Spinnaker # :nodoc:
+  # Model to access data from the 'spinnaker_visits' table.
+  class Visit < ActiveRecord::Base
+    self.table_name = VISITS_TABLE
+    self.primary_key = "id"
+    belongs_to :page
+  end
+end

data/lib/spinnaker/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
-module Spinnaker
-  VERSION = "0.0.1"
+class Spinnaker
+  VERSION = "0.1.0"
 end

data/lib/spinnaker.rb

@@ -1,8 +1,102 @@
 # frozen_string_literal: true
 
+require "active_record"
+require "json"
+require "nokogiri"
+require "rack"
+require "yaml"
+
+require_relative "spinnaker/api"
+require_relative "spinnaker/migrate"
+require_relative "spinnaker/models/helper"
+require_relative "spinnaker/models/page"
+require_relative "spinnaker/models/visit"
 require_relative "spinnaker/version"
 
-module Spinnaker
+# Spinnaker is a rack middleware used to log page hits to a SQL database.
+class Spinnaker
+  # Error template.
   class Error < StandardError; end
-  # Your code goes here...
+
+  # return [String] CIDR range to listen on for the API, default is ['127.0.0.1'].
+  @@listen = [IPAddr.new("127.0.0.1")]
+
+  # return [String] Endpoint for the API, default is 'spinnaker'.
+  @@endpoint = "spinnaker"
+
+  def initialize(app = nil)
+    database
+    @api = API.new
+    @app = app
+  end
+
+  def call(env)
+    status, headers, body = @app.call(env)
+    req = Rack::Request.new(env)
+
+    if req.path.start_with? "/#{@@endpoint}"
+      return [403, {}, ["Forbidden"]] unless its_from_a_valid_ip(req.ip)
+
+      return @api.handle(req.path.delete_prefix("/#{@@endpoint}").delete_prefix("/"))
+    end
+
+    record(req.path, status, body)
+    [status, headers, body]
+  end
+
+  def listen=(val)
+    @@listen.clear
+    if val.instance_of?(Array)
+      val.each do |v|
+        @@listen << IPAddr.new(v)
+      end
+    else
+      @@listen << IPAddr.new(val)
+    end
+  end
+
+  def endpoint=(val)
+    @@endpoint = val
+  end
+
+  private
+
+  def database
+    conn = ENV.fetch("DATABASE_URL", { adapter: "sqlite3", database: "db/test.db" })
+    ActiveRecord::Base.establish_connection(conn)
+    SpinnakerMigrations.migrate(:create_pages) unless ActiveRecord::Base.connection.table_exists? PAGES_TABLE
+    SpinnakerMigrations.migrate(:create_visits) unless ActiveRecord::Base.connection.table_exists? VISITS_TABLE
+  end
+
+  def its_from_a_valid_ip(ip)
+    @@listen.each do |addr|
+      return true if addr.include? IPAddr.new(ip)
+    end
+
+    false
+  end
+
+  def record(path, status, body)
+    # Check if the page exists
+    exists = (status >= 400 && status < 500 ? false : true)
+
+    # Create an entry in the database
+    page = Page.find_or_create_by!(path: path)
+    page.update!(title: Nokogiri::HTML(body[0]).xpath("//title")&.text, exists: exists)
+    Visit.create(page: page, timestamp: Time.now)
+
+    # Determine if there is any meta data to add for the page
+    metainfo(page, path)
+  end
+
+  def metainfo(page, path)
+    meta = nil
+    if ["/", ""].include?(path)
+      meta = "Index"
+    elsif path.include?(".xml") || path.include?(".atom")
+      meta = "Feed"
+    end
+
+    page.update!(meta: meta)
+  end
 end

metadata

@@ -1,30 +1,37 @@
 --- !ruby/object:Gem::Specification
 name: spinnaker
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.1.0
 platform: ruby
 authors:
 - Rory Dudley
-autorequire:
+autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-07-28 00:00:00.000000000 Z
+date: 2023-10-23 00:00:00.000000000 Z
 dependencies: []
-description: A simple server that can serve Duck apps (https://rubygems.org/gems/duck).
+description: Track page hits for rack apps and store in a database.
 email:
 - rory.dudley@gmail.com
 executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- ".rspec"
 - ".rubocop.yml"
+- CHANGELOG.md
 - LICENSE.txt
 - README.md
 - Rakefile
+- config.ru
 - lib/spinnaker.rb
+- lib/spinnaker/api.rb
+- lib/spinnaker/migrate.rb
+- lib/spinnaker/models/helper.rb
+- lib/spinnaker/models/page.rb
+- lib/spinnaker/models/visit.rb
 - lib/spinnaker/version.rb
 - sig/spinnaker.rbs
-- spinnaker.gemspec
 homepage: https://github.com/pinecat/spinnaker
 licenses:
 - MIT
@@ -33,7 +40,7 @@ metadata:
   homepage_uri: https://github.com/pinecat/spinnaker
   source_code_uri: https://github.com/pinecat/spinnaker
   changelog_uri: https://github.com/pinecat/spinnaker
-post_install_message:
+post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
@@ -48,8 +55,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.17
-signing_key:
+rubygems_version: 3.4.21
+signing_key: 
 specification_version: 4
-summary: A server for Ruby/Duck applications.
+summary: Track page hits for rack apps.
 test_files: []

data/spinnaker.gemspec

@@ -1,40 +0,0 @@
-# frozen_string_literal: true
-
-require_relative "lib/spinnaker/version"
-
-Gem::Specification.new do |spec|
-  spec.name = "spinnaker"
-  spec.version = Spinnaker::VERSION
-  spec.authors = ["Rory Dudley"]
-  spec.email = ["rory.dudley@gmail.com"]
-
-  spec.summary = "A server for Ruby/Duck applications."
-  spec.description = "A simple server that can serve Duck apps (https://rubygems.org/gems/duck)."
-  spec.homepage = "https://github.com/pinecat/spinnaker"
-  spec.license = "MIT"
-  spec.required_ruby_version = ">= 2.6.0"
-
-  spec.metadata["allowed_push_host"] = "https://rubygems.org"
-
-  spec.metadata["homepage_uri"] = spec.homepage
-  spec.metadata["source_code_uri"] = "https://github.com/pinecat/spinnaker"
-  spec.metadata["changelog_uri"] = "https://github.com/pinecat/spinnaker"
-
-  # Specify which files should be added to the gem when it is released.
-  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
-  spec.files = Dir.chdir(__dir__) do
-    `git ls-files -z`.split("\x0").reject do |f|
-      (File.expand_path(f) == __FILE__) ||
-        f.start_with?(*%w[bin/ test/ spec/ features/ .git .circleci appveyor Gemfile])
-    end
-  end
-  spec.bindir = "exe"
-  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
-  spec.require_paths = ["lib"]
-
-  # Uncomment to register a new dependency of your gem
-  # spec.add_dependency "example-gem", "~> 1.0"
-
-  # For more information and examples about making a new gem, check out our
-  # guide at: https://bundler.io/guides/creating_gem.html
-end