content_signals 0.1.0 → 0.1.1

data/READY_TO_GEM.md

@@ -0,0 +1,375 @@
+# ✅ Ready to Extract into Gem
+
+## What You Already Have (90% Complete)
+
+### ✅ Complete Specification
+- [COUNTERS.MD](COUNTERS.MD) - 1,500+ lines of detailed implementation
+- All services designed and documented
+- Database schema with migrations
+- API endpoints defined
+- Analytics queries specified
+- Testing requirements outlined
+
+### ✅ Core Features Designed
+1. **PageViewTracker Service** - Visitor identification, data compilation, Redis deduplication
+2. **TrackPageViewJob** - Background async processing with retry logic
+3. **VisitorLocationService** - MaxMind GeoLite2 integration
+4. **DeviceDetectorService** - Browser gem, hybrid app detection
+5. **AnalyticsQueryService** - Top countries, cities, devices, time series
+6. **PageView Model** - Polymorphic trackable, tenant scoping, analytics methods
+7. **Trackable Concern** - Include in any model (Page, Profile, Event)
+8. **Configuration DSL** - Multitenancy, Redis, MaxMind setup
+
+### ✅ Multi-Tenant Support
+- Optional `tenant_id` on all tables
+- Automatic tenant scoping with default_scope
+- ActsAsTenant & Apartment gem compatibility
+- Tenant-aware Redis keys
+- Cross-tenant admin analytics
+
+### ✅ Hybrid Mobile App Support
+- Capacitor, Cordova, React Native, Flutter detection
+- Custom headers: X-App-Platform, X-App-Version, X-Device-ID
+- URL parameter fallbacks
+- WebView user agent detection
+- Persistent device_id tracking
+
+### ✅ Architecture Decisions Made
+- Redis for 24-hour unique visitor deduplication (optional)
+- PostgreSQL for persistent PageView records
+- Background jobs for async processing
+- Counter caching for performance
+- Bot/crawler exclusion
+- Admin user exclusion (optional)
+
+### ✅ Performance Validated
+- Redis: 1,000 requests/second (only 0.2-1% capacity)
+- Background jobs: Non-blocking
+- Counter cache: Instant `page_views_count` access
+- Indexed queries: Fast analytics
+- Optional Redis: Works PostgreSQL-only
+
+---
+
+## What Needs to Be Done (10% Remaining)
+
+### 🔨 Implementation Tasks (10-12 hours)
+
+#### 1. Gem Scaffolding (30 minutes)
+```bash
+bundle gem modest_analytics --test=rspec --ci=github --mit
+```
+- Update gemspec (dependencies, description)
+- Create lib/modest_analytics.rb entry point
+- Create lib/modest_analytics/engine.rb Rails engine
+- Set version to 0.1.0
+
+#### 2. Copy Code from COUNTERS.MD (4 hours)
+Simply copy these 8 components from specification to gem:
+
+| Component | From COUNTERS.MD | To Gem |
+|-----------|------------------|---------|
+| PageView model | Lines 990-1050 | lib/modest_analytics/models/page_view.rb |
+| PageViewTracker | Lines 860-940 | lib/modest_analytics/services/page_view_tracker.rb |
+| TrackPageViewJob | Lines 950-980 | lib/modest_analytics/jobs/track_page_view_job.rb |
+| VisitorLocationService | Lines 420-480 | lib/modest_analytics/services/visitor_location_service.rb |
+| DeviceDetectorService | Lines 570-620 | lib/modest_analytics/services/device_detector_service.rb |
+| AnalyticsQueryService | Lines 1090-1180 | lib/modest_analytics/services/analytics_query_service.rb |
+| Trackable concern | Lines 1040-1070 | lib/modest_analytics/concerns/trackable.rb |
+| Configuration | Lines 172-189 | lib/modest_analytics/configuration.rb |
+
+**No new code needed** - just copy & paste with minor namespace adjustments!
+
+#### 3. Migrations (30 minutes)
+Copy migration from COUNTERS.MD lines 700-750:
+- `db/migrate/001_create_modest_analytics_page_views.rb`
+
+#### 4. Generator (1 hour)
+Create `lib/generators/modest_analytics/install_generator.rb`:
+- Copy migrations to app
+- Create initializer
+- Display setup instructions
+
+#### 5. Tests (3 hours)
+Write RSpec tests for:
+- PageView model (scopes, methods)
+- All 4 services (mocked)
+- Background job
+- Integration test (end-to-end)
+
+#### 6. Documentation (2 hours)
+- README.md (installation, quick start, examples)
+- CHANGELOG.md (v0.1.0 release notes)
+- docs/CONFIGURATION.md
+- docs/MULTITENANCY.md
+- docs/HYBRID_APPS.md
+
+#### 7. Polish & Release (1 hour)
+- RuboCop code style
+- Test in Stejar locally
+- Build & publish to RubyGems
+- Push to GitHub
+- Create v0.1.0 release
+
+---
+
+## File-by-File Mapping
+
+### From COUNTERS.MD → Gem Structure
+
+```
+COUNTERS.MD (1,500 lines)
+├── Lines 1-80: Overview & Requirements
+│   → README.md introduction
+│
+├── Lines 172-278: Multi-Tenant Support
+│   → lib/modest_analytics/configuration.rb
+│   → docs/MULTITENANCY.md
+│
+├── Lines 280-380: Hybrid Mobile App Support
+│   → docs/HYBRID_APPS.md
+│
+├── Lines 382-540: Geolocation Services
+│   → lib/modest_analytics/services/visitor_location_service.rb
+│
+├── Lines 570-620: Device Detection
+│   → lib/modest_analytics/services/device_detector_service.rb
+│
+├── Lines 700-750: Database Migrations
+│   → db/migrate/001_create_modest_analytics_page_views.rb
+│
+├── Lines 860-940: PageViewTracker
+│   → lib/modest_analytics/services/page_view_tracker.rb
+│
+├── Lines 950-980: TrackPageViewJob
+│   → lib/modest_analytics/jobs/track_page_view_job.rb
+│
+├── Lines 990-1070: PageView Model + Trackable Concern
+│   → lib/modest_analytics/models/page_view.rb
+│   → lib/modest_analytics/concerns/trackable.rb
+│
+├── Lines 1090-1180: AnalyticsQueryService
+│   → lib/modest_analytics/services/analytics_query_service.rb
+│
+└── Lines 1400-1461: Analytics Queries Examples
+    → README.md examples section
+```
+
+---
+
+## Dependencies Already Defined
+
+### Required
+```ruby
+# In gemspec
+spec.add_dependency "rails", ">= 7.0"
+spec.add_dependency "maxmind-geoip2"  # IP geolocation
+spec.add_dependency "browser"          # Device detection
+```
+
+### Optional (Peer Dependencies)
+```ruby
+# User's Gemfile
+gem 'redis'          # For unique visitor deduplication
+gem 'acts_as_tenant' # For multitenancy (optional)
+gem 'apartment'      # Alternative multitenancy (optional)
+```
+
+---
+
+## Quick Start Guide (After Publishing)
+
+### 1. Install in Stejar CMS
+
+```ruby
+# Gemfile
+gem 'modest_analytics'
+
+# Terminal
+bundle install
+rails generate modest_analytics:install
+rails db:migrate
+
+# app/models/stejar/page.rb
+module Stejar
+  class Page < ApplicationRecord
+    include ModestAnalytics::Trackable
+  end
+end
+
+# app/controllers/stejar/pages_controller.rb
+def show
+  @page = Stejar::Page.find(params[:id])
+  @page.track_view(user: current_user, request: request)
+end
+
+# app/views/stejar/pages/show.html.erb
+<p><%= @page.page_views_count %> views</p>
+<p><%= @page.analytics.unique_views_today %> unique today</p>
+```
+
+### 2. Install in Eventya
+
+```ruby
+# app/models/profile.rb
+class Profile < ApplicationRecord
+  include ModestAnalytics::Trackable
+end
+
+# app/models/event.rb
+class Event < ApplicationRecord
+  include ModestAnalytics::Trackable
+end
+
+# Controllers
+@profile.track_view(user: current_user, request: request)
+@event.track_view(user: current_user, request: request)
+
+# Views
+<%= @profile.page_views_count %> profile views
+<%= @event.analytics.top_countries.first %> most popular country
+```
+
+---
+
+## Testing Strategy
+
+### Unit Tests (50% of testing time)
+```ruby
+# spec/models/page_view_spec.rb
+describe ModestAnalytics::PageView do
+  it { should belong_to(:trackable) }
+  it { should belong_to(:tenant).optional }
+  it { should belong_to(:user).optional }
+
+  describe '.today' do
+    # Test scope
+  end
+
+  describe '.unique_count' do
+    # Test analytics method
+  end
+end
+
+# spec/services/page_view_tracker_spec.rb
+describe ModestAnalytics::PageViewTracker do
+  describe '#track' do
+    it 'increments page_views_count'
+    it 'enqueues background job'
+    it 'tracks unique view if first today'
+    it 'skips duplicate view same day'
+  end
+
+  describe '#identify_visitor' do
+    it 'prefers user_id'
+    it 'falls back to device_id'
+    it 'falls back to cookie'
+    it 'falls back to IP hash'
+  end
+end
+```
+
+### Integration Tests (30% of testing time)
+```ruby
+# spec/integration/tracking_spec.rb
+describe 'End-to-end tracking' do
+  it 'tracks page view from web browser' do
+    page = create(:page)
+    user = create(:user)
+
+    # Simulate controller action
+    request = mock_request(ip: '1.2.3.4', user_agent: 'Mozilla...')
+    page.track_view(user: user, request: request)
+
+    expect(page.page_views_count).to eq(1)
+    expect(TrackPageViewJob).to have_been_enqueued
+
+    # Process job
+    perform_enqueued_jobs
+
+    page_view = ModestAnalytics::PageView.last
+    expect(page_view.trackable).to eq(page)
+    expect(page_view.user).to eq(user)
+    expect(page_view.country_code).to be_present
+  end
+
+  it 'tracks from Capacitor mobile app' do
+    # Test hybrid app detection
+  end
+
+  it 'respects tenant isolation' do
+    # Test multitenancy
+  end
+end
+```
+
+### Manual Testing (20% of testing time)
+1. Install in dummy Rails app
+2. Track views on test pages
+3. Check analytics queries
+4. Test MaxMind download
+5. Test Redis caching
+6. Test hybrid app headers
+
+---
+
+## Success Metrics
+
+### Technical Metrics
+- [ ] All tests passing (RSpec green)
+- [ ] Code coverage >80%
+- [ ] RuboCop violations: 0
+- [ ] Gem builds successfully
+- [ ] Installs in Stejar without errors
+- [ ] Installs in Eventya without errors
+
+### Release Metrics
+- [ ] Published to RubyGems.org
+- [ ] GitHub repo with README
+- [ ] v0.1.0 release created
+- [ ] License file (MIT) included
+- [ ] Changelog with release notes
+
+### Usage Metrics (First Week)
+- [ ] Stejar using gem in production
+- [ ] Eventya using gem in production
+- [ ] 0-1 GitHub issues opened (low bug count)
+- [ ] 5+ stars on GitHub (if announced)
+
+---
+
+## Risk Assessment
+
+### Low Risk ✅
+- **Code quality** - Already designed, just needs copying
+- **Dependencies** - Minimal, well-maintained gems
+- **Scope** - Small, focused gem (single responsibility)
+- **Use cases** - Clear, well-defined (Stejar, Eventya)
+
+### Medium Risk ⚠️
+- **Maintenance** - Will need occasional updates (1-2 hours/month)
+- **Support** - GitHub issues from other users
+- **Documentation** - Must be clear for external users
+- **Breaking changes** - Need semantic versioning
+
+### Mitigation Strategies
+1. **Set expectations** - README "nights-and-weekends project"
+2. **Issue templates** - Guide users to self-service
+3. **Close stale issues** - 30 days without response
+4. **Encourage PRs** - "Code contributions welcome"
+5. **Version carefully** - Use semver, document breaking changes
+
+---
+
+## Decision Matrix
+
+| Factor | Build Now | Extract Later |
+|--------|-----------|---------------|
+| **Time to production** | 2 days + gem | Immediate (in Stejar) |
+| **Code reuse** | ✅ Single gem | ❌ Duplicate in Eventya |
+| **Maintenance** | ⚠️ GitHub issues | ✅ Internal only |
+| **Learning** | ✅ Gem authoring | ❌ No new skills |
+| **Community impact** | ✅ Help others | ❌ Private code |
+| **Resume value** | ✅ Open source | ❌ Not visible |
+| **Flexibility** | ⚠️ Breaking changes hard | ✅ Easy iteration |
+| **Testing** | ✅ Community feedback | ⚠️ Only your use cases |

data/STRUCTURE.md

@@ -0,0 +1,108 @@
+# Content Signals Gem - Project Structure
+
+## ✅ Completed Setup
+
+### Directory Structure
+```
+lib/
+├── content_signals.rb              # Main entry point
+├── content_signals/
+│   ├── version.rb                  # Version constant
+│   ├── configuration.rb            # Configuration DSL
+│   ├── engine.rb                   # Rails engine integration
+│   ├── models/                     # ActiveRecord models
+│   ├── services/                   # Business logic services
+│   ├── concerns/                   # ActiveSupport concerns
+│   └── jobs/                       # ActiveJob background jobs
+└── generators/
+    └── content_signals/            # Rails generators
+
+spec/
+├── spec_helper.rb                  # Basic RSpec config
+├── rails_helper.rb                 # Rails + DB config
+├── support/
+│   ├── schema.rb                   # Test database schema
+│   ├── models.rb                   # Test models (Page, Profile, User)
+│   └── request_helpers.rb          # Mock request helpers
+├── models/                         # Model specs
+├── services/                       # Service specs
+└── jobs/                          # Job specs
+```
+
+### Dependencies Installed
+- **Rails** >= 7.0
+- **maxmind-geoip2** ~> 1.1 (IP geolocation)
+- **browser** ~> 5.0 (device detection)
+- **sqlite3** >= 2.1 (testing)
+- **rspec** ~> 3.0 (testing framework)
+
+### Configuration System
+```ruby
+ContentSignals.configure do |config|
+  config.multitenancy = true
+  config.current_tenant_method = :current_tenant_id
+  config.tenant_model = 'Account'
+  config.redis_enabled = true
+  config.redis_namespace = 'content_signals'
+  config.maxmind_db_path = Rails.root.join('db', 'GeoLite2-City.mmdb')
+  config.track_bots = false
+  config.track_admins = false
+end
+```
+
+### Test Database Schema
+- ✅ `pages` table (test trackable model)
+- ✅ `profiles` table (test trackable model)
+- ✅ `users` table
+- ✅ `content_signals_page_views` table (full schema from specs)
+
+### Testing Setup
+- ✅ RSpec configured
+- ✅ In-memory SQLite database
+- ✅ Transaction rollback between tests
+- ✅ Mock request helpers
+- ✅ Test models defined
+- ✅ All tests passing (7 examples, 0 failures)
+
+## Next Steps
+
+### 1. Core Models
+- [ ] `ContentSignals::PageView` model with scopes
+- [ ] Test specs for PageView model
+
+### 2. Services
+- [ ] `PageViewTracker` - Main tracking service
+- [ ] `VisitorLocationService` - MaxMind geolocation
+- [ ] `DeviceDetectorService` - Browser detection
+- [ ] `AnalyticsQueryService` - Query interface
+- [ ] Test specs for all services
+
+### 3. Background Jobs
+- [ ] `TrackPageViewJob` - Async processing
+- [ ] Test specs for job
+
+### 4. Concerns
+- [ ] `Trackable` concern for models
+- [ ] Test specs for concern
+
+### 5. Generators
+- [ ] Install generator (migrations, initializer)
+- [ ] Test generator output
+
+### 6. Documentation
+- [ ] Update README with usage examples
+- [ ] Add CHANGELOG entry
+- [ ] Create migration templates
+
+## Running Tests
+```bash
+bundle exec rspec                    # Run all tests
+bundle exec rspec spec/models/       # Run model tests
+bundle exec rspec spec/services/     # Run service tests
+```
+
+## Current Status
+✅ Gem structure complete
+✅ Testing environment working
+✅ Configuration system tested
+⏳ Ready to implement core functionality

data/lib/content_signals/concerns/trackable_page_views.rb

@@ -0,0 +1,88 @@
+# frozen_string_literal: true
+
+module ContentSignals
+  module TrackablePageViews
+    extend ActiveSupport::Concern
+
+    included do
+      after_action :track_page_view, only: [:show]
+    end
+
+    private
+
+    def track_page_view
+      return if should_skip_tracking?
+
+      trackable = find_trackable_for_tracking
+      return unless trackable
+
+      # Ensure visitor cookie is set
+      ensure_visitor_cookie
+
+      PageViewTracker.track(
+        trackable: trackable,
+        user: current_user_for_tracking,
+        request: request
+      )
+    rescue => e
+      Rails.logger.error "Failed to track page view: #{e.message}"
+      # Don't raise - tracking failures shouldn't break the request
+    end
+
+    def ensure_visitor_cookie
+      return if cookies.signed[:visitor_id].present?
+      return if current_user_for_tracking.present? # Don't need cookie for authenticated users
+
+      cookies.signed[:visitor_id] = {
+        value: "visitor_#{SecureRandom.uuid}",
+        expires: 2.years.from_now,
+        httponly: true,
+        same_site: :lax
+      }
+    rescue => e
+      Rails.logger.error "Failed to set visitor cookie: #{e.message}"
+    end
+
+    def should_skip_tracking?
+      bot_request? ||
+      admin_viewing? ||
+      preview_mode?
+    end
+
+    def find_trackable_for_tracking
+      # Look for common instance variable names
+      # Override this method in your controller if you use different naming
+      @page || @post || @article || @profile || @event || @trackable
+    end
+
+    def current_user_for_tracking
+      # Override this method if you use a different method name
+      current_user if respond_to?(:current_user, true)
+    end
+
+    def bot_request?
+      return false unless defined?(Browser)
+
+      browser = Browser.new(request.user_agent)
+      browser.bot? || browser.search_engine?
+    rescue
+      false
+    end
+
+    def admin_viewing?
+      user = current_user_for_tracking
+      return false unless user
+
+      # Check common admin patterns
+      user.respond_to?(:admin?) && user.admin?
+    rescue
+      false
+    end
+
+    def preview_mode?
+      params[:preview].present? || session[:preview_mode]
+    rescue
+      false
+    end
+  end
+end

data/lib/content_signals/configuration.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module ContentSignals
+  class Configuration
+    attr_accessor :multitenancy,
+                  :current_tenant_method,
+                  :tenant_model,
+                  :redis_enabled,
+                  :redis_namespace,
+                  :maxmind_db_path,
+                  :track_bots,
+                  :track_admins
+
+    def initialize
+      @multitenancy = false
+      @current_tenant_method = :current_tenant_id
+      @tenant_model = nil
+      @redis_enabled = true
+      @redis_namespace = "content_signals"
+      @maxmind_db_path = default_maxmind_path
+      @track_bots = false
+      @track_admins = false
+    end
+
+    def multitenancy?
+      @multitenancy
+    end
+
+    def redis_enabled?
+      @redis_enabled && defined?(Redis)
+    end
+
+    private
+
+    def default_maxmind_path
+      return nil unless defined?(Rails) && Rails.respond_to?(:root) && Rails.root
+
+      Rails.root.join("db", "GeoLite2-City.mmdb")
+    end
+  end
+end

data/lib/content_signals/engine.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require "rails"
+require "active_record"
+
+module ContentSignals
+  class Engine < ::Rails::Engine
+    isolate_namespace ContentSignals
+
+    config.generators do |g|
+      g.test_framework :rspec
+      g.fixture_replacement :factory_bot
+      g.factory_bot dir: "spec/factories"
+    end
+
+    # Eager load models, services, jobs, concerns
+    config.eager_load_paths += %W[
+      #{root}/lib/content_signals/models
+      #{root}/lib/content_signals/services
+      #{root}/lib/content_signals/jobs
+      #{root}/lib/content_signals/concerns
+    ]
+
+    initializer "content_signals.active_job" do
+      config.after_initialize do
+        if defined?(ActiveJob)
+          require_relative "jobs/track_page_view_job" if File.exist?(File.join(__dir__, "jobs", "track_page_view_job.rb"))
+        end
+      end
+    end
+  end
+end
+

data/lib/content_signals/jobs/track_page_view_job.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module ContentSignals
+  class TrackPageViewJob < ActiveJob::Base
+    queue_as :default
+
+    # Retry on database errors but not on validation errors
+    retry_on ActiveRecord::Deadlocked, wait: 5.seconds, attempts: 3
+    discard_on ActiveRecord::RecordInvalid
+
+    def perform(trackable_type, trackable_id, user_id, tracking_data)
+      PageView.create!(
+        tenant_id: tracking_data['tenant_id'] || tracking_data[:tenant_id],
+        trackable_type: trackable_type,
+        trackable_id: trackable_id,
+        user_id: user_id,
+        visitor_id: tracking_data['visitor_id'] || tracking_data[:visitor_id],
+        ip_address: tracking_data['ip_address'] || tracking_data[:ip_address],
+        user_agent: tracking_data['user_agent'] || tracking_data[:user_agent],
+        referrer: tracking_data['referrer'] || tracking_data[:referrer],
+        country_code: tracking_data['country_code'] || tracking_data[:country_code],
+        country_name: tracking_data['country_name'] || tracking_data[:country_name],
+        city: tracking_data['city'] || tracking_data[:city],
+        region: tracking_data['region'] || tracking_data[:region],
+        latitude: tracking_data['latitude'] || tracking_data[:latitude],
+        longitude: tracking_data['longitude'] || tracking_data[:longitude],
+        locale: tracking_data['locale'] || tracking_data[:locale],
+        device_type: tracking_data['device_type'] || tracking_data[:device_type],
+        browser: tracking_data['browser'] || tracking_data[:browser],
+        os: tracking_data['os'] || tracking_data[:os],
+        app_platform: tracking_data['app_platform'] || tracking_data[:app_platform],
+        app_version: tracking_data['app_version'] || tracking_data[:app_version],
+        device_id: tracking_data['device_id'] || tracking_data[:device_id],
+        viewed_at: tracking_data['viewed_at'] || tracking_data[:viewed_at] || Time.current
+      )
+    rescue => e
+      Rails.logger.error "Failed to track page view: #{e.message}"
+      Rails.logger.error e.backtrace.join("\n")
+      # Optionally send to error tracking service (Sentry, Rollbar, etc.)
+      raise unless Rails.env.production? # Re-raise in non-production for debugging
+    end
+  end
+end