provenance 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 272ac0a7de2193660ff6e31bc2e5e1ef79f0809837057c3cce8df4b56c00e5f1
+  data.tar.gz: e34d95e42e38eb2c1e40d238cc981b2ee0d7c06829d24b9c134d9eae5b7cc41d
+SHA512:
+  metadata.gz: f789e8800b2bd3082424a4f6567ba981f3b0d7f732ab8ec1bfde3f1f0d66dbed31290b4ad30a067491d27e6d2c6ed58490ffad662321a1083f34ee2b4fa30bd8
+  data.tar.gz: '03285aa742eb534df8d103659467b004dc5a040991351072d37e7120a16e144697172987cf54742ec11a259cd3482d21dd214d89df8f271e64806cfbb52518c2'

data/CHANGELOG.md

@@ -0,0 +1,28 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [1.0.0] - 2026-06-05
+
+### Added
+
+- Initial public release.
+- Model change tracking (create / update / destroy) via ActiveRecord callbacks,
+  grouped per request and per transaction.
+- Controller-level auditing with automatic event-type generation.
+- Error reporting through `audit_error(error, status)`.
+- Bulk operation tracking for `update_all` / `delete_all` (opt-in).
+- `has_and_belongs_to_many` join-table change tracking.
+- Recursive sensitive-data filtering with global and per-model attribute lists.
+- Pluggable value providers (username, roles, remote IP, origin IP, session id).
+- Configurable delivery hooks for shipping audit events to any sink.
+- Transaction-aware delivery: events are flushed only after every transaction
+  has committed, and discarded on rollback.
+
+[Unreleased]: https://github.com/inikalaev/provenance/compare/v1.0.0...HEAD
+[1.0.0]: https://github.com/inikalaev/provenance/releases/tag/v1.0.0

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ivan Nikolaev
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,333 @@
+<div align="center">
+
+# 📜 Provenance
+
+### A drop-in audit trail for Rails — every user action and model change, captured.
+
+[![CI](https://github.com/inikalaev/provenance/actions/workflows/ci.yml/badge.svg)](https://github.com/inikalaev/provenance/actions/workflows/ci.yml)
+[![Gem Version](https://img.shields.io/gem/v/provenance.svg)](https://rubygems.org/gems/provenance)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.2-CC342D.svg)](.ruby-version)
+
+</div>
+
+---
+
+Provenance watches your Rails app and records **who did what, to which record, when** — then ships
+a structured event anywhere you want: a log pipeline, a data warehouse, an external SIEM, or just
+`Rails.logger`. You wire it in once; it stays out of your controllers and models.
+
+```ruby
+# An audit event Provenance produces for a successful request
+{
+  "event_type": "create_users",
+  "status": 201,
+  "username": "admin@example.com",
+  "remote_ip": "203.0.113.1",
+  "message": {
+    "count": 1,
+    "changes": [
+      { "model": "User", "model_id": 42, "action": "create",
+        "changes": { "attributes": { "email": "user@example.com", "password": "[FILTERED]" } } }
+    ]
+  },
+  "source": "myapp_production"
+}
+```
+
+## ✨ Features
+
+- 🧾 **Model change tracking** — `create` / `update` / `destroy` captured straight from ActiveRecord callbacks.
+- 🌐 **Controller auditing** — one `around_action` records every change made during a request and emits a single event.
+- 🧬 **Transaction-aware** — changes are grouped per transaction, flushed only after every transaction commits, and **discarded on rollback**.
+- 💥 **Error reporting** — emit a dedicated audit event for failed requests with `audit_error`.
+- 🗂️ **Bulk operations** — opt-in tracking for `update_all` / `delete_all`, which normally bypass callbacks.
+- 🔗 **`has_and_belongs_to_many`** — join-table writes are tracked through SQL notifications.
+- 🛡️ **Sensitive-data filtering** — recursive `[FILTERED]` redaction with global and per-model attribute lists.
+- 🔌 **Pluggable providers & hooks** — decide how to resolve the actor and where events are delivered.
+- 🪶 **Fail-safe** — auditing never breaks the underlying request or operation.
+
+## 📦 Installation
+
+Add it to your `Gemfile`:
+
+```ruby
+gem "provenance"
+```
+
+Then install:
+
+```bash
+bundle install
+```
+
+## 🚀 Quick start
+
+### 1. Configure the initializer
+
+```ruby
+# config/initializers/provenance.rb
+require "provenance"
+
+Provenance.configure do |config|
+  config.source_name = "myapp_#{Rails.env}"
+  config.sensitive_attributes = %w[password password_confirmation token secret_key api_key]
+
+  # Auditing is disabled in the test environment by default. Override if needed:
+  # config.enabled = true
+end
+
+# How to resolve the actor and request metadata (each receives the controller):
+Provenance.setup_username_provider(->(controller) { controller.current_user&.email })
+Provenance.setup_roles_provider(->(controller) { controller.current_user&.roles || [] })
+Provenance.setup_remote_ip_provider(->(controller) { controller.request.remote_ip })
+Provenance.setup_origin_ip_provider(->(controller) { ENV["SERVER_IP"] || "127.0.0.1" })
+Provenance.setup_session_id_provider(->(controller) { controller.request.headers["Authorization"]&.split(" ")&.last })
+
+# Where audit events go (you can register more than one hook):
+Provenance.config.add_audit_hook do |audit_data|
+  Rails.logger.info("AUDIT: #{audit_data.to_json}")
+end
+```
+
+### 2. Mix the concerns in
+
+```ruby
+# app/controllers/application_controller.rb
+class ApplicationController < ActionController::API
+  include Provenance::Auditable       # records changes per request
+  include Provenance::ErrorReporting  # adds audit_error(error, status)
+end
+```
+
+```ruby
+# app/models/application_record.rb
+class ApplicationRecord < ActiveRecord::Base
+  self.abstract_class = true
+  include Provenance::Trackable
+end
+```
+
+That's it. Every write that happens inside a request now produces an audit event.
+
+## 🧭 How it works
+
+```
+Request ─▶ Auditable (around_action)
+              │  opens a Journal for this request
+              ▼
+        Trackable callbacks  ──▶  Journal  ◀──  BulkOperations / HABTM SQL
+        (create/update/destroy)   (grouped by transaction)
+              │
+              ▼
+        all transactions committed?
+              │ yes                       │ rollback
+              ▼                           ▼
+        audit hooks receive          changes discarded
+        one assembled event
+```
+
+The `Journal` lives in fiber-local storage for the duration of the request, so concurrent requests
+never see each other's changes.
+
+## 💥 Error logging
+
+Call `audit_error` from your rescue handlers to record failures:
+
+```ruby
+def render_errors(errors, status: :unprocessable_entity)
+  audit_error(errors, status)
+  render json: { errors: Array(errors) }, status: status
+end
+
+rescue_from ActiveRecord::RecordNotFound do
+  audit_error("Not found", :not_found)
+  head :not_found
+end
+```
+
+Symbolic statuses (`:not_found`, `:unauthorized`, `:forbidden`, `:unprocessable_entity`, `:conflict`)
+are mapped to their numeric codes automatically; anything else defaults to `500`.
+
+## 🛡️ Sensitive data filtering
+
+Filtered values are replaced with `[FILTERED]` — in model attributes, request params, and nested
+hashes/arrays alike.
+
+```ruby
+# Global (applies everywhere)
+Provenance.configure do |config|
+  config.sensitive_attributes = %w[password token api_key]
+end
+
+# Per-model (takes priority over the global list)
+class Payment < ApplicationRecord
+  sensitive_attributes :card_number, :cvv, :token
+end
+```
+
+```ruby
+# in                                  # out
+{ user: {                            { user: {
+    email: "user@example.com",           email: "user@example.com",
+    password: "secret",                  password: "[FILTERED]",
+    profile: { api_key: "abc123" }       profile: { api_key: "[FILTERED]" }
+} }                                  } }
+```
+
+## 🗂️ Bulk operations
+
+`update_all` and `delete_all` skip ActiveRecord callbacks, so they are tracked separately and must be
+enabled explicitly:
+
+```ruby
+Provenance.configure do |config|
+  config.track_bulk_operations = true   # default: false
+  config.bulk_operations_max_ids = 1000 # cap ids per record; over the cap sets truncated: true
+end
+```
+
+The affected ids are collected with a `pluck` **before** the statement runs, so factor in one extra
+query on large result sets. Operations outside an HTTP request (migrations, rake tasks, background
+jobs) are not tracked. A bulk change looks like:
+
+```ruby
+{
+  model: "Comment",
+  model_ids: ["101", "102"],
+  action: "bulk_update",      # or "bulk_delete"
+  count: 2,
+  changes: { status: "deleted", deleted_at: "2026-06-05T12:00:00Z" }
+}
+```
+
+## 🔗 has_and_belongs_to_many
+
+Join-table inserts and deletes never trigger model callbacks, so Provenance observes them through
+`sql.active_record` notifications and folds them into the owner's change as an `*_ids` update. No extra
+setup is required beyond including `Provenance::Trackable` in the participating models.
+
+> **Note:** the SQL reconstruction for HABTM is tuned for PostgreSQL bind placeholders (`$1`, `$2`).
+> Insert tracking is portable; delete tracking depends on that placeholder style.
+
+## ⚙️ Fine-tuning
+
+### Skip auditing per action
+
+```ruby
+class UsersController < ApplicationController
+  skip_audit_logging :index, :show       # no event at all
+  skip_model_change_tracking :index      # event, but without model diffs
+end
+```
+
+### Custom event types
+
+```ruby
+class SessionsController < ApplicationController
+  custom_audit_event_type :create, "user_login"
+  custom_audit_event_type :destroy, "user_logout"
+end
+```
+
+### Automatic event-type generation
+
+When you don't override it, Provenance derives the event type from the controller and action:
+
+| Action            | Event type                | Example (`UsersController`)        |
+| ----------------- | ------------------------- | ---------------------------------- |
+| `index`           | `read_{controller}`       | `read_users`                       |
+| `show`            | `show_{singular}`         | `show_user`                        |
+| `create`          | `create_{controller}`     | `create_users`                     |
+| `update`          | `update_{singular}`       | `update_user`                      |
+| `destroy`         | `destroy_{singular}`      | `destroy_user`                     |
+| _custom action_   | `{action}_{controller}`   | `archive_users`                    |
+
+Namespaced controllers are flattened with `_`, e.g. `Admin::UsersController#index` → `read_admin_users`.
+
+### Delivery hooks
+
+```ruby
+Provenance.config.add_audit_hook { |event| ExternalAuditService.deliver(event) }
+Provenance.config.add_audit_hook { |event| Rails.logger.info("AUDIT: #{event.to_json}") }
+Provenance.config.clear_audit_hooks  # remove all hooks
+```
+
+## 🔧 Configuration reference
+
+| Option                      | Default              | Description                                              |
+| --------------------------- | -------------------- | ------------------------------------------------------- |
+| `source_name`               | `"app_#{Rails.env}"` | Identifies the emitting application in every event.     |
+| `sensitive_attributes`      | `[]`                 | Global attribute names to redact.                       |
+| `enabled`                   | `!Rails.env.test?`   | Master switch for the whole pipeline.                   |
+| `track_bulk_operations`     | `false`              | Track `update_all` / `delete_all`.                      |
+| `bulk_operations_max_ids`   | `1000`               | Max ids recorded per bulk change.                       |
+| `audit_hooks`               | `[]`                 | Delivery callbacks (use `add_audit_hook`).              |
+
+Providers: `username`, `roles`, `remote_ip`, `origin_ip`, `session_id` — each set via
+`Provenance.setup_<name>_provider(callable)`.
+
+## 📐 Event structure
+
+**Successful request**
+
+```json
+{
+  "timestamp": "2026-06-05T12:00:00.000Z",
+  "event_type": "create_users",
+  "status": 201,
+  "message": {
+    "count": 1,
+    "changes": [
+      {
+        "model": "User",
+        "model_id": 123,
+        "action": "create",
+        "changes": { "attributes": { "email": "user@example.com", "name": "John Doe" } },
+        "timestamp": "2026-06-05T12:00:00.000Z"
+      }
+    ],
+    "params": { "user": { "email": "user@example.com" } }
+  },
+  "username": "admin@example.com",
+  "remote_ip": "203.0.113.1",
+  "origin_ip": "192.168.1.100",
+  "session_id": "token123",
+  "roles": ["admin"],
+  "request_id": "req-123",
+  "source": "myapp_production"
+}
+```
+
+**Failed request** (via `audit_error`)
+
+```json
+{
+  "timestamp": "2026-06-05T12:00:00.000Z",
+  "event_type": "create_users",
+  "status": "422",
+  "message": {
+    "error_type": "ActiveRecord::RecordInvalid",
+    "error_message": "Validation failed: Email has already been taken",
+    "params": { "user": { "email": "invalid" } }
+  },
+  "username": "admin@example.com",
+  "source": "myapp_production"
+}
+```
+
+## 🧪 Development
+
+```bash
+bundle install
+bundle exec rspec     # run the test suite
+bundle exec rubocop   # lint
+```
+
+## 🤝 Contributing
+
+Bug reports and pull requests are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## 📄 License
+
+Released under the [MIT License](LICENSE).

data/lib/provenance/configuration.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Provenance
+  # Holds the global configuration: the data source name, the list of sensitive
+  # attributes, the delivery hooks and the value providers used to enrich every
+  # audit event.
+  class Configuration
+    attr_accessor :source_name, :sensitive_attributes, :audit_hooks, :enabled,
+      :username_provider, :roles_provider, :remote_ip_provider, :origin_ip_provider, :session_id_provider,
+      :track_bulk_operations, :bulk_operations_max_ids
+
+    def initialize
+      @source_name = "app_#{Rails.env}"
+      @sensitive_attributes = []
+      @audit_hooks = []
+      @enabled = !Rails.env.test?
+      @track_bulk_operations = false
+      @bulk_operations_max_ids = 1000
+      @username_provider = nil
+      @roles_provider = nil
+      @remote_ip_provider = nil
+      @origin_ip_provider = nil
+      @session_id_provider = nil
+    end
+
+    def add_audit_hook(&block)
+      @audit_hooks << block
+    end
+
+    def clear_audit_hooks
+      @audit_hooks.clear
+    end
+  end
+end

data/lib/provenance/context.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: true
+
+require "securerandom"
+
+module Provenance
+  # Per-request scratch space backed by fiber-local storage. Holds the active
+  # journal, the deferred delivery callback and the request/response metadata
+  # for the duration of a single request.
+  class Context
+    class << self
+      def journal
+        Thread.current[:provenance_journal]
+      end
+
+      def journal=(value)
+        Thread.current[:provenance_journal] = value
+      end
+
+      def pending_audit_log
+        Thread.current[:provenance_pending_log]
+      end
+
+      def pending_audit_log=(callback)
+        Thread.current[:provenance_pending_log] = callback
+      end
+
+      def request_id
+        Thread.current[:provenance_request_id]
+      end
+
+      def request_id=(value)
+        Thread.current[:provenance_request_id] = value
+      end
+
+      def response_status
+        Thread.current[:provenance_response_status]
+      end
+
+      def response_status=(value)
+        Thread.current[:provenance_response_status] = value
+        Thread.current[:provenance_request_completed] = true
+      end
+
+      def request_completed?
+        Thread.current[:provenance_request_completed] || false
+      end
+
+      def cleanup
+        Thread.current[:provenance_journal] = nil
+        Thread.current[:provenance_pending_log] = nil
+        Thread.current[:provenance_request_id] = nil
+        Thread.current[:provenance_send_scheduled] = nil
+        Thread.current[:provenance_response_status] = nil
+        Thread.current[:provenance_request_completed] = nil
+      end
+    end
+  end
+end

data/lib/provenance/journal.rb

@@ -0,0 +1,123 @@
+# frozen_string_literal: true
+
+module Provenance
+  # Accumulates model and bulk changes for the current request, grouped by
+  # transaction key so they can be discarded on rollback and flushed together
+  # once every transaction has completed.
+  class Journal
+    attr_reader :changes
+
+    def initialize
+      @changes = []
+      @active_transactions = Set.new
+    end
+
+    def register_transaction(transaction_id)
+      return if transaction_id.nil?
+
+      @active_transactions.add(transaction_id)
+    end
+
+    def complete_transaction(transaction_id)
+      return if transaction_id.nil?
+
+      @active_transactions.delete(transaction_id)
+    end
+
+    def add_change(model, action, data)
+      change_data = {
+        model: model.class.name,
+        model_id: model.id,
+        action: action.to_s,
+        changes: filter_sensitive_data(data, model.class),
+        timestamp: Time.current.utc.iso8601(3)
+      }
+
+      change_data[:transaction_id] = data[:transaction_id] if data.is_a?(Hash) && data[:transaction_id]
+
+      @changes << change_data
+    end
+
+    def add_bulk_change(model_class, action, ids, updates, truncated: false, transaction_id: nil)
+      change_data = {
+        model: model_class.name,
+        model_ids: ids,
+        action: action.to_s,
+        count: ids.size,
+        changes: updates.nil? ? {} : filter_sensitive_data(updates, model_class),
+        timestamp: Time.current.utc.iso8601(3)
+      }
+      change_data[:truncated] = true if truncated
+      change_data[:transaction_id] = transaction_id if transaction_id
+
+      @changes << change_data
+    end
+
+    def remove_changes_for_transaction(transaction_id)
+      return if transaction_id.nil?
+
+      @changes.reject! { |change| change[:transaction_id] == transaction_id }
+      @active_transactions.delete(transaction_id)
+    end
+
+    def all_transactions_completed?
+      @active_transactions.empty?
+    end
+
+    def to_h
+      {
+        count: @changes.size,
+        changes: @changes
+      }
+    end
+
+    def clear!
+      @changes.clear
+      @active_transactions.clear
+    end
+
+    def empty?
+      @changes.empty?
+    end
+
+    def present?
+      @changes.any?
+    end
+
+    private
+
+    def filter_sensitive_data(data, model_class = nil)
+      sensitive_attrs = model_sensitive_attributes(model_class)
+      return data if sensitive_attrs.empty?
+
+      case data
+      when Hash
+        filtered_data = {}
+        data.each do |key, value|
+          filtered_data[key] = if key == "transaction_id"
+                                 value
+                               elsif sensitive_attrs.include?(key.to_s)
+                                 "[FILTERED]"
+                               else
+                                 filter_sensitive_data(value, model_class)
+                               end
+        end
+        filtered_data
+      when Array
+        data.map { |item| filter_sensitive_data(item, model_class) }
+      else
+        data
+      end
+    end
+
+    def model_sensitive_attributes(model_class)
+      attributes = model_class&.attribute_names
+      if model_class.respond_to?(:sensitive_attributes_list)
+        attrs = model_class.sensitive_attributes_list
+        attributes = attrs.map(&:to_s)
+      end
+
+      (Provenance.config.sensitive_attributes + attributes).map(&:to_s)
+    end
+  end
+end