ree_lib 1.3.12 → 1.3.14

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e10f8d9ba3558f200f081f325d2a7f10b81696459c8fbdd9054fd4e0b72c72ed
-  data.tar.gz: fc2f2bba979d98082ed296caeeb8823cf1cef3201cbda6244adb131e7ccc5db1
+  metadata.gz: 2ebed252b6557db13e60268884fd6b1a0f405f256d28aaf18953764886ab6cb0
+  data.tar.gz: e6328fda6013213c588f834b5a0782c271c76686b8303a0f458cfac5e926c80d
 SHA512:
-  metadata.gz: 4cbd59039f6e3b238db9c88763e75f753a567c13edd080eda2e6fafbed838e3c61a71d9ee40af35b7042f58cee315f5aa3c7cdf8be8b2e7ae339f9ed7883906d
-  data.tar.gz: e0dddd85f84fec535514ebbec51e78425511b32a8ee877b65fdf4ac31f513717f9c12ab80d4772d7649ad1856ad625bf1d967e9c56d78823dc1435196b438c85
+  metadata.gz: 06ad16d1ca9361397b0e324d8ab0d9a3a064337831384f45616dd500efb88367c84be252537d203e07ec08cb109e8f976980ff39ac2de315f96938a21edc697e
+  data.tar.gz: fa3b6b910f9855585af6f0e985720db3773d7483779f65d3907badc16711e5bfdd159abcfcd8135466a5a2c57a75645da44c79ba532e0bcd9bf777d50aa7bdd5

data/Gemfile.lock

@@ -9,7 +9,7 @@ PATH
 PATH
   remote: .
   specs:
-    ree_lib (1.3.12)
+    ree_lib (1.3.14)
       bigdecimal
       binding_of_caller
       i18n

data/lib/ree_lib/packages/ree_audit/README.md

@@ -0,0 +1,204 @@
+# ree_audit
+
+An opt-in trail of audited action calls.
+
+The package answers one question: **who called what, when, and how did it end.**
+It says nothing about what happened inside the action, and nothing about files
+downloaded straight from object storage — see *Boundaries* below.
+
+## Why it exists
+
+An application that serves an internal (admin) API eventually has to answer a
+customer, an auditor or a regulator: *did your staff look at my data, and when?*
+Answering that from request logs does not work — they rotate, they contain
+secrets, and they cannot be handed to a customer. This package produces a
+structured record per audited call that an application can persist wherever it
+wants and export as-is.
+
+## Off by default
+
+`AUDIT_ENABLED` defaults to `false`, and an application that never passes
+`audit:` to the `ree_routes` plugin behaves exactly as it did before the gem was
+bumped. Both properties are covered by specs: a shared framework must not start
+recording anything because someone upgraded a dependency.
+
+When audit is disabled, `Audit#around` costs a single `if`.
+
+## Wiring it up
+
+```ruby
+class MyApp < ReeRoda::App
+  link :audit, from: :ree_audit, import: -> { Audit }
+  link :my_sink, from: :my_package, import: -> { MySink }
+
+  AUDIT = Audit.new          # singleton bean
+  AUDIT.sink = MySink.new    # the one and only mutation, at boot
+
+  plugin :ree_routes, audit: AUDIT
+end
+```
+
+A sink implements one method:
+
+```ruby
+class MySink < ReeAudit::Sink
+  def write(event)
+    # persist it
+  end
+end
+```
+
+The package ships `NullSink` (default) and `LoggerSink` (dev). A sink that
+raises never fails the request: `Audit#write_safely` catches it and reports
+through `ree_logger`.
+
+## What gets audited
+
+Nothing is audited until the route says so:
+
+```ruby
+get "api/v1/admin/users" do
+  visibility :internal
+  audit true              # recorded
+end
+
+get "api/v1/admin/health" do
+  visibility :internal
+  audit false             # deliberately silent
+end
+
+get "api/v1/organizations/:id/balance" do
+  audit :on_annotation    # the owner calls it too — see below
+end
+```
+
+The flag is declared at the route and nowhere else. Deriving it from
+`visibility` would work just as well and read far worse: whether a call lands in
+the access trail is exactly the kind of fact a reader must see where the route
+is written, not infer from a flag that means something else.
+
+Forgetting it is not possible where it matters: like `visibility`, `audit` is
+mandatory on admin paths — a route whose path carries an `admin` segment fails
+at boot until it declares one. So the audited set cannot drift away from the set
+of routes that reach client data.
+
+### `audit :on_annotation`
+
+Some routes are called by two different people: the owner of the data, reading
+its own, and a staff member reading someone else's. Which one it was is known
+only after the permission check has run, inside the action — so the event is
+always assembled, and it reaches the sink only if the code annotated it:
+
+```ruby
+def call(organization_id, user_id)
+  admin = platform_admins.active.by_user(user_id).first
+
+  if admin
+    # this call was made by staff, and that is what makes it worth recording
+    annotate_audit(organization_id: organization_id, staff_id: admin.id)
+  end
+  ...
+end
+```
+
+`audit true` on such a route would bury the handful of staff calls under every
+ordinary request the owners make; leaving it unaudited would lose them.
+Annotating before the check, not after, keeps a refusal in the trail as well:
+`Event#recordable?` is decided in `ensure`, when the annotation is already there.
+
+## The event
+
+| field | meaning |
+|---|---|
+| `action_name`, `package_name` | which action ran |
+| `summary`, `sections` | route metadata, useful for grouping and for classifying sensitivity |
+| `request_method`, `path`, `request_path` | `path` is the DSL template (stable), `request_path` is the actual path (shows the object) |
+| `params` | what the action received, after filtering |
+| `accessor` | the authenticated object — the application decides how to read it |
+| `status` | `:ok`, `:denied`, `:error` |
+| `error_type`, `error_message` | present when the call failed; message truncated to 512 chars |
+| `started_at`, `duration_ms` | timing (monotonic clock) |
+| `annotations` | whatever the application attached during the call |
+
+`:denied` means the action raised a `ReeErrors::Error` whose type is
+`:permission`. A refusal to show client data is as much an audit fact as
+showing it.
+
+There is **no HTTP status** on the event, and that is not an oversight: the
+wrapper sits around the action call, while Roda sets the status afterwards (and,
+on an exception, in the application's `error` block). Observing it here is
+impossible, and deriving it from the error type would put a guess into an audit
+trail. `status` carries the fact instead.
+
+## Annotations
+
+Product code attaches what only it knows:
+
+```ruby
+class MyPackage::AdminUploadQuery
+  action :admin_upload_query do
+    link :annotate_audit, from: :ree_audit
+  end
+
+  def call(access, attrs)
+    upload = find_upload(attrs)
+    annotate_audit(organization_id: upload.organization_id,
+                   subject: {type: :upload, id: upload.uuid})
+    ...
+  end
+end
+```
+
+`annotate_audit` returns `nil` when audit is off or the code was reached from
+outside an audited route, so it is always safe to call.
+
+Use the `annotate_audit` fn rather than `ReeAudit.annotate` directly. A bare
+constant works, but Ree only verifies a declared package dependency for
+`link`/`import` — so removing `depends_on :ree_audit` as "unused" would compile
+fine and blow up with `NameError` in production. With `link`, the same removal
+fails loudly at load time.
+
+## Params filtering
+
+Two limits, applied before the event reaches the sink:
+
+* keys containing any `filter_words` are replaced with `'FILTERED'` **at any
+  depth** (the logger only inspects the top level);
+* anything nested deeper than `AUDIT_MAX_PARAMS_DEPTH` becomes
+  `'[TRUNCATED_DEPTH]'`, and a payload whose JSON exceeds
+  `AUDIT_MAX_PARAMS_BYTES` is replaced with `{truncated: true, size: N}`.
+
+This is a blacklist, and it is the framework's job: it protects any host
+application from leaking a credential. An application that exports the trail to
+its customers should apply its own **whitelist** on top — a blacklist silently
+passes through a key someone adds six months from now.
+
+## Nested calls
+
+Events live on a per-fiber stack (`ReeAudit::Context`), so an audited action may
+call another one without the inner call stealing the outer one's annotations.
+The stack is unwound in `ensure`, including when the action raises — a leftover
+event would attach one client's context to the next request served by the same
+fiber.
+
+## Configuration
+
+| variable | default | meaning |
+|---|---|---|
+| `AUDIT_ENABLED` | `false` | master switch |
+| `AUDIT_MAX_PARAMS_BYTES` | `4096` | size limit for the serialized params |
+| `AUDIT_MAX_PARAMS_DEPTH` | `4` | nesting limit |
+| `AUDIT_FILTER_WORDS` | see `Config::DEFAULT_FILTER_WORDS` | comma-separated |
+
+## Boundaries
+
+The package answers *who called what*. It does not see anything that bypasses
+the application: direct database access, an object-storage console, or a file
+fetched with a pre-signed URL that was handed out earlier. An application that
+needs those answers has to record them where they happen.
+
+## Specs
+
+```
+cd ree_lib/lib/ree_lib && bundle exec ree spec ree_audit
+```

data/lib/ree_lib/packages/ree_audit/package/ree_audit/beans/audit.rb

@@ -56,7 +56,7 @@ class ReeAudit::Audit
         (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_monotonic) * 1000
       ).round
       ReeAudit::Context.pop
-      write_safely(event)
+      write_safely(event) if event.recordable?
     end
   end
 

data/lib/ree_lib/packages/ree_audit/package/ree_audit/event.rb

@@ -9,7 +9,7 @@ class ReeAudit::Event
   attr_accessor :action_name, :package_name, :summary, :sections,
                 :request_method, :path, :request_path, :params, :accessor,
                 :status, :error_type, :error_message, :started_at,
-                :duration_ms, :annotations
+                :duration_ms, :annotations, :on_annotation
 
   class << self
     # Keyword-only on purpose: callers live in another package and must not
@@ -23,7 +23,8 @@ class ReeAudit::Event
       path: nil,
       request_path: nil,
       params: {},
-      accessor: nil
+      accessor: nil,
+      on_annotation: false
     )
       event = new
 
@@ -37,6 +38,7 @@ class ReeAudit::Event
       event.params = params || {}
       event.accessor = accessor
       event.annotations = {}
+      event.on_annotation = on_annotation
 
       event
     end
@@ -47,6 +49,7 @@ class ReeAudit::Event
     @params = {}
     @annotations = {}
     @status = :ok
+    @on_annotation = false
   end
 
   def ok?
@@ -57,6 +60,14 @@ class ReeAudit::Event
     status == :denied
   end
 
+  # An `on_annotation` event is the same call seen from two sides: the client
+  # asking for its own data, and a staff member asking for the client's. Only
+  # the code that resolved the caller can tell them apart, and it says so by
+  # annotating — so an untouched event of that kind is not worth recording.
+  def recordable?
+    !on_annotation || !annotations.empty?
+  end
+
   def to_h
     {
       action_name: action_name,

data/lib/ree_lib/packages/ree_audit/spec/ree_audit/beans/audit_spec.rb

@@ -158,4 +158,41 @@ RSpec.describe :audit do
   it "ignores an annotation made outside an audited call" do
     expect(ReeAudit.annotate(permission: :access_uploads)).to be_nil
   end
+
+  context "an :on_annotation event" do
+    let(:event) {
+      ReeAudit::Event.build(action_name: :balance_query, on_annotation: true)
+    }
+
+    it "drops the call nobody annotated" do
+      result = audit.around(event) { :action_result }
+
+      expect(result).to eq(:action_result)
+      expect(test_sink.events).to be_empty
+    end
+
+    it "writes the call the action annotated" do
+      audit.around(event) do
+        ReeAudit.annotate(platform_admin_id: 7)
+        :action_result
+      end
+
+      expect(test_sink.events.size).to eq(1)
+      expect(test_sink.events.first.annotations).to eq({platform_admin_id: 7})
+    end
+
+    it "writes a refusal that was annotated before the raise" do
+      klass = permission_error(:no_access)
+
+      expect {
+        audit.around(event) do
+          ReeAudit.annotate(platform_admin_id: 7)
+          raise klass.new("not allowed")
+        end
+      }.to raise_error(klass)
+
+      expect(test_sink.events.size).to eq(1)
+      expect(test_sink.events.first.status).to eq(:denied)
+    end
+  end
 end

data/lib/ree_lib/packages/ree_roda/package/ree_roda/plugins/ree_routes.rb

@@ -146,7 +146,8 @@ class Roda
                       path: route.path,
                       request_path: r.path,
                       params: filtered_params,
-                      accessor: accessor
+                      accessor: accessor,
+                      on_annotation: route.audit_on_annotation?
                     )
 
                     audit.around(audit_event) do

data/lib/ree_lib/packages/ree_roda/spec/ree_roda/plugins/ree_routes_audit_spec.rb

@@ -49,6 +49,34 @@ RSpec.describe "ree_routes audit" do
       end
     end
 
+    # Клиентская ручка глазами сотрудника: код опознаёт служебное обращение и
+    # помечает его аннотацией.
+    class ReeRodaAuditTest::StaffCmd
+      include ReeActions::DSL
+
+      action :staff_cmd
+
+      def call(access, attrs)
+        ReeAudit.annotate(staff_id: 42)
+        {result: "ok"}
+      end
+    end
+
+    class ReeRodaAuditTest::DeniedStaffCmd
+      include ReeActions::DSL
+
+      action :denied_staff_cmd do
+        link :permission_error, from: :ree_errors
+      end
+
+      PermissionErr = permission_error(:no_access, msg: "not allowed")
+
+      def call(access, attrs)
+        ReeAudit.annotate(staff_id: 42)
+        raise PermissionErr
+      end
+    end
+
     class ReeRodaAuditTest::BoomCmd
       include ReeActions::DSL
 
@@ -71,6 +99,14 @@ RSpec.describe "ree_routes audit" do
           summary "Internal route"
           sections "Admin"
           visibility :internal
+          audit true
+          action :ok_cmd, **opts
+        end
+
+        get "api/internal_undeclared" do
+          summary "Internal route that never mentions audit"
+          sections "Admin"
+          visibility :internal
           action :ok_cmd, **opts
         end
 
@@ -95,10 +131,32 @@ RSpec.describe "ree_routes audit" do
           action :ok_cmd, **opts
         end
 
+        get "api/on_annotation_client" do
+          summary "Client reading its own data"
+          sections "Public"
+          audit :on_annotation
+          action :ok_cmd, **opts
+        end
+
+        get "api/on_annotation_staff" do
+          summary "Staff reading someone else's data"
+          sections "Public"
+          audit :on_annotation
+          action :staff_cmd, **opts
+        end
+
+        get "api/on_annotation_staff_denied" do
+          summary "Staff refused someone else's data"
+          sections "Public"
+          audit :on_annotation
+          action :denied_staff_cmd, **opts
+        end
+
         get "api/internal_denied" do
           summary "Internal route refusing access"
           sections "Admin"
           visibility :internal
+          audit true
           action :denied_cmd, **opts
         end
 
@@ -106,6 +164,7 @@ RSpec.describe "ree_routes audit" do
           summary "Internal route blowing up"
           sections "Admin"
           visibility :internal
+          audit true
           action :boom_cmd, **opts
         end
       end
@@ -225,6 +284,13 @@ RSpec.describe "ree_routes audit" do
     expect(event.accessor).to eq({user: "visitor"})
   end
 
+  it "leaves an internal route that never declared audit out of the trail" do
+    get "api/internal_undeclared"
+
+    expect(last_response.status).to eq(200)
+    expect(test_sink.events).to be_empty
+  end
+
   it "leaves a silenced internal route out of the trail" do
     get "api/internal_silenced"
 
@@ -247,6 +313,31 @@ RSpec.describe "ree_routes audit" do
     expect(test_sink.events.first.status).to eq(:ok)
   end
 
+  it "leaves an unannotated :on_annotation route out of the trail" do
+    get "api/on_annotation_client"
+
+    expect(last_response.status).to eq(200)
+    expect(test_sink.events).to be_empty
+  end
+
+  it "writes an :on_annotation route once the action annotated it" do
+    get "api/on_annotation_staff"
+
+    expect(last_response.status).to eq(200)
+    expect(test_sink.events.size).to eq(1)
+    expect(test_sink.events.first.annotations).to eq({staff_id: 42})
+  end
+
+  # Отказ сотруднику — такой же факт журнала, как и показ данных: аннотация
+  # успевает лечь до того, как действие бросит исключение.
+  it "writes an annotated :on_annotation route that ended in a refusal" do
+    get "api/on_annotation_staff_denied"
+
+    expect(last_response.status).to eq(403)
+    expect(test_sink.events.size).to eq(1)
+    expect(test_sink.events.first.status).to eq(:denied)
+  end
+
   it "records a refusal and still answers 403" do
     get "api/internal_denied"
 

data/lib/ree_lib/packages/ree_routes/package/ree_routes/dsl.rb

@@ -97,6 +97,14 @@ module ReeRoutes
           )
         end
 
+        if !builder.get_route.audit_declared? && admin_path?(path)
+          raise ArgumentError.new(
+            "admin route #{path} must declare audit explicitly: " \
+            "`audit true` to record the call in the access trail, " \
+            "or `audit false` if the route touches nothing worth recording"
+          )
+        end
+
         route = builder.get_route
 
         @dsl.link(route.action.name, from: route.action.package_name)

data/lib/ree_lib/packages/ree_routes/package/ree_routes/route.rb

@@ -16,12 +16,27 @@ class ReeRoutes::Route
     !public?
   end
 
-  # Internal routes are audited unless told otherwise. `visibility` is
-  # mandatory for admin paths (see DSL#define_route), so this list can never
-  # drift away from the set of routes that actually touch client data — which a
-  # hand-maintained list of audited routes inevitably would.
+  # Auditing is declared at the route and nowhere else. Deriving it from
+  # `visibility` would work just as well and read far worse: whether a call
+  # lands in the access trail is exactly the kind of fact that must be visible
+  # where the route is written, not inferred from a flag that means something
+  # else. `audit` is mandatory on admin paths (see DSL#define_route), so the
+  # audited set still cannot drift away from the routes that touch client data.
   def audited?
-    @audit.nil? ? internal? : @audit
+    @audit == true || @audit == :on_annotation
+  end
+
+  def audit_declared?
+    !@audit.nil?
+  end
+
+  # `audit :on_annotation` — a public route that the client calls for its own
+  # data and a staff member calls for someone else's. The event is always
+  # assembled, but it only reaches the sink if the code annotated the call;
+  # auditing such a route unconditionally would bury the staff access under
+  # every ordinary client request.
+  def audit_on_annotation?
+    @audit == :on_annotation
   end
 
   def valid?

data/lib/ree_lib/packages/ree_routes/package/ree_routes/route_builder.rb

@@ -6,6 +6,7 @@ class ReeRoutes::RouteBuilder
   Redirect = Struct.new(:path, :code)
 
   VISIBILITIES = [:public, :internal].freeze
+  AUDIT_MODES = [true, false, :on_annotation].freeze
 
   def initialize
     @route = ReeRoutes::Route.new
@@ -44,10 +45,18 @@ class ReeRoutes::RouteBuilder
     @route.summary = str
   end
 
-  # Needed in exactly two cases: silencing a noisy internal route, and turning
-  # the trail on for a public route that reaches into someone else's data.
-  contract Bool => Bool
+  # Needed in exactly three cases: silencing a noisy internal route, turning
+  # the trail on for a public route that reaches into someone else's data, and
+  # `:on_annotation` for a route both the owner of the data and a staff member
+  # can call — see Route#audit_on_annotation?.
+  contract Or[Bool, Symbol] => Or[Bool, Symbol]
   def audit(value)
+    if !AUDIT_MODES.include?(value)
+      raise ArgumentError.new(
+        "audit should be one of #{AUDIT_MODES.inspect}, got #{value.inspect}"
+      )
+    end
+
     @route.audit = value
   end
 

data/lib/ree_lib/packages/ree_routes/spec/ree_routes/dsl_spec.rb

@@ -76,7 +76,14 @@ RSpec.describe ReeRoutes::DSL, type: [:autoclean] do
         default_warden_scope :user
 
         get "internal_default" do
-          summary "Internal route, audited by default"
+          summary "Internal route that says it is audited"
+          action :cmd, from: :ree_routes_test
+          visibility :internal
+          audit true
+        end
+
+        get "internal_undeclared" do
+          summary "Internal route that never mentions audit"
           action :cmd, from: :ree_routes_test
           visibility :internal
         end
@@ -98,6 +105,12 @@ RSpec.describe ReeRoutes::DSL, type: [:autoclean] do
           action :cmd, from: :ree_routes_test
           audit true
         end
+
+        get "public_on_annotation" do
+          summary "Public route the owner and a staff member both call"
+          action :cmd, from: :ree_routes_test
+          audit :on_annotation
+        end
       end
     end
   end
@@ -131,8 +144,22 @@ RSpec.describe ReeRoutes::DSL, type: [:autoclean] do
     }
 
     expect(routes["internal_default"].audited?).to eq(true)
+    expect(routes["internal_undeclared"].audited?).to eq(false)
+    expect(routes["internal_undeclared"].audit_declared?).to eq(false)
     expect(routes["internal_silenced"].audited?).to eq(false)
+    expect(routes["internal_silenced"].audit_declared?).to eq(true)
     expect(routes["public_default"].audited?).to eq(false)
     expect(routes["public_audited"].audited?).to eq(true)
+    expect(routes["public_on_annotation"].audited?).to eq(true)
+
+    expect(routes["internal_default"].audit_on_annotation?).to eq(false)
+    expect(routes["public_audited"].audit_on_annotation?).to eq(false)
+    expect(routes["public_on_annotation"].audit_on_annotation?).to eq(true)
   }
+
+  it "refuses an audit mode it does not know" do
+    expect {
+      ReeRoutes::RouteBuilder.new.audit(:on_annotations)
+    }.to raise_error(ArgumentError, /audit should be one of/)
+  end
 end

data/lib/ree_lib/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ReeLib
-  VERSION = "1.3.12"
+  VERSION = "1.3.14"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ree_lib
 version: !ruby/object:Gem::Version
-  version: 1.3.12
+  version: 1.3.14
 platform: ruby
 authors:
 - Ruslan Gatiyatov
@@ -277,6 +277,7 @@ files:
 - lib/ree_lib/packages/ree_array/spec/ree_array/functions/wrap_spec.rb
 - lib/ree_lib/packages/ree_array/spec/spec_helper.rb
 - lib/ree_lib/packages/ree_audit/.rspec
+- lib/ree_lib/packages/ree_audit/README.md
 - lib/ree_lib/packages/ree_audit/package/ree_audit.rb
 - lib/ree_lib/packages/ree_audit/package/ree_audit/beans/audit.rb
 - lib/ree_lib/packages/ree_audit/package/ree_audit/config.rb