lethe 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 1b7cb2467505c4efcd2901a83a887f829641721fc6efbd812810df4836f8e38c
+  data.tar.gz: 3d93eb10b84a500315d85a687528321e098a09eada43d264db7865cba14d44e7
+SHA512:
+  metadata.gz: c7a9e090623334708e50d611ab37d68d8c57bfe2a5372bd6bdca222f7b34b3cf3db050e6c381f731e650f2c06851cb43f9093a91cc161792de50795ce0ea419d
+  data.tar.gz: b5351ebc5369d48142aa7e0e8f343a70530e90ad4ba13481a3688881c61b9e99246bf6b083975145c96d264e4987acd666107b9784c5ddfe532dbc54e692a0d1

data/CHANGELOG.md

@@ -0,0 +1,41 @@
+# Changelog
+
+All notable changes to this project are documented in this file. The format
+is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the
+project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## 0.1.0 - 2026-10-27
+
+The first release.
+
+### Added
+
+- `Lethe.define` to declare, per subject such as `User`, what happens to the
+  data that points at it: `destroy`, `scrub`, `keep`, `ignore`,
+  `destroy_attachments`, and `subject` for its own row.
+- Finding what points at a subject from foreign keys, associations, join
+  tables, polymorphic associations, Active Storage attachments and, as a
+  guess, column names.
+- `bin/rails lethe:check`, with text, GitHub Actions and JSON output, and
+  `STRICT=1` to fail on guesses.
+- `bin/rails lethe:export`, which writes the policies as Markdown and JSON,
+  and `CHECK=1` to fail when they are out of date.
+- `Lethe.check!` for tests.
+- `Lethe.erase!` to carry the policies out in one transaction, refusing
+  while the check has problems; `destroy ..., cascade: true`, Procs in
+  `set:`, and `before_erase` / `after_erase` hooks.
+- `Lethe.plan` to see the steps without changing anything, and
+  `Lethe.leftovers` to prove in tests that nothing else was left behind.
+- `bin/rails lethe:sweep` to delete kept rows once `keep ..., for:` or
+  `scrub ..., for:` has passed, through the model with `cascade: true`, and
+  `bin/rails generate lethe:install` for the table it needs.
+- The check also fails on a polymorphic "table.column" without its type,
+  `cascade: true` without a model, and rows kept with a foreign key to a
+  subject row that is deleted.
+- Enum names in `where:` and `set:`, read through the application's model;
+  a name the enum does not have fails the check.
+- A definition for an STI subclass is used for its records, and its kept
+  rows are swept by its own retention.
+- Inside a Rails app, `config/lethe.rb` and every model are loaded on first
+  use, so `Lethe.erase!` works from a controller and sees what CI sees.
+- Messages and exports in English and Japanese.

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 KAIBYAKU K.K
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,326 @@
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="docs/logo/lethe-logo-vertical-dark.svg">
+    <img src="docs/logo/lethe-logo-vertical.svg" alt="lethe: a time to keep, and a time to cast away." width="300">
+  </picture>
+</p>
+
+# lethe [![CI](https://github.com/kaibyaku-jp/lethe/actions/workflows/ci.yml/badge.svg)](https://github.com/kaibyaku-jp/lethe/actions/workflows/ci.yml)
+
+A time to keep, and a time to cast away.
+
+When someone deletes their account, every table that points at them needs a
+decision: delete the rows, overwrite what is personal, or keep them for a
+reason. lethe finds those tables in your Rails app, checks that each one has
+a declared policy, fails CI when a new table has none, and carries the
+policies out when the account goes.
+
+> **Status:** early, before its first release. Until 1.0, minor versions may
+> change how things work; [CHANGELOG.md](CHANGELOG.md) says what changed.
+
+Lethe is the river of forgetting in Greek myth: in Plato's *Republic*, souls
+drink from it before they are born again.
+
+
+## Features
+
+### Finds What Points at a User
+
+Foreign keys, `has_many` and `has_one`, `belongs_to` in other models,
+join tables, polymorphic associations, Active Storage attachments and,
+as a guess, columns named like `user_id` or `*_user_id`.
+
+### Fails on What Nobody Decided
+
+A table added last week with a `user_id` and no policy fails the check. So
+does a policy for a table or column that no longer exists.
+
+### Writes It Down for People
+
+`lethe:export` writes what happens to each kind of data as a Markdown table,
+in English or Japanese, for your privacy policy, app store review or support
+team. CI can check that it is up to date, so every change shows up in review.
+
+### Erases as Declared
+
+`Lethe.erase!(user)` deletes, overwrites and keeps exactly what the
+policies say, in one transaction, and refuses to run while the check has
+problems. `Lethe.leftovers(user)` proves in a test that nothing else was
+left behind.
+
+### Reads the Schema to Check
+
+`lethe:check` and `lethe:export` read the schema and your models'
+associations only, never a row. Only `erase!`, `plan`, `leftovers` and
+`lethe:sweep` touch data, and only when your code calls them.
+
+
+## Supported Versions
+
+* Ruby 3.4, 4.0
+
+* Rails 7.2, 8.0, 8.1
+
+* SQLite, MySQL and PostgreSQL
+
+
+## Installation
+
+Put this in your Gemfile:
+
+```ruby
+group :development, :test do
+  gem "lethe"
+end
+```
+
+To erase accounts with it in production, put it outside the group instead.
+It adds the rake tasks and the methods below, and nothing else: no
+callbacks, middleware or patches to Rails.
+
+To keep rows for a while (`keep ..., for:`), lethe records each erasure in a
+table of its own. Create it, with a `config/lethe.rb` to start from:
+
+```sh
+% bin/rails generate lethe:install
+% bin/rails db:migrate
+```
+
+
+## Declaring Policies
+
+Write the policies in `config/lethe.rb`:
+
+```ruby
+Lethe.define User do
+  # The users row itself: destroy: true, or soft_delete: with a column.
+  subject soft_delete: :deleted_at, scrub: {nickname: "Deleted user", email: nil}
+
+  # Associations of User, by name.
+  destroy :ratings, :favorites, :notifications
+  scrub   :posts, set: {body: ""}, reason: "Quoted posts keep their row"
+  scrub   :questions, where: {status: "answered"}, set: {body: ""}
+  destroy :questions, where: {status: %w[pending declined]}
+  scrub   :orders, set: {phone: nil, address: nil}, for: 7.years, reason: "Bookkeeping"
+
+  # Anything without an association, as "table.column".
+  destroy "follows.follower_id", "follows.following_id"
+  keep    "muted_users.muted_user_id", reason: "Other people's settings"
+
+  destroy_attachments :avatar
+
+  # A guess that does not point at User.
+  ignore "audit_logs.admin_user_id", reason: "Points at AdminUser"
+end
+```
+
+| policy | means |
+| --- | --- |
+| `destroy` | The rows are deleted. |
+| `scrub` | The rows stay, with the columns in `set:` overwritten; `for:` says how long. |
+| `keep` | The rows stay as they are. Needs a `reason:`; `for:` says how long. |
+| `ignore` | The column does not point at the subject. Needs a `reason:`. |
+| `destroy_attachments` | The subject's Active Storage attachments are deleted. |
+| `subject` | How the subject's own row goes. |
+
+`where:` limits a policy to some rows, so the same table can have one policy
+per condition. lethe does not check that the conditions cover every row.
+Enum names work in `where:` and `set:` as they do with your model, and a
+name the enum does not have fails the check.
+
+An STI subclass, such as `Admin < User`, can have a definition of its own.
+It is used instead of the base class's for records of that class, so it
+must declare everything.
+
+Declare more subjects, such as `Organization`, the same way. Declarations
+for the same subject add up.
+
+
+## Checking
+
+```sh
+% bin/rails lethe:check
+User: 41 references (association 36, attachment 3, column name 4, foreign key 31)
+
+✓ 37 have a policy
+✗ chat_messages.user_id             no policy (found by foreign key, association, column name)
+? question_blocks.blocked_user_id   guessed from the column name; declare a policy, or `ignore` it with a reason if it does not point at User
+! config/lethe.rb:14 :badges        User has no association :badges
+
+2 problems, 1 to confirm.
+```
+
+| mark | means | fails the check |
+| --- | --- | --- |
+| `✓` | Has a policy. | no |
+| `✗` | Certainly points at the subject, with no policy. | yes |
+| `!` | A declaration is wrong or names something that is gone. | yes |
+| `?` | Might point at the subject, from its name alone, with no policy. | with `STRICT=1` |
+
+It exits with 0 when there are no problems, 1 when there are, and 2 when
+`config/lethe.rb` cannot be loaded.
+
+| option | meaning |
+| --- | --- |
+| `FORMAT=github` | GitHub Actions annotations; `FORMAT=json` for tools |
+| `STRICT=1` | Fail on `?` too |
+| `LOCALE=ja` | Messages in Japanese |
+
+In a test instead:
+
+```ruby
+it "has a policy for every table that points at a user" do
+  Lethe.check!
+end
+```
+
+Inside a Rails app, lethe loads `config/lethe.rb` the first time it is
+needed, and loads every model once where `eager_load` is off, so that
+development and test see the same associations as CI.
+
+
+## Exporting
+
+```sh
+% bin/rails lethe:export LOCALE=ja
+lethe: wrote docs/lethe.md
+lethe: wrote docs/lethe.json
+```
+
+```markdown
+| Data | When they leave | Kept for | Reason |
+| --- | --- | --- | --- |
+| The User itself | Kept as deleted (deleted_at), with nickname, email overwritten |  |  |
+| Avatar (attachment) | Deleted |  |  |
+| Order | Kept, with phone, address overwritten | 7 years | Bookkeeping |
+```
+
+Names come from your app's translations, such as `activerecord.models.order`.
+Nothing is written while the check has problems. `CHECK=1` compares instead
+of writing and fails when the files are out of date; run it in CI:
+
+```sh
+bin/rails lethe:check lethe:export CHECK=1 FORMAT=github
+```
+
+
+## Erasing
+
+```ruby
+class AccountsController < ApplicationController
+  def destroy
+    Lethe.erase!(current_user)
+    reset_session
+    head :no_content
+  end
+end
+```
+
+`erase!` checks the subject's definition first and raises `Lethe::Unclean`
+without touching anything when it has problems. Then, in one transaction,
+it runs the `before_erase` hooks, deletes rows (those with foreign keys to
+other deleted tables first), overwrites columns, marks or deletes the
+subject's own row, runs the `after_erase` hooks and records the erasure. If
+anything fails, all of it is rolled back. Attachments are detached in the
+transaction and their files purged after it commits.
+
+| erasing | how |
+| --- | --- |
+| `destroy` | `delete_all` on the table, with no callbacks; `cascade: true` uses the model's `destroy_all`, so `dependent:` associations and attachments go too |
+| `scrub` | `update_all`; a Proc in `set:` is called for each row, given the row if it takes an argument |
+| `keep` | Nothing now; `lethe:sweep` deletes the rows once `for:` has passed (as does `scrub ..., for:`) |
+| polymorphic | The type always becomes part of the condition, so erasing User 5 never touches Post 5's rows |
+
+```ruby
+Lethe.define User do
+  subject soft_delete: :deleted_at,
+    scrub: {screen_name: ->(user) { "deleted_#{user.id}_#{SecureRandom.hex(8)}" }}
+  destroy :episodes, cascade: true
+  destroy "ratings.target_id", where: {target_type: "User"}
+
+  before_erase { |user| Follow.decrement_counters_for(user) }
+  after_erase { |user| DeleteFirebaseAccountJob.perform_later(user.firebase_uid) }
+end
+```
+
+Hooks run inside the transaction. Anything that cannot be rolled back,
+such as a call to another service, belongs in a job enqueued from
+`after_erase`.
+
+`Lethe.plan(user)` lists the steps and how many rows each touches, without
+changing anything. `erase!` returns the rows per table and policy, and
+sends an `erase.lethe` notification with the subject's class, id and those
+counts, never anything personal. Files that could not be purged after the
+commit are logged and listed as `purge_failures` (blob ids) rather than
+raised, since the erasure itself has already committed.
+
+Running `erase!` again is safe: what is gone stays gone, and a soft
+delete keeps the time of the first erasure.
+
+### Proving It in Tests
+
+```ruby
+it "leaves nothing of a user behind but what we keep" do
+  user = create(:user, :with_everything)
+  Lethe.erase!(user)
+
+  expect(Lethe.leftovers(user)).to be_empty
+end
+```
+
+`leftovers` lists rows that should be gone, columns that should have been
+overwritten (values from a Proc are not compared), attachments, and the
+subject's row.
+
+### Sweeping
+
+Run this daily, from cron or a recurring job:
+
+```sh
+% bin/rails lethe:sweep
+lethe: swept 3 from User :inquiries
+```
+
+It deletes the rows of each `keep ..., for:` and `scrub ..., for:` whose
+time has passed since their subject was first erased, one subject at a time
+in a transaction. If other rows still point at them through a foreign key,
+it stops there; add `cascade: true` to delete through the model and its
+`dependent:` associations instead.
+
+
+## Configuring lethe
+
+In `config/initializers/lethe.rb`, or anywhere loaded before the tasks run:
+
+```ruby
+Lethe.configure do |config|
+  config.definition_path = "config/lethe.rb"   # the default
+  config.export_path = "docs/lethe"            # .md and .json are added
+  config.skip_tables = %w[audit_logs]          # besides Rails' own
+  config.guess_columns[User] = %w[follower_id following_id questioner_id]
+  config.purge = :later                        # or :now, for erased attachments' files
+end
+```
+
+
+## Questions, Feedback
+
+Open an issue on GitHub. Report security problems privately as described in
+[SECURITY.md](SECURITY.md).
+
+
+## Contributing to lethe
+
+Fork, fix, then send a pull request. [CONTRIBUTING.md](CONTRIBUTING.md) has the
+details.
+
+```sh
+% nix develop          # or use Ruby 3.4+ without Nix
+% bundle install
+% bundle exec rake
+```
+
+
+## Copyright
+
+Copyright (c) 2026- KAIBYAKU K.K. See [LICENSE](LICENSE) for further details.

data/lib/generators/lethe/install/install_generator.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record"
+
+module Lethe
+  module Generators
+    # bin/rails generate lethe:install: the lethe_erasures table and a
+    # config/lethe.rb to start from.
+    class InstallGenerator < Rails::Generators::Base
+      include ActiveRecord::Generators::Migration
+
+      source_root File.expand_path("templates", __dir__)
+
+      def create_migration_file
+        migration_template "create_lethe_erasures.rb.tt", File.join(db_migrate_path, "create_lethe_erasures.rb")
+      end
+
+      def create_definitions
+        template "lethe.rb.tt", "config/lethe.rb", skip: true
+      end
+    end
+  end
+end

data/lib/generators/lethe/install/templates/create_lethe_erasures.rb.tt

@@ -0,0 +1,14 @@
+# One row each time lethe erases a subject or sweeps its kept rows: the
+# subject's class and id, and the number of rows per table. Nothing personal.
+class CreateLetheErasures < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    create_table :lethe_erasures do |t|
+      t.string :subject_type, null: false
+      t.string :subject_id, null: false
+      t.string :kind, null: false
+      t.text :counts, null: false
+      t.datetime :created_at, null: false
+    end
+    add_index :lethe_erasures, [:subject_type, :subject_id]
+  end
+end

data/lib/generators/lethe/install/templates/lethe.rb.tt

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+# What happens to the data that points at a user when they leave.
+# bin/rails lethe:check lists what still needs a policy.
+Lethe.define "User" do
+  subject destroy: true
+end

data/lib/lethe/checker.rb

@@ -0,0 +1,205 @@
+# frozen_string_literal: true
+
+module Lethe
+  # Compares one definition with what Discovery finds, and returns a Result
+  # for every reference and for every declaration that is wrong.
+  class Checker
+    def initialize(definition, models:, connection:, configuration:)
+      @definition = definition
+      @models = models
+      @connection = connection
+      @configuration = configuration
+    end
+
+    def call
+      @subject = resolve_subject
+      return [invalid(nil, "subject_not_a_model", location: first_location, subject: subject_name)] unless @subject
+
+      discovery = Discovery.new(@subject, models: models, connection: @connection, configuration: @configuration)
+      @tables = discovery.tables
+      by_key, problems = declared_keys
+
+      results = problems + contradictions(by_key)
+      results.concat(discovery.call.map { |reference| result_for(reference, by_key.delete(reference.key) || []) })
+      # Declared and valid, but not something Discovery found: trust the declaration.
+      results.concat(by_key.map { |key, policies| result_for(Reference.column(*key.split(".", 2), :declared), policies) })
+      results << subject_result
+      results.concat(blocked_by_foreign_keys(results))
+    end
+
+    # The schema questions Rules asks.
+    def column_names(table)
+      @column_names ||= {}
+      @column_names[table] ||= @connection.columns(table).map(&:name)
+    end
+
+    def model_for_table(table)
+      models.find do |model|
+        !model.abstract_class? && model.table_name == table && model.base_class == model
+      rescue ActiveRecord::ActiveRecordError
+        false
+      end
+    end
+
+    def primary_key(table)
+      @connection.primary_key(table)
+    end
+
+    private
+
+    def subject_name
+      @definition.subject_name
+    end
+
+    def models
+      @models || Lethe.application_models
+    end
+
+    def resolve_subject
+      model = subject_name.safe_constantize
+      (model.is_a?(Class) && model < ActiveRecord::Base && !model.abstract_class?) ? model : nil
+    end
+
+    def first_location
+      @definition.policies.first&.location || @definition.subject_policy&.location
+    end
+
+    # Returns [key, nil], or [nil, an :invalid Result].
+    def resolve(policy)
+      target = policy.target
+      case target.kind
+      when :association then resolve_association(policy)
+      when :attachment then resolve_attachment(policy)
+      else resolve_column(policy, target.table, target.column)
+      end
+    end
+
+    def resolve_association(policy)
+      name = policy.target.name
+      reflection = @subject.reflect_on_association(name)
+      return [nil, invalid(policy, "no_association", name: name)] unless reflection
+      return [nil, invalid(policy, "through_association", name: name)] if reflection.through_reflection?
+
+      case reflection.macro
+      when :has_many, :has_one
+        table = begin
+          reflection.klass.table_name
+        rescue NameError, ArgumentError
+          return [nil, invalid(policy, "association_class_missing", name: name)]
+        end
+        resolve_column(policy, table, reflection.foreign_key.to_s)
+      when :has_and_belongs_to_many
+        resolve_column(policy, reflection.join_table.to_s, reflection.foreign_key.to_s)
+      else
+        [nil, invalid(policy, "belongs_to_association", name: name)]
+      end
+    end
+
+    def resolve_attachment(policy)
+      name = policy.target.name
+      found = @subject.respond_to?(:reflect_on_attachment) && @subject.reflect_on_attachment(name)
+      found ? ["attachment:#{name}", nil] : [nil, invalid(policy, "no_attachment", name: name)]
+    end
+
+    def resolve_column(policy, table, column)
+      return [nil, invalid(policy, "no_table", table: table)] unless @tables.include?(table)
+      return [nil, invalid(policy, "no_column", table: table, column: column)] unless column_names(table).include?(column)
+
+      ["#{table}.#{column}", nil]
+    end
+
+    # { "table.column" => [valid policies] }, and the problems with the rest.
+    def declared_keys
+      rules = Rules.new(@subject, self) { |policy, message_key, **values| invalid(policy, message_key, **values) }
+      problems = []
+      by_key = Hash.new { |hash, key| hash[key] = [] }
+      @definition.policies.each do |policy|
+        key, error = resolve(policy)
+        next problems << error if error
+
+        found = rules.call(policy, key)
+        found.empty? ? by_key[key] << policy : problems.concat(found)
+      end
+      [by_key, problems]
+    end
+
+    # When the subject's row is deleted, rows that stay and point at it
+    # through a foreign key would make the delete fail.
+    def blocked_by_foreign_keys(results)
+      return [] unless @definition.subject_policy&.destroy
+
+      results.filter_map do |result|
+        next unless result.status == :covered && result.reference&.vias&.include?(:foreign_key)
+
+        staying = result.policies.find { |policy| %i[keep scrub].include?(policy.action) }
+        invalid(staying, "kept_rows_block_destroy", key: result.key) if staying
+      end
+    end
+
+    # Two policies for the same rows that say different things. The rows stay
+    # covered; the second declaration is reported.
+    def contradictions(by_key)
+      by_key.flat_map do |key, policies|
+        unconditional = policies.select { |policy| policy.where.nil? }
+        next [] if unconditional.map(&:action).uniq.size < 2
+
+        unconditional.drop(1).map do |policy|
+          invalid(policy, "contradiction", key: key, other: unconditional.first.location)
+        end
+      end
+    end
+
+    def result_for(reference, policies)
+      status =
+        if policies.empty?
+          reference.certain? ? :missing : :guess
+        elsif policies.all?(&:ignore?)
+          :ignored
+        else
+          :covered
+        end
+      Result.new(
+        status: status, subject: subject_name, key: reference.key, reference: reference,
+        policies: policies, message: nil, location: policies.first&.location
+      )
+    end
+
+    def subject_result
+      policy = @definition.subject_policy
+      message = policy ? subject_problem(policy) : nil
+      Result.new(
+        status: if policy.nil?
+                  :missing
+                else
+                  (message ? :invalid : :covered)
+                end, subject: subject_name, key: "subject",
+        reference: nil, policies: [], message: message, location: policy&.location
+      )
+    end
+
+    # What is wrong with the subject declaration, as [message key, values], or nil.
+    def subject_problem(policy)
+      table = @subject.table_name
+      columns = column_names(table)
+      return ["subject_both", {}] if policy.destroy && policy.soft_delete
+      return ["subject_neither", {}] if !policy.destroy && policy.soft_delete.nil?
+      if policy.soft_delete && !columns.include?(policy.soft_delete.to_s)
+        return ["unknown_column", {option: "soft_delete", table: table, column: policy.soft_delete.to_s}]
+      end
+      return ["subject_scrub_not_hash", {}] unless policy.scrub.is_a?(Hash)
+
+      unknown = policy.scrub.keys.map(&:to_s) - columns
+      return ["unknown_column", {option: "scrub", table: table, column: unknown.first}] if unknown.any?
+
+      column, value = Enums.unknown(@subject, policy.scrub).first
+      ["unknown_enum_value", {option: "scrub", table: table, column: column, value: value.inspect}] if column
+    end
+
+    def invalid(policy, message_key, location: policy&.location, subject: subject_name, **values)
+      Result.new(
+        status: :invalid, subject: subject, key: policy ? policy.target.to_s : subject, reference: nil,
+        policies: policy ? [policy] : [], message: [message_key, values], location: location
+      )
+    end
+  end
+end