test_budget 0.0.0 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bb6e4e3b3722807341b35ea3446e54c5bf989295702c4670388a8c7e337fff1d
-  data.tar.gz: 6ec3ab375a2c22c6b57659b8931b3aef08019e321ea98aca4f9db8e5c681cd2c
+  metadata.gz: 0e19638c3b0f42996339e6a676005e07193ee34017b67c636f241ec04c91b59d
+  data.tar.gz: 68fff54ed7f1f4e888e18bb15db65888de38b5c76e6d5ff8fc3188cafdeb83d0
 SHA512:
-  metadata.gz: '08c2a82158b3202c4fa5353d59bb98e0990d3f5d9cd71ea1bfdbcacb399565c8b60e6448b5277f7342e6f8166520c6d684e204a23f661511c6a3c641b7d08984'
-  data.tar.gz: fff5a694e1d95454a6eb95343b981b470b936454e6c2286aa810c7e8f2eca24f8de6dc77debc9308b60e3bd43f5cddf46336871f599bc2c3ead542f0e2a2354e
+  metadata.gz: ab0d1bef70892fa3ae89a504e380498307e97646b2f0a98dbd8ec20bd5cb858d1e99b5725fd043d7cfecf822cadf5a6e9c0a1e8760fe89791057b09916f3d0d6
+  data.tar.gz: 5ef882443f34709c32785eb39ca669ea50a46fafee70e173ceaae534cb2da30845b53ba893d79554670dac5a30b63e9f1ac55263fc2e43029ed7082e049abac1

data/.test_budget.yml

@@ -0,0 +1,4 @@
+timings_path: tmp/test_timings.json
+
+suite:
+  max_duration: 1

data/README.md

@@ -1,39 +1,267 @@
-# TestBudget
+# Test Budget
 
-TODO: Delete this and the text below, and describe your gem
+> Prevent slow tests from creeping into your suite.
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/test_budget`. To experiment with that code, run `bin/console` for an interactive prompt.
+Test suites get slow one test at a time. By the time you notice, your CI takes
+40 minutes and nobody wants to touch it.
 
-## Installation
+Test Budget is **a linter for test performance**. It reads your test results
+after the run, checks durations against configured budgets, and fails if
+anything goes over.
 
-TODO: Replace `UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG` with your gem name right after releasing it to RubyGems.org. Please do not do it earlier due to security reasons. Alternatively, replace this section with instructions to install your gem from git if you don't plan to release to RubyGems.org.
+It doesn't change how your tests run. It just tells you when they're too slow —
+before it gets worse.
 
-Install the gem and add to the application's Gemfile by executing:
+## Install
+
+Add to your Gemfile:
+
+```ruby
+gem "test_budget", github: "thoughtbot/test_budget"
+```
+
+> [!NOTE]
+> Test Budget currently supports **RSpec only**. Minitest support is not yet available.
+
+## Quick start
+
+Generate a starter config from an existing RSpec JSON results file:
+
+```bash
+bundle exec test_budget init tmp/test_timings.json
+```
+
+When given a results file, `init` derives budgets from your actual data (99th
+percentile + 10% tolerance, rounded to the nearest 0.5s). If you don't have a
+results file yet, run without arguments to generate a config with Rails
+defaults:
+
+```bash
+bundle exec test_budget init
+```
+
+Use `--force` to overwrite an existing `.test_budget.yml`.
+
+`estimate` is an alias for `init`. Use whichever name feels right:
+
+```bash
+bundle exec test_budget estimate tmp/test_timings.json
+```
+
+Then run the audit after your tests:
+
+```bash
+bundle exec test_budget audit
+```
+
+Example output:
+
+```
+Test budget: 1 violation(s) found
+
+  1) spec/system/signup_spec.rb -- creates account (11.20s) exceeds system limit (6.00s)
+     To allowlist, run:
+     bundle exec test_budget allowlist spec/system/signup_spec.rb:15 --reason "<reason>"
+```
+
+## RSpec JSON results file
+
+> [!TIP]
+> You can skip this step and run `test_budget init` without a results
+> file to get started with Rails defaults right away.
+
+Add to your RSpec configuration or CI command:
+
+```bash
+bundle exec rspec --format json --out tmp/test_timings.json
+```
+
+Or combine with your usual formatter:
+
+```bash
+bundle exec rspec --format progress --format json --out tmp/test_timings.json
+```
+
+### Parallel test runners
+
+If you use [`parallel_tests`][parallel_tests],
+`$TEST_ENV_NUMBER` in command arguments is replaced per worker (empty string for
+worker 1, `2` for worker 2, etc.). Use it to write a separate output file per
+worker:
 
 ```bash
-bundle add UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+bundle exec parallel_rspec -- --format json --out 'tmp/test_timings$TEST_ENV_NUMBER.json'
+```
+
+This produces `test_timings.json`, `test_timings2.json`, `test_timings3.json`,
+etc. Then set your `timings_path` to a glob pattern:
+
+```yaml
+timings_path: "tmp/test_timings*.json"
 ```
 
-If bundler is not being used to manage dependencies, install the gem by executing:
+If you use [`flatware`][flatware], each worker appends its results to the same
+output file. Test Budget handles this automatically:
 
 ```bash
-gem install UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+flatware rspec --format json --out tmp/test_timings.json
 ```
 
-## Usage
+## Configuration
 
-TODO: Write usage instructions here
+The `init` command creates a `.test_budget.yml` in your project root. You can
+also create one manually:
 
-## Development
+```yaml
+timings_path: tmp/test_timings.json
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+suite:
+  max_duration: 300 # seconds
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+per_test_case:
+  default: 2
+  system: 6
+  request: 3
+  model: 1.5
 
-## Contributing
+allowlist:
+  - test_case: "spec/services/invoice_pdf_spec.rb -- generates PDF with line items"
+    reason: "PDF generation is inherently slow, tracking in JIRA-1234"
+    expires_on: "2025-06-01"
+```
+
+- **`timings_path`** (required) — path (or glob pattern) to the RSpec JSON output file(s).
+- **`suite.max_duration`** — total duration budget for the entire suite.
+- **`per_test_case.default`** — default per-test limit. Applies to any type without a specific limit.
+- **`per_test_case.<type>`** — per-test limit for a specific type. Types are inferred from file paths by singularizing the directory name (`spec/models/` -> `model`, `spec/features/` -> `feature`, `spec/system/` -> `system`, etc).
+- **`allowlist`** — known slow tests to skip. Each entry requires an `expires_on` date (YYYY-MM-DD). Expired entries stop exempting their tests. Use this as a temporary escape hatch, not a permanent solution.
+
+> [!IMPORTANT]
+> At least one limit (`suite.max_duration`, `per_test_case.default`, or a type-specific limit) must be configured.
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/test_budget.
+## Audit
+
+```bash
+bundle exec test_budget audit
+```
+
+Use `--budget` to point to a different config file:
+
+```bash
+bundle exec test_budget audit --budget config/test_budget.yml
+```
+
+Use `--tolerant` to apply a 10% tolerance to all limits. This is useful on
+shared CI infrastructure where CPU contention causes small fluctuations in test
+durations:
+
+```bash
+bundle exec test_budget audit --tolerant
+```
+
+With `--tolerant`, a test only fails if it exceeds the limit by more than 10%
+(e.g., a 5s limit becomes an effective 5.5s limit).
+
+Exit code is `0` when all tests are within budget, `1` when there are violations.
+
+## Allowlist
+
+You can allowlist individual tests via the CLI:
+
+```bash
+bundle exec test_budget allowlist spec/models/user_spec.rb:10 --reason "Tracking in JIRA-1234"
+```
+
+Entries are created with a 60-day expiration by default. Edit the `expires_on`
+date in the YAML file if you need a different window.
+
+### Pruning obsolete entries
+
+Over time, allowlisted tests may be fixed or removed. Use `prune` to clean up
+entries that are no longer needed:
+
+```bash
+bundle exec test_budget prune
+```
+
+This removes **stale** entries (test no longer exists) and **unnecessary** entries
+(test is now within budget). The `audit` command also warns about these entries
+so you know when it's time to prune.
+
+### Example output
+
+```
+Test budget: 2 violation(s) found
+
+  1) spec/models/user_spec.rb -- User#full_name (2.50s) exceeds model limit (1.00s)
+     To allowlist, run:
+     bundle exec test_budget allowlist spec/models/user_spec.rb:10 --reason "<reason>"
+
+  2) Suite total (650.00s) exceeds limit (600.00s)
+```
+
+## CI integration
+
+Run the audit after your test suite:
+
+```yaml
+# .github/workflows/ci.yml
+- run: bundle exec rspec --format progress --format json --out tmp/test_timings.json
+- run: bundle exec test_budget audit
+```
+
+The second step fails the build if any test exceeds its budget.
+
+## I have violations. Now what?
+
+Violations mean tests are slower than you decided they should be. You have
+options:
+
+- **Make the tests faster.** This is the best option. Look for unnecessary
+  setup, N+1 queries, slow external calls that could be stubbed. Can the same
+  behavior be exercised with a faster test type? (e.g. system -> request,
+  request -> model)
+- **Split the work.** A test doing too much can often be broken into
+  focused scenarios.
+- **Parallelize.** Tools like [`parallel_tests`][parallel_tests] and
+  [`flatware`][flatware] reduce wall time without changing individual test
+  durations, but consider also setting per-test budgets to keep individual tests
+  honest.
+- **Upgrade infrastructure.** Faster CI (or developer) machines buy time.
+- **Allowlist temporarily.** If a fix isn't immediate, add the test to the
+  allowlist and create a ticket. This keeps the budget enforced for everything
+  else.
+
+The goal isn't zero violations on day one. It's to stop the bleeding and make
+test performance visible.
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+The gem is available as open source under the terms of the [MIT
+License](https://opensource.org/licenses/MIT).
+
+## Code of Conduct
+
+Everyone interacting in the Test Budget project's codebases, issue trackers,
+chat rooms and mailing lists is expected to follow the [code of
+conduct](https://github.com/thoughtbot/test_budget/blob/main/CODE_OF_CONDUCT.md).
+
+<!-- START /templates/footer.md -->
+
+## About thoughtbot
+
+![thoughtbot](https://thoughtbot.com/thoughtbot-logo-for-readmes.svg)
+
+This repo is maintained and funded by thoughtbot, inc. The names and logos for
+thoughtbot are trademarks of thoughtbot, inc.
+
+We love open source software! See [our other projects][community]. We are
+[available for hire][hire].
+
+[community]: https://thoughtbot.com/community?utm_source=github&utm_medium=readme&utm_campaign=test_budget
+[hire]: https://thoughtbot.com/hire-us?utm_source=github&utm_medium=readme&utm_campaign=test_budget
+
+<!-- END /templates/footer.md -->
+
+[parallel_tests]: https://github.com/grosser/parallel_tests
+[flatware]: https://github.com/briandunn/flatware

data/Rakefile

@@ -6,3 +6,4 @@ require "rspec/core/rake_task"
 RSpec::Core::RakeTask.new(:spec)
 
 task default: :spec
+task build: :spec

data/exe/test_budget

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "test_budget"
+
+exit TestBudget::CLI.new.call(ARGV)

data/lib/test_budget/allowlist.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+require "date"
+
+module TestBudget
+  class Allowlist
+    def initialize(raw_entries = [])
+      @entries = raw_entries.map do |entry|
+        Entry.new(
+          test_case_key: entry["test_case"],
+          reason: entry["reason"],
+          expires_on: parse_date(entry["expires_on"])
+        )
+      end
+    end
+
+    def allowed?(key)
+      @entries.find { |e| e.matches?(key) }
+    end
+
+    def add(key, reason:, expires_on:)
+      raise Error, "#{key} is already allowlisted" if allowed?(key)
+
+      entry = Entry.new(test_case_key: key, reason: reason, expires_on: expires_on)
+      @entries << entry
+      entry
+    end
+
+    def prune(test_run, budget)
+      removed, @entries = @entries.partition { |entry| entry.obsolete?(test_run, budget) }
+      removed
+    end
+
+    def to_a
+      @entries.dup
+    end
+
+    private
+
+    def parse_date(raw)
+      Date.parse(raw.to_s)
+    rescue Date::Error, TypeError
+      raise Error, "expires_on is required and must be a valid date (YYYY-MM-DD)"
+    end
+
+    Entry = Data.define(:test_case_key, :reason, :expires_on) do
+      def expired? = Date.today > expires_on
+
+      def matches?(key)
+        test_case_key == key
+      end
+
+      def obsolete?(test_run, budget)
+        test_case = test_run.test_cases.find { |tc| tc.key == test_case_key }
+
+        if test_case.nil?
+          :stale
+        elsif test_case.within?(budget)
+          :unnecessary
+        end
+      end
+
+      def to_h
+        {"test_case" => test_case_key, "reason" => reason, "expires_on" => expires_on.to_s}
+      end
+    end
+  end
+end

data/lib/test_budget/audit.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module TestBudget
+  class Audit
+    Result = Data.define(:violations, :warnings) do
+      def passed? = violations.empty?
+    end
+
+    def initialize(budget_path:, tolerant: false)
+      @budget_path = budget_path
+      @tolerant = tolerant
+    end
+
+    def perform
+      budget = Budget.load(@budget_path)
+      test_run = Parser::Rspec.parse(budget.timings_path)
+      result = Auditor.new(budget, tolerant: @tolerant).audit(test_run)
+      Reporter.new(budget_path: @budget_path).report(result)
+
+      result
+    end
+  end
+end

data/lib/test_budget/auditor.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+module TestBudget
+  class Auditor
+    TOLERANCE = 0.1
+
+    def initialize(budget, tolerant: false)
+      @budget = tolerant ? budget.inflate_by(TOLERANCE) : budget
+    end
+
+    def audit(test_run)
+      Audit::Result.new(
+        violations: violations_for(test_run),
+        warnings: warnings_for(test_run)
+      )
+    end
+
+    private
+
+    def violations_for(test_run)
+      violations = test_run.test_cases.filter_map { |tc| tc.violation_for(@budget) }
+      violations << test_run.over?(@budget)
+      violations.compact
+    end
+
+    def warnings_for(test_run)
+      @budget.allowlist.to_a.filter_map do |entry|
+        kind = entry.obsolete?(test_run, @budget)
+        Warning.new(entry: entry, kind: kind) if kind
+      end
+    end
+  end
+end

data/lib/test_budget/budget/estimate.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+require "yaml"
+
+module TestBudget
+  class Budget
+    class Estimate
+      DEFAULT_TIMINGS_PATH = "tmp/test_timings.json"
+      TOLERANCE = 0.10
+      PER_TEST_CASE_DEFAULTS = {"default" => 3, "system" => 6, "request" => 3, "model" => 1.5}.freeze
+
+      def initialize(timings_path: nil, force: false)
+        @timings_path = timings_path
+        @force = force
+      end
+
+      def generate
+        guard_existing_config!
+
+        if @timings_path
+          generate_from_results
+        else
+          generate_defaults
+        end
+
+        puts "Created #{DEFAULT_BUDGET_PATH}"
+      end
+
+      private
+
+      def guard_existing_config!
+        return if @force || !File.exist?(DEFAULT_BUDGET_PATH)
+
+        raise TestBudget::Error, "#{DEFAULT_BUDGET_PATH} already exists. Use --force to overwrite."
+      end
+
+      def generate_from_results
+        test_run = Parser::Rspec.parse(@timings_path)
+        suite_budget = (test_run.suite_duration * (1 + TOLERANCE)).ceil
+        per_test_case_limits = derive_per_test_case(test_run.test_cases)
+
+        budget = build_budget(
+          timings_path: @timings_path,
+          suite_max_duration: suite_budget,
+          per_test_case_limits: per_test_case_limits
+        )
+        budget.save
+      end
+
+      def generate_defaults
+        build_budget(
+          timings_path: DEFAULT_TIMINGS_PATH,
+          per_test_case_limits: PER_TEST_CASE_DEFAULTS
+        ).save
+      end
+
+      def build_budget(timings_path:, per_test_case_limits:, suite_max_duration: nil)
+        types = per_test_case_limits.except("default").transform_keys(&:to_sym)
+
+        Budget.new(
+          path: DEFAULT_BUDGET_PATH,
+          timings_path: timings_path,
+          suite: Budget::Suite.new(max_duration: suite_max_duration),
+          per_test_case: Budget::PerTestCase.new(
+            default: per_test_case_limits["default"],
+            types: types
+          ),
+          allowlist: Allowlist.new
+        )
+      end
+
+      def derive_per_test_case(test_cases)
+        grouped = test_cases.group_by { |tc| tc.type.to_s }
+
+        PER_TEST_CASE_DEFAULTS.merge(
+          grouped.transform_values { |cases|
+            Statistics.p99(cases.map(&:duration), tolerance: TOLERANCE)
+          }
+        )
+      end
+    end
+  end
+end

data/lib/test_budget/budget.rb

@@ -0,0 +1,107 @@
+# frozen_string_literal: true
+
+require "date"
+require "yaml"
+
+module TestBudget
+  class Budget < Data.define(:path, :timings_path, :suite, :per_test_case, :allowlist)
+    DEFAULT_EXPIRATION_DAYS = 60
+
+    Suite = Data.define(:max_duration)
+    PerTestCase = Data.define(:default, :types)
+
+    def self.estimate(...) = Estimate.new(...).generate
+
+    def self.load(path)
+      config = YAML.safe_load_file(path) || {}
+      suite_config = config["suite"] || {}
+      per_test_case_config = config["per_test_case"] || {}
+      types = per_test_case_config.except("default").transform_keys(&:to_sym)
+
+      budget = new(
+        path: path,
+        timings_path: config["timings_path"],
+        suite: Suite.new(max_duration: suite_config["max_duration"]),
+        per_test_case: PerTestCase.new(
+          default: per_test_case_config["default"],
+          types: types
+        ),
+        allowlist: Allowlist.new(config["allowlist"] || [])
+      )
+
+      raise TestBudget::Error, "timings_path is required in budget file" unless budget.timings_path
+      raise TestBudget::Error, "No limits configured. Set suite.max_duration or per_test_case limits" unless budget.limits_set?
+
+      budget
+    rescue Errno::ENOENT
+      raise TestBudget::Error, "Budget file not found: #{path}"
+    end
+
+    def exempt?(test_case)
+      entry = allowlist.allowed?(test_case.key)
+
+      entry && !entry.expired?
+    end
+
+    def limit_for(test_case)
+      per_test_case.types[test_case.type] || per_test_case.default
+    end
+
+    def add_to_allowlist(locator, reason:)
+      test_run = Parser::Rspec.parse(timings_path)
+      test_case = TestCase.find_by_location!(test_run.test_cases, locator)
+
+      allowlist.add(test_case.key, reason: reason, expires_on: Date.today + DEFAULT_EXPIRATION_DAYS).tap { save }
+    end
+
+    def prune_allowlist
+      test_run = Parser::Rspec.parse(timings_path)
+      removed = allowlist.prune(test_run, self)
+      save if removed.any?
+      removed
+    end
+
+    def inflate_by(percent)
+      factor = 1 + percent
+
+      with(
+        suite: Suite.new(max_duration: suite.max_duration&.*(factor)),
+        per_test_case: PerTestCase.new(
+          default: per_test_case.default&.*(factor),
+          types: per_test_case.types.transform_values { |v| v * factor }
+        )
+      )
+    end
+
+    def limits_set?
+      suite.max_duration || per_test_case.default || per_test_case.types.any?
+    end
+
+    def save
+      File.write(path, YAML.dump(to_h))
+    end
+
+    private
+
+    def to_h
+      deep_compact_blank(
+        "timings_path" => timings_path,
+        "suite" => {"max_duration" => suite.max_duration},
+        "per_test_case" => {
+          "default" => per_test_case.default,
+          **per_test_case.types.transform_keys(&:to_s)
+        },
+        "allowlist" => allowlist.to_a.map(&:to_h)
+      )
+    end
+
+    def deep_compact_blank(hash)
+      hash.each_with_object({}) do |(key, value), result|
+        value = deep_compact_blank(value) if value.is_a?(Hash)
+        next if value.nil?
+        next if value.respond_to?(:empty?) && value.empty?
+        result[key] = value
+      end
+    end
+  end
+end

data/lib/test_budget/cli.rb

@@ -0,0 +1,133 @@
+# frozen_string_literal: true
+
+require "optparse"
+require "argument_parser"
+
+module TestBudget
+  class CLI
+    def call(argv)
+      args = argv.dup
+
+      return print_help if args.empty? || (args & %w[--help -h]).any?
+      return print_version if (args & %w[--version -v]).any?
+
+      parsed = command_parser.parse!(args)
+
+      case parsed[:command]
+      in "audit" then run_audit(args)
+      in "allowlist" then run_allowlist(args)
+      in "prune" then run_prune(args)
+      in "init" then run_init(args)
+      in "help" then print_help
+      end
+    rescue ArgumentParser::ParseError, TestBudget::Error, OptionParser::MissingArgument => e
+      warn e.message
+      1
+    end
+
+    private
+
+    def command_parser
+      ArgumentParser.build do
+        required :command, pattern: {"audit" => "audit", "allowlist" => "allowlist", "prune" => "prune",
+                                     "init" => "init", "estimate" => "init", "help" => "help"}
+      end
+    end
+
+    def print_help
+      puts help_text
+      0
+    end
+
+    def print_version
+      puts "test_budget #{TestBudget::VERSION}"
+      0
+    end
+
+    def help_text
+      <<~HELP
+        Usage: test_budget <command> [options]
+
+        Commands:
+          audit       Check test results against budget
+          allowlist   Exclude a test from budget checks
+          prune       Remove obsolete allowlist entries
+          init        Generate starter .test_budget.yml config
+          estimate    Alias for init
+          help        Show this help message
+
+        Options:
+          -h, --help     Show this help message
+          -v, --version  Show version
+      HELP
+    end
+
+    def run_audit(args)
+      budget_path = DEFAULT_BUDGET_PATH
+      tolerant = false
+
+      OptionParser.new do |opts|
+        opts.banner = "Usage: test_budget audit [options]"
+        opts.on("--budget PATH", "Path to budget file") { |path| budget_path = path }
+        opts.on("--tolerant", "Apply 10% tolerance to limits") { tolerant = true }
+      end.parse!(args)
+
+      result = Audit.new(budget_path: budget_path, tolerant: tolerant).perform
+
+      result.passed? ? 0 : 1
+    end
+
+    def run_allowlist(args)
+      budget_path = DEFAULT_BUDGET_PATH
+      reason = nil
+
+      OptionParser.new do |opts|
+        opts.banner = "Usage: test_budget allowlist FILE:LINE --reason REASON [options]"
+        opts.on("--budget PATH", "Path to budget file") { |path| budget_path = path }
+        opts.on("--reason REASON", "Reason for allowlisting") { |r| reason = r }
+      end.parse!(args)
+
+      locator = args.shift
+      raise Error, "--reason is required for allowlist" unless reason
+      raise Error, "locator (e.g., spec/file_spec.rb:10) is required" unless locator
+
+      entry = Budget.load(budget_path).add_to_allowlist(locator, reason: reason)
+      puts "Allowlisted: #{entry.test_case_key}"
+
+      0
+    end
+
+    def run_prune(args)
+      budget_path = DEFAULT_BUDGET_PATH
+
+      OptionParser.new do |opts|
+        opts.banner = "Usage: test_budget prune [options]"
+        opts.on("--budget PATH", "Path to budget file") { |path| budget_path = path }
+      end.parse!(args)
+
+      removed = Budget.load(budget_path).prune_allowlist
+
+      if removed.any?
+        puts "#{removed.size} obsolete allowlist #{(removed.size == 1) ? "entry" : "entries"} removed"
+      else
+        puts "No obsolete allowlist entries found"
+      end
+
+      0
+    end
+
+    def run_init(args)
+      force = false
+
+      OptionParser.new do |opts|
+        opts.banner = "Usage: test_budget init [timings_file] [options]"
+        opts.on("--force", "Overwrite existing config") { force = true }
+      end.parse!(args)
+      timings_path = args.shift
+
+      Budget.estimate(timings_path: timings_path, force: force)
+
+      0
+    end
+  end
+end

data/lib/test_budget/parser/rspec.rb

@@ -0,0 +1,53 @@
+# frozen_string_literal: true
+
+require "json"
+
+module TestBudget
+  module Parser
+    module Rspec
+      extend self
+
+      def parse(pattern)
+        paths = Dir.glob(pattern)
+        raise TestBudget::Error, "No timing files found matching: #{pattern}" if paths.empty?
+
+        groups = paths.flat_map { |path| parse_file(path) }
+        TestRun.new(
+          test_cases: groups.flatten,
+          suite_duration: groups.map { |g| g.sum(&:duration) }.max || 0
+        )
+      end
+
+      private
+
+      def parse_file(path)
+        read_json_objects(path).map do |data|
+          data["examples"].map do |example|
+            TestCase.new(
+              file: example["file_path"],
+              name: example["full_description"],
+              duration: example["run_time"],
+              status: example["status"],
+              line_number: example["line_number"]
+            )
+          end
+        end
+      end
+
+      def read_json_objects(path)
+        content = File.read(path)
+        [JSON.parse(content)]
+      rescue JSON::ParserError
+        parse_concatenated_json(content, path)
+      end
+
+      CONCATENATED_JSON_BOUNDARY = /(?<=\})(?=\{)/
+
+      def parse_concatenated_json(content, path)
+        content.split(CONCATENATED_JSON_BOUNDARY).map { |chunk| JSON.parse(chunk) }
+      rescue JSON::ParserError => e
+        raise TestBudget::Error, "Invalid JSON in #{path}: #{e.message}"
+      end
+    end
+  end
+end

data/lib/test_budget/reporter.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+module TestBudget
+  class Reporter
+    def initialize(budget_path:)
+      @budget_path = budget_path
+    end
+
+    def report(result)
+      violations = result.violations
+
+      if violations.empty?
+        puts "Test budget: all clear"
+      else
+        puts "Test budget: #{violations.size} violation(s) found\n\n"
+
+        violations.each_with_index do |violation, index|
+          puts "  #{index + 1}) #{violation.message}"
+
+          if (locator = violation.locator)
+            budget_flag = (@budget_path == ".test_budget.yml") ? "" : " --budget #{@budget_path}"
+            puts "     To allowlist, run:"
+            puts "     bundle exec test_budget allowlist #{locator}#{budget_flag} --reason \"<reason>\""
+          end
+        end
+      end
+
+      result.warnings.each do |warning|
+        warn "Warning: #{warning.message}\n\n"
+      end
+    end
+  end
+end

data/lib/test_budget/statistics.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+module TestBudget
+  module Statistics
+    def self.p99(values, tolerance: 0) = percentile(0.99, values, tolerance: tolerance)
+
+    private_class_method def self.percentile(rank, values, tolerance: 0)
+      sorted = values.sort
+      n = sorted.size
+      return apply_tolerance(sorted[0], tolerance) if n == 1
+
+      index = rank * (n - 1)
+      lower = sorted[index.floor]
+      upper = sorted[index.ceil]
+      result = lower + (upper - lower) * (index - index.floor)
+
+      apply_tolerance(result, tolerance)
+    end
+
+    private_class_method def self.apply_tolerance(value, tolerance)
+      ceil_to_half(value * (1 + tolerance))
+    end
+
+    private_class_method def self.ceil_to_half(value)
+      (value * 2).ceil / 2.0
+    end
+  end
+end

data/lib/test_budget/test_case.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+
+module TestBudget
+  class TestCase < Data.define(:file, :name, :duration, :status, :line_number)
+    def initialize(file:, name:, duration:, status:, line_number:)
+      super(file: file.delete_prefix("./"), name: name, duration: duration, status: status, line_number: line_number)
+    end
+
+    IRREGULAR_TYPES = {
+      "policies" => :policy,
+      "factories" => :factory,
+      "queries" => :query
+    }.freeze
+
+    def type
+      dir = file[%r{^spec/([^/]+)/}, 1]
+      return :default unless dir
+
+      IRREGULAR_TYPES[dir] || dir.chomp("s").to_sym
+    end
+
+    def key
+      "#{file} -- #{name}"
+    end
+
+    def violation_for(budget)
+      return unless violates?(budget)
+
+      Violation.new(test_case: self, duration: duration, limit: budget.limit_for(self), kind: :per_test_case)
+    end
+
+    def violates?(budget) = over?(budget) && !exempted?(budget)
+
+    def over?(budget) = !within?(budget)
+
+    def within?(budget)
+      limit = budget.limit_for(self)
+      return true if limit.nil?
+
+      duration <= limit
+    end
+
+    def exempted?(budget)
+      budget.exempt?(self)
+    end
+
+    def self.find_by_location!(test_cases, locator)
+      file, line = parse_locator(locator)
+      match = test_cases.find { |tc| tc.file == file && tc.line_number == line }
+
+      raise Error, "No test case found at #{locator}" unless match
+
+      match
+    end
+
+    def self.parse_locator(locator)
+      m = locator.match(/\A(.+):(\d+)\z/)
+      raise Error, "Line number required in locator (e.g., spec/file_spec.rb:10)" unless m
+
+      [m[1], m[2].to_i]
+    end
+
+    private_class_method :parse_locator
+  end
+end

data/lib/test_budget/test_run.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module TestBudget
+  class TestRun < Data.define(:test_cases, :suite_duration)
+    def over?(budget)
+      max = budget.suite.max_duration
+      return if max.nil?
+      return if suite_duration <= max
+
+      Violation.new(test_case: nil, duration: suite_duration, limit: max, kind: :suite)
+    end
+  end
+end

data/lib/test_budget/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module TestBudget
-  VERSION = "0.0.0"
+  VERSION = "0.1.0"
 end

data/lib/test_budget/violation.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+module TestBudget
+  Violation = Data.define(:test_case, :duration, :limit, :kind) do
+    def message
+      if kind == :suite
+        "Suite total time (%.2fs) exceeds limit (%.2fs)" % [duration, limit]
+      else
+        format("%s (%.2fs) exceeds %s limit (%.2fs)", test_case.key, duration, test_case.type, limit)
+      end
+    end
+
+    def locator
+      return nil if kind == :suite
+
+      "#{test_case.file}:#{test_case.line_number}"
+    end
+  end
+end

data/lib/test_budget/warning.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module TestBudget
+  Warning = Data.define(:entry, :kind) do
+    def message
+      reason_line = entry.reason ? "\nreason: #{entry.reason}" : ""
+      "obsolete allowlist entry (#{kind})\n#{entry.test_case_key}#{reason_line}"
+    end
+  end
+end

data/lib/test_budget.rb

@@ -1,8 +1,22 @@
 # frozen_string_literal: true
 
 require_relative "test_budget/version"
+require_relative "test_budget/allowlist"
+require_relative "test_budget/test_case"
+require_relative "test_budget/violation"
+require_relative "test_budget/warning"
+require_relative "test_budget/budget"
+require_relative "test_budget/test_run"
+require_relative "test_budget/auditor"
+require_relative "test_budget/reporter"
+require_relative "test_budget/parser/rspec"
+require_relative "test_budget/audit"
+require_relative "test_budget/statistics"
+require_relative "test_budget/budget/estimate"
+require_relative "test_budget/cli"
 
 module TestBudget
   class Error < StandardError; end
-  # Your code goes here...
+
+  DEFAULT_BUDGET_PATH = ".test_budget.yml"
 end

metadata

@@ -1,30 +1,65 @@
 --- !ruby/object:Gem::Specification
 name: test_budget
 version: !ruby/object:Gem::Version
-  version: 0.0.0
+  version: 0.1.0
 platform: ruby
 authors:
 - Matheus Richard
 bindir: exe
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
-dependencies: []
-description: You have a time budget for tests. This gem enforces it.
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: argument_parser
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+description: Set a time budget for your tests (and your whole suite) and have it automatically
+  enforced in CI. No more slow tests sneaking into your suite!
 email:
 - matheusrichardt@gmail.com
-executables: []
+executables:
+- test_budget
 extensions: []
 extra_rdoc_files: []
 files:
+- ".test_budget.yml"
 - CHANGELOG.md
 - LICENSE.txt
 - README.md
 - Rakefile
+- exe/test_budget
 - lib/test_budget.rb
+- lib/test_budget/allowlist.rb
+- lib/test_budget/audit.rb
+- lib/test_budget/auditor.rb
+- lib/test_budget/budget.rb
+- lib/test_budget/budget/estimate.rb
+- lib/test_budget/cli.rb
+- lib/test_budget/parser/rspec.rb
+- lib/test_budget/reporter.rb
+- lib/test_budget/statistics.rb
+- lib/test_budget/test_case.rb
+- lib/test_budget/test_run.rb
 - lib/test_budget/version.rb
+- lib/test_budget/violation.rb
+- lib/test_budget/warning.rb
+homepage: https://github.com/thoughtbot/test_budget
 licenses:
 - MIT
-metadata: {}
+metadata:
+  homepage_uri: https://github.com/thoughtbot/test_budget
+  source_code_uri: https://github.com/thoughtbot/test_budget
+  changelog_uri: https://github.com/thoughtbot/test_budget/blob/main/CHANGELOG.md
 rdoc_options: []
 require_paths:
 - lib