drive 0.5.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 99a4adcee91a1d02137f10f2ffff028ee3bcabd96896d550c71e81c443ad47b8
-  data.tar.gz: 2a5e5b329808e3e315464d70e1e8fe1357a5736632d96dde4077f3f36a57518e
+  metadata.gz: ba84684bcb7ceb627421453571e0976616e77b7756ef5677ae79f7837b23945b
+  data.tar.gz: 2068762d2e4c2ec3c94eee70b35cc14b6270d74504c55d02d4be4bc2ccf7b280
 SHA512:
-  metadata.gz: 57b24c0022b7476eb3db58ab1108ee86283f5397a49704c39626f751c8158baad8d35404ceebafedaf53bbefc912d5a99e65ddbe6cd6cfea546b8417b20320d8
-  data.tar.gz: eeaa76fd2453f73da3332ded46d662d4165b458ea9db7359582ec6c05b980db96446eaccf47ed880893d1f4298fa37876c9fb573a7b9d399953d43daa98a29a4
+  metadata.gz: 15717e1c2263a8335350fe3d6bcba8d9807df283f6c72180b4e273d6912ef39a1f82172798e7265f189ae92c39b5689ba6cbdf3fd25e08c0be09b1a0aa4b21c0
+  data.tar.gz: 881c0fdbefb2d8466f1443d6d867c1965964b52f68801fd1000cf4117e5c300086f05b1a8cef048558ced13eae2398c4ed38dcb84e4f0cb5663d6e671d5f2541

data/CHANGELOG.md

@@ -6,6 +6,50 @@ For more information about changelogs, check [Keep a Changelog](http://keepachan
 
 ## [Unreleased]
 
+## 0.7.0 - 2026-10-16
+
+* [BREAKING] `Mirrored`, `Counted`, `Run`, `AccessFile` and `Drive::TestHelper` are the `mirroring` gem's
+
+  An app that mirrors adds `gem 'mirroring'` and declares each mirrored model with
+  `mirroring key: :code` (`key: false` where the source names none) in place of `include Mirrored`
+  and `KEY`, and includes `Mirroring::TestHelper` in its tests. `Run.record`, `mirror`,
+  `include Counted` and `AccessFile` keep their names. Drive no longer needs PostgreSQL, nor ships
+  the migrations of runs: an app keeps the `runs` table it has.
+
+## 0.6.0 - 2026-10-09
+
+* [BREAKING] `include RateLimited` sets no limit by itself: a controller calls `rate_limited`
+
+  `rate_limited` keeps 20 a minute and 200 a day; `rate_limited minute: 120, day: nil,
+  scope: 'public'` sets an app's own, counted across every controller of the scope. A client over
+  the limit is told so in words, with status 429, instead of an error the app would report. A host
+  may say it its own way by defining `refuse_flood(minute:, day:)`, and `crawler?` is a helper.
+
+* [FEATURE] `Crawler` also believes Cloudflare's verified bots, where `EDGE_KEY` is set
+
+  A request whose `X-Edge-Key` is `EDGE_KEY` and whose `X-Verified-Bot` is `true` is a crawler.
+  While a search engine's ranges cannot be fetched, no crawler naming it is believed, and the
+  request is served rather than failing.
+
+* [FEATURE] The same error pages on every app
+
+  `Drive::ErrorsController` answers 400, 404, 422 and 500 at the paths Rails keeps for them, and
+  `/down.html`, as `config.exceptions_app` unless the app sets its own. Delete the app's
+  `public/400.html`, `404.html`, `422.html` and `500.html`: a static page wins over the route.
+
+* [FEATURE] What every app behind Cloudflare sets alike
+
+  Cloudflare's addresses (`Drive::Cloudflare::RANGES`) are trusted proxies, unless the app sets
+  its own. `Drive::Cloudflare` purges pages by tag or everything, and warms pages, with
+  `CLOUDFLARE_ZONE_ID` and `CLOUDFLARE_API_TOKEN`. "aircraft" is its own plural, and FAA, DRS,
+  NTSB and XML are acronyms (a constant spelled `Drs` is now `DRS`).
+
+* [FEATURE] `NNumber`, `Designation` and `Serial`
+
+  An N-number as the FAA writes it (`normalize`, `parse`, `valid?`, `display`, `to_param`); a
+  designation's and a part number's key (`Designation.key`, `Designation.part_key`), the same as
+  the SQL the refinery keys its tables with; a serial number's prefix and number (`Serial.parts`).
+
 ## 0.5.0 - 2026-10-07
 
 * [FEATURE] `RateLimited` no longer limits Claude's, ChatGPT's or Perplexity's search crawler

data/README.md

@@ -1,20 +1,19 @@
 # Drive
 
-A Rails engine for an app that keeps a copy of what somebody else publishes — a government's
-files, a monthly database — changing only the rows the source did, and counting them.
-
-- **`Mirrored`**: a table copying one of the source's tables field for field, as text. A refresh
-  stages the source's table, then inserts, updates and deletes only the rows the source did.
-- **`Counted`**: a table written a record at a time, such as a document fetched by URL.
-- **`Run`**: one refresh, in one transaction, the only one running, with how many rows it
-  inserted, updated and deleted.
-- **`AccessFile`**: stages a table of a Microsoft Access file, read with `mdb-export`.
-- **`RateLimited`**: 20 requests a minute and 200 a day per address, for public pages. A search
-  engine's own crawler has no limit.
-- **`Crawler`**: whether a request comes from the crawler of Google, Bing, Apple, Claude, ChatGPT
-  or Perplexity: its address is in the ranges they publish, fetched once a day.
-
-It needs PostgreSQL.
+A Rails engine for the easyFAA apps. Each sits behind Cloudflare and shows the same error pages.
+A copy of the tables a source publishes is the [`mirroring`](https://github.com/easyfaa/mirroring)
+gem's.
+
+- **`RateLimited`**: `rate_limited` holds an address to 20 pages a minute and 200 a day, or the
+  app's own limits, and tells it so in words. A search engine's own crawler has no limit.
+- **`Crawler`**: whether a request comes from a search engine's crawler: Cloudflare vouches for
+  it (where `EDGE_KEY` is set), or its address is in the ranges Google, Bing, Apple, Claude,
+  ChatGPT or Perplexity publish, fetched once a day.
+- **`Drive::Cloudflare`**: purges pages by tag or everything, and warms pages. Its addresses are
+  the app's trusted proxies.
+- **Error pages**: the same page for 400, 404, 422 and 500 on every app, and `/down.html`.
+- **`NNumber`**, **`Designation`**, **`Serial`**: an N-number, a designation's key and a serial
+  number, as every app reads them.
 
 ## How to install
 
@@ -24,55 +23,10 @@ gem install drive
 
 ```ruby
 # Gemfile
-gem 'drive', '~> 0.5.0'
-```
-
-`~> 0.5.0` takes every 0.5.x and stops short of 0.6, which may break what 0.5 does.
-
-Then copy its migrations, of runs, and run them:
-
-```bash
-bin/rails drive:install:migrations db:migrate
-```
-
-## How to use
-
-A mirrored table has the source's fields as text columns, plus `source`, `row_key` and
-`row_hash`, unique on `source` and `row_key`. Its model says which fields tell a row apart:
-
-```ruby
-class Engine < ApplicationRecord
-  include Mirrored
-
-  KEY = %w[code].freeze # [] where the source names none: a row is then told apart by all it says
-end
+gem 'drive', '~> 0.7.0'
 ```
 
-A refresh stages the source's table and mirrors it, inside a run:
-
-```ruby
-Run.record 'engines' do
-  staged = AccessFile.new('ENGINE.accdb').stage('ENGINE', Engine.connection)
-  Engine.mirror staged, source: 'ENGINE'
-end
-```
-
-A table written a record at a time counts each save and delete in the run at hand:
-
-```ruby
-class Publication < ApplicationRecord
-  include Counted
-end
-```
-
-Runs commit one at a time, so another app reading the tables directly sees either the last
-run's rows or the one before's, never a mix.
-
-In tests, `Drive::TestHelper#stage` stages rows as a source's file would:
-
-```ruby
-ActiveSupport.on_load(:active_support_test_case) { include Drive::TestHelper }
-```
+`~> 0.7.0` takes every 0.7.x and stops short of 0.8, which may break what 0.7 does.
 
 ## Development
 

data/app/controllers/concerns/rate_limited.rb

@@ -1,10 +1,40 @@
-# The limits of a page anybody may ask for, by the address asking: 20 a minute, 200 a day. A search
-# engine's own crawler has none, so it can list every page.
+# The limits of a page anybody may ask for, by the address asking, which a search engine's own
+# crawler (Crawler) is never held to, so it can list every page. Refused, a client is told why in
+# words, with status 429, rather than with an error the app would report.
+#
+#   rate_limited                         # 20 a minute and 200 a day
+#   rate_limited minute: 120, day: nil, scope: 'public'
 module RateLimited
   extend ActiveSupport::Concern
 
   included do
-    rate_limit to: 20, within: 1.minute, name: 'minute', unless: -> { Crawler.verified? request }
-    rate_limit to: 200, within: 1.day, name: 'day', unless: -> { Crawler.verified? request }
+    helper_method :crawler? if respond_to? :helper_method
+  end
+
+  class_methods do
+    # @param minute [Integer, nil] pages an address may ask for a minute.
+    # @param day [Integer, nil] pages an address may ask for a day.
+    # @param scope [String, nil] the controllers counted together; each its own if nil.
+    def rate_limited(minute: 20, day: 200, scope: nil)
+      { minute: [minute, 1.minute], day: [day, 1.day] }.each do |name, (to, within)|
+        next unless to
+
+        rate_limit(to:, within:, name: name.to_s, scope:, unless: -> { crawler? },
+                   with: -> { refuse_flood minute:, day: })
+      end
+    end
+  end
+
+private
+
+  # Whether a search engine vouches for this client, asked once a request.
+  def crawler? = defined?(@crawler) ? @crawler : @crawler = Crawler.verified?(request)
+
+  # What a client asking too often is told. A host may say it its own way.
+  def refuse_flood(minute:, day:)
+    limits = { minute:, day: }.compact.map { |per, count| "#{count} a #{per}" }
+    render plain: 'Too many requests. Each address may ask for so many pages: ' \
+                  "#{limits.join ' and '}.\n",
+           status: :too_many_requests
   end
 end

data/app/controllers/drive/errors_controller.rb

@@ -0,0 +1,33 @@
+module Drive
+  # The pages every app shows for 400, 404, 422 and 500 (config.exceptions_app, set by the
+  # engine). Plain ActionController::Base, not the host's ApplicationController: a 500 is exactly
+  # the moment the database or a session might be what broke, so nothing here may depend on either.
+  class ErrorsController < ActionController::Base # rubocop:disable Rails/ApplicationController
+    layout 'drive/errors'
+
+    # Heading and sentence by status.
+    MESSAGES = {
+      400 => ['Bad request', 'The request could not be understood.'],
+      404 => ['Page not found', 'You may have mistyped the address, or the page may have moved.'],
+      422 => ['Can’t process that', 'The change was rejected, often because a page had been ' \
+                                    'left open too long. Going back and trying again usually ' \
+                                    'fixes it.',],
+      500 => ['Something went wrong', 'We’ve been notified and are looking into it. Try again ' \
+                                      'in a moment.',],
+    }.freeze
+
+    # The page for the status the route names, whatever format was asked for.
+    def show
+      @status = params[:status].to_i
+      @heading, @message = MESSAGES.fetch(@status, MESSAGES.fetch(500))
+      render status: @status, formats: :html
+    end
+
+    # The page shown while the app is down for maintenance.
+    def down
+      @heading = 'Down for maintenance'
+      @message = 'easyFAA is being updated and will be back in a few minutes.'
+      render :show, formats: :html
+    end
+  end
+end

data/app/models/crawler.rb

@@ -1,8 +1,10 @@
 require 'ipaddr'
 require 'net/http'
 
-# A search engine's own crawler, told apart from a client that only names one in its user agent:
-# its address is in the ranges the search engine publishes for it.
+# A search engine's own crawler, told apart from a client that only names one in its user agent.
+# Either Cloudflare vouches for it (a verified bot, in `X-Verified-Bot`, beside the `X-Edge-Key`
+# only Cloudflare and the app know, so a request that skips Cloudflare cannot forge it), or its
+# address is in the ranges its search engine publishes for it.
 module Crawler
   # The ranges each search engine publishes, by the user agents of its crawlers.
   RANGES = {
@@ -16,13 +18,24 @@ module Crawler
   }.freeze
 
   # @param request [ActionDispatch::Request] request a page was asked with.
-  # @return [Boolean] whether it came from a crawler, its ranges fetched once a day.
-  def self.verified?(request)
+  # @return [Boolean] whether it came from a crawler.
+  def self.verified?(request) = vouched?(request) || published?(request)
+
+  # @return [Boolean] whether Cloudflare says it verified the bot, where EDGE_KEY is set.
+  def self.vouched?(request)
+    key = ENV['EDGE_KEY'].presence or return false
+    ActiveSupport::SecurityUtils.secure_compare(request.headers['X-Edge-Key'].to_s, key) &&
+      request.headers['X-Verified-Bot'] == 'true'
+  end
+
+  # @return [Boolean] whether the user agent names a crawler whose ranges hold the address.
+  def self.published?(request)
     url = RANGES.find { |agent, _| request.user_agent.to_s.match? agent }&.last
     url.present? && ranges(url).any? { it.include? request.remote_ip }
   end
 
-  private_class_method def self.ranges(url)
+  # The ranges at that address, fetched once a day; none while it cannot be reached.
+  def self.ranges(url)
     Rails.cache.fetch(['crawler', url], expires_in: 1.day) do
       uri = URI url
       body = Net::HTTP.start(uri.host, use_ssl: true, open_timeout: 2, read_timeout: 2) do |http|
@@ -30,5 +43,8 @@ module Crawler
       end
       JSON.parse(body)['prefixes'].map { IPAddr.new it.values.first }
     end
+  rescue Net::OpenTimeout, Net::ReadTimeout, SocketError, SystemCallError, JSON::ParserError
+    []
   end
+  private_class_method :vouched?, :published?, :ranges
 end

data/app/models/designation.rb

@@ -0,0 +1,15 @@
+# A model, engine or part number as every app keys it, so one written two ways is found as one:
+# what the refinery writes (models.designation_key, by the same rule in SQL) and the site reads.
+module Designation
+  # @param designation [String, nil] as an AD, a data sheet or the registry writes it.
+  # @return [String] upcased, nothing after a parenthesis, letters, digits, slashes and pluses
+  #   only, and a zero after the leading letters read as the letter O: "O320" for "0-320 (A)".
+  def self.key(designation)
+    designation.to_s.upcase.sub(/\s*\(.*\z/, '').gsub(%r{[^A-Z0-9/+]}, '')
+               .sub(/\A([A-Z]*)0(?=\d)/, '\1O')
+  end
+
+  # @param part [String, nil] a part number as a shop or an AD writes it.
+  # @return [String] upcased, without dashes or spaces: "ST07" for "st-07".
+  def self.part_key(part) = part.to_s.upcase.delete('- ')
+end

data/app/models/n_number.rb

@@ -0,0 +1,29 @@
+# A U.S. registration (an N-number) as text: as the FAA writes one, however it was typed.
+module NNumber
+  # One the FAA could assign: N, then one to five characters, a digit 1 to 9 first, up to four
+  # more digits, and at most two letters last, never I or O.
+  ASSIGNABLE = /\AN(?=.{1,5}\z)[1-9]\d{0,4}[A-HJ-NP-Z]{0,2}\z/
+  # One as older records write it: a digit 1 to 9 after the N, then any four letters or digits.
+  WRITTEN = /\AN[1-9][A-Z0-9]{0,4}\z/
+
+  # @param typed [String, nil] as somebody typed it: "n-714ad", "714AD".
+  # @return [String] with its N, upcased, letters and digits only: "N714AD".
+  def self.normalize(typed) = typed.to_s.upcase.delete('^A-Z0-9').sub(/\A(?!N)/, 'N')
+
+  # @param typed [String, nil]
+  # @return [String, nil] normalized, where it reads as a U.S. registration at all ("A6FKF", another
+  #   country's, does not).
+  def self.parse(typed) = normalize(typed).then { it if it.match? WRITTEN }
+
+  # @param n_number [String] normalized.
+  # @return [Boolean] whether the FAA could assign it.
+  def self.valid?(n_number) = n_number.to_s.match?(ASSIGNABLE)
+
+  # @param n_number [String] as stored: "N714AD" or "N-714AD".
+  # @return [String] as a page shows it, the N joined to the rest: "N714AD".
+  def self.display(n_number) = n_number.to_s.sub(/\AN-?/, 'N')
+
+  # @param n_number [String] normalized.
+  # @return [String] in an address, without its N: "714AD".
+  def self.to_param(n_number) = n_number.to_s.delete_prefix('N')
+end

data/app/models/serial.rb

@@ -0,0 +1,11 @@
+# A serial number as the registry or a data sheet writes it, cut into what comes before its last
+# run of digits and that number, so that two are compared as the sheets mean them: "15279406" is
+# within "S/N 15279406 thru 15285594", and "A-100" within "S/N A-100 and up".
+module Serial
+  # @param serial [String, nil]
+  # @return [Array(String, Integer), Array(nil, nil)] the prefix, dashes dropped, and the number.
+  def self.parts(serial)
+    match = serial.to_s.upcase.delete(' ').match(/\A(.*?)(\d+)\z/) or return [nil, nil]
+    [match[1].delete('-'), match[2].to_i]
+  end
+end

data/app/views/drive/errors/show.html.erb

@@ -0,0 +1,15 @@
+<% content_for :title, "#{@heading} | easyFAA" %>
+<h1>
+  <svg viewBox="0 0 64 64" aria-hidden="true">
+    <g stroke="currentColor" stroke-width="2.5" stroke-linejoin="round">
+      <polygon points="60,8 4,30 25,37" fill="#ffffff"/>
+      <polygon points="60,8 25,37 31,57" fill="#cfe0f3"/>
+    </g>
+    <g stroke="#7a93b3" stroke-width="1.6" stroke-linecap="round">
+      <line x1="16" y1="28.5" x2="36" y2="21"/><line x1="21" y1="31.5" x2="44" y2="22.5"/><line x1="27" y1="33" x2="46" y2="25.5"/>
+    </g>
+    <path d="M10 44 C14 50 18 53 24 55" fill="none" stroke="currentColor" stroke-width="2" stroke-dasharray="3 3" stroke-linecap="round"/>
+  </svg>
+  <%= @heading %>
+</h1>
+<p><%= @message %></p>

data/app/views/layouts/drive/errors.html.erb

@@ -0,0 +1,26 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<meta name="color-scheme" content="light dark">
+<meta name="robots" content="noindex">
+<title><%= content_for(:title) || "easyFAA" %></title>
+<style>
+:root{color-scheme:light;--bg:#ffffff;--ink:#15202e;--muted:#50575d;--primary:#1a70d0}
+@media (prefers-color-scheme: dark){:root{color-scheme:dark;--bg:#131415;--ink:#f3f5f8;--muted:#abb2b8;--primary:#689dff}}
+*{box-sizing:border-box}
+body{margin:0;min-height:100svh;display:flex;flex-direction:column;align-items:center;justify-content:center;background:var(--bg);color:var(--ink);font:500 17px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif;text-align:center;padding:40px 24px}
+a{color:var(--primary)}
+h1{display:flex;align-items:center;justify-content:center;font-size:clamp(1.5rem,5.5vw,2.25rem);margin:0}
+h1 svg{width:1.1em;height:1.1em;margin-right:0.3em;flex-shrink:0}
+p{color:var(--muted);margin:18px 0 0;max-width:32rem}
+footer{margin-top:40px;font-size:15px;color:var(--muted)}
+footer a{color:inherit;text-decoration:none}
+</style>
+</head>
+<body>
+<%= yield %>
+<footer><a href="https://easyfaa.com/">easyFAA</a></footer>
+</body>
+</html>

data/lib/drive/cloudflare.rb

@@ -0,0 +1,67 @@
+require 'ipaddr'
+require 'net/http'
+
+module Drive
+  # The Cloudflare zone in front of every easyFAA app: its addresses, and what it keeps of the
+  # pages, forgotten once a night's data has changed them. Nothing is purged where the zone is
+  # not named (CLOUDFLARE_ZONE_ID and CLOUDFLARE_API_TOKEN), as in development and test.
+  module Cloudflare
+    # The addresses Cloudflare reaches an app from, as https://www.cloudflare.com/ips lists them.
+    RANGES = %w[
+      173.245.48.0/20 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 141.101.64.0/18
+      108.162.192.0/18 190.93.240.0/20 188.114.96.0/20 197.234.240.0/22 198.41.128.0/17
+      162.158.0.0/15 104.16.0.0/13 104.24.0.0/14 172.64.0.0/13 131.0.72.0/22
+      2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32 2405:8100::/32
+      2a06:98c0::/29 2c0f:f248::/32
+    ].map { IPAddr.new it }.freeze
+    # Cloudflare takes up to a hundred tags a request.
+    BATCH = 100
+
+    # @return [Boolean] whether a zone and a token were given.
+    def self.configured? = zone.present? && token.present?
+
+    # Forgets the pages carrying any of these tags, a hundred tags a request.
+    # @param tags [Array<String>]
+    # @return [void]
+    def self.purge(tags) = tags.each_slice(BATCH) { post tags: it }
+
+    # Forgets every page the zone holds, of every app in it.
+    # @return [void]
+    def self.purge_everything = post(purge_everything: true)
+
+    # Asks the zone for each address, which is what makes it keep the answer, so the first reader
+    # of the morning is not the one who waits.
+    # @param urls [Array<String>]
+    # @return [Integer] how many were answered.
+    def self.warm(urls)
+      urls.count do |url|
+        uri = URI url
+        get = Net::HTTP::Get.new(uri, 'User-Agent' => 'easyfaa-cache-warmer')
+        options = { use_ssl: uri.scheme == 'https', read_timeout: 60 }
+        Net::HTTP.start(uri.host, uri.port, **options) { it.request(get).is_a? Net::HTTPSuccess }
+      end
+    end
+
+    # Asks Cloudflare to purge, raising when it says no.
+    def self.post(body)
+      uri = URI "https://api.cloudflare.com/client/v4/zones/#{zone}/purge_cache"
+      response = Net::HTTP.post uri, body.to_json, 'Authorization' => "Bearer #{token}",
+                                                   'Content-Type' => 'application/json'
+      return if response.is_a? Net::HTTPSuccess
+
+      raise "Cloudflare purge failed: #{response.code} #{response.body}"
+    end
+
+    # The zone, from the environment or the credentials.
+    def self.zone = setting('CLOUDFLARE_ZONE_ID', :zone_id)
+
+    # The token, which may purge the zone's cache and nothing else.
+    def self.token = setting('CLOUDFLARE_API_TOKEN', :api_token)
+
+    # A setting from the environment, else from the credentials' `cloudflare:`.
+    def self.setting(name, key)
+      ENV[name].presence || Rails.application.credentials.dig(:cloudflare, key)
+    end
+    private_class_method :post, :zone, :token, :setting
+  end
+end

data/lib/drive/engine.rb

@@ -1,7 +1,27 @@
 module Drive
-  # Lends the host its models (Run, Mirrored, Counted, AccessFile, Crawler), RateLimited and the
-  # migrations of runs (`bin/rails drive:install:migrations`).
+  # Lends the host its models (Crawler, NNumber, Designation, Serial), RateLimited and the error
+  # pages.
   class Engine < ::Rails::Engine
     engine_name 'drive'
+
+    # Every request reaches the app through Cloudflare: trusted as proxies beside the private
+    # ones, its addresses are skipped, and `request.remote_ip` is the visitor's own, for every
+    # rate limit and log line.
+    initializer 'drive.trusted_proxies', before: :build_middleware_stack do |app|
+      app.config.action_dispatch.trusted_proxies ||=
+        ActionDispatch::RemoteIp::TRUSTED_PROXIES + Drive::Cloudflare::RANGES
+    end
+
+    # The same pages for 400, 404, 422 and 500 on every app (Drive::ErrorsController), at the
+    # paths Rails keeps for them, and the page shown while an app is down for maintenance.
+    initializer 'drive.errors' do |app|
+      app.config.exceptions_app ||= app.routes
+      app.routes.prepend do
+        %w[400 404 422 500].each do |status|
+          get status, to: 'drive/errors#show', defaults: { status: status.to_i }
+        end
+        get 'down.html', to: 'drive/errors#down', format: false
+      end
+    end
   end
 end

data/lib/drive/inflections.rb

@@ -0,0 +1,9 @@
+# The words every easyFAA app names alike: more than one aircraft is still "aircraft", whatever
+# precedes it ("NTSB::Aircraft", "accident_aircraft"), and the agencies and formats are the acronyms
+# they are. Set when the gem loads, before any of the app's constants is named.
+ActiveSupport::Inflector.inflections(:en) do |inflect|
+  inflect.uncountable 'aircraft'
+  inflect.plural(/(aircraft)\z/i, '\1')
+  inflect.singular(/(aircraft)\z/i, '\1')
+  %w[FAA DRS NTSB XML].each { inflect.acronym it }
+end

data/lib/drive/version.rb

@@ -1,4 +1,4 @@
 module Drive
   # Version of the gem, read by the gemspec.
-  VERSION = '0.5.0'.freeze
+  VERSION = '0.7.0'.freeze
 end

data/lib/drive.rb

@@ -1,7 +1,9 @@
 require 'drive/version'
+require 'drive/inflections'
+require 'drive/cloudflare'
 require 'drive/engine'
 
-# Keeps a copy of the tables another source publishes, changing only the rows the source did,
-# and counts what each refresh changed (Mirrored, Counted, Run).
+# What every easyFAA app in front of Cloudflare needs alike: Drive::Cloudflare, Crawler,
+# RateLimited, the error pages, NNumber, Designation and Serial.
 module Drive
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: drive
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.7.0
 platform: ruby
 authors:
 - claudiob
@@ -9,34 +9,6 @@ bindir: bin
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
-- !ruby/object:Gem::Dependency
-  name: csv
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: pg
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 - !ruby/object:Gem::Dependency
   name: rails
   requirement: !ruby/object:Gem::Requirement
@@ -51,9 +23,8 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '8.1'
-description: 'A Rails engine for apps that mirror a public source table for table:
-  each refresh changes only the rows the source did, in one run, and counts how many
-  it inserted, updated and deleted.'
+description: 'A Rails engine for the easyFAA apps: each sits behind Cloudflare and
+  shows the same error pages, limits, crawlers and N-numbers.'
 email:
 - claudiob@users.noreply.github.com
 executables: []
@@ -64,17 +35,17 @@ files:
 - MIT-LICENSE
 - README.md
 - app/controllers/concerns/rate_limited.rb
-- app/models/access_file.rb
-- app/models/concerns/counted.rb
-- app/models/concerns/mirrored.rb
+- app/controllers/drive/errors_controller.rb
 - app/models/crawler.rb
-- app/models/drive/current.rb
-- app/models/run.rb
-- db/migrate/20261005100000_create_runs_and_changes.rb
-- db/migrate/20261006100000_drop_changes.rb
+- app/models/designation.rb
+- app/models/n_number.rb
+- app/models/serial.rb
+- app/views/drive/errors/show.html.erb
+- app/views/layouts/drive/errors.html.erb
 - lib/drive.rb
+- lib/drive/cloudflare.rb
 - lib/drive/engine.rb
-- lib/drive/test_helper.rb
+- lib/drive/inflections.rb
 - lib/drive/version.rb
 homepage: https://github.com/easyfaa/drive
 licenses:
@@ -97,7 +68,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.3
+rubygems_version: 4.0.20
 specification_version: 4
-summary: Keep a copy of what a source publishes, changing only the rows it changed.
+summary: 'What the easyFAA apps share: error pages, Cloudflare, limits and N-numbers.'
 test_files: []

data/app/models/access_file.rb

@@ -1,47 +0,0 @@
-require 'csv'
-
-# A Microsoft Access database, as DRS and the NTSB publish theirs, read with mdb-export (mdbtools).
-class AccessFile
-  # The mdb-export command, writing dates with their century, which it leaves out by default.
-  EXPORT = ['mdb-export', '-D', '%Y-%m-%d', '-T', '%Y-%m-%d %H:%M:%S'].freeze
-
-  # @param path [String] .accdb or .mdb file.
-  def initialize(path)
-    @path = path
-  end
-
-  # Copies a table into a temporary one of text columns, dropped when the transaction commits:
-  # "AD Number" is ad_number, and the table's own id is access_id.
-  # @param table [String] table name in the Access file, such as AD_DATA or events.
-  # @param connection [ActiveRecord::ConnectionAdapters::PostgreSQLAdapter] inside a transaction.
-  # @return [String] temporary table.
-  def stage(table, connection)
-    staged = "staged_#{column table}"
-    IO.popen [*EXPORT, @path, table], 'rb' do |io|
-      columns = CSV.parse_line(io.gets.force_encoding(Encoding::UTF_8)).map { column it }
-      connection.execute "DROP TABLE IF EXISTS #{staged}"
-      connection.execute <<~SQL.squish
-        CREATE TEMP TABLE #{staged} (#{columns.map { %("#{it}" text) }.join ', '}) ON COMMIT DROP
-      SQL
-      copy io, staged, connection.raw_connection
-    end
-    staged
-  end
-
-private
-
-  def column(name)
-    column = name.strip.downcase.gsub(/[^a-z0-9]+/, '_').delete_prefix('_').delete_suffix('_')
-    # An Access table's own counter, named apart from the id Rails gives every row.
-    column == 'id' ? 'access_id' : column
-  end
-
-  def copy(io, staged, raw)
-    raw.copy_data "COPY #{staged} FROM STDIN WITH (FORMAT csv)" do
-      while (chunk = io.read(1 << 20))
-        # Some narratives carry null bytes, which PostgreSQL cannot keep in text.
-        raw.put_copy_data chunk.delete("\x00")
-      end
-    end
-  end
-end

data/app/models/concerns/counted.rb

@@ -1,17 +0,0 @@
-# A table written a record at a time, each save and delete counted in the current run.
-module Counted
-  extend ActiveSupport::Concern
-
-  included do
-    after_save :count_upsert, if: :saved_changes?
-    after_destroy :count_delete
-  end
-
-private
-
-  def count_upsert
-    Run.current.tally(**(previously_new_record? ? { inserted: 1 } : { updated: 1 }))
-  end
-
-  def count_delete = Run.current.tally(deleted: 1)
-end

data/app/models/concerns/mirrored.rb

@@ -1,90 +0,0 @@
-# A table copying a source's table field for field as text, changing only the rows the source did.
-module Mirrored
-  extend ActiveSupport::Concern
-
-  # The columns this app adds to what the source published.
-  OWN = %w[id source row_key row_hash created_at updated_at].freeze
-
-  included do
-    # A source's own `type` column is data, not Rails' single-table inheritance.
-    self.inheritance_column = nil
-  end
-
-  class_methods do
-    # Brings the rows from one source in line with a table staged from it, in the current run.
-    # @param staged [String] temporary table holding what the source publishes.
-    # @param source [String] file or document type the rows come from.
-    # @return [Hash{Symbol => Integer}] how many rows were inserted, updated and deleted.
-    def mirror(staged, source:)
-      stage_incoming staged
-      counts = { inserted: inserted(source), updated: updated(source), deleted: deleted(source) }
-      apply source
-      counts.tap { Run.current.tally(**it) }
-    end
-
-  private
-
-    def fields = column_names - OWN
-
-    def stage_incoming(staged)
-      present = connection.columns(staged).map(&:name)
-      picked = fields.map { |field| present.include?(field) ? %(s."#{field}") : 'NULL::text' }
-      hash = "md5(ROW(#{picked.join ', '})::text)"
-      keys = self::KEY.map { %(s."#{it}") }
-      connection.execute 'DROP TABLE IF EXISTS mirror_incoming'
-      connection.execute <<~SQL.squish
-        CREATE TEMP TABLE mirror_incoming ON COMMIT DROP AS
-        SELECT DISTINCT ON (row_key) * FROM (
-          SELECT #{picked.zip(fields).map { |value, field| %(#{value} AS "#{field}") }.join ', '},
-                 #{keys.any? ? "concat_ws('|', #{keys.join ', '})" : hash} AS row_key,
-                 #{hash} AS row_hash
-          FROM #{staged} s
-        ) rows ORDER BY row_key, row_hash
-      SQL
-      connection.execute 'CREATE INDEX ON mirror_incoming (row_key)'
-      connection.execute 'ANALYZE mirror_incoming'
-    end
-
-    def inserted(source)
-      rows <<~SQL.squish
-        FROM mirror_incoming i
-        WHERE NOT EXISTS (SELECT 1 FROM #{quoted_table_name} t
-                          WHERE t.source = #{connection.quote source} AND t.row_key = i.row_key)
-      SQL
-    end
-
-    def updated(source)
-      rows <<~SQL.squish
-        FROM mirror_incoming i JOIN #{quoted_table_name} t USING (row_key)
-        WHERE t.source = #{connection.quote source} AND t.row_hash <> i.row_hash
-      SQL
-    end
-
-    def deleted(source)
-      rows <<~SQL.squish
-        FROM #{quoted_table_name} i
-        WHERE i.source = #{connection.quote source}
-          AND NOT EXISTS (SELECT 1 FROM mirror_incoming n WHERE n.row_key = i.row_key)
-      SQL
-    end
-
-    def rows(from) = connection.select_value("SELECT count(*) #{from}").to_i
-
-    def apply(source)
-      quoted = fields.map { connection.quote_column_name it }
-      connection.delete <<~SQL.squish
-        DELETE FROM #{quoted_table_name} t WHERE t.source = #{connection.quote source}
-        AND NOT EXISTS (SELECT 1 FROM mirror_incoming n WHERE n.row_key = t.row_key)
-      SQL
-      connection.execute <<~SQL.squish
-        INSERT INTO #{quoted_table_name} AS t
-          (source, row_key, row_hash, #{quoted.join ', '}, created_at, updated_at)
-        SELECT #{connection.quote source}, row_key, row_hash, #{quoted.join ', '}, now(), now()
-        FROM mirror_incoming
-        ON CONFLICT (source, row_key) DO UPDATE SET row_hash = excluded.row_hash,
-          #{quoted.map { "#{it} = excluded.#{it}" }.join ', '}, updated_at = now()
-        WHERE t.row_hash <> excluded.row_hash
-      SQL
-    end
-  end
-end

data/app/models/drive/current.rb

@@ -1,7 +0,0 @@
-module Drive
-  # The run of the refresh at hand, which every change belongs to.
-  class Current < ActiveSupport::CurrentAttributes
-    # The run every change of the refresh at hand belongs to.
-    attribute :run
-  end
-end

data/app/models/run.rb

@@ -1,33 +0,0 @@
-# One refresh of a source: when it started and finished, and how many rows it changed.
-class Run < ApplicationRecord
-  # The lock every refresh takes, so runs commit one at a time.
-  LOCK = 1_729_001
-
-  # Refreshes a source inside one transaction, the only one running, recording what it changed.
-  # @param source [String] what is refreshed, such as "registry".
-  # @return [Run] finished run.
-  def self.record(source)
-    transaction do
-      connection.execute "SELECT pg_advisory_xact_lock(#{LOCK})"
-      run = create! source:, started_at: Time.current
-      Drive::Current.set(run:) { yield run }
-      run.tap { it.update! finished_at: Time.current }
-    end
-  end
-
-  # @return [Run] run of the refresh at hand.
-  def self.current = Drive::Current.run
-
-  # Adds what a table's refresh did to this run's counts.
-  # @param inserted [Integer] rows inserted.
-  # @param updated [Integer] rows updated.
-  # @param deleted [Integer] rows deleted.
-  def tally(inserted: 0, updated: 0, deleted: 0)
-    self.inserted += inserted
-    self.updated += updated
-    self.deleted += deleted
-  end
-
-  # @return [String] source and start, as the runs' table names a run.
-  def to_s = "#{source} #{started_at.to_fs :db}"
-end

data/db/migrate/20261005100000_create_runs_and_changes.rb

@@ -1,28 +0,0 @@
-# Each refresh of a source (a run), and every row it inserted, changed or deleted (a change),
-# which is the feed another app follows by the change's id.
-class CreateRunsAndChanges < ActiveRecord::Migration[8.1]
-  def change
-    create_enum :operation, %w[ upsert delete ]
-
-    create_table :runs do |t|
-      t.text :source, null: false
-      t.integer :inserted, default: 0, null: false
-      t.integer :updated, default: 0, null: false
-      t.integer :deleted, default: 0, null: false
-      t.datetime :started_at, null: false
-      t.datetime :finished_at
-      t.virtual :name, type: :text, as: "source || ' #' || id", stored: true
-    end
-
-    create_table :changes do |t|
-      t.enum :operation, enum_type: :operation, null: false
-      t.text :kind, null: false
-      t.text :source, null: false
-      t.text :row_key, null: false
-      t.references :run, null: false, foreign_key: true
-      t.jsonb :record
-      t.datetime :created_at, null: false
-      t.index %i[ kind row_key ]
-    end
-  end
-end

data/db/migrate/20261006100000_drop_changes.rb

@@ -1,16 +0,0 @@
-class DropChanges < ActiveRecord::Migration[8.1]
-  def change
-    drop_table :changes do |t|
-      t.enum :operation, enum_type: :operation, null: false
-      t.text :kind, null: false
-      t.text :source, null: false
-      t.text :row_key, null: false
-      t.references :run, null: false, foreign_key: true
-      t.jsonb :record
-      t.datetime :created_at, null: false
-      t.index %i[kind row_key]
-    end
-
-    drop_enum :operation, %w[upsert delete]
-  end
-end

data/lib/drive/test_helper.rb

@@ -1,20 +0,0 @@
-module Drive
-  # What a host's tests need to mirror rows without a source's file.
-  module TestHelper
-    # Stages rows as a source's file would, a text column per field, for a model to mirror.
-    # @param table [String] temporary table to create.
-    # @param rows [Array<Hash{String => String}>] the rows, each by field.
-    # @return [String] the table.
-    def stage(table, rows)
-      columns = rows.flat_map(&:keys).uniq
-      connection = ActiveRecord::Base.connection
-      connection.execute "DROP TABLE IF EXISTS #{table}"
-      connection.execute "CREATE TEMP TABLE #{table} (#{columns.map { %("#{it}" text) }.join ', '})"
-      rows.each do |row|
-        values = columns.map { connection.quote row[it] }
-        connection.execute "INSERT INTO #{table} VALUES (#{values.join ', '})"
-      end
-      table
-    end
-  end
-end