airtable-orm 0.2.3 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 628a8ae1f053bb9b1f598608785b9a09062797d152e234d7cd2e2b1a4fefb982
-  data.tar.gz: 30813eda1cfb49e6e74418be674490526697264c3d67b6d6f6edd0d3f194c810
+  metadata.gz: daa9deb3cda25ce52f78e6b63b887cf08df87e68751d7068c0843d72964d10e3
+  data.tar.gz: 3bfcca81437c8a530de3eb5e92f893d45945dace06c1c6b105c4f797491b87f8
 SHA512:
-  metadata.gz: ba0c3abab62e8f8931c6f7ca3f84fe35b343e5341df5ea9652f641c1b8b6d146241be1cd3b554d84232b2d44a1651c475aee2f5abb8580a4b75b2faf9c971806
-  data.tar.gz: 3394db605f8b6910d0e41a40c9087ba74daa36b3b5b60d5db65487fa870d4dbc4e0c56c4aed43d5476473e327fc7cf6bd9b6856078578a6809d78b9bc6a5fc49
+  metadata.gz: '07087e501496c4029784c2f29cae6779214473d636ad8d38cb7f000ff4f1cfc1c8be1d9c973f15f5949afb00df9bbbbe7ffee76851c7f333c129414dfdb849ac'
+  data.tar.gz: ca66c7ef561dba5dbb48f9c6e8f57d5cc38f9a16965d4c9a88560038d8a548f5f79b8aaf1414213dc3a6c1b4b77e46854170114dd49e8e50a85affde7d746555

data/CHANGELOG.md

@@ -1,5 +1,44 @@
 ## [Unreleased]
 
+## [0.3.0] - 2026-11-15
+
+Schema caching rework. Minor (not patch) because `Schema.fetch` and `Model.clear_schema_cache`
+change observable behaviour and new public configuration is added — see Changed below.
+
+### Fixed
+
+- Schema changes now reach running processes without a restart (other processes need a shared
+  `config.cache`, e.g. `Rails.cache`). The per-model memo used to live
+  for the whole process, so neither `Schema.clear_cache` nor store expiry reached running puma or
+  worker processes: a new single-select option stayed invisible (and failed
+  `inclusion: field_options(...)` validations) until a restart.
+
+### Changed
+
+- The base schema is memoized **per base** in each process (`Schema::Memo`, one per `base_id`)
+  instead of per model class: `Schema.fetch(base_id)` reads the memo, then `config.cache`, then
+  `/v0/meta`. The memo is trusted for `config.schema_memo_ttl`; `/v0/meta` is still called only
+  on a store miss.
+- If a reload fails with `ApiError` or `ConnectionError`, `Schema.fetch` keeps serving the previous
+  copy, logs a warning to `config.logger` and retries after 30 seconds (or the TTL, if shorter).
+  Without a copy the error
+  is raised. `fetch(force_refresh: true)` raises on failure, leaving the copy as the fallback.
+- `Schema.clear_cache(base_id)` also expires this process's memo, so the next read reloads (the
+  copy stays as the outage fallback); with a shared store, every other process follows within the
+  TTL.
+- `Model.clear_schema_cache` now drops the memo of the model's whole base (shared by every model
+  on it), fallback copy included. `Airtable::ORM.configure` and `Airtable::ORM.reset!` drop every
+  memo.
+
+### Added
+
+- `config.schema_memo_ttl` (default `5.minutes`; `nil`/`0` disables the memo).
+- `config.schema_cache_expiry` (default `24.hours`, the previous hard-coded value) sets how long
+  the schema lives in `config.cache`. `Schema::CACHE_EXPIRY` is deprecated and no longer read.
+- `Schema.clear_memo(base_id)` and `Schema.reset_memos!` drop in-process memos.
+- Explicit `concurrent-ruby ~> 1.1` dependency (ActiveSupport already pulls in a compatible
+  version).
+
 ## [0.2.3] - 2026-09-24
 
 ### Fixed

data/README.md

@@ -53,7 +53,9 @@ end
 | `rate_limit` | `5` | Requests/second per base (Airtable throttles for 30 s above 5) |
 | `logger` | null logger | Batch-update failure reporting |
 | `http_logger` | `nil` | Set to a `Logger` to log Faraday requests |
-| `cache` | in-process `MemoryCache` | Schema cache; anything responding to `fetch`/`delete` |
+| `cache` | in-process `MemoryCache` | Schema cache; anything responding to `fetch(key, expires_in:)`/`delete` |
+| `schema_cache_expiry` | `24.hours` | How long the base schema lives in `cache` before `/v0/meta` is re-read |
+| `schema_memo_ttl` | `5.minutes` | How long each process trusts its in-memory schema before re-reading `cache`; `nil`/`0` disables the memo |
 
 `Airtable::ORM.configured?` returns whether an `api_key` is set — handy for gating enqueue hooks so test/CI environments never touch the network.
 
@@ -75,10 +77,30 @@ end
 - **Querying:** `all`, `where(formula:, sort:, max_records:)`, `first`, `find`, `find_many`, `find_by`/`find_by!`, `count`. Symbol conditions build (and escape) Airtable formulas; pagination is automatic.
 - **Persistence:** `save`/`save!`, `create`/`create!`, `update`/`update!`, `destroy`, `reload`, plus `Model.update_many(records)` batching 10 per PATCH with a `BatchResult` (`updated`/`skipped`/`failed`).
 - **Associations:** `has_many`/`belongs_to` readers, writers, `add_*`/`remove_*`, memoization and `Collection#preload` for eager loading.
-- **Schema:** table/field metadata from `/v0/meta/bases`, cached via `config.cache`; `Model.field_options(:state)` reads select options.
+- **Schema:** table/field metadata from `/v0/meta/bases`, cached via `config.cache`; `Model.field_options(:state)` reads select options. See [Schema caching](#schema-caching).
 - **`#url`** — deeplink to the record in the Airtable UI.
 - **`normalizes`** — available on ActiveModel 8.0+ (the DSL doesn't exist in 7.1).
 
+## Schema caching
+
+The base schema (tables, fields, select options) is cached on two levels, both keyed by base ID:
+
+1. **Store** — `config.cache` holds the indexed base schema for `config.schema_cache_expiry`;
+   `/v0/meta` is only called on a miss.
+2. **Per-process memo** — one memo per base, shared by every model on it, trusted for
+   `config.schema_memo_ttl` and then re-read through the store. If the re-read fails with an API
+   or connection error, the previous copy keeps serving (logged as a warning) and the re-read is
+   retried after 30 seconds (or the TTL, if shorter). With the memo disabled (`nil`/`0`), every
+   schema lookup reads the store.
+
+After changing the schema in Airtable (e.g. a new single-select option), call
+`Airtable::ORM::Schema.clear_cache(base_id)` — or `Schema.fetch(base_id, force_refresh: true)` to
+refetch right away (it raises if Airtable is unreachable). The calling process sees the new schema
+on its next read (barring a reload already in flight); with a **shared** store (e.g. `Rails.cache`) every other process sees it within
+`schema_memo_ttl`, no restart needed. With the default in-process `MemoryCache`, other processes
+refresh only when their own entry expires. `Model.clear_schema_cache` (or
+`Schema.clear_memo(base_id)`) drops this process's memo for the base, without a fallback copy.
+
 ## Errors
 
 Consumers rescue `Airtable::ORM::*` only — the HTTP client (Faraday today) never leaks, so swapping transports is not a breaking change.

data/lib/airtable/orm/attributes.rb

@@ -42,22 +42,21 @@ module Airtable
           ORM.config.table_id(table_name)
         end
 
-        # Get schema for this table
-        # Memoized at class level to avoid repeated cache lookups
+        # Schema of this table; caching (memo, store, fallback) lives in Schema, keyed by base_id.
         def schema
           return {} unless table_name
 
-          @schema ||= Airtable::ORM::Schema.fetch(base_id)[table_id] || raise(
+          Airtable::ORM::Schema.fetch(base_id)[table_id] || raise(
             Airtable::ORM::ConfigurationError,
             "No table #{table_id.inspect} (#{table_name.inspect}) in the fetched schema for base " \
             "#{base_id.inspect} — check config.tables against the Airtable base"
           )
         end
 
-        # Clear the memoized schema cache
-        # Useful for testing or when the schema changes
+        # Drops this process's schema memo for the model's base (shared by every model on it,
+        # fallback copy included) and the field mapping.
         def clear_schema_cache
-          @schema = nil
+          Airtable::ORM::Schema.clear_memo(base_id)
           @field_mapping = nil
         end
 
@@ -83,8 +82,9 @@ module Airtable
         def schema_options
           return {} unless field_mapping.present?
 
+          table_schema = schema
           field_mapping.each_with_object({}) do |(symbol, field_id), options|
-            field_info = schema.dig(:fields, field_id)
+            field_info = table_schema.dig(:fields, field_id)
             next unless field_info
 
             case field_info[:type]

data/lib/airtable/orm/configuration.rb

@@ -10,7 +10,8 @@ module Airtable
     class Configuration
       attr_accessor :api_key, :base_id, :tables, :api_url,
                     :open_timeout, :read_timeout, :rate_limit,
-                    :logger, :http_logger, :cache
+                    :logger, :http_logger, :cache,
+                    :schema_cache_expiry, :schema_memo_ttl
 
       def initialize
         @tables = {}
@@ -21,6 +22,8 @@ module Airtable
         @logger = Logger.new(IO::NULL)
         @http_logger = nil             # set to a Logger to enable Faraday request logging
         @cache = MemoryCache.new       # schema cache default; hosts inject Rails.cache
+        @schema_cache_expiry = 24.hours # schema lifetime in `cache` before /v0/meta is re-read
+        @schema_memo_ttl = 5.minutes    # per-process memo before re-reading `cache`; nil/0 = off
       end
 
       # Example: table_fields(:case) => { _id: "id", advisors: "fldgU5KU1qlN8eJ3v", ... }
@@ -41,7 +44,7 @@ module Airtable
       # API on every Schema.fetch (and burn the 5 RPS budget on metadata). Hosts inject a real
       # store (Rails.cache); this one honours only :expires_in. The mutex is held across the
       # fetch block (all keys serialize behind it, and nesting fetch calls would deadlock) —
-      # fine for the rare schema refresh this exists for.
+      # fine, since nothing fetches from inside the schema fetch block.
       class MemoryCache
         def initialize
           @mutex = Mutex.new

data/lib/airtable/orm/schema/memo.rb

@@ -0,0 +1,69 @@
+# frozen_string_literal: true
+
+module Airtable
+  module ORM
+    class Schema
+      # Per-process memo of ONE base's indexed schema in front of the cache store (Schema keeps
+      # one per base_id), trusted for config.schema_memo_ttl. If a reload fails with an API or
+      # connection error, the previous copy keeps serving and the reload is retried after
+      # FALLBACK_RETRY. No locking: concurrent reloads just re-read the store, and the last write
+      # wins (entries are frozen, an ivar write is atomic).
+      class Memo
+        Entry = Struct.new(:schema, :expires_at)
+        FALLBACK_RETRY = 30 # seconds (capped by the TTL)
+
+        def self.clock
+          Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        end
+
+        def initialize(base_id)
+          @base_id = base_id
+          @entry = nil
+        end
+
+        # Memoized schema, reloading via the block once the TTL has elapsed.
+        def fetch
+          return yield unless memoizing?
+
+          entry = @entry
+          return entry.schema if entry && self.class.clock < entry.expires_at
+
+          store(yield, ttl)
+        rescue ApiError, ConnectionError => e
+          raise unless entry&.schema
+
+          ORM.config.logger.warn("Airtable schema refresh for base #{@base_id} failed, " \
+                                 "serving the previous copy: #{e.class}: #{e.message}")
+          store(entry.schema, [FALLBACK_RETRY, ttl].min)
+        end
+
+        # Reload via the block now; raises on failure, leaving the previous copy as the fallback.
+        def refresh
+          memoizing? ? store(yield, ttl) : yield
+        end
+
+        # Force a reload on next read but keep the copy as the outage fallback.
+        def expire!
+          entry = @entry
+          @entry = Entry.new(entry.schema, -Float::INFINITY).freeze if entry
+        end
+
+        private
+
+        def memoizing?
+          ttl.positive?
+        end
+
+        def ttl
+          ORM.config.schema_memo_ttl.to_f # nil → 0.0 (off)
+        end
+
+        # Stamped after the load, so the TTL starts once the value is in hand.
+        def store(schema, expires_in)
+          @entry = Entry.new(schema, self.class.clock + expires_in).freeze
+          schema
+        end
+      end
+    end
+  end
+end

data/lib/airtable/orm/schema.rb

@@ -1,24 +1,61 @@
 # frozen_string_literal: true
 
+require "concurrent/map"
+
 module Airtable
   module ORM
     class Schema
       CACHE_KEY = "airtable:schema"
+      # Superseded by config.schema_cache_expiry; no longer read.
       CACHE_EXPIRY = 24.hours
+      deprecate_constant :CACHE_EXPIRY
+
+      MEMOS = Concurrent::Map.new # base_id → Memo
+      private_constant :MEMOS
 
       class << self
+        # Indexed base schema: the per-process memo (config.schema_memo_ttl), then the cache store
+        # (config.schema_cache_expiry), then /v0/meta. force_refresh deletes the store entry,
+        # expires the memo and refetches; it raises on failure, leaving the memo's copy as the
+        # outage fallback.
         def fetch(base_id, force_refresh: false)
-          clear_cache(base_id) if force_refresh
+          base_id = base_id.to_s
+          memo = memo_for(base_id)
+          return memo.fetch { read_store(base_id) } unless force_refresh
 
-          cache.fetch(cache_key(base_id), expires_in: CACHE_EXPIRY) { fetch_from_api(base_id) }
+          clear_cache(base_id)
+          memo.refresh { read_store(base_id) }
         end
 
+        # Deletes the store entry and expires this process's memo (it keeps its copy as the outage
+        # fallback); with a shared store, other processes follow once their schema_memo_ttl elapses.
         def clear_cache(base_id)
-          cache.delete(cache_key(base_id))
+          cache.delete(cache_key(base_id)).tap { MEMOS[base_id.to_s]&.expire! }
+        end
+
+        # Drops this process's memo for the base, fallback copy included.
+        def clear_memo(base_id)
+          MEMOS.delete(base_id.to_s)
+          nil
+        end
+
+        # Drops every memo in this process (ORM.configure and ORM.reset!).
+        def reset_memos!
+          MEMOS.clear
         end
 
         private
 
+        def memo_for(base_id)
+          MEMOS.compute_if_absent(base_id) { Memo.new(base_id) }
+        end
+
+        def read_store(base_id)
+          cache.fetch(cache_key(base_id), expires_in: ORM.config.schema_cache_expiry) do
+            fetch_from_api(base_id)
+          end
+        end
+
         def fetch_from_api(base_id)
           response = client.connection.get("/v0/meta/bases/#{base_id}/tables")
           parsed_response = response.body
@@ -31,8 +68,9 @@ module Airtable
         end
 
         # Index the FULL base schema — no filtering by the tables configured at fetch time.
-        # The cached payload outlives the config (24h, shared store in Rails hosts), so a
-        # fetch-time filter would hide tables added to config after the cache warmed up.
+        # The cached payload outlives the config (config.schema_cache_expiry, a shared store in
+        # Rails hosts), so a fetch-time filter would hide tables added to config after the cache
+        # warmed up.
         # Reads are keyed lookups (fetch(base_id)[table_id]), so no read-time filter is needed.
         def indexed_schema(parsed_response)
           tables = parsed_response.deep_symbolize_keys[:tables]

data/lib/airtable/orm/version.rb

@@ -2,6 +2,6 @@
 
 module Airtable
   module ORM
-    VERSION = "0.2.3"
+    VERSION = "0.3.0"
   end
 end

data/lib/airtable/orm.rb

@@ -33,6 +33,7 @@ module Airtable
         # The client memoizes api_key/timeouts/rate_limit at first request — without this
         # reset a later configure (e.g. rotating the API key) would be silently ignored.
         Http::Client.reset!
+        Schema.reset_memos! # a new cache store or tables must not be served from old memos
       end
 
       # True when an API key is configured — hosts gate network-touching hooks on this.
@@ -43,6 +44,7 @@ module Airtable
       def reset!
         @config = nil
         Http::Client.reset!
+        Schema.reset_memos!
       end
     end
   end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: airtable-orm
 version: !ruby/object:Gem::Version
-  version: 0.2.3
+  version: 0.3.0
 platform: ruby
 authors:
 - Roman Sklenar
@@ -29,6 +29,20 @@ dependencies:
     - - "<"
       - !ruby/object:Gem::Version
         version: '9'
+- !ruby/object:Gem::Dependency
+  name: concurrent-ruby
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
 - !ruby/object:Gem::Dependency
   name: faraday
   requirement: !ruby/object:Gem::Requirement
@@ -105,6 +119,7 @@ files:
 - lib/airtable/orm/persistence.rb
 - lib/airtable/orm/querying.rb
 - lib/airtable/orm/schema.rb
+- lib/airtable/orm/schema/memo.rb
 - lib/airtable/orm/testing.rb
 - lib/airtable/orm/testing/stub_helpers.rb
 - lib/airtable/orm/version.rb