scout-essentials 1.8.7 → 1.9.0

data/doc/developer/AnnotationSystem.md

@@ -0,0 +1,184 @@
+# Annotation System
+
+Annotations attach named metadata to **ordinary Ruby objects** without
+wrapping them and without changing their class. The same mechanism is what
+makes `Path`, `Resource` and `NamedArray` work, so understanding it is a
+prerequisite for [Path Resolution](PathResolution.md) and
+[Persistence and Resources](PersistenceAndResources.md).
+
+Everything on this page is backed by `tmp/rewrite_C/probe_01..04.rb` and
+probes P36–P42 in `research/behavior-probes.md`, run against the current
+`lib/scout/annotation*.rb`.
+
+## Load path
+
+`lib/scout-essentials.rb` does **not** require `scout/annotation` directly.
+`Annotation` becomes available because `scout/path` requires it
+(`lib/scout/path.rb:1`), so `require 'scout-essentials'` gives you the
+constant (`Object.const_defined?(:Annotation) => true`, probe_03) — but a
+library that only wants annotations can `require 'scout/annotation'` on its
+own.
+
+## The DSL
+
+```ruby
+module SampleInfo
+  extend Annotation
+  annotation :organism, :tissue
+end
+```
+
+`extend Annotation` turns the module into an **AnnotationModule**. The
+`annotation` class-level call does three things (annotation_module.rb):
+
+- declares the attribute list, kept in module state `@annotations` and read
+  with `SampleInfo.annotations` (there is **no `ANNOTATIONS` constant**);
+- defines reader/writer methods (`#organism`, `#organism=`) for when the
+  module is mixed into an object;
+- registers the module so `Annotation.setup` can resolve type names.
+
+## `setup` and the binding rules
+
+`AnnotationModule#setup(obj, values = nil, &block)` is the constructor. The
+values may be given as a Hash or positionally, and a block is an alternative
+source for the object itself:
+
+```ruby
+SampleInfo.setup('S003', organism: 'Human', tissue: 'Liver')
+SampleInfo.setup('S004', %w[Human Liver])          # positional, same order
+SampleInfo.setup { 'S005' }                         # block-as-object
+```
+
+Rules verified in probe_01/probe_02:
+
+- **In-place extension.** `setup` calls `obj.extend SampleInfo` on the object
+  you pass and returns **that same object** (`setup(s).equal?(s) => true`) —
+  except when the object is **frozen**, in which case `obj.dup` is annotated
+  and returned instead; always use the return value of `setup`.
+- **TypeError → un-annotated.** Objects that cannot hold singleton methods
+  (`Integer` literals, symbols) raise `TypeError: can't define singleton`;
+  `setup` rescues it and returns the *plain* object with no metadata.
+- **`:annotation_types` is reserved.** Each annotated object gets an
+  `@annotation_types` array; declaring `annotation :annotation_types` collides
+  with it (probe_01: writing to it raised `NoMethodError` against `nil`).
+
+## Introspection and serialisation
+
+State lives on the object, the type list on the object too:
+
+```ruby
+s = SampleInfo.setup('S003', organism: 'Human', tissue: 'Liver')
+
+SampleInfo.annotations           # => [:organism, :tissue]   (module state)
+s.annotation_types               # => [SampleInfo]           (module objects!)
+Annotation.is_annotated?(s)      # => true
+s.is_a?(SampleInfo)              # => true  (extend really happened)
+s.is_a?(String)                  # => true  (class unchanged)
+```
+
+`annotation_types` holds the **module objects**, never their names —
+`s.annotation_types.include?(SampleInfo)` is the correct membership test.
+
+`AnnotatedObject` (mixed into every annotated object) provides:
+
+```ruby
+s.annotation_hash   # => {:organism=>"Human", :tissue=>"Liver"}
+s.annotation_info   # => {:organism=>"Human", :tissue=>"Liver",
+                    #     :annotation_types=>[SampleInfo], :annotated_array=>false}
+s.serialize         # => {...same, :literal=>"S003"}   (values purged recursively)
+s.annotation_id     # => "860c73f490edb8115064663b7f579d73"
+```
+
+- `annotation_hash` — the declared attributes only;
+- `annotation_info` — plus `:annotation_types` and `:annotated_array`;
+- `serialize` — `annotation_info` merged with `:literal` (the object itself),
+  with every value passed through `Annotation.purge` (annotated_object.rb:23);
+  this is what the `:annotation` serialisation driver consumes. It is **only a
+  Hash here**; the TSV form (`Annotation.tsv` / `Annotation.load_tsv`) lives in
+  scout-gear, not in this repo (attribution table in
+  [Architecture](Architecture.md));
+- `annotation_id` (aliased `id`) — `Misc.digest([self, annotation_info])`.
+
+### Module-level helpers
+
+- `Annotation.is_annotated?(obj)` — true if any annotation module is mixed in.
+- `Annotation.purge(obj)` — **recursive**: for an annotated object it calls the
+  instance `#purge`; for Arrays/Hashes it purges each element; otherwise it
+  returns the object unchanged.
+- `Annotation.setup(obj, "A|B", hash)` — the generic deserialiser. The type
+  string is split on `|`, each name resolved with `Object.const_get`. **Unknown
+  names only warn and are skipped** (`Log.warn "Annotation NoSuchAnnotation not
+  defined"`, probe_01) — no exception.
+
+### Instance-side helpers
+
+- `#purge` — returns a **copy**: it removes `@annotation_types`, `@annotations`
+  and every attribute ivar from a `dup` (and purges nested values) and returns
+  it. The object's own class is kept (a purged `SampleInfo` String is still a
+  `String`, no longer `is_a?(SampleInfo)`); always use the return value.
+- `#make_array` — wraps `self` in a **one-element Array** carrying the same
+  annotations and extends it with `AnnotatedArray` (annotated_object.rb:75-80);
+  it does not annotate the receiver's elements.
+- `#annotate(other)` — copies the current annotations onto another object.
+
+## Round-trips and copies
+
+- **Marshal round-trips** annotations (P42, probe_01): the singleton modules
+  survive dump/load.
+- **Only `dup` loses them; `clone` keeps them** (probe_02:
+  `dup is_annotated? => false`, `clone is_annotated? => true, .a => 1` —
+  `clone` copies the singleton class, `dup` does not): to re-annotate a `dup`
+  copy the metadata explicitly with `annotation_hash` →
+  `Annotation.setup` / `MyModule.setup`, or call `#annotate` on the copy.
+
+## `AnnotatedArray`
+
+`extend AnnotatedArray` on an annotated Array — the pattern used throughout
+`test/scout/annotation/test_array.rb` — makes the *elements* carry the
+container's annotations:
+
+```ruby
+arr = SampleInfo.setup(%w[S001 S002], organism: 'Human')
+arr.extend AnnotatedArray
+arr[0].organism        # => "Human"
+arr.first.organism     # => "Human"
+```
+
+Overrides provided in `lib/scout/annotation/array.rb`:
+
+- `[]` `(pos, clean = false)` — the element is re-annotated unless the second
+  argument is truthy, in which case it is returned clean (probe_09: a fresh
+  array's `fresh[0, true]` is an un-annotated `String` while `fresh[1]` is
+  annotated);
+- `first`, `last`, `each_with_index`, `each`, `inject`, `collect`, `select` —
+  re-annotated;
+- `compact`, `uniq`, `flatten`, `reverse`, `sort_by` — re-annotated;
+- `subset(list)`, `remove(list)` — set operations (`&`, `-`) with
+  re-annotation.
+
+**Limits** — live probe `tmp/rewrite_C/probe_09_annotated_array.rb` (method
+owners plus actual results): `map`, `zip`, `filter_map`, `flat_map`,
+`each_slice`, `values_at` and `count` are **not overridden** (their owner is
+`Array`/`Enumerable`) and return plain results with no annotations — `ary.map
+{ |x| x }` yields `[false, false, false]` under `Annotation.is_annotated?`,
+while `ary.each` yields `[true, true, true]`, and `zip` keeps annotations only
+on the container-side elements.
+
+**Requirement:** elements must be extendable. `AnnotatedArray` over an Array
+of `Integer`s raises `TypeError: can't define singleton` (probe_09) — use
+Strings or other extendable objects.
+
+## NamedArray is a separate thing
+
+`NamedArray` (`lib/scout/named_array.rb`) is an Annotation module over Arrays
+giving field-name access to positions. It does **not** extend
+`AnnotatedArray` and is not a String type. It needs an explicit
+`require 'scout/named_array'`. See [Annotating
+Data](../user/AnnotatingData.md).
+
+## Related
+
+- [Path Resolution](PathResolution.md) — `Path` is an annotated module.
+- [Persistence and Resources](PersistenceAndResources.md) — `Resource` is an
+  annotated module; `:annotation` serialisation uses `serialize`.
+- [Annotating Data](../user/AnnotatingData.md) — the user-facing API.

data/doc/developer/Architecture.md

@@ -0,0 +1,147 @@
+# Architecture
+
+This page describes how the parts of scout-essentials fit together: the
+module dependency graph, the load order, and where the boundaries to the
+rest of the Scout ecosystem are. Everything here is derived from the source
+under `lib/`.
+
+## Modules and dependencies
+
+```
+IndiferentHash  (no internal dependencies)
+      ^  ^
+      |  \-----------------\
+Annotation  (standalone)     \
+      ^                       \
+      |                        \
+Misc::Digest <- Path  -----> Resource (annotation-based)
+      ^           ^               ^  ^   ^
+      |           |               |  |   \
+  TmpFile --> Open --> CMD --> ConcurrentStream
+                ^          ^
+                |          |
+              Lockfile   (required BY CMD, not the other way round)
+```
+
+Reading the edges as "requires":
+
+- `IndiferentHash` requires nothing else internally.
+- `Annotation` is a standalone mixin framework; `Path` and `Resource`
+  *extend* it, which is how `Path`/`Resource` metadata (pkgdir, libdir,
+  `@where`, `@original`, ...) attaches to plain Strings. This
+  **annotation-based unification** means a `Path` object is just a String
+  carrying annotations, and `Resource` is a module (not a class) whose
+  class-level state (`pkgdir`, `rake_dirs`, `path_maps`, `map_order`,
+  `lock_dir`) is read through those annotations.
+- `Path` requires `Misc::Digest` (for `digest` naming) and the annotation
+  framework.
+- `TmpFile` requires `Open`; the arrow therefore points **TmpFile → Open**
+  (not "Open depends on TmpFile").
+- `Open` requires `Path` and `CMD`; `Open::Remote` shells out through
+  `CMD`. `Open` *uses* `TmpFile` helpers at runtime but does not require the
+  file.
+- `CMD` requires `ConcurrentStream` (and `Open::Stream`); `ConcurrentStream`
+  does not require `CMD`.
+- `Resource` requires `Log` and `Path` only — it does **not** depend on
+  `Persist`.
+- `Persist` requires `Open` (and through it `CMD`); it never touches
+  `Resource`.
+- `Log` colors come from `term-ansicolor`; there is **no `Log::Color`
+  constant**.
+- The `Lockfile` implementation is **vendored** at
+  `lib/scout/open/lock/lockfile.rb` and used by `Open.lock`.
+
+## Load order and entry points
+
+`lib/scout-essentials.rb` is the **only** entry point; there is no
+`lib/scout.rb`:
+
+```ruby
+require_relative 'scout/exceptions'
+require_relative 'scout/indiferent_hash'
+require_relative 'scout/tmpfile'
+require_relative 'scout/log'
+require_relative 'scout/path'
+require_relative 'scout/simple_opt'
+require_relative 'scout/resource'
+require_relative 'scout/resource/scout'
+require_relative 'scout/persist'
+require_relative 'scout/config'
+```
+
+Notes on the cascade:
+
+- `Annotation`, `Misc`, `CMD`, `ConcurrentStream`, `Open` and `Persist`'s
+  sub-files are pulled in indirectly (e.g. `scout/path` requires
+  `scout/annotation`; `scout/persist` requires `scout/persist/serialize`,
+  `scout/persist/open`, `scout/persist/path`).
+- `scout/resource/scout` defines `Scout` (`extend Resource`, `pkgdir =
+  'scout'`, `Resource.default_resource = Scout`) and loads
+  `Path.load_path_maps(Scout.etc['path_maps'])`.
+- `Misc::NamedArray` and `Misc::Hook` are **not** auto-loaded; they live in
+  `lib/scout/named_array.rb` and `lib/scout/misc/hook.rb` and are required
+  explicitly by the file that needs them.
+- `scout/config` runs last and provides the `Scout::Config` module (not a top-level `Config` constant).
+
+## Lock namespaces
+
+There are three distinct lock directories, all under `$HOME/.scout/tmp`:
+
+| Namespace | Constant/definition | Used by |
+|-----------|---------------------|---------|
+| `tmp/persist_locks` | `Persist.lock_dir` | `Persist.persist` while (re)computing a cache entry |
+| `tmp/produce_locks` | `Resource.default_lock_dir` | `Resource#produce` while materializing a resource |
+| `tmp/sensible_write_locks` | `Open.sensible_write_lock_dir` | `Open.sensible_write` atomic writes |
+
+`Persist.lock` does **not exist** — the locking primitive is
+`Open.lock(filename, &block)`, built on the vendored `Lockfile`.
+
+## Fork model
+
+`CMD` runs subprocesses through `Open3`, wiring stdout/stderr into
+`ConcurrentStream`s so that streaming consumers can read while the process
+is alive. Aborting a `CMD` kills the child process and aborts the attached
+streams. `Resource#produce` for rake claims forks a `ScoutRake` invocation,
+so a failed rake does not leave partial files behind
+(`Open.sensible_write` removes them).
+
+## Packaging
+
+`scout-essentials.gemspec` is generated by juwelier from the `Juwelier::Tasks`
+block in the `Rakefile` ("DO NOT EDIT THIS FILE DIRECTLY"). That block lists
+four runtime dependencies — `term-ansicolor`, `yaml`, `rake`, `listen` — and
+the checked-in gemspec mirrors them as `add_runtime_dependency` entries
+(lines 172-175). Each is actually used from `lib/`: `term/ansicolor`
+(`scout/log/color.rb`), `yaml` (`scout/persist/open.rb`,
+`scout/log/progress/report.rb`), `rake` (`scout/resource/produce/rake.rb`),
+`listen` (`scout/open/util.rb`, loaded lazily when a watcher is requested).
+Nothing else is required at runtime; in particular scout-essentials does
+not depend on scout-gear, rbbt-util, or any TSV/Workflow/Step
+implementation.
+
+## Ecosystem boundaries and attribution
+
+scout-essentials is a **substrate library**. The following well-known Scout
+concepts are **not** implemented here; each row says where they actually
+live:
+
+| Concept | Where it lives |
+|---------|----------------|
+| `TSV`, `TSV::Dumper`, `Annotation.tsv` | scout-gear (`scout/tsv`) |
+| `Workflow`, `Task`, `Step`, `Step#info` / `.info` files | scout-gear (`scout/workflow`) |
+| Job/step persistence beyond `Persist` (`.info` directory, `files/`) | scout-gear |
+| HPC / scheduler (`slurm`, `lsf`, `pbs`, orchestrators) | scout-gear / scout-rig |
+| `notify` / `send_email` helpers | scout-gear |
+| `Bgzf` (blocked gzip) | scout-gear |
+| `deep_indifferent` | **does not exist anywhere in the ecosystem** |
+| Scout CLI (`scout` executable, `scout_commands/`) | scout-gear |
+| rbbt-util shims (e.g. `rbbt-util` compatibility aliases) | UNVERIFIED — not present in this repo's `lib/` |
+
+When writing documentation or examples for this repo, do not present those
+as features of scout-essentials.
+
+## Related
+
+- [Path Resolution](PathResolution.md) for the `Path`/`Resource` machinery.
+- [Persistence and Resources](PersistenceAndResources.md) for `Persist` and
+  `Resource#produce` contracts.

data/doc/developer/Configuration.md

@@ -0,0 +1,238 @@
+# Configuration
+
+`Scout::Config` is the registry behind every option the Scout stack reads:
+it is a flat `key -> [tokens, value]` store with one rule — **among the
+entries whose token matches the caller's lookup context, the entry with the
+LOWEST priority number wins**.
+
+Source: `lib/scout/config.rb` (183 lines, whole file), plus
+`lib/scout/resource/scout.rb` for `Scout.etc` (`Path.setup("etc")` — see
+[PathResolution.md](PathResolution.md)).
+
+## The token priority table
+
+`Scout::Config.token_priority(token)` (`lib/scout/config.rb:38-67`) splits a
+token on `::` and returns `[token, priority]`:
+
+| token | priority | who sets it |
+|---|---|---|
+| `line:<file>:<line>` | 1 | derived from the caller of `Config.get` |
+| `file:<file>` | 2 | derived from the caller of `Config.get` |
+| `task:<name>` | 3 | passed by higher layers (e.g. a workflow task) |
+| `workflow:<name>` | 4 | passed by higher layers |
+| anything else | 10 | free-form custom tokens |
+| `key:` / `key:<name>` | 20 | the key itself; LOWEST priority |
+| `token::N` | N | explicit numeric suffix always wins over the defaults |
+
+Verified by `tmp/rewrite_D/probe_11_token_probe.rb`:
+
+```text
+workflow:x   => ["workflow:x", 4]
+task:x       => ["task:x", 3]
+file:x       => ["file:x", 2]
+line:x       => ["line:x", 1]
+key:zz       => ["key:zz", 20]
+other:x      => ["other:x", 10]
+explicit ::5 => ["workflow:x", 5]
+```
+
+Lower number = higher precedence, because `Config.get` picks
+`priorities.sort_by{|p,v| p}.first` (`lib/scout/config.rb:132`). The `key:`
+token (20) is deliberately the weakest, so any context entry can override a
+plain value.
+
+## A worked example
+
+A common misreading is "workflow overrides task". The numbers say the
+opposite — `line` (1) outranks `workflow` (4). `tmp/rewrite_D/probe_13_get_internals.rb`
+stacks all four tokens on one key and shows the chain end-to-end:
+
+```ruby
+def fresh(k); Scout::Config::CACHE.delete(k); end
+
+k='q1'; fresh k
+Scout::Config.set({k=>'wfv'},'workflow:W')
+Scout::Config.set({k=>'taskv'},'task:T')
+Scout::Config.set({k=>'filev'},'file:F')
+Scout::Config.set({k=>'linev'},'line:L')
+Scout::Config.get(k,'workflow:W','task:T','file:F','line:L')
+# => "linev"   (line=1 wins)
+
+k='q2'; fresh k
+Scout::Config.set({k=>'wfv'},'workflow:W')
+Scout::Config.set({k=>'taskv'},'task:T')
+Scout::Config.set({k=>'filev'},'file:F')
+Scout::Config.get(k,'workflow:W','task:T','file:F')
+# => "filev"   (file=2 wins)
+
+k='q3'; fresh k
+Scout::Config.set({k=>'wfv'},'workflow:W')
+Scout::Config.set({k=>'taskv'},'task:T')
+Scout::Config.get(k,'workflow:W','task:T')
+# => "taskv"   (task=3 wins)
+```
+
+Within one priority bucket the LAST `set` wins (`priorities[prio].unshift`,
+`lib/scout/config.rb:83`; verified `probe_13` `two task entries => "second"`).
+
+Explicit suffixes punch through the table, both ways — `::1` promotes an
+entry above every default token, `::30` buries it below `key:` (probe_13,
+cases `q6`/`q7`). Note the suffix is per-entry, not per-key: it re-prices one
+`set` call, it does not change the token for later calls.
+
+### Implicit caller tokens
+
+Every `Config.get` appends `file:<caller>` and `line:<caller>:<n>` derived
+from `caller` (`lib/scout/config.rb:119-127`), after filtering out frames
+matching the legacy rbbt regexes (see *Legacy internals* below). Entries set
+with those tokens match without being named in the call (verified
+`probe_12`, `matched by implicit caller file token => "v"`).
+
+## `etc/config` files
+
+`Scout.etc.config` is `Path.setup("etc").config` — typically
+`$HOME/.scout/etc/config`. Lines are `key value token...`, `#` comments,
+whitespace-separated (`load_file`, `lib/scout/config.rb:15-25`):
+
+```text
+# comment
+key value token
+```
+
+`load_config` walks `Path.setup("etc").config.find_all.reverse`
+(`lib/scout/config.rb:27-31`). `find_all` returns paths in `map_order`
+(`current, user, home, local, global, usr, lib, fast, cache, bulk, ...`), so
+the **reverse** order loads the *most specific* file (e.g. `$PWD/etc/config`)
+**first** and `$HOME/.scout/etc/config` last. Two files setting the same key
+with the same (implicit `key:`) token were tested by
+`tmp/rewrite_D/probe_23_findall_order.rb`:
+
+```text
+map_order    => [:current, :user, :home, ...]
+find_all     => [/tmp/.../etc/config, /home/.../.scout/etc/config]
+loaded order => find_all.reverse  (HOME first, CWD last)
+CACHE['shared'] => [key-token fromHOME, key-token fromCWD]
+get('shared')   => "fromCWD-currentmap"
+```
+
+The `current` map wins even though it was loaded last, because `add_entry`
+**appends** and `match` `unshift`s each matching value, so the last-loaded
+entry lands at the *front* of the priority bucket and `get` takes
+`priorities.sort_by.first`. In short: **later-loaded (more specific) files
+win ties**.
+
+`Config.set({key => value}, *tokens)` is the programmatic equivalent; with
+one key it auto-adds the `key:<name>` token (`add_entry`,
+`lib/scout/config.rb:11-19`).
+
+## `Config.get` value post-processing
+
+`Config.get(key, *tokens)` (`lib/scout/config.rb:94-155`) does four things
+after resolving the winner:
+
+1. **`'false'` becomes `false`** — the ONLY string-to-object coercion.
+   Verified in `tmp/rewrite_D/probe_10_env.rb`: `'false'` → `false`
+   (FalseClass), but `'TRUE'`, `'0'` and `'42'` all stay Strings.
+2. **`'nil'` becomes `nil`** (probe_10).
+3. **`env:VAR[,VAR2]`** in the resolved value is replaced by the first set
+   environment variable, and `get` returns `nil` if none is set (probe_10,
+   `'from-env'`).
+4. Every `get` appends `[key, value, tokens]` to `GOT_KEYS` — an audit trail
+   of what was asked and answered (probe_10). `with_config` snapshots and
+   restores it.
+
+Options hash as last argument:
+
+- `:default => v` — used when no entry matches.
+- `:env => 'VAR1,VAR2'` — first set env var becomes the default (probe_10).
+
+A `Symbol` key raises `TypeError` — the cache is keyed by `key.to_s`, but
+`CACHE[key.to_s]` is called after `match` has already used the raw key;
+`Config.get(:symbol)` raises `TypeError: no implicit conversion of Symbol
+into String` (probe_10). Use String keys.
+
+## `with_config` and `process_config`
+
+`with_config { ... }` (`lib/scout/config.rb:157-166`) dups every `CACHE`
+bucket and `GOT_KEYS`, yields, and restores both in an `ensure`. Anything
+`set` inside the block is rolled back — including sets made by code you
+call (probe_10: `inside => "inside"`, `after => "outside"`). It is the
+mechanism that keeps config changes from leaking out of a sub-operation.
+
+`Config.process_config(config)` (`lib/scout/config.rb:168-180`) interprets
+a CLI `--config` argument, in this order:
+
+1. existing file path → `load_file`
+2. name of an existing `Scout.etc.config_profile[config]` file → `load_file`
+3. otherwise a `"key value token..."` string. The `::N` suffix is mandatory
+   in the stored token: any token without one gets `prio = "0"`, i.e. the
+   token is re-emitted as `token::0`.
+
+Verified by `tmp/rewrite_D/probe_17b.rb` / `probe_17c.rb`. The subtle part
+of case 3 is that the stored token keeps the suffix, so `process_config
+"k3 cli workflow::0"` stores the token `workflow::0`, whose *name* is the
+bare string `workflow` at priority 0 (stronger than `task`=3,
+`file`=2, `line`=1, `key:`=20), and it does **not** match a call like
+`get('k3', 'workflow:W')`, because the token *names* differ
+(`workflow::0` vs `workflow:W`). Quoting a concrete outcome:
+
+```text
+CACHE[k3]                       => [[["workflow::0", "key:k3"], "cli"]]
+token_priority('workflow::0')   => ["workflow", 0]
+get(k3)                         => "cli"     (via the implicit key: token)
+get(k3, 'workflow')             => "cli"     (prio 0 beats key:=20)
+get(k3, 'workflow:W')           => nil       (token name mismatch)
+```
+
+So a CLI `-config "key val workflow::0"` overrides anything except entries
+carrying an explicit smaller `::N` — but only for lookups that pass the bare
+`workflow` token.
+
+## Environment variables read by the gem
+
+Verified by grep over `lib/` and, where noted, executed probes
+(`tmp/rewrite_D/probe_10_env.rb` covers the `env:`/`:env` machinery; the
+rest are read directly from source):
+
+| variable | read by | effect |
+|---|---|---|
+| `SCOUT_LOG` | `lib/scout/log.rb:35` | sets default log severity |
+| `SCOUT_LOG_INSIST` | `lib/scout/misc/insist.rb:35` | logs each insist retry exception |
+| `SCOUT_NOCOLOR` | `lib/scout/log/color.rb:137` (also `misc/format.rb:28`) | disables colors / unicode seconds glyph |
+| `SCOUT_NO_PROGRESS` | `lib/scout/log/progress.rb:11` | disables progress bars |
+| `SCOUT_ORIGINAL_STACK` | `lib/scout/log.rb:261,319` | keeps original exception backtrace |
+| `SCOUT_DEBUG_PID` | `lib/scout/log.rb:178` | prefixes messages with the pid |
+
+All are tested with `== 'true'` (except `SCOUT_LOG`, a severity name), so
+any other value is false.
+
+`Config.get` itself reads no env var unless `:env` is given or the value is
+an `env:` string.
+
+## Legacy internals
+
+The caller-token filter in `Config.get` (`lib/scout/config.rb:119-127`)
+still names `rbbt/...` files in its regexes. That is a leftover from the
+upstream project this gem was extracted from; the effect here is only that
+frames whose path matches those patterns are skipped when deriving
+`file:`/`line:` tokens. Nothing else in `scout-essentials` depends on it
+(marked INTERNAL in the audit matrix).
+
+## Thread-safety
+
+There is no mutex anywhere in `lib/scout/config.rb` (`Scout::Config` has no
+`Mutex` ivar; verified `tmp/rewrite_D/probe_09_threads_fork.rb`). `CACHE`
+and `GOT_KEYS` are plain shared structures and `add_entry` does
+read-modify-write (`CACHE[key] ||= []; CACHE[key] << ...`). Concurrent
+`set`/`get` from threads is therefore unprotected; the gem's own processes
+are fork-based, not thread-based (see
+[LockingAndConcurrency.md](LockingAndConcurrency.md)).
+
+## Related pages
+
+- [Architecture.md](Architecture.md) — where config sits in the layering.
+- [CommandLineOptions.md](../user/CommandLineOptions.md) — the user-facing
+  side of `process_config`.
+- [PathResolution.md](PathResolution.md) — `Scout.etc`, `find_all` order,
+  `etc/config` search path.
+- [CachingResults.md](../user/CachingResults.md) — what reads `Config.get`.