custmatch-server 0.1.0__tar.gz

custmatch_server-0.1.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

custmatch_server-0.1.0/NOTICE

@@ -0,0 +1,24 @@
+custmatch
+Copyright 2026 Andrew Goodchild
+
+Licensed under the Apache License, Version 2.0 (see LICENSE).
+
+patches/ditto-mps-and-fixes.patch modifies Ditto (https://github.com/megagonlabs/ditto),
+Copyright Megagon Labs, licensed under the Apache License, Version 2.0. The patch contains
+lines of Ditto's source as diff context.
+
+No third-party datasets are included in the current tree. (Earlier commits in the history
+contained the nickname list below and a Wikidata name list, CC0, since removed.) custmatch
+downloads it into ~/.cache/custmatch (or $CUSTMATCH_DATA) when first needed:
+
+- nicknames.csv: the carltonnorthern/nicknames list (https://github.com/carltonnorthern/nicknames,
+  commit ed160b3), Copyright its contributors, licensed under the Apache License, Version 2.0.
+  Downloaded unmodified on first use and checked against its SHA-256.
+
+
+Other datasets used by the research scripts are not included either. Scripts download or generate them (FEBRL and
+historical_50k via recordlinkage and splink, the Magellan / DeepMatcher and MatchGPT
+benchmarks from their own repositories, synthetic people via pseudopeople, name frequencies
+from the US Census and Social Security via the FiveThirtyEight data repository (CC BY 4.0)
+for the Australian test file), each under its own licence; see docs/reproducing.md. Files in results/ are metrics and logs computed from
+them.

custmatch_server-0.1.0/PKG-INFO

@@ -0,0 +1,289 @@
+Metadata-Version: 2.4
+Name: custmatch-server
+Version: 0.1.0
+Summary: custmatch as a self-hosted service: scheduled batch runs, real-time matching, a steward dashboard
+Author: Andrew Goodchild
+License-Expression: Apache-2.0
+Project-URL: Repository, https://github.com/andrewgoodchild/custmatch
+Project-URL: Documentation, https://github.com/andrewgoodchild/custmatch/blob/main/server/README.md
+Project-URL: Issues, https://github.com/andrewgoodchild/custmatch/issues
+Keywords: entity resolution,record linkage,customer matching,master data management,data stewardship
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python :: 3
+Classifier: Framework :: FastAPI
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Database
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+Requires-Dist: custmatch>=0.1.0
+Requires-Dist: sqlalchemy>=2
+Requires-Dist: alembic
+Requires-Dist: fastapi
+Requires-Dist: uvicorn
+Requires-Dist: psycopg[binary]
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: httpx; extra == "test"
+Dynamic: license-file
+
+# custmatch-server: operator guide
+
+custmatch-server runs custmatch as a service next to your data: it keeps your customer records
+and their loads in one database, matches them in batch on a schedule, answers "who is this?" in
+real time, and gives data stewards a dashboard to review and correct matches. One image, one
+database (Postgres in production, SQLite for a trial). How it works is below; what it was
+measured at is in the report's chapter on the service
+([docs/report/12-the-service.md](https://github.com/andrewgoodchild/custmatch/blob/main/docs/report/12-the-service.md)).
+
+| Process | Command | What it does |
+| --- | --- | --- |
+| API and dashboard | `custmatch-server serve` | Lookups, real-time matching, overrides, the dashboard at `/dashboard` |
+| Scheduler | `custmatch-server schedule ...` | Loads files dropped in an inbox, runs the batch at set times, checks health, alerts |
+| One-off jobs | `load`, `batch`, `key`, `check` | The same work by hand |
+
+All processes share the database (`CUSTMATCH_DB_URL`) and a data directory for run files
+(`CUSTMATCH_DATA_DIR`). Run one API process and one scheduler; the API scales to more copies for
+lookups, but only one holds the write lease for real-time matching at a time.
+
+## How it works
+
+```
+                 ┌─────────────────────────── custmatch-server ──────────────────────────┐
+  files ──► inbox│ scheduler ── load ──────────────┐                                     │
+                 │     │                           ▼                                     │
+                 │     └─ batch run ──publish──► ┌──────────── the database ───────────┐ │
+                 │       (the custmatch          │ source records and every version,   │ │
+                 │        library)               │ loads, runs (which one is current), │ │
+  your apps ────►│ API: lookups, POST /match ◄──►│ customers, links, golden records,   │ │
+                 │   matcher: one writer,        │ history, steward decisions,         │ │
+                 │   index in memory             │ audit log, API keys                 │ │
+  stewards ─────►│ dashboard ◄──────────────────►└─────────────────────────────────────┘ │
+                 └───────────────────────────────────────────────────────────────────────┘
+```
+
+- **One database holds everything shared:** the source records and their loads, the runs and
+  which one is current, the results, stewards' decisions and the matcher's lock. Postgres for
+  production (many readers and writers, advisory locks, managed everywhere); SQLite on a
+  persistent disk for a trial or one small team. The code targets both, so moving from trial to
+  production is configuration.
+- **Loads keep history.** A load is full (records missing from it are marked deleted, not
+  removed) or incremental. The current version of every record is kept, and every earlier
+  version, so "what did this record look like at load N" has an answer. Only the mapped fields
+  are stored.
+- **The batch run is the truth.** It is `custmatch run` over every live record - loaded from
+  files or arrived through the API - with labels and stewards' decisions read from the database,
+  and customer IDs carried over from the current run. It publishes in **one transaction**:
+  readers see the old run or the new one, never a mix. The current run and one before it are kept.
+- **Real-time matching is provisional.** `POST /match` scores a record against the current run's
+  model in memory - a compact index of hashed blocking keys, built from the batch blocker's own
+  key definition - and joins it to the customer it fits: it must match at least half of that
+  customer's records, not just one. It merges a second customer only on a link of 0.9 or more.
+  Exactly one process writes (a lease); lookups keep working without it. Stewards' decisions,
+  trusted IDs, the twin guard and duplicate-free sources apply at once; the next batch run
+  reconciles everything, and changes reach users as merges and splits in the history.
+- **The tables** (`custmatch_server/schema.py`, created by versioned migrations):
+
+| Table | Holds |
+| --- | --- |
+| `loads` | Every load: source, origin (a file or `api:<day>`), full or incremental, and what it inserted, updated, deleted and left unchanged |
+| `source_records`, `source_record_versions` | The current version of every record, with a content hash and `deleted_at`; and every earlier version |
+| `runs`, `current_run` | The catalog of runs, and the one readers see |
+| `customers`, `record_customer`, `links`, `customer_events` | Per run: golden records, which customer each record is in, the accepted links with evidence, and merges and splits (partitioned by run on Postgres) |
+| `realtime_records`, `realtime_merges`, `realtime_moves` | The API's matching decisions, replayed when the matcher restarts, until a batch run reconciles them |
+| `overrides`, `steward_log`, `golden_pins`, `review_claims` | Stewards' decisions and their approval, the audit log (with each action's note), pinned golden values, and review pairs held by the steward working on them |
+| `api_keys`, `leases` | Hashed API keys with roles, and the matcher's write lease |
+
+## Start it
+
+A trial on your machine, with Postgres and the demo data:
+
+```sh
+docker compose -f server/docker-compose.yml up -d --build db api
+docker compose -f server/docker-compose.yml run --rm api key create me --role admin   # shown once: keep it
+```
+
+Load the demo (or your own files, below) and run a first batch - one scheduler pass, done by hand:
+
+```sh
+for s in web app store; do cp examples/demo/$s.csv server/inbox/$s.full.csv; done
+docker compose -f server/docker-compose.yml run --rm scheduler \
+  schedule /demo/config.json --labels /demo/labels.csv --inbox /inbox --once
+```
+
+Then start the scheduler, which from now on loads what arrives in the inbox and runs the batch
+at 02:00 UTC (only one scheduler works at a time, so a `--once` pass while it runs does nothing
+and says so):
+
+```sh
+docker compose -f server/docker-compose.yml up -d scheduler
+```
+
+Then open http://localhost:8765/dashboard and paste a key with the `steward` (or `admin`) role.
+
+The image holds the library's default model only. For the opt-in `"matcher_model":
+"forest_catboost"` (docs/report/03-which-matcher.md "Which classifier"), build it with CatBoost: `CUSTMATCH_EXTRAS=catboost
+docker compose -f server/docker-compose.yml build`, or `docker build --build-arg CUSTMATCH_EXTRAS=catboost
+...`. A run trained with it cannot be served by an image without it; the API says so.
+
+For production, run the same image three ways (API, scheduler, and Postgres you already manage)
+on whatever runs containers for you, with your config and labels mounted where the commands
+point. Without Docker: `pip install ./ ./server` from the repository.
+
+## Configuration
+
+| Variable | Default | Meaning |
+| --- | --- | --- |
+| `CUSTMATCH_DB_URL` | `sqlite:///custmatch.db` | The database. Postgres: `postgresql+psycopg://user:password@host/db` |
+| `CUSTMATCH_DATA_DIR` | `data` (`/data` in the image) | Run files: each run's inputs, outputs and model. Shared by every process |
+| `CUSTMATCH_AUTH` | on | `off` answers anyone who can reach the API: a local trial only |
+| `CUSTMATCH_ALERT_WEBHOOK` | none | Where the scheduler posts alerts: any URL taking a JSON POST (`{"text": ..., "checks": [...]}`; Slack incoming webhooks read `text`) |
+| `CUSTMATCH_MAX_RUN_AGE_HOURS` | 36 | Warn when the current run is older |
+| `CUSTMATCH_MAX_CUSTOMER_CHANGE` | 5 | Warn when the customer count moves by more than this % between runs |
+| `CUSTMATCH_MAX_REVIEW` | 1000 | Warn when the review queue is longer |
+| `CUSTMATCH_FOUR_EYES` | off | `on`: a steward's decisions, merges, splits and revocations wait until a different steward approves them (Decisions tab). Needs authentication on |
+| `CUSTMATCH_DB_PASSWORD` | `custmatch` | The Compose file's Postgres password: set it for anything but a local trial |
+
+The matching itself is configured by the custmatch config file (sources, columns, rules) and the
+labels file: see the [guide](https://github.com/andrewgoodchild/custmatch/blob/main/docs/guide.md). The database creates and upgrades its own tables
+when any process starts.
+
+## API keys
+
+```sh
+custmatch-server key create NAME --role ROLE   # prints the key once; only its hash is stored
+custmatch-server key list
+custmatch-server key revoke NAME               # stops working within 30 seconds
+```
+
+| Role | Can |
+| --- | --- |
+| `read` | Look up records and customers, read runs, loads, decisions and health |
+| `match` | Also match records in real time (`POST /match`) |
+| `steward` | Also record decisions (`POST /overrides`): the dashboard's role |
+| `admin` | Everything |
+
+Give each person and each system its own key: a steward's decisions are recorded under their
+key's name. `server/sql/roles.sql` sketches least-privilege database roles for Postgres.
+
+## Getting records in
+
+**The inbox** (with the scheduler running): drop a file named `<source>.<mode>.<csv|parquet>` -
+`web.full.csv`, `store.incremental.2026-10-05.parquet` - where `source` is a source in your config
+and `mode` is:
+
+- `full`: the source's complete contents. Records missing from it are marked deleted (kept, with
+  their history, but left out of matching).
+- `incremental`: new and changed records only.
+
+The scheduler waits until a file's size has stopped changing, loads it, and moves it to
+`inbox/done/` (or `inbox/failed/`, with an alert). The image runs as user 10001: on Linux, make the
+inbox folder writable by it (`chown 10001 inbox`). Each load is recorded: what it inserted,
+updated, deleted and left unchanged, and every version of every record.
+
+**By hand:** `custmatch-server load CONFIG SOURCE FILE [--full]`.
+
+**Through the API:** records matched with `POST /match` and `"write": true` are stored as that
+day's API load and included in the next batch run.
+
+## Batch runs and the schedule
+
+`custmatch-server schedule CONFIG --labels LABELS --inbox DIR --batch-at 02:00` loads the inbox as
+files arrive and runs the batch at 02:00 UTC daily (repeat `--batch-at` for more). A batch run
+matches every live record, keeps customer IDs stable from the previous run, publishes in one
+transaction (readers see the old run or the new one, never a mix), keeps the previous run for
+rollback, and runs the health checks. Only one scheduler works at a time, even if two are started.
+
+With your own scheduler (cron, a Kubernetes CronJob, Airflow), run
+`custmatch-server schedule ... --once` instead: one pass - load the inbox, run a batch, check -
+exiting 0 (ok), 1 (warnings) or 2 (failed).
+
+**Stewards' decisions train the model.** Each "same" or "different" a steward records overrides
+that pair at once, and the next batch run also trains on it. Measured on the demo
+(`server/learning_test.py`, `results/server_learning.json`): starting from 200 random labels,
+three rounds of answering the review queue lifted clustered F1 from 79.5 to 88.8 when decisions
+train the model, against 84.1 when they only override their own pairs.
+
+## The dashboard
+
+`/dashboard`, with a `steward` key:
+
+- **Overview**: health, the current run, loads, recent decisions.
+- **Model & data**: the threshold and accuracy estimates, score distribution, what the model
+  relies on, completeness and common values by source, records too sparse to link, blocking,
+  runs compared.
+- **Customers**: search for a person; browse customers worth a second look (largest, weakest
+  link, ...); sample links by score. A customer shows its golden record, its records as a network
+  of links, and why each link scored what it did.
+- **Review**: the uncertain pairs, most uncertain first: same person, different people, or skip
+  (keys S, D, K). Filter by source, the model's call or score; a list mode decides many at once;
+  a note goes with each decision. Pairs shown to one steward are held from the others for 15
+  minutes.
+- **Decisions**: every decision with its author and note; revoke one, approve or reject proposals
+  (with `CUSTMATCH_FOUR_EYES=on`), download them as CSV; steward activity, including how often
+  each steward overturned the model.
+
+In a customer: golden-record lineage (which records hold each value) and pinning a value, which
+beats survivorship in this and every later run; merging with another customer and splitting a
+record out, both previewed before they happen and applied at once; the household; every steward
+action on its records. Under Customers, **Compare** puts any two records side by side and says
+whether blocking would ever compare them (why they did not match); **Find possible matches**
+scores a person you type in, without saving anything. Every panel has an (i) explaining what it
+shows, how it is calculated and what good looks like.
+
+## Monitoring
+
+`GET /health` (no key) says the process is up. `GET /status` and `custmatch-server check` run the
+health checks; the scheduler runs them after every batch and posts anything not ok to
+`CUSTMATCH_ALERT_WEBHOOK`.
+
+| Check | Warns or fails when |
+| --- | --- |
+| run | there is no published run, or the latest run failed |
+| run_age | the current run is older than `CUSTMATCH_MAX_RUN_AGE_HOURS` |
+| customers | the customer count moved more than `CUSTMATCH_MAX_CUSTOMER_CHANGE` % between runs: look at the run's merges and splits before trusting it |
+| review | the review queue is over `CUSTMATCH_MAX_REVIEW`, or growing fast |
+| loads | a load failed in the last day, or has run for over two hours |
+| realtime | a real-time record has waited longer than the run age limit for a batch run |
+
+## Backup and restore
+
+Back up two things together: the database and the data directory.
+
+```sh
+pg_dump -Fc "$DATABASE" > custmatch-$(date +%F).dump        # Postgres
+tar czf custmatch-runs-$(date +%F).tgz -C "$CUSTMATCH_DATA_DIR" runs
+```
+
+With SQLite, stop the processes and copy the `.db` file (or use `sqlite3 custmatch.db ".backup copy.db"`).
+Restore both from the same time: the database names the run directories it uses. Only the
+current run and the one before it are kept, so older run directories can be deleted or archived.
+
+## Upgrading
+
+1. Back up (above).
+2. Stop the scheduler, so no batch run is in progress.
+3. Start the new image's processes: the database upgrades itself at start (migrations).
+4. Run `custmatch-server check`.
+
+Migrations do not run backwards automatically; to go back, restore the backup and the old image.
+
+## Sizing
+
+Measured with Postgres on a laptop (`server/scale_test.py`, `results/server_scale.json`), per
+million records: loads about 1 minute, a batch run about 2.5 minutes (2-3 GB of memory), the
+real-time matcher starts in 6 seconds and holds about 0.5 GB, a real-time match takes 24 ms
+(median) and a lookup 7 ms; the database grows by about 1.8 GB. The batch library has run 10
+million records on one 16-core machine (docs/report/06-blocking-and-scale.md, "Larger files").
+
+## Troubleshooting
+
+| Symptom | Look at |
+| --- | --- |
+| `401` from every call | The key: `custmatch-server key list` (revoked?); the header is `Authorization: Bearer <key>` |
+| `409 another matcher holds the write lease` | Another API process is the writer; send writes to one process, or wait 60 s after stopping the old one |
+| A file stays in the inbox | Its name: `<source>.<full|incremental>.<csv|parquet>`, with a source from the config |
+| A file lands in `inbox/failed/` | The scheduler's log and the alert: usually a missing column or an unknown source |
+| "customers" warns after a run | Customers' events in the dashboard (Customers, then a customer's history); a new source or a data problem can merge many customers at once |
+| The review queue keeps growing | More labels help most: answering the queue trains the next run |