@engine9/schemas 1.10.1 → 1.11.3

package/README.md

@@ -7,7 +7,7 @@ this code as-is.
 
 ## Documentation
 
-Each schema plugin is documented in its own `README.md`. That file is the human-readable contract: what the schema plugin is for, the data model, inbound/outbound behavior, search, **predefined segments**, and UI. Keep implementation details in code comments; keep audience and membership rules in the README. Dashboards live on `@engine9/plugins/reports/<area>`, not in this package.
+Each schema plugin is documented in its own `README.md`. That file is the human-readable contract: what the schema plugin is for, the data model, inbound/outbound behavior, segment definition clauses, **predefined segments**, and UI. Keep implementation details in code comments; keep audience and membership rules in the README. Dashboards live on `@engine9/plugins/reports/<area>`, not in this package.
 
 Schema plugins with predefined segments:
 
@@ -22,9 +22,9 @@ See [`skills/create-engine9-plugin/SKILL.md`](../skills/create-engine9-plugin/SK
 
 - **Plugin uniqueness:** `SchemaWorker.install` reuses an existing `plugin` row when the path is unique: `metadata.unique` if set, otherwise `@engine9/schemas/*` (except `person_custom`, which is `unique: false`). Native plugins that set `unique: true` behave the same. Third-party plugins and `person_custom` may have more than one instance per path; pass `id` to update an existing non-unique row.
 
-- **Public module surface:** Export only the standard building blocks documented in [`skills/create-engine9-plugin/SKILL.md`](../skills/create-engine9-plugin/SKILL.md): for example `metadata`, optional `schema`, `transforms`, `search`, `segments`, `metrics`, and the default object that aggregates them. Do **not** export `reports` from schema plugins — dashboards belong on `@engine9/plugins/reports/<area>`. Do **not** export ad hoc hooks (such as custom install helpers or segment–plugin wiring) from schema plugins; the server is responsible for any special install-time behavior tied to a specific schema plugin path.
+- **Public module surface:** Export only the standard building blocks documented in [`skills/create-engine9-plugin/SKILL.md`](../skills/create-engine9-plugin/SKILL.md): for example `metadata`, optional `schema`, `transforms`, `segmentDefinitionClauses`, `segments`, `metrics`, and the default object that aggregates them. Do **not** export `reports` from schema plugins — dashboards belong on `@engine9/plugins/reports/<area>`. Do **not** export ad hoc hooks (such as custom install helpers or segment–plugin wiring) from schema plugins; the server is responsible for any special install-time behavior tied to a specific schema plugin path.
 
-- **Segments, universes, and `plugin_id`:** Saved segments use `segment.plugin_id` = the schema plugin (or plugin) that **owns the definition**. A definition’s **`universe`** describes the span of source data searched across; input EQL entries select timeline stores, while person/table entries can restrict eligible people. Explicit `pluginId` values inside `search` options may narrow that span further, so a segment can evaluate data tied to other deployed plugins without changing its owning `plugin_id`. Export bundles use the same universe vocabulary for table and input-file artifacts.
+- **Segments, universes, and `plugin_id`:** Saved segments use `segment.plugin_id` = the schema plugin (or plugin) that **owns the definition**. A definition’s **`universe`** describes the span of source data searched across; input EQL entries select timeline stores, while person/table entries can restrict eligible people. Explicit `pluginId` values inside a clause's options may narrow that span further, so a segment can evaluate data tied to other deployed plugins without changing its owning `plugin_id`. Export bundles use the same universe vocabulary for table and input-file artifacts.
 
 ## License
 

package/channels/email/README.md

@@ -1,10 +1,10 @@
 # Email Channel Schema Plugin
 
-The `@engine9/schemas/channels/email` plugin defines email-channel search and engagement segments. It does not create its own tables. Segment builds read `global_message`, `input`, and `timeline` rows produced by email message plugins.
+The `@engine9/schemas/channels/email` plugin defines the email-channel segment definition clause and engagement segments. It does not create its own tables. Segment builds read `global_message`, `input`, and `timeline` rows produced by email message plugins.
 
 It depends on `@engine9/schemas/person` because membership is always a set of people.
 
-## Search
+## Segment definition clauses
 
 `emailEngagement` finds people who opened or clicked email in a rolling window.
 
@@ -14,13 +14,13 @@ It depends on `@engine9/schemas/person` because membership is always a set of pe
 | `timelineEntryType` | `EMAIL_OPEN` or `EMAIL_CLICK` |
 | `pluginId` | Optional. When set, only that plugin's inputs are included. When empty, scope comes from the segment universe instead |
 
-The search looks at `timeline` rows whose `entry_type_id` matches the engagement type and whose `ts` is in the window. It always joins `timeline.input_id = input.id`, so only timeline entries for inputs present in the build are counted.
+The clause looks at `timeline` rows whose `entry_type_id` matches the engagement type and whose `ts` is in the window. It always joins `timeline.input_id = input.id`, so only timeline entries for inputs present in the build are counted.
 
 ## Segments
 
 Predefined audiences shipped with this schema plugin. On install, each key becomes a `segment` row whose definition path is `@engine9/schemas/channels/email:segments:<key>`.
 
-Every segment here uses the same **universe** and the `emailEngagement` search. The universe and the search do different jobs, and both are required:
+Every segment here uses the same **universe** and the `emailEngagement` clause. The universe and the clause do different jobs, and both are required:
 
 - **Universe** chooses *which email sends* can contribute entries: messages on the email channel published in the last **90 days** (`global_message.channel = 'email'` and `publish_date` within 90 days). Those message ids become the `input` rows for the build.
 - **Search** chooses *which people* from those sends belong in the audience: a recent open or click on the timeline, inside the segment's rolling window.

package/channels/email/index.js

@@ -1,4 +1,4 @@
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 import segments, { personSegmentTableName, universeEmailPublished90d } from './segments.js';
 
 const metadata = {
@@ -9,13 +9,13 @@ const metadata = {
 };
 
 export { metadata };
-export { search };
+export { segmentDefinitionClauses };
 export { segments };
 export { personSegmentTableName };
 export { universeEmailPublished90d };
 export default {
   metadata,
-  search,
+  segmentDefinitionClauses,
   segments,
   personSegmentTableName,
   universeEmailPublished90d

package/channels/email/{search.js → segmentDefinitionClauses.js}

@@ -3,7 +3,7 @@ import { TIMELINE_ENTRY_TYPES } from '@engine9/input-tools';
 /**
  * Build EQL for email engagement (opens / clicks) within a rolling window.
  *
- * The search always joins `timeline` → `input` on `timeline.input_id = input.id`.
+ * The clause always joins `timeline` → `input` on `timeline.input_id = input.id`.
  * During a segment build the DuckDB `input` table is populated exclusively with
  * the input ids discovered by the segment's `universe` entry (e.g.
  * `universeEmailPublished90d`), so the join scopes results to the universe even
@@ -60,8 +60,9 @@ export const emailEngagement = {
     type: 'object',
     properties: {
       pluginId: {
-        title: 'Plugin ID',
-        type: 'string'
+        title: 'Plugin',
+        type: 'string',
+        resolveOptionsUsing: 'plugin'
       },
       windowDays: {
         title: 'Window (days)',
@@ -81,6 +82,7 @@ export const emailEngagement = {
   }
 };
 
+export const segmentDefinitionClauses = { emailEngagement };
 export default {
-  emailEngagement
+  segmentDefinitionClauses
 };

package/channels/email/segments.js

@@ -1,4 +1,4 @@
-const SEGMENT_SEARCH_PATH = '@engine9/schemas/channels/email:search:emailEngagement';
+const EMAIL_ENGAGEMENT_CLAUSE_PATH = '@engine9/schemas/channels/email:segment_definition_clause:emailEngagement';
 
 /**
  * EQL over the message table: rows published in the last 90 days on the email channel.
@@ -6,12 +6,12 @@ const SEGMENT_SEARCH_PATH = '@engine9/schemas/channels/email:search:emailEngagem
  * scanned for timeline parquet.
  *
  * During a segment build the input ids returned here are copied into the DuckDB `input`
- * table.  The search EQL (see `emailEngagement` in `search.js`) always joins
+ * table.  The clause EQL (see `emailEngagement` in `segmentDefinitionClauses.js`) always joins
  * `timeline.input_id = input.id`, so only timeline rows whose input is in **this**
  * universe are considered — even when `pluginId` is empty.  This means:
  *
  * - The **universe** controls *which* message/input stores supply timeline data.
- * - The **search** applies time-window and entry-type filters *within* that scope.
+ * - The **clause** applies time-window and entry-type filters *within* that scope.
  *
  * Both layers are required for correct segment membership counts.
  */
@@ -35,7 +35,7 @@ export function personSegmentTableName(tablePrefix, segmentName) {
 /**
  * Build a segment definition for email engagement (opens or clicks).
  *
- * `pluginId` is left empty intentionally: the search handler in `search.js`
+ * `pluginId` is left empty intentionally: the `emailEngagement` clause
  * always joins `timeline` → `input`, and the segment `universe` determines
  * which input rows are present in DuckDB.  This scopes the time-window query
  * to the correct set of message inputs without hard-coding a plugin id.
@@ -44,10 +44,10 @@ function emailEngagementSegment(name, windowDays, timelineEntryType) {
   return {
     name,
     universe: [universeEmailPublished90d],
-    search: {
+    definition: {
       and: [
         {
-          path: SEGMENT_SEARCH_PATH,
+          path: EMAIL_ENGAGEMENT_CLAUSE_PATH,
           options: {
             pluginId: '',
             windowDays,
@@ -61,7 +61,7 @@ function emailEngagementSegment(name, windowDays, timelineEntryType) {
 
 /**
  * Rolling-window engagement segments.  `pluginId` is empty so scope follows the
- * segment `universe` (the `input` table join in the search EQL restricts to
+ * segment `universe` (the `input` table join in the clause EQL restricts to
  * universe-provided message input ids).
  */
 export const email_openers_30d = emailEngagementSegment('30-day email openers', 30, 'EMAIL_OPEN');

package/event/README.md

@@ -56,11 +56,11 @@ None in this package. Native event / calendar plugins should upsert `event` and
 
 None.
 
-## Search
+## Segment definition clauses
 
 | Key | Path | Purpose |
 | --- | --- | --- |
-| `responses` | `@engine9/schemas/event:search:responses` | Filter people by `person_event.response` and optional `eventId`. |
+| `responses` | `@engine9/schemas/event:segment_definition_clause:responses` | Filter people by `person_event.response` and optional `eventId`. |
 
 ## Segments
 

package/event/index.js

@@ -1,5 +1,5 @@
 import schema from './schema.js';
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 
 const metadata = {
   name: '@engine9/schemas/event',
@@ -9,9 +9,9 @@ const metadata = {
   schemas: ['schema.js']
 };
 
-export { metadata, schema, search };
+export { metadata, schema, segmentDefinitionClauses };
 export default {
   metadata,
   schema,
-  search
+  segmentDefinitionClauses
 };

package/event/{search.js → segmentDefinitionClauses.js}

@@ -46,6 +46,7 @@ export const responses = {
   }
 };
 
+export const segmentDefinitionClauses = { responses };
 export default {
-  responses
+  segmentDefinitionClauses
 };

package/package.json

@@ -1,11 +1,11 @@
 {
   "name": "@engine9/schemas",
-  "version": "1.10.1",
+  "version": "1.11.3",
   "type": "module",
   "description": "Engine9 standard schemas: shared tables, transforms, searches, and segments",
   "main": "index.js",
   "scripts": {
-    "test": "node --test person/upsert.test.js person_address/normalize.test.js person_email/upsert.test.js person_hash/hash.test.js person_name/upsert.test.js person_phone/upsert.test.js stacks/stacks.test.js transaction/core/attribute.test.js"
+    "test": "node --test person/upsert.test.js person_address/normalize.test.js person_address/segmentDefinitionClauses.test.js person_email/upsert.test.js person_hash/hash.test.js person_name/upsert.test.js person_phone/upsert.test.js stacks/stacks.test.js timeline/segmentDefinitionClauses.test.js transaction/core/attribute.test.js transaction/core/segmentDefinitionClauses.test.js"
   },
   "engines": {
     "node": ">=18"

package/person/index.js

@@ -21,11 +21,10 @@ const transforms = {
   // Saved export definitions still call @engine9/schemas/person:transforms:appendName.
   appendName
 };
-export const search = {
-  person: {
-    title: 'Primary Info',
+const person = {
+    title: 'Ids',
     form: {
-      title: 'Primary Info',
+      title: 'Ids',
       type: 'object',
       properties: {
         ids: {
@@ -71,8 +70,8 @@ export const search = {
         }
       };
     }
-  }
-};
+  };
+export const segmentDefinitionClauses = { person };
 export { metadata };
 export { schema };
 export { transforms };
@@ -82,5 +81,5 @@ export default {
   schema,
   transforms,
   metrics,
-  search
+  segmentDefinitionClauses
 };

package/person_address/index.js

@@ -2,6 +2,7 @@ import upsertPersonAddress from './transforms/inbound/upsert_tables.js';
 import normalize from './transforms/inbound/normalize.js';
 import appendAddress from './transforms/outbound/appendAddress.js';
 import schema from './schema.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 
 const metadata = {
   name: '@engine9/schemas/person_address',
@@ -23,8 +24,10 @@ export const transforms = {
 
 export { metadata };
 export { schema };
+export { segmentDefinitionClauses };
 export default {
   metadata,
   schema,
-  transforms
+  transforms,
+  segmentDefinitionClauses
 };

package/person_address/segmentDefinitionClauses.js

@@ -0,0 +1,98 @@
+import { normalizeCountry, normalizePostalCode, normalizeRegion } from './normalize.js';
+
+function blank(value) {
+  return value == null || String(value).trim() === '';
+}
+
+/* Stored regions are normalized per country; without a country, accept US then Canadian names. */
+function regionFor(region, country) {
+  if (country) return normalizeRegion(region, country);
+  const us = normalizeRegion(region, 'US');
+  const upper = String(region).toUpperCase().replace(/\./g, '').trim();
+  return us !== upper ? us : normalizeRegion(region, 'CA');
+}
+
+/* Without a country, read US ZIPs and Canadian postal codes by shape; leave anything else as typed. */
+function postalCountryFor(postalCode, country) {
+  if (country) return country;
+  if (/^\d{5}(-?\d{4})?$/.test(postalCode)) return 'US';
+  if (/^[A-Z]\d[A-Z] ?\d[A-Z]\d$/.test(postalCode)) return 'CA';
+  return null;
+}
+
+function equals(column, value) {
+  return {
+    type: 'EQUALS',
+    values: [{ ref: { column } }, { value: { value } }]
+  };
+}
+
+export const addresses = {
+  title: 'Addresses',
+  description:
+    'People with an address in a region (state / province) and/or postal code. One value per field; combine several with an or group.',
+  form: {
+    title: 'Addresses',
+    type: 'object',
+    properties: {
+      region: {
+        title: 'Region',
+        description: "State or province code or name, e.g. 'VA' or 'Virginia'",
+        type: 'string'
+      },
+      postalCode: {
+        title: 'Postal code',
+        description: "Exact postal code, or a LIKE pattern when it contains %, e.g. '220%'",
+        type: 'string'
+      },
+      country: {
+        title: 'Country',
+        description: "ISO country code or name, e.g. 'US'. Optional; narrows the match and how region and postal code are read.",
+        type: 'string'
+      }
+    },
+    required: []
+  },
+  optionsToEQL(options = {}) {
+    const { region, postalCode, country: countryOption } = options;
+    const conditions = [];
+    const text = [];
+    const country = blank(countryOption) ? null : normalizeCountry(countryOption);
+    if (!blank(region)) {
+      const value = regionFor(region, country);
+      conditions.push(equals('region', value));
+      text.push(`in ${value}`);
+    }
+    if (!blank(postalCode)) {
+      const raw = String(postalCode).trim().toUpperCase();
+      if (raw.includes('%')) {
+        conditions.push({
+          type: 'LIKE',
+          values: [{ ref: { column: 'postal_code' } }, { value: { value: raw } }]
+        });
+        text.push(`with postal code matching ${raw}`);
+      } else {
+        const value = normalizePostalCode(raw, postalCountryFor(raw, country));
+        conditions.push(equals('postal_code', value));
+        text.push(`with postal code ${value}`);
+      }
+    }
+    if (country) {
+      conditions.push(equals('country', country));
+      text.push(`in country ${country}`);
+    }
+    return {
+      text: ['Has an address', ...text].join(' '),
+      eql: {
+        table: 'person_address',
+        columns: ['person_id'],
+        conditions
+      }
+    };
+  }
+};
+
+export const segmentDefinitionClauses = { addresses };
+export default {
+  segmentDefinitionClauses
+};

package/person_address/segmentDefinitionClauses.test.js

@@ -0,0 +1,46 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { addresses } from './segmentDefinitionClauses.js';
+
+const valuesByColumn = (eql) =>
+  Object.fromEntries(eql.conditions.map((c) => [c.values[0].ref.column, [c.type, c.values[1].value.value]]));
+
+test('region names normalize to the stored code', () => {
+  const { text, eql } = addresses.optionsToEQL({ region: 'Virginia' });
+  assert.equal(eql.table, 'person_address');
+  assert.deepEqual(eql.columns, ['person_id']);
+  assert.deepEqual(valuesByColumn(eql), { region: ['EQUALS', 'VA'] });
+  assert.equal(text, 'Has an address in VA');
+});
+
+test('Canadian province names normalize without a country', () => {
+  const { eql } = addresses.optionsToEQL({ region: 'Ontario' });
+  assert.deepEqual(valuesByColumn(eql), { region: ['EQUALS', 'ON'] });
+});
+
+test('postal codes normalize by shape, or match a LIKE pattern', () => {
+  assert.deepEqual(valuesByColumn(addresses.optionsToEQL({ postalCode: '22201-1234' }).eql), {
+    postal_code: ['EQUALS', '22201']
+  });
+  assert.deepEqual(valuesByColumn(addresses.optionsToEQL({ postalCode: 'k1a0b1' }).eql), {
+    postal_code: ['EQUALS', 'K1A 0B1']
+  });
+  assert.deepEqual(valuesByColumn(addresses.optionsToEQL({ postalCode: 'SW1A 1AA' }).eql), {
+    postal_code: ['EQUALS', 'SW1A 1AA']
+  });
+  assert.deepEqual(valuesByColumn(addresses.optionsToEQL({ postalCode: '220%' }).eql), {
+    postal_code: ['LIKE', '220%']
+  });
+});
+
+test('country narrows the match and drives region normalization', () => {
+  const { text, eql } = addresses.optionsToEQL({ region: 'virginia', country: 'United States' });
+  assert.deepEqual(valuesByColumn(eql), { region: ['EQUALS', 'VA'], country: ['EQUALS', 'US'] });
+  assert.equal(text, 'Has an address in VA in country US');
+});
+
+test('no options means any address', () => {
+  const { text, eql } = addresses.optionsToEQL({});
+  assert.deepEqual(eql.conditions, []);
+  assert.equal(text, 'Has an address');
+});

package/person_email/README.md

@@ -51,9 +51,9 @@ The plugin provides outbound transforms for appending email data to batches:
 - `appendEmail` looks up `person_email` rows by `person_id` and appends `email` to each batch row. It can be filtered by `subscriptionStatus`, and it preserves any existing `email` value already on the row.
 - `appendEmailHash` appends `email_hash_v1` by hashing the row's trimmed, lowercased `email` when the hash is not already present.
 
-## Search
+## Segment definition clauses
 
-The `emails` search lets users find people who have an email address, optionally filtered by:
+The `emails` clause selects people who have an email address, optionally filtered by:
 
 - `subscriptionStatus`, using the same values as `subscription_status`
 - `emailMatch`, matched against the `email` column with `LIKE`
@@ -70,7 +70,7 @@ Predefined audiences shipped with this schema plugin. On install, each key becom
 | **Definition path** | `@engine9/schemas/person_email:segments:subscribers` |
 | **Who is included** | People with at least one email whose `subscription_status` is `Subscribed` |
 | **Who is excluded** | People whose emails are only `Not Subscribed`, `Unsubscribed`, `Bouncing`, or `Spam` |
-| **How it is built** | The `emails` search with `subscriptionStatus: Subscribed` |
+| **How it is built** | The `emails` clause with `subscriptionStatus: Subscribed` |
 | **Universe** | None — membership is current `person_email` rows, not a time window or message input set |
 
 A person with mixed statuses is included if any of their addresses is `Subscribed`.

package/person_email/index.js

@@ -1,7 +1,7 @@
 import schema from './schema.js';
 import extractEmailHashes from './transforms/inbound/extract_identifiers.js';
 import upsertPersonEmail from './transforms/inbound/upsert_tables.js';
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 import appendEmail from './transforms/outbound/appendEmail.js';
 import appendEmailHash from './transforms/outbound/appendEmailHash.js';
 import segments from './segments.js';
@@ -24,12 +24,12 @@ export const transforms = {
 };
 export { metadata };
 export { schema };
-export { search };
+export { segmentDefinitionClauses };
 export { segments };
 export default {
   metadata,
   schema,
-  search,
+  segmentDefinitionClauses,
   segments,
   transforms
 };

package/person_email/{search.js → segmentDefinitionClauses.js}

@@ -56,6 +56,7 @@ export const emails = {
     };
   }
 };
+export const segmentDefinitionClauses = { emails };
 export default {
-  emails
+  segmentDefinitionClauses
 };

package/person_email/segments.js

@@ -1,9 +1,9 @@
 export const subscribers = {
   name: 'Email Subscribers',
-  search: {
+  definition: {
     and: [
       {
-        path: '@engine9/schemas/person_email:search:emails',
+        path: '@engine9/schemas/person_email:segment_definition_clause:emails',
         options: {
           subscriptionStatus: 'Subscribed'
         }

package/person_hash/README.md

@@ -65,6 +65,6 @@ The `upsertPersonHash` transform writes only hash columns to `person_hash_email`
 
 `PersonWorker.search` `emails` / `phones` match `person_email` / `person_phone` when those tables exist, and also hash the query (or accept SHA-256 / MD5 hex) against `person_hash_email` / `person_hash_phone`. If the caller passed emails or phones and neither table exists, the clause matches nobody (`1=0`). Related summaries include `email_hashes` / `phone_hashes` when those tables are present.
 
-The plugin also exports `search.emailHashes` / `search.phoneHashes` for the plugin search tree (`@engine9/schemas/person_hash:search:emailHashes`).
+The plugin exports `segmentDefinitionClauses.emailHashes` and `segmentDefinitionClauses.phoneHashes` for segment definitions (`@engine9/schemas/person_hash:segment_definition_clause:emailHashes`).
 
 Default warehouse export includes the hash tables (export skips them when they do not exist). When utilities/limited-pii `exclude_pii` is set and `tables` is not explicit, `person_name` / `person_email` / `person_phone` / `person_address` are omitted from that default list.

package/person_hash/index.js

@@ -1,5 +1,5 @@
 import schema from './schema.js';
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 import extractContactHashes from './transforms/inbound/extract_identifiers.js';
 import upsertPersonHash from './transforms/inbound/upsert_tables.js';
 import appendEmailHash from './transforms/outbound/appendEmailHash.js';
@@ -28,10 +28,10 @@ export const transforms = {
 
 export { metadata };
 export { schema };
-export { search };
+export { segmentDefinitionClauses };
 export default {
   metadata,
   schema,
   transforms,
-  search
+  segmentDefinitionClauses
 };

package/person_hash/{search.js → segmentDefinitionClauses.js}

@@ -80,7 +80,7 @@ export const phoneHashes = {
   }
 };
 
+export const segmentDefinitionClauses = { emailHashes, phoneHashes };
 export default {
-  emailHashes,
-  phoneHashes
+  segmentDefinitionClauses
 };

package/person_name/README.md

@@ -54,9 +54,9 @@ Primary key: `person_id`.
 
 `@engine9/schemas/person:transforms:appendName` is the same transform, kept so saved export definitions keep working.
 
-## Search
+## Segment definition clauses
 
-`names` matches people by `given_name` and/or `family_name` with `LIKE`. Searching with neither finds everyone who has a name row.
+`names` matches people by `given_name` and/or `family_name` with `LIKE`. With neither, it selects everyone who has a name row.
 
 ## Upgrading accounts created before this plugin
 
@@ -66,4 +66,4 @@ Before this plugin, the names were `person.given_name` and `person.family_name`.
 - `npx e9core upgrade --db <connection>` for a SQLite or MySQL database
 - `npx e9core upgrade --url https://<site>` for a Cloudflare site (it posts to `/api/admin/upgrade` with `E9_ADMIN_API_KEY`)
 
-The upgrade installs this plugin and copies every non-blank name into `person_name`. It then refreshes the `person` plugin's inbound snapshot and drops both columns from `person`. On a PII-free account it does not install this plugin. It drops the columns only when they are empty, or when you pass `discard_names`.
+The upgrade installs this plugin and copies every non-blank name into `person_name`. It then refreshes the `person` plugin's inbound snapshot, so later loads write `person_name` only. The columns stay on `person` with the names as of the upgrade. Dropping them is a separate, opt-in step (`drop_name_columns`), for when nothing reads them. On a PII-free account the upgrade does not install this plugin, and the opt-in drop removes the columns only when they are empty, or when you also pass `discard_names`.

package/person_name/index.js

@@ -1,7 +1,7 @@
 import schema from './schema.js';
 import upsertPersonName from './transforms/inbound/upsert_tables.js';
 import appendName from './transforms/outbound/appendName.js';
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 const metadata = {
   name: '@engine9/schemas/person_name',
   unique: true,
@@ -19,10 +19,10 @@ export const transforms = {
 };
 export { metadata };
 export { schema };
-export { search };
+export { segmentDefinitionClauses };
 export default {
   metadata,
   schema,
-  search,
+  segmentDefinitionClauses,
   transforms
 };

package/person_name/{search.js → segmentDefinitionClauses.js}

@@ -42,6 +42,7 @@ export const names = {
     };
   }
 };
+export const segmentDefinitionClauses = { names };
 export default {
-  names
+  segmentDefinitionClauses
 };

package/person_phone/README.md

@@ -49,9 +49,9 @@ The inbound `upsert` transform writes rows to `person_phone`.
 
 `appendPhoneHash` hashes the row's normalized `phone` into `phone_hash_v1` when the hash is not already present.
 
-## Search
+## Segment definition clauses
 
-The `phones` search lets users find people who have a phone number, optionally filtered by:
+The `phones` clause selects people who have a phone number, optionally filtered by:
 
 - `smsStatus`, using the same values as `sms_status`
 - `callStatus`, using the same values as `call_status`

package/person_phone/index.js

@@ -1,5 +1,5 @@
 import schema from './schema.js';
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 import segments from './segments.js';
 import extractPhoneHashes from './transforms/inbound/extract_identifiers.js';
 import upsertPersonPhone from './transforms/inbound/upsert_tables.js';
@@ -22,12 +22,12 @@ export const transforms = {
 };
 export { metadata };
 export { schema };
-export { search };
+export { segmentDefinitionClauses };
 export { segments };
 export default {
   metadata,
   schema,
   transforms,
-  search,
+  segmentDefinitionClauses,
   segments
 };

package/person_phone/{search.js → segmentDefinitionClauses.js}

@@ -74,6 +74,7 @@ export const phones = {
     };
   }
 };
+export const segmentDefinitionClauses = { phones };
 export default {
-  phones
+  segmentDefinitionClauses
 };

package/person_phone/segments.js

@@ -1,6 +1,6 @@
 export const textable = {
   name: 'Textable People',
-  search: {
+  definition: {
     and: [
       {
         table: 'person_phone',

package/person_remote/index.js

@@ -14,17 +14,17 @@ const metadata = {
     upsert: ['upsertPersonRemote']
   }
 };
-export const search = {
-  all: {
-    title: 'Remote people',
-    name: 'All remote people',
+const all = {
+    title: 'Remote Ids',
+    name: 'All remote ids',
     form: {
-      title: 'Remote people',
+      title: 'Remote Ids',
       type: 'object',
       properties: {
         pluginId: {
-          title: 'Plugin ID',
-          type: 'string'
+          title: 'Plugin',
+          type: 'string',
+          resolveOptionsUsing: 'plugin'
         }
       },
       required: []
@@ -39,8 +39,8 @@ export const search = {
         }
       ]
     })
-  }
-};
+  };
+export const segmentDefinitionClauses = { all };
 export const transforms = {
   extractRemotePersonIds,
   upsertPersonRemote,
@@ -53,6 +53,6 @@ export default {
   metadata,
   schema,
   metrics,
-  search,
+  segmentDefinitionClauses,
   transforms
 };

package/plugin/schema.js

@@ -35,7 +35,7 @@ export const tables = [
       id: 'id_uuid',
       plugin_id: 'foreign_uuid',
       name: 'string',
-      value: 'string',
+      value: 'text',
       created_at: 'created_at',
       modified_at: 'modified_at'
     },

package/segment/index.js

@@ -1,5 +1,5 @@
 import schema from './schema.js';
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 import upsert from './transforms/inbound/upsert.js';
 const metadata = {
   name: '@engine9/schemas/segment',
@@ -12,10 +12,10 @@ export const transforms = {
 };
 export { metadata };
 export { schema };
-export { search };
+export { segmentDefinitionClauses };
 export default {
   metadata,
   schema,
-  search,
+  segmentDefinitionClauses,
   transforms
 };

package/segment/schema.js

@@ -23,15 +23,15 @@ export const tables = [
       category: 'string', // A arbitrary type, user facing, not to be used for logic
       definition_path: 'string', // a definition file path, will be used for configuring
       stars: 'int', //number of stars, good for UI organizing
-      search: 'json',
+      definition: 'json',
       build_type: {
         type: 'enum',
         nullable: false,
         default_value: 'list',
         values: [
           'list', //static list of people
-          'search', // built by the defined search
-          'scheduled_search', // built on a schedule by the search
+          'definition', // built by the segment definition
+          'scheduled_definition', // built on a schedule by the segment definition
           'remote_count', // no person records, just counts, pulled from remote site
           'remote', // pulled from remote plugin site
           'manual' // created outside normal processing

package/segment/{search.js → segmentDefinitionClauses.js}

@@ -6,8 +6,8 @@ export const segment = {
     properties: {
       segmentId: {
         title: 'Segment',
-        description: '',
-        type: 'string'
+        type: 'string',
+        resolveOptionsUsing: 'segment'
       }
     },
     required: []
@@ -59,6 +59,7 @@ export const segment = {
     return { text, eql };
   }
 };
+export const segmentDefinitionClauses = { segment };
 export default {
-  segment
+  segmentDefinitionClauses
 };

package/stacks/limited-pii/stack.json

@@ -8,8 +8,8 @@
     "@engine9/schemas/person",
     "@engine9/schemas/person_remote",
     "@engine9/schemas/segment",
-    "@engine9/schemas/timeline",
     "@engine9/schemas/source_code",
+    "@engine9/schemas/timeline",
     "@engine9/schemas/transaction/core",
     "@engine9/schemas/person_hash"
   ],

package/stacks/stacks.test.js

@@ -12,6 +12,11 @@ test('standard stack includes plaintext contact plugins and does not exclude the
   assert.ok(standard.include.includes('@engine9/schemas/person_address'));
   assert.ok(standard.include.includes('@engine9/schemas/timeline'));
   assert.ok(standard.include.includes('@engine9/schemas/source_code'));
+  assert.ok(
+    standard.include.indexOf('@engine9/schemas/source_code') <
+      standard.include.indexOf('@engine9/schemas/timeline'),
+    'source_code must install before timeline (timeline depends on it)'
+  );
   assert.ok(standard.include.includes('@engine9/schemas/transaction/core'));
   assert.ok(!standard.include.includes('@engine9/plugins/reports/people'));
   assert.deepEqual(standard.exclude, []);

package/stacks/standard/stack.json

@@ -12,8 +12,8 @@
     "@engine9/schemas/person_email",
     "@engine9/schemas/person_phone",
     "@engine9/schemas/person_address",
-    "@engine9/schemas/timeline",
     "@engine9/schemas/source_code",
+    "@engine9/schemas/timeline",
     "@engine9/schemas/transaction/core",
     "@engine9/schemas/transaction/contact_details"
   ],

package/timeline/index.js

@@ -1,9 +1,11 @@
 import schema from './schema.js';
 import upsert from './transforms/inbound/upsert_tables.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 const metadata = {
   name: '@engine9/schemas/timeline',
   dependencies: {
-    '@engine9/schemas/person': '>=1.9.0'
+    '@engine9/schemas/person': '>=1.9.0',
+    '@engine9/schemas/source_code': '>=1.9.0'
   }
 };
 export const transforms = {
@@ -11,8 +13,10 @@ export const transforms = {
 };
 export { metadata };
 export { schema };
+export { segmentDefinitionClauses };
 export default {
   metadata,
   schema,
-  transforms
+  transforms,
+  segmentDefinitionClauses
 };

package/timeline/segmentDefinitionClauses.js

@@ -0,0 +1,56 @@
+function sourceCodeCondition(sourceCode) {
+  const value = String(sourceCode);
+  const type = value.includes('%') ? 'LIKE' : 'EQUALS';
+  return {
+    type,
+    values: [{ ref: { column: 'source_code' } }, { value: { value } }]
+  };
+}
+
+/**
+ * People with at least one timeline entry whose source_code matches.
+ *
+ * Options:
+ * - sourceCode: exact code, or LIKE pattern when it contains %
+ */
+export const sourceCode = {
+  title: 'Timeline source code',
+  description:
+    'People with a timeline entry credited to a source code. Exact match, or a LIKE pattern when the value contains %.',
+  form: {
+    title: 'Timeline source code',
+    type: 'object',
+    properties: {
+      sourceCode: {
+        title: 'Source code',
+        description: 'Exact source code, or a LIKE pattern when it contains %',
+        type: 'string'
+      }
+    },
+    required: ['sourceCode']
+  },
+  optionsToEQL(options = {}) {
+    const { sourceCode: code } = options;
+    if (!code) throw new Error('sourceCode is required');
+    const matchLabel = String(code).includes('%') ? 'matching ' : '';
+    return {
+      text: `Has a timeline entry with source code ${matchLabel}${code}`,
+      eql: {
+        table: 'timeline',
+        columns: ['person_id'],
+        joins: [
+          {
+            table: 'source_code_dictionary',
+            join_eql: 'timeline.source_code_id=source_code_dictionary.source_code_id'
+          }
+        ],
+        conditions: [sourceCodeCondition(code)]
+      }
+    };
+  }
+};
+
+export const segmentDefinitionClauses = { sourceCode };
+export default {
+  segmentDefinitionClauses
+};

package/timeline/segmentDefinitionClauses.test.js

@@ -0,0 +1,37 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { sourceCode, segmentDefinitionClauses } from './segmentDefinitionClauses.js';
+
+test('timeline sourceCode is the only segment definition clause', () => {
+  assert.deepEqual(Object.keys(segmentDefinitionClauses), ['sourceCode']);
+});
+
+test('timeline sourceCode requires sourceCode', () => {
+  assert.throws(() => sourceCode.optionsToEQL({}), /sourceCode/);
+});
+
+test('timeline sourceCode exact match joins the dictionary', () => {
+  const { text, eql } = sourceCode.optionsToEQL({ sourceCode: 'AU_TEST' });
+  assert.equal(text, 'Has a timeline entry with source code AU_TEST');
+  assert.equal(eql.table, 'timeline');
+  assert.deepEqual(eql.columns, ['person_id']);
+  assert.deepEqual(eql.joins, [
+    {
+      table: 'source_code_dictionary',
+      join_eql: 'timeline.source_code_id=source_code_dictionary.source_code_id'
+    }
+  ]);
+  assert.deepEqual(eql.conditions, [
+    {
+      type: 'EQUALS',
+      values: [{ ref: { column: 'source_code' } }, { value: { value: 'AU_TEST' } }]
+    }
+  ]);
+});
+
+test('timeline sourceCode uses LIKE when the value contains %', () => {
+  const { text, eql } = sourceCode.optionsToEQL({ sourceCode: 'AUTH_%' });
+  assert.equal(text, 'Has a timeline entry with source code matching AUTH_%');
+  assert.equal(eql.conditions[0].type, 'LIKE');
+  assert.equal(eql.conditions[0].values[1].value.value, 'AUTH_%');
+});

package/transaction/core/README.md

@@ -1,6 +1,6 @@
 # Transaction Core Schema Plugin
 
-The `@engine9/schemas/transaction/core` plugin stores payment and gift transactions for people and provides the common transaction search, metrics, and segment used by imports, reports, and exports.
+The `@engine9/schemas/transaction/core` plugin stores payment and gift transactions for people and provides the common transaction segment definition clauses, metrics, and segment used by imports, reports, and exports.
 
 It depends on `@engine9/schemas/person` because every transaction belongs to a person through `person_id`.
 
@@ -53,10 +53,9 @@ The inbound `upsert` transform writes rows to `transaction`.
 
 `appendTransactionSummary` looks up a person's transactions and attaches summary fields such as count, revenue, first/last/smallest/largest gifts, and recurring totals.
 
-## Search
+## Segment definition clauses
 
-- `all` finds people who have any transaction, optionally filtered by `plugin_id` and a `start` / `end` window on `ts` (ISO datetimes or relative expressions such as `-30d`).
-- `minimum` finds people who have a transaction of at least a given `amount`.
+- `all` selects people who have any transaction, optionally filtered by `plugin_id`, a `start` / `end` window on `ts` (ISO datetimes or relative expressions such as `-30d`), and `minAmount` / `maxAmount` (inclusive) on `amount`. Every filter applies to the same transaction, so `{ start: '2026-01-01', minAmount: 100 }` is one gift of at least 100 this year.
 
 ## Metrics
 
@@ -74,7 +73,7 @@ Predefined audiences shipped with this schema plugin. On install, each key becom
 | **Definition path** | `@engine9/schemas/transaction/core:segments:customers` |
 | **Who is included** | People who have at least one row in `transaction` |
 | **Who is excluded** | People with no transactions |
-| **How it is built** | A `transaction` table search that selects `person_id` with no extra filters |
+| **How it is built** | The `all` clause, which selects `person_id` from `transaction` with no extra filters |
 | **Universe** | None — any transaction from any plugin, date, or amount counts |
 
 Refunds, recurring gifts, and $0 rows are not filtered out. If a person has ever appeared on a transaction row, they are a customer.

package/transaction/core/index.js

@@ -1,5 +1,5 @@
 import schema from './schema.js';
-import search from './search.js';
+import { segmentDefinitionClauses } from './segmentDefinitionClauses.js';
 import metrics from './metrics.js';
 import segments from './segments.js';
 import upsert from './transforms/inbound/upsert_tables.js';
@@ -19,13 +19,13 @@ export const transforms = {
 export { metadata };
 export { schema };
 export { metrics };
-export { search };
+export { segmentDefinitionClauses };
 export { segments };
 export default {
   metadata,
   schema,
   metrics,
   transforms,
-  search,
+  segmentDefinitionClauses,
   segments
 };

package/transaction/core/{search.js → segmentDefinitionClauses.js}

@@ -1,5 +1,15 @@
 import { relativeDate, isValidDate } from '@engine9/input-tools';
 
+function blank(value) {
+  return value == null || String(value).trim() === '';
+}
+
+function amountOption(name, value) {
+  const n = Number(value);
+  if (!Number.isFinite(n)) throw new Error(`Invalid ${name}: ${value}`);
+  return n;
+}
+
 export const all = {
   title: 'Transactions',
   form: {
@@ -7,7 +17,9 @@ export const all = {
     type: 'object',
     properties: {
       plugin_id: {
-        type: 'string'
+        title: 'Plugin',
+        type: 'string',
+        resolveOptionsUsing: 'plugin'
       },
       start: {
         title: 'Start',
@@ -20,12 +32,22 @@ export const all = {
         type: 'string',
         description:
           'Optional; only include rows with ts < end (exclusive upper bound). Parsed with relativeDate; supports ISO datetimes or relative expressions.'
+      },
+      minAmount: {
+        title: 'Minimum amount',
+        type: 'number',
+        description: 'Optional; only include rows with amount >= minAmount.'
+      },
+      maxAmount: {
+        title: 'Maximum amount',
+        type: 'number',
+        description: 'Optional; only include rows with amount <= maxAmount.'
       }
     },
     required: []
   },
   optionsToEQL: (options) => {
-    const { plugin_id, start, end } = options;
+    const { plugin_id, start, end, minAmount, maxAmount } = options;
     const conditions = [];
     let text = 'Has any transactions';
     if (plugin_id) {
@@ -44,6 +66,16 @@ export const all = {
       conditions.push(`transaction.ts<'${e.toISOString()}'`);
       text += ` with ts < ${e.toISOString()}`;
     }
+    if (!blank(minAmount)) {
+      const min = amountOption('minAmount', minAmount);
+      conditions.push(`transaction.amount>=${min}`);
+      text += ` with amount >= ${min}`;
+    }
+    if (!blank(maxAmount)) {
+      const max = amountOption('maxAmount', maxAmount);
+      conditions.push(`transaction.amount<=${max}`);
+      text += ` with amount <= ${max}`;
+    }
     return {
       text,
       eql: {
@@ -60,28 +92,7 @@ export const all = {
     };
   }
 };
-export const minimum = {
-  title: 'Minimum transaction amount',
-  form: {
-    title: 'Minimum transaction amount',
-    type: 'object',
-    properties: {
-      amount: {
-        type: 'number'
-      },
-      filterType: {
-        type: 'string'
-      }
-    },
-    required: []
-  },
-  optionsToEQL: (options) => ({
-    table: 'transaction',
-    columns: ['person_id'],
-    conditions: [`amount>=${parseFloat(options.amount)}`]
-  })
-};
+export const segmentDefinitionClauses = { all };
 export default {
-  all,
-  minimum
+  segmentDefinitionClauses
 };

package/transaction/core/segmentDefinitionClauses.test.js

@@ -0,0 +1,23 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { all, segmentDefinitionClauses } from './segmentDefinitionClauses.js';
+
+test('all is the only transaction clause', () => {
+  assert.deepEqual(Object.keys(segmentDefinitionClauses), ['all']);
+});
+
+test('amount bounds apply to the same transaction as the date window', () => {
+  const { text, eql } = all.optionsToEQL({ start: '2026-01-01', minAmount: 100, maxAmount: '500' });
+  assert.deepEqual(eql.conditions, [
+    "transaction.ts>='2026-01-01T00:00:00.000Z'",
+    'transaction.amount>=100',
+    'transaction.amount<=500'
+  ]);
+  assert.equal(text, 'Has any transactions with ts >= 2026-01-01T00:00:00.000Z with amount >= 100 with amount <= 500');
+});
+
+test('blank amounts are ignored and invalid amounts throw', () => {
+  assert.deepEqual(all.optionsToEQL({ minAmount: '', maxAmount: null }).eql.conditions, []);
+  assert.throws(() => all.optionsToEQL({ minAmount: 'abc' }), /Invalid minAmount: abc/);
+  assert.throws(() => all.optionsToEQL({ maxAmount: '1; drop table person' }), /Invalid maxAmount/);
+});

package/transaction/core/segments.js

@@ -1,6 +1,6 @@
 export const customers = {
   name: 'Customers',
-  search: {
+  definition: {
     and: [
       {
         table: 'transaction',