@dforge-core/dforge-mcp 0.1.11 → 0.1.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,62 @@ All notable changes to `@dforge-core/dforge-mcp`. This project uses semver-ish
4
4
  `0.1.0-rc.N` pre-release tags; the published version is set at publish time via
5
5
  the release workflow, so committed `package.json` versions are placeholders.
6
6
 
7
+ ## 0.1.12
8
+
9
+ > Release prerequisite: publish `@dforge-core/metadata@0.0.10` first (adds the
10
+ > `domains.schema.json` schema + the domain-backed column branch in
11
+ > `entity.schema.json`), then `pnpm install` + `sync-schemas` here so the
12
+ > vendored schemas match.
13
+
14
+ ### Skill
15
+
16
+ - **Column domains** — new `references/column-domains.md`: a reusable field type
17
+ (base datatype + control + shared option list) declared in `domains.json` and
18
+ referenced from a column via a single `"domain": "module_cd.domain_cd"` key
19
+ instead of restating the type. Covers the owned-field rules (restating a
20
+ domain-owned field fails install), scalar-only constraint, cross-module
21
+ dependency requirement, and the localize-once payoff. Added to the SKILL
22
+ loading-policy table and cross-linked from `field-types.md`.
23
+ - **Document extraction / OCR** — new `references/document-extraction.md`: the
24
+ `ocrExtract()` forms (v1 raw string, v2 inline `schema` + `{ mode: 'extract' }`,
25
+ v2 `{ profile }`), the `logic/extraction_profiles.json` registry (profile
26
+ lifecycle + `detect` rules), and the `detectDocument(rawText)` doc-type
27
+ auto-detect built-in. The DSL reference (`dforge://docs/dsl`) gains an
28
+ "External integration → Document extraction" section documenting both built-ins.
29
+ - **Dropdown / domain option localization** (`translations.md`): per-option labels
30
+ are translatable (opt-in) under `entities.<e>.fields.<f>.options` (value-keyed,
31
+ string shorthand or `{label,icon,color}` partial override — `value` never
32
+ translated), and a column domain's label + shared option labels are translated
33
+ once under a top-level `domains` section, inherited by every consuming column.
34
+ - **Formulas** (`formulas.md`) — corrected against the real parser and runtime,
35
+ after a reported `F` column that rendered empty on every row with no error
36
+ anywhere. The doc had documented functions the engine never implemented
37
+ (`ISNULL`, `NULLIF`, `MOD`, `LTRIM`, `RTRIM`) — since the parser accepts any
38
+ `NAME(...)` as a call, those installed cleanly and then rendered blank. They
39
+ are implemented now, and an unknown function is rejected at module install.
40
+ Also fixed: `==`, `%` and `"double-quoted strings"` are **parse errors**
41
+ (four examples used them); `CONTAINS`/`STARTS_WITH`/`ENDS_WITH` are infix
42
+ operators, not functions; and the documented `CASE` signature was actually
43
+ `SWITCH`'s — `CASE([priority], 'high', 3, …)` returns `'high'` for every row,
44
+ because the real `CASE` takes condition/result pairs. Adds the previously
45
+ undocumented `LEFT`/`RIGHT`/`FROUND`/`MID`/`INDEX_OF`/`SPLIT` and the
46
+ conversion functions, so the list now matches the engine exactly, plus notes
47
+ on date comparison, 0-based string indexing, and out-of-domain math. Every
48
+ function is now implemented client-side *and* SQL-side, so a column reads the
49
+ same in a grid and in a report.
50
+
51
+ ### Resources
52
+
53
+ - New `dforge://schema/domains` (JSON Schema for `domains.json`).
54
+ - New `dforge://reference/column-domains` and `dforge://reference/document-extraction`.
55
+
56
+ ### Schema
57
+
58
+ - Re-vendored `entity.schema.json` — adds the "Domain-backed column" branch
59
+ (`required: ["domain"]`, forbidding restatement of domain-owned fields) and the
60
+ `domain` field property.
61
+ - Vendored `domains.schema.json` (new) from `@dforge-core/metadata@0.0.10`.
62
+
7
63
  ## 0.1.11
8
64
 
9
65
  > Release prerequisite: publish `@dforge-core/metadata@0.0.9` first (adds the
package/dist/server.js CHANGED
@@ -34366,6 +34366,11 @@ var resources = [
34366
34366
  "Entity JSON schema",
34367
34367
  "JSON Schema for entity files under entities/*.json (description, dbObject, toString, traits, fields)."
34368
34368
  ),
34369
+ schema(
34370
+ "domains",
34371
+ "Column domains JSON schema",
34372
+ 'JSON Schema for domains.json \u2014 reusable column domains (base datatype + control + sizing + shared option list) referenced from a field via "domain": "module_cd.domain_cd".'
34373
+ ),
34369
34374
  schema(
34370
34375
  "data-views",
34371
34376
  "Data view JSON schema",
@@ -34444,6 +34449,7 @@ var resources = [
34444
34449
  reference("field-types", "Field types \u2014 fieldTypeCd (UI control) vs dbDatatype (SQL type), and the correct value for each. Load before adding any field."),
34445
34450
  reference("flags", "Column flags \u2014 only V/I/E/M/H; valid combos (VEM/VE/V/EM/I) and why 'VEMHI' is invalid. Load before setting any flags."),
34446
34451
  reference("column-types", "The FK + Reference two-column pattern (the #1 source of broken modules) and Set columns. Load before any relation."),
34452
+ reference("column-domains", "Column domains: a reusable field type (datatype + control + shared option list) referenced via a single `domain` key. Use when the same enum/type repeats across columns. Load before authoring domains.json or a domain-backed column."),
34447
34453
  reference("formulas", "Formula columns (columnType 'F'): baseDatatypeCd, no dbDatatype, flags 'V', and the formula expression grammar."),
34448
34454
  reference("traits", "Entity traits (identity, audit, audit-full, ...). identity \u2192 PK is '{entity}_id'; don't redefine trait columns."),
34449
34455
  reference("data-views", "Data views: dataSources array, columns, the order string-array ('order': ['-col','col']), and specialized view configs."),
@@ -34455,6 +34461,7 @@ var resources = [
34455
34461
  reference("settings", "Module settings shape."),
34456
34462
  reference("jobs", "Scheduled jobs: cron, timeout, jobClass, and the no-record-context constraint."),
34457
34463
  reference("number-sequences", "Number sequences for reference numbers / codes."),
34464
+ reference("document-extraction", "OCR / document extraction: ocrExtract() (v2 schema-driven + profile forms), logic/extraction_profiles.json, and the detectDocument() doc-type auto-detect built-in. Load before wiring any OCR/intake action."),
34458
34465
  reference("print-templates", "Liquid HTML print templates."),
34459
34466
  reference("translations", "Translation files: locale-keyed, every trait-provided field needs an entry."),
34460
34467
  reference("queries", "Pre-built saved queries."),
package/package.json CHANGED
@@ -1,45 +1,45 @@
1
1
  {
2
- "name": "@dforge-core/dforge-mcp",
3
- "version": "0.1.11",
4
- "description": "MCP server for dForge module authoring. Exposes scaffold/pack/install tools and schema resources so AI agents (Claude Code, Cursor, Zed) can create and ship dForge modules.",
5
- "license": "MIT",
6
- "homepage": "https://github.com/iash44/dForge-core",
7
- "repository": {
8
- "type": "git",
9
- "url": "git+https://github.com/dforge-core/dforge-mcp.git"
10
- },
11
- "bin": {
12
- "dforge-mcp": "dist/server.js"
13
- },
14
- "files": [
15
- "dist/",
16
- "docs/",
17
- "resources/",
18
- "skills/",
19
- "README.md",
20
- "CHANGELOG.md"
21
- ],
22
- "main": "dist/server.js",
23
- "engines": {
24
- "node": ">=18"
25
- },
26
- "scripts": {
27
- "build": "tsup",
28
- "sync-schemas": "node scripts/vendor-schemas.cjs",
29
- "prepublishOnly": "node scripts/vendor-schemas.cjs && tsup",
30
- "typecheck": "tsc --noEmit",
31
- "test": "vitest run"
32
- },
33
- "dependencies": {
34
- "@dforge-core/dforge-cli": "^0.2.7",
35
- "@dforge-core/metadata": "^0.0.9",
36
- "@modelcontextprotocol/sdk": "^1.29.0",
37
- "zod": "^4.4.3"
38
- },
39
- "devDependencies": {
40
- "@types/node": "^22.20.1",
41
- "tsup": "^8.5.1",
42
- "typescript": "^6.0.3",
43
- "vitest": "^3.2.7"
44
- }
2
+ "name": "@dforge-core/dforge-mcp",
3
+ "version": "0.1.13",
4
+ "description": "MCP server for dForge module authoring. Exposes scaffold/pack/install tools and schema resources so AI agents (Claude Code, Cursor, Zed) can create and ship dForge modules.",
5
+ "license": "MIT",
6
+ "homepage": "https://github.com/iash44/dForge-core",
7
+ "repository": {
8
+ "type": "git",
9
+ "url": "git+https://github.com/dforge-core/dforge-mcp.git"
10
+ },
11
+ "bin": {
12
+ "dforge-mcp": "dist/server.js"
13
+ },
14
+ "files": [
15
+ "dist/",
16
+ "docs/",
17
+ "resources/",
18
+ "skills/",
19
+ "README.md",
20
+ "CHANGELOG.md"
21
+ ],
22
+ "main": "dist/server.js",
23
+ "engines": {
24
+ "node": ">=18"
25
+ },
26
+ "scripts": {
27
+ "build": "tsup",
28
+ "sync-schemas": "node scripts/vendor-schemas.cjs",
29
+ "prepublishOnly": "node scripts/vendor-schemas.cjs && tsup",
30
+ "typecheck": "tsc --noEmit",
31
+ "test": "vitest run"
32
+ },
33
+ "dependencies": {
34
+ "@dforge-core/dforge-cli": "^0.2.7",
35
+ "@dforge-core/metadata": "^0.0.10",
36
+ "@modelcontextprotocol/sdk": "^1.29.0",
37
+ "zod": "^4.4.3"
38
+ },
39
+ "devDependencies": {
40
+ "@types/node": "^22.20.1",
41
+ "tsup": "^8.5.1",
42
+ "typescript": "^6.0.3",
43
+ "vitest": "^3.2.7"
44
+ }
45
45
  }
@@ -417,6 +417,32 @@ var result = callService('studio.validate', { module_cd: [module_cd] })
417
417
  if (!result.isValid) { error('Validation failed: ' + result.errors[0]) }
418
418
  ```
419
419
 
420
+ ### Document extraction (OCR)
421
+
422
+ Full guide + intake patterns: `dforge://reference/document-extraction`.
423
+
424
+ #### `ocrExtract(fileField, endpointBaseUrl, schemaOrHints?, opts?) → string | object`
425
+ Send a file to the OCR service and get structured fields back. Three forms:
426
+
427
+ - **v1 (raw):** `ocrExtract([doc_file], getSetting('ocr_endpoint_url'))` → JSON **string** (you `JSON.parse`). Optional third arg = hints (array | string | object).
428
+ - **v2 inline schema:** `ocrExtract([doc_file], url, schema, { mode: 'extract' })` → **parsed** `{ fields, confidence?, unmatched, raw_text, meta }`.
429
+ - **v2 profile:** `ocrExtract([doc_file], url, null, { profile: 'module_cd.profile_cd' })` → same parsed shape, schema from `logic/extraction_profiles.json`.
430
+
431
+ v2 is selected **only** by `opts.mode` (`'extract' | 'summary' | 'raw'`) or `opts.profile` — a bare schema object without `opts.mode` stays on the v1 (raw string) contract. `opts.profile` implies `mode: 'extract'` and is mutually exclusive with an inline `schema`.
432
+
433
+ ```dsl
434
+ var r = ocrExtract([doc_file], getSetting('ocr_endpoint_url'), null, { profile: 'proc.bill_of_lading' })
435
+ insert('bill_of_lading', { bl_no: r.fields.bl_no, issue_date: r.fields.issue_date })
436
+ ```
437
+
438
+ #### `detectDocument(rawText) → object | null`
439
+ Score a document's text against every reachable extraction profile's `detect` rules and return the best match at/above its threshold: `{ profile, docType, score }` (profile module-qualified, ready for `ocrExtract(..., { profile })`) — or `null`. Advisory: malformed rules are skipped, let the human confirm.
440
+
441
+ ```dsl
442
+ var det = detectDocument(data.raw_text) // { profile: 'proc.bill_of_lading', docType: 'bill_of_lading', score } | null
443
+ if (det != null) { var r = ocrExtract([doc_file], getSetting('ocr_endpoint_url'), null, { profile: det.profile }) }
444
+ ```
445
+
420
446
  ### Utility
421
447
 
422
448
  #### `entityLink(entityCd, record, description?) → link value`
@@ -64,6 +64,7 @@ The phase column below indicates the **typical** use. During a backtrack, the ba
64
64
  |---|---|---|
65
65
  | Add any field | `dforge://reference/field-types`, `dforge://reference/flags`, `dforge://example/entities/todo_item.json` | — |
66
66
  | Add a Reference or Set column | `dforge://reference/column-types` (FK+Reference pattern), `dforge://example/entities/todo_item.json` | — |
67
+ | Reuse a field type / enum across columns (a `domain`) | `dforge://reference/column-domains` | `dforge://schema/domains` |
67
68
  | Add a formula column | `dforge://reference/formulas` | — |
68
69
  | Add a trait | `dforge://reference/traits` | — |
69
70
  | Add a data view (grid / list) | `dforge://reference/data-views`, `dforge://example/ui/data_views.json` | — |
@@ -78,6 +79,8 @@ The phase column below indicates the **typical** use. During a backtrack, the ba
78
79
  | Add security roles or folders | `dforge://reference/security`, `dforge://reference/filters`, `dforge://example/security/roles.json` | — |
79
80
  | Add a report | `dforge://reference/reports` | `dforge://schema/reports` |
80
81
  | Add a print template | `dforge://reference/print-templates` | — |
82
+ | Wire OCR / document extraction (`ocrExtract` / `detectDocument`) | `dforge://reference/document-extraction` | `dforge://docs/dsl` (built-ins) |
83
+ | Localize dropdown options / a column domain | `dforge://reference/translations` (§Dropdown option labels / §Column domain labels) | — |
81
84
  | Add translations | `dforge://reference/translations` | — |
82
85
  | Add a number sequence | `dforge://reference/number-sequences` | — |
83
86
  | Add module settings | `dforge://reference/settings` | — |
@@ -0,0 +1,94 @@
1
+ # Column Domains Reference
2
+
3
+ A **column domain** is a reusable, named field type — a base datatype, a control, sizing, and a shared option list under one code. Columns reference it instead of restating the definition. Use one whenever the same enum or typed column repeats across entities (a `Draft/Posted/Reversed` status, a currency code, a priority, an account type).
4
+
5
+ Lives in: `domains.json` at the module root (a flat map `domain_cd → definition`).
6
+
7
+ Schema: `dforge://schema/domains`.
8
+
9
+ ## Why
10
+
11
+ Without domains, every "status" column restates the same `fieldTypeCd: "dropdown"` + the full `params.options` list — and, if you localize, the same option translations, once per column. A domain declares that once; every consuming column inherits the type, the control, the options, **and** their translations. Localize `fin.doc_status` a single time and all its uses update together.
12
+
13
+ ## `domains.json`
14
+
15
+ ```json
16
+ {
17
+ "doc_status": {
18
+ "description": "Document Status",
19
+ "baseDatatypeCd": "string",
20
+ "dbDatatype": "varchar",
21
+ "maxLen": 20,
22
+ "fieldTypeCd": "dropdown",
23
+ "params": {
24
+ "options": [
25
+ { "value": "draft", "label": "Draft" },
26
+ { "value": "posted", "label": "Posted", "color": "#0a0" }
27
+ ]
28
+ }
29
+ }
30
+ }
31
+ ```
32
+
33
+ | Key | Required | Notes |
34
+ |---|---|---|
35
+ | `baseDatatypeCd` | **yes** | What a consuming column's DDL is generated from (`string`, `number`, `date`, …). |
36
+ | `description` | no | Label. Localizable — see below. |
37
+ | `dbDatatype` | no | Physical Postgres type; omit to infer from `fieldTypeCd`. |
38
+ | `fieldTypeCd` | no | Control (`dropdown`, `text`, `number`, …). |
39
+ | `maxLen` / `precision` | no | Sizing for string / numeric domains. |
40
+ | `params` | no | Shared field-type config — most importantly `options` for `dropdown`/`radio`/`flags`. |
41
+
42
+ ## Using a domain in an entity
43
+
44
+ A column swaps its whole type block for a single `domain` key — qualified as `module_cd.domain_cd`, or bare for a domain in the current module (unqualified resolves the current module first, then tenant-created domains):
45
+
46
+ ```json
47
+ "status": {
48
+ "domain": "fin.doc_status",
49
+ "flags": "VEM",
50
+ "orderNum": 40,
51
+ "description": "Status"
52
+ }
53
+ ```
54
+
55
+ The column keeps what is genuinely its own — `description` (its label), `flags`, `orderNum`, `isNullable`, `isPk`, `columnGroupCd`. The domain supplies the rest.
56
+
57
+ ## Rules (these fail the install)
58
+
59
+ 1. **Don't restate what the domain owns.** Declaring `dbDatatype`, `fieldTypeCd`, `maxLen`, `precision`, `baseDatatypeCd` or `params` alongside `domain` is rejected, naming the conflicting key — it is never silently overridden. Domains are authoritative.
60
+ 2. **Scalar columns only.** A reference (`R`) or set (`S`) column can't use a domain — it has no datatype for the domain to supply. Drop the `domain` key on those.
61
+ 3. **Declare the dependency for cross-module use.** `"domain": "fin.doc_status"` resolves by looking the domain up in the installed tenant, so declare `fin` in the manifest `dependencies` — dependencies drive install order, and without it nothing guarantees `fin` is installed first, so the resolve fails with "references unknown column domain".
62
+
63
+ ## Localizing a domain (and its options)
64
+
65
+ Translate a domain **once** in `translations/<locale>.json` under a top-level `domains` section — its label and its per-option labels flow to every consuming column:
66
+
67
+ ```json
68
+ "domains": {
69
+ "doc_status": {
70
+ "label": "Belegstatus",
71
+ "options": { "draft": "Entwurf", "posted": { "label": "Gebucht", "color": "#0a0" } }
72
+ }
73
+ }
74
+ ```
75
+
76
+ An option override is a partial merged over the base — a bare string is shorthand for `{ "label": … }`; `value` is the stored code and is never translated. See `translations.md` ("Dropdown option localization").
77
+
78
+ ## How it resolves (mental model)
79
+
80
+ - **Install** materializes the domain's structural fields (datatype/control/sizing) onto each consuming column, because DDL generation needs them locally.
81
+ - **Runtime** resolves `params` (the options) — and their per-culture overrides — from the domain, not the column. That is what lets a shared list be authored and translated once.
82
+
83
+ You don't act on this split; just know that the option list stays on the domain, so **don't** paste `params.options` onto a domain-backed column (the install rejects it).
84
+
85
+ ## When NOT to use a domain
86
+
87
+ - A one-off column whose type/options appear nowhere else — inline it (`fieldTypeCd` + `params.options`) and skip the indirection.
88
+ - Anything non-scalar (references, sets, formulas).
89
+
90
+ ## Common mistakes
91
+
92
+ - Putting `params.options` on the column **and** naming a `domain` → install error. Options belong on the domain.
93
+ - Using a cross-module domain without declaring the dependency → "unknown column domain" at install.
94
+ - Expecting per-column option overrides — the domain is authoritative in v1; narrow with `optionSets` (which filters by `value`) if you need a conditional subset.
@@ -0,0 +1,113 @@
1
+ # Document Extraction Reference (OCR)
2
+
3
+ dForge turns uploaded documents (PDFs, scans, photos) into structured records via a generic OCR service plus a per-module **field schema**. The load-bearing principle: **OCR is schema-agnostic, the module owns the schema.** The platform service is a write-once "document + schema → fields" engine; every new document type is a new schema in *your* module, never a change to the service.
4
+
5
+ Two DSL built-ins do the work, both host functions callable from `logic/actions/*.dsl`:
6
+
7
+ - `ocrExtract(...)` — send a file to the OCR endpoint, get structured fields back.
8
+ - `detectDocument(rawText)` — score a document's text against your profiles' detect rules to guess its type.
9
+
10
+ The OCR endpoint URL is a module setting (commonly `ocr_endpoint_url`), read with `getSetting('ocr_endpoint_url')`.
11
+
12
+ ## `ocrExtract()`
13
+
14
+ Three forms, from raw to managed:
15
+
16
+ **v1 — raw bundle** (back-compat; returns a JSON **string**, you `JSON.parse` it):
17
+ ```
18
+ var raw = ocrExtract([doc_file], getSetting('ocr_endpoint_url')) // hints optional (array | string | object)
19
+ ```
20
+
21
+ **v2 — inline schema** (explicit opt-in via `opts.mode`; returns the **parsed** object):
22
+ ```
23
+ var r = ocrExtract([doc_file], getSetting('ocr_endpoint_url'), schema, { mode: 'extract' })
24
+ // r → { fields, confidence?, unmatched, raw_text, meta }
25
+ ```
26
+
27
+ **v2 — extraction profile** (the managed path; schema lives in a row, not inline):
28
+ ```
29
+ var r = ocrExtract([doc_file], getSetting('ocr_endpoint_url'), null, { profile: 'proc.bill_of_lading' })
30
+ ```
31
+
32
+ Notes:
33
+ - v2 is selected **only** by `opts.mode` (`'extract' | 'summary' | 'raw'`) or `opts.profile`. A bare object third arg *without* `opts.mode` stays on the v1 hints contract — so passing a schema object alone does **not** switch to v2.
34
+ - `opts.profile` implies `mode: 'extract'` and is mutually exclusive with an inline `schema`. Unqualified profile codes resolve the action's own module first, then tenant-created rows. Missing/inactive profiles fail loudly.
35
+ - The parsed response is `{ fields, confidence?, unmatched, raw_text, meta }` — map `r.fields.<name>` straight into `insert()`.
36
+
37
+ ## The `schema:` block (inline)
38
+
39
+ Declare the field schema next to the mapping, in a `schema:` block in the action `.dsl`. It compiles verbatim into `var schema = {…}`, so it's a **JS object-literal body** — entries comma-separated, **string values quoted** (`type: 'string'`, not `type: string`, which fails the compile with `'string' is not defined`):
40
+
41
+ ```
42
+ schema:
43
+ bl_no: { type: 'string', description: "Bill of lading number (label 'B.L. No.')" },
44
+ container_no: { type: 'string', description: 'ISO 6346 container number' },
45
+ issue_date: { type: 'date' }
46
+
47
+ execute:
48
+ var r = ocrExtract([doc_file], getSetting('ocr_endpoint_url'), schema, { mode: 'extract' })
49
+ var bl = insert('bill_of_lading', { bl_no: r.fields.bl_no, issue_date: r.fields.issue_date })
50
+ if (r.fields.container_no != null) {
51
+ insert('container', { bill_of_lading_id: bl.bill_of_lading_id, container_no: r.fields.container_no })
52
+ }
53
+ ```
54
+
55
+ Each field descriptor carries a `type` (`string` | `date` | `number` | …) and an optional `description` — the human-language hint the recognizer/LLM uses to find it. Keep descriptions concrete ("label 'B.L. No.'", "ISO 6346 container number").
56
+
57
+ ## Extraction profiles — `logic/extraction_profiles.json`
58
+
59
+ The managed evolution of the inline `schema:` block: the same field schema stored as a row in `dForge.extraction_profile` (a platform-managed entity), so it's editable in the admin UI and reusable across actions. Mirrors the print-template lifecycle.
60
+
61
+ ```json
62
+ {
63
+ "bill_of_lading": {
64
+ "description": "Ocean bill of lading",
65
+ "docType": "bill_of_lading",
66
+ "schema": {
67
+ "bl_no": { "type": "string", "description": "Bill of lading number (label 'B.L. No.')" },
68
+ "container_no": { "type": "string", "description": "ISO 6346 container number" },
69
+ "issue_date": { "type": "date" }
70
+ },
71
+ "detect": {
72
+ "signals": [
73
+ { "regex": "bill of lading", "weight": 3 },
74
+ { "regex": "B/?L\\s*No", "weight": 2 }
75
+ ],
76
+ "threshold": 4
77
+ }
78
+ }
79
+ }
80
+ ```
81
+
82
+ - **Keyed by `profile_cd`** (no `.` — it collides with the `module.profile` qualified form).
83
+ - **Registered on install** by `(module_id, profile_cd)`; profiles dropped from the manifest are **reaped** on upgrade; uninstall removes the module's rows. Tenant-created profiles (`module_id IS NULL`) are authored in the admin UI and never touched by install.
84
+ - Reference at runtime with `ocrExtract(..., null, { profile: 'module_cd.profile_cd' })`.
85
+
86
+ ## `detectDocument()` — doc-type auto-detect
87
+
88
+ A profile's optional `detect` rules let the platform guess a document's type before you extract. `detectDocument(rawText)` scores every reachable profile's `signals` (case-insensitive regexes, summed weights, per-pattern timeout) and returns the best profile at/above its `threshold`:
89
+
90
+ ```
91
+ execute:
92
+ var det = detectDocument(data.raw_text) // → { profile: 'proc.bill_of_lading', docType: 'bill_of_lading', score } | null
93
+ if (det != null && det.docType != null) { [doc_type] = det.docType }
94
+ var r = ocrExtract([doc_file], getSetting('ocr_endpoint_url'), null, { profile: det.profile })
95
+ ```
96
+
97
+ - Returns `{ profile, docType, score }` (the profile code is already module-qualified, ready to pass to `ocrExtract`) or `null`.
98
+ - **Advisory by design** — malformed rules are skipped, and the returned type is a suggestion; let the human confirm before committing an irreversible mapping.
99
+ - `rawText` is typically `ocrExtract`'s `raw_text` (or a v1 raw bundle's text) from a first pass.
100
+
101
+ ## Typical intake flow
102
+
103
+ 1. User uploads a file to an intake record (a `file`/`image` column).
104
+ 2. An action runs OCR once to get `raw_text`, calls `detectDocument(raw_text)` to suggest the type, and stores the suggestion for the user to confirm.
105
+ 3. On confirm, `ocrExtract(..., { profile })` returns structured `fields`; the action `insert()`s the target record(s), applying resolution transforms (`lookupRef`, `matchCatalog`, `parseMoney`) for anything that needs turning into a DB ref.
106
+
107
+ ## Common mistakes
108
+
109
+ - Passing a schema object as the third arg **without** `{ mode: 'extract' }` — that's still v1 (raw string), not v2.
110
+ - Unquoted values in a `schema:` block (`type: string`) — compile fails; quote them.
111
+ - A `profile_cd` containing `.` — rejected (reserved for the `module.profile` form).
112
+ - Treating `detectDocument` as authoritative — it's advisory; confirm the type before an irreversible write.
113
+ - Forgetting the OCR endpoint is a **setting** — read it with `getSetting(...)`, don't hardcode.
@@ -10,6 +10,8 @@ Source of truth: `server/database/system-modules/metadata/seed-data/field_types.
10
10
  >
11
11
  > Reference (`columnType: "R"`) and formula (`columnType: "F"`) columns never get a `dbDatatype` at all.
12
12
 
13
+ > **Repeating the same type across columns?** If several columns share a datatype + control + option list (a `Draft/Posted/…` status, a currency, a priority), define a **column domain** once and reference it with a single `domain` key instead of restating `fieldTypeCd` + `params.options` each time — the options (and their translations) are then authored once for all consumers. See `dforge://reference/column-domains`.
14
+
13
15
  ## String-backed
14
16
 
15
17
  | `fieldTypeCd` | `baseDatatypeCd` | Description | Common `dbDatatype` | Params |
@@ -18,11 +18,14 @@ The same grammar applies everywhere.
18
18
 
19
19
  - **Field references**: `[column_name]` — the value of another column on the same entity
20
20
  - **Navigation**: `[reference_column].[field]` — traverse a reference to a related entity's field
21
- - **Literals**: numbers (`42`, `3.14`), strings (`'hello'` or `"hello"`), booleans (`true`, `false`), null (`null`)
22
- - **Operators**: `+`, `-`, `*`, `/`, `%`, `==`, `!=`, `<`, `>`, `<=`, `>=`, `AND`, `OR`, `NOT`
23
- - **Function calls**: `FUNCTION_NAME(arg1, arg2, ...)` — uppercase by convention
21
+ - **Literals**: numbers (`42`, `3.14`), strings in **single quotes** (`'hello'`; embed a quote by doubling it: `'it''s'`), booleans (`true`, `false`), null (`null`). Double quotes are **not** string delimiters — `"hello"` is a lex error
22
+ - **Operators**: `+`, `-`, `*`, `/`, `=`, `!=`, `<`, `>`, `<=`, `>=`, `AND`, `OR`, `NOT`, `IN`, `BETWEEN … AND …`, `CONTAINS`, `STARTS_WITH`, `ENDS_WITH`. Equality is a **single** `=` — `==` is a syntax error. There is no `%` operator; use `MOD(a, b)`
23
+ - **Function calls**: `FUNCTION_NAME(arg1, arg2, ...)` — uppercase by convention. Only the functions listed below exist; any other name is rejected at module install (see "Unknown functions" below)
24
24
  - **Parentheses** for grouping
25
25
 
26
+ `CONTAINS` / `STARTS_WITH` / `ENDS_WITH` are infix operators, not functions — write
27
+ `[note] CONTAINS 'urgent'`, never `CONTAINS([note], 'urgent')`.
28
+
26
29
  ## Examples
27
30
 
28
31
  ```
@@ -34,15 +37,17 @@ The same grammar applies everywhere.
34
37
 
35
38
  [account].[billing_address].[country]
36
39
 
37
- [status] == "active" AND [balance] > 0
40
+ [status] = 'active' AND [balance] > 0
41
+
42
+ TODAY() > [due_date] AND [total_paid] < [total]
38
43
 
39
- CASE([priority], "high", 3, "medium", 2, "low", 1, 0)
44
+ SWITCH([priority], 'high', 3, 'medium', 2, 'low', 1, 0)
40
45
 
41
- IF([total] > 1000, "large", "small")
46
+ IF([total] > 1000, 'large', 'small')
42
47
 
43
- COALESCE([nickname], [first_name], "Unknown")
48
+ COALESCE([nickname], [first_name], 'Unknown')
44
49
 
45
- FORMAT([created_date], "yyyy-MM-dd")
50
+ FORMAT([created_date], 'yyyy-MM-dd')
46
51
  ```
47
52
 
48
53
  ## Built-in functions
@@ -50,21 +55,27 @@ FORMAT([created_date], "yyyy-MM-dd")
50
55
  ### String functions
51
56
 
52
57
  - `CONCAT(a, b, ...)` — concatenate strings
53
- - `LEN(s)` — string length
58
+ - `LEN(s)` — string length (`LENGTH` is an alias)
59
+ - `LEFT(s, n)`, `RIGHT(s, n)` — leading / trailing `n` characters
54
60
  - `UPPER(s)`, `LOWER(s)` — case conversion
55
61
  - `TRIM(s)`, `LTRIM(s)`, `RTRIM(s)` — whitespace removal
56
- - `SUBSTRING(s, start, len)` — substring
57
- - `REPLACE(s, find, with)` — replace substring
58
- - `CONTAINS(s, find)` — boolean substring check
59
- - `STARTSWITH(s, prefix)`, `ENDSWITH(s, suffix)` — boolean
60
- - `FORMAT(value, pattern)` — format dates/numbers
62
+ - `SUBSTRING(s, start, end)` — substring by 0-based offsets, `end` exclusive
63
+ - `MID(s, start, len)` — substring by 0-based offset and length
64
+ - `INDEX_OF(s, find)`, `LAST_INDEX_OF(s, find)` — 0-based position, `-1` when absent
65
+ - `SPLIT(s, separator)` — split into a list
66
+ - `REPLACE(s, find, with)` — replace **every** occurrence
67
+ - Substring / prefix / suffix tests are the **operators** `CONTAINS`, `STARTS_WITH`, `ENDS_WITH` (`[note] CONTAINS 'x'`), not functions
68
+ - `FORMAT(date, 'pattern')` — format a **date** as text with .NET-style tokens (case-sensitive: `MM` = month, `mm` = minute): `yyyy`, `yy`, `MM`/`M`, `dd`/`d`, `HH`/`H`, `mm`/`m`, `ss`/`s`; single-letter variants unpadded, other characters pass through. Pattern must be a string literal. Dates only — numbers are not supported. Works client-side and SQL-time (reports)
61
69
 
62
70
  ### Number functions
63
71
 
64
72
  - `ABS(n)`, `ROUND(n, digits)`, `FLOOR(n)`, `CEIL(n)`, `CEILING(n)`
65
- - `MIN(a, b)`, `MAX(a, b)` — binary min/max
73
+ - `MIN(a, b, ...)`, `MAX(a, b, ...)` — smallest / largest argument; nulls are ignored
66
74
  - `POW(base, exp)`, `SQRT(n)`
67
- - `MOD(a, b)` — modulo
75
+ - `MOD(a, b)` — modulo; a zero divisor is null, not an error
76
+ - `TRUNC(n)`, `SIGN(n)`, `EXP(n)`, `LOG(n)` (natural), `LOG10(n)`, `LOG2(n)`, `LOG1P(n)`
77
+ - `FROUND(n)` — round to single (32-bit float) precision
78
+ - Trigonometry: `SIN`, `COS`, `TAN`, `ASIN`, `ACOS`, `ATAN`, `ATAN2(y, x)`, and the hyperbolic `SINH`, `COSH`, `TANH`, `ASINH`, `ATANH`
68
79
 
69
80
  ### Date functions
70
81
 
@@ -76,17 +87,67 @@ FORMAT([created_date], "yyyy-MM-dd")
76
87
  - `DATE(y, m, d)` — build a date from parts; `DATE(v)` casts a value to a date
77
88
  - `DATEADD(d, count, unit)` — add time; unit is a **string literal**: `'DAY'`, `'HOUR'`, `'MINUTE'`, `'SECOND'`, `'MONTH'`, `'YEAR'` (plural forms accepted)
78
89
  - `DATEDIFF(d1, d2, unit)` — difference `d2 − d1` in the given unit (same unit list). `DAY`/`HOUR`/`MINUTE`/`SECOND` floor the elapsed time; `MONTH`/`YEAR` are calendar-component deltas. E.g. days overdue: `DATEDIFF([due_date], TODAY(), 'DAY')`
79
- - `STARTMONTH(d?)`, `ENDMONTH(d?)`, `STARTQUARTER(d?)`, `ENDQUARTER(d?)`, `STARTYEAR(d?)`, `ENDYEAR(d?)`, `STARTNEXTMONTH(d?)` — period boundaries; the argument is optional and defaults to today
90
+ - `STARTMONTH(d?)`, `ENDMONTH(d?)`, `STARTQUARTER(d?)`, `ENDQUARTER(d?)`, `STARTYEAR(d?)`, `ENDYEAR(d?)`, `STARTNEXTMONTH(d?)` — period boundaries. **Omitting** the argument means today; passing one that is null gives null (a row with no date renders blank, it does not fall back to today)
80
91
 
81
92
  The whole date family evaluates identically **client-side and SQL-time** (reports, filters, sorts on formula columns) — the SQL translator mirrors the client runtime, including `MONTH` 1-based and `WEEKDAY` 0=Sunday. Unit arguments must be string literals; an unknown unit fails loud with the accepted list.
82
93
 
83
94
  ### Logical functions
84
95
 
85
- - `IF(cond, then, else)` — ternary
86
- - `CASE(expr, val1, result1, val2, result2, ..., default)` — multi-branch
96
+ - `IF(cond, then, else)` — ternary (`IIF` is an alias)
97
+ - `CASE(cond1, result1, cond2, result2, ..., default)` — first **condition** that holds wins
98
+ - `SWITCH(expr, val1, result1, val2, result2, ..., default)` — first **value** that matches
99
+ `expr` wins. Note the difference: `CASE` takes conditions, `SWITCH` takes values to compare
87
100
  - `COALESCE(a, b, c, ...)` — first non-null
88
101
  - `NULLIF(a, b)` — null if equal, else a
89
- - `ISNULL(x)` — boolean null check
102
+ - `ISNULL(x)` — boolean **null check** (returns true/false). This is *not* T-SQL's two-arg "replace null with" — that is `COALESCE(x, fallback)`
103
+
104
+ ### Conversion and context functions
105
+
106
+ - `TEXT(v)` — as text (null becomes `''`)
107
+ - `NUMBER(v)` — as a number, or null when the value isn't numeric (including a blank string)
108
+ - `BOOLEAN(v)` — null, `''` and `0` are false; everything else is true
109
+ - `JSON_GET(json, 'key' [, type])` — read a key out of a JSON column. The optional type is a
110
+ string literal: `'text'` (default), `'number'`/`'numeric'`, `'int'`/`'integer'`/`'bigint'`,
111
+ `'bool'`/`'boolean'`, `'date'`, `'datetime'`/`'timestamp'`/`'timestamptz'`
112
+ - `CURRENT_USER_ID()` — the id of the requesting user
113
+
114
+ ### Client-side vs SQL-time evaluation
115
+
116
+ An `F` column is evaluated **client-side** in grids and cards, but translated to **SQL** when a
117
+ report selects it, or when it is used in a filter or sort. **Every function listed here is
118
+ implemented in both**, so a column reads the same in a grid and in a report; a test executes
119
+ each one against PostgreSQL and diffs the result against the client runtime.
120
+
121
+ Two deliberate points of agreement, because the engines disagree by nature:
122
+
123
+ - **Out-of-domain math is null, not an error.** `SQRT(-1)`, `LOG(0)`, `ASIN(2)`, `EXP(1000)`,
124
+ `POW(0, -1)`, `MOD(x, 0)` — JavaScript would answer NaN or Infinity and PostgreSQL would
125
+ raise (failing the whole report). Both return null, so the cell renders blank.
126
+ - **String indexing is JavaScript-flavoured, not SQL's.** `SUBSTRING(s, start, end)` and
127
+ `MID(s, start, len)` take a **0-based** start, and `SUBSTRING`'s third argument is an **end
128
+ offset**, not a length. `INDEX_OF` / `LAST_INDEX_OF` return a 0-based index, or `-1`.
129
+
130
+ Known remaining difference: `CONCAT` on a numeric column renders PostgreSQL's declared scale
131
+ (`-3.0`) where the client renders the JavaScript number (`-3`). Use `FORMAT`/`TEXT` when the
132
+ exact rendering matters.
133
+
134
+ `CURRENT_USER_ID()` needs the requesting user, so it resolves in grids, reports, filters and
135
+ `select()`, but not in a context with no user (an unattended job).
136
+
137
+ ### Unknown functions
138
+
139
+ The parser treats any `NAME(...)` as a call, so a misspelled or non-existent function is a
140
+ *grammatically valid* formula. Module install rejects it with the offending name (and a "did
141
+ you mean" suggestion where one is close). Before that check existed, such a formula installed
142
+ cleanly and then rendered as an **empty cell on every row**, with no error in the server log,
143
+ the browser console, or the install output.
144
+
145
+ ### Comparing dates
146
+
147
+ Date comparisons work directly — `TODAY() > [due_date]`, `[due_date] BETWEEN STARTMONTH() AND
148
+ ENDMONTH()`, `[due_date] = TODAY()` — and combine with other conditions through `AND` / `OR`
149
+ like any other boolean. A bare `date` column is treated as a **local calendar date**, matching
150
+ `TODAY()` and `DATE(y, m, d)`.
90
151
 
91
152
  ### Aggregation over a child set — use a Generated (`G`) column, not `F`
92
153
 
@@ -127,6 +188,8 @@ and the JSON shape: `column-types.md` → "Roll-up totals over child rows — us
127
188
 
128
189
  Navigation works through `columnType: "R"` columns (reference columns). Chains of length 1 are **synchronous** and resolved instantly. Chains of length ≥ 2 are **asynchronous** — the formula engine resolves them after the initial data load.
129
190
 
191
+ **SQL-time (reports, filters, sorts):** single-hop navigation (`[ref].[field]`) is translated to a `LEFT JOIN` and computes in reports and query datasets. Multi-hop chains and `$[Setting]` references are **not** available at SQL time — the column returns NULL there and the response carries a warning naming the column. Keep report-bound formulas to single-hop navigation.
192
+
130
193
  ## Sync vs async formulas
131
194
 
132
195
  - **Sync formula**: pure local math, no navigation, or one-level navigation that can be JOINed. Evaluated on load and on every edit to a dependency.
@@ -85,6 +85,8 @@ Translation files mirror the module's content structure. Each translatable objec
85
85
  | `actions` | Action labels, descriptions, and param labels | Action codes (keys from `ui/actions.json`) → `{ label, desc, params: { param_cd: { label } } }` |
86
86
  | `reports` | Report dataset captions and param labels | Report codes → `{ datasets: { ds_cd: { caption } }, params: { param_cd: { label } } }` |
87
87
  | `entities.<e>.constraints` | Check/unique constraint violation messages (opt-in — warned, not enforced) | Constraint codes → `{ message }` under the owning entity |
88
+ | `entities.<e>.fields.<f>.options` | Dropdown/radio/flags **option labels** (opt-in) | Option `value` → localized label (string) or `{ label, icon, color }` |
89
+ | `domains` | Column domain labels **and** their shared option labels (opt-in) | Domain codes → `{ label, options: { value: … } }` |
88
90
 
89
91
  ### Constraint violation messages ARE translatable (opt-in)
90
92
 
@@ -105,6 +107,47 @@ A check/unique constraint declares a user-facing `message` in the entity JSON (`
105
107
  - **Opt-in, never mandatory** (unlike roles/labels, which completeness *enforces*). If the manifest lists `supportedLocales` and a constraint `message` has no override for one of those (non-English) locales, install emits a **non-fatal warning** naming the missing `entities.<e>.constraints.<c>.message` keys (printed by the CLI, returned in the marketplace install response). Install still commits with the base message as the fallback. `dforge_module_validate` surfaces the same gap pre-flight as a warning.
106
108
  - Only constraints that declare a `message` are scanned; English (`en`/`en-*`) is the base and is never warned; extension entities (`"extends": "..."`) are skipped — their constraint translations belong with the foreign module's files.
107
109
 
110
+ ### Dropdown option labels ARE translatable (opt-in)
111
+
112
+ The `options` on a `dropdown`/`radio`/`flags` column are the **base (fallback) labels**. To translate the labels a user actually sees, add an `options` map next to the field's `label`, keyed by the option's **value** (the stored code):
113
+
114
+ ```json
115
+ "entities": {
116
+ "position": {
117
+ "fields": {
118
+ "status": {
119
+ "label": "Status",
120
+ "options": {
121
+ "on_hold": { "label": "In Wartestellung", "color": "#f90" },
122
+ "draft": "Entwurf"
123
+ }
124
+ }
125
+ }
126
+ }
127
+ }
128
+ ```
129
+
130
+ - An override is a **partial merged over the base** — a bare string is shorthand for `{ "label": … }`; any field it omits (icon/color) falls through to the authored option. Only `label`, `icon`, `color` are translatable; **`value` is the stored code and is never translated** (translations match on it).
131
+ - **Opt-in, never mandatory** — an option with no override keeps its authored label; nothing warns or fails.
132
+ - Resolved once at metadata-load time and cached per culture, so the localized labels show up identically in the grid, the form, **and** print templates (`_fmt.<col>` / bare field refs) — which previously always rendered the base English label.
133
+ - `optionSets` conditional narrowing is unaffected: it filters by `value`, which translation never touches.
134
+
135
+ ### Column domain labels + options ARE translatable (opt-in) — and shared
136
+
137
+ If a column uses a **domain** (see `column-domains.md`), translate the domain **once** under a top-level `domains` section — its label and per-option labels then flow to *every* column that references it, instead of once per column:
138
+
139
+ ```json
140
+ "domains": {
141
+ "doc_status": {
142
+ "label": "Belegstatus",
143
+ "options": { "draft": "Entwurf", "posted": { "label": "Gebucht", "color": "#0a0" } }
144
+ }
145
+ }
146
+ ```
147
+
148
+ - Keys are domain codes from this module's `domains.json`. The `options` map uses the same value-keyed, partial-override shape as field options above.
149
+ - This is the whole point of domains for localization: the shared option list is authored and translated in one place. Don't also translate the consuming columns' options — they inherit from the domain.
150
+
108
151
  ### Roles ARE translated — and completeness is enforced
109
152
 
110
153
  Role labels are **required and localized** (this changed — older docs said roles were skipped; they are not). `SecurityRegistrar` gives every role a `res_id`, so `TranslationRegistrar` wires the role label exactly like entities/views/menus: