@peter.prib/renderengine 0.12.0 → 0.14.0

package/README.md

@@ -1,296 +1,454 @@
-# RenderEngine
-
-**An ArchiMate modeller, a data modeller, a requirements story map, a diagram editor and a chart library that run
-in any browser tab, with no install and no runtime dependencies.**
-
-Open an Archi model, edit it the way you would in Archi, and save it back to the same `.archimate` file, without
-installing anything. Model data at three levels, governed and checked the way DAMA-DMBOK2 has it, with banking at
-BIAN level built in. Or put the same editors in your own app: it's a JavaScript library and a set of workbench
-parts, not a desktop application.
-
-![The ArchiMate modeller: a layered view of an online shop, the model tree by layer with each type's picture, and the palette](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/archimate-modeller.png)
-
----
-
-## Try it
-
-```bash
-git clone https://gitlab.com/peter.prib/renderengine.git
-cd renderengine
-npm start          # the ArchiMate modeller
-npm run diagram    # the diagram editor (flowcharts, UML and more)
-npm run datamodel  # the data modeller: an enterprise ontology and a retail bank at BIAN level
-```
-
-No server is needed, and nothing needs installing first, because the built bundles are in `dist/`. You can also
-open the pages directly:
-
-- `examples/archimate.html`: the ArchiMate modeller
-- `examples/diagram-editor.html`: the diagram editor
-- `examples/data-model.html`: the data modeller
-
-The models they open come from [modelengine](https://gitlab.com/peter.prib/modelengine), kept apart from the tools: with
-it checked out beside renderengine, the pages open on its models (File ▸ Open from ▸ modelengine lists them by domain);
-without it they start empty.
-
-All work in current Chrome, Edge, Firefox and Safari. Save writes back to the file you opened in Chrome and Edge;
-other browsers download the file instead.
-
----
-
-## The ArchiMate modeller
-
-### Works with Archi
-
-- **Open and save Archi files** (`.archimate`, and archives that include pictures). Save writes back to the file you
-  opened; Save As picks a new one. Unsaved changes are marked (•), and the page asks before you leave or close them.
-- **Kept in the browser, backed up to the cloud.** Every model is kept in the browser as you edit it, and those open
-  reopen next time, with no file saved. File ▸ This browser backs them all up to a folder you choose once: pick one
-  that OneDrive, Google Drive or Dropbox syncs and the backups reach the cloud (each named with the date and time,
-  plus one under its plain name); browsers that cannot write to a folder download them instead.
-- **Import a spreadsheet** into the open model: choose the name, type, id, documentation, property and "related
-  to" columns; existing elements are updated, not duplicated (Archi's elements.csv imports as it is).
-- **draw.io export**: every view a page, in draw.io's ArchiMate shapes, with the elements' properties.
-- **Open Group Exchange Format** import and export, the standard way into Sparx, BiZZdesign and other tools.
-  Checked with Archi 5.8, which validates and opens the files.
-- **Every element Archi receives is a valid ArchiMate 3.2 class.** A type Archi lacks is written as its nearest
-  3.2 type, and its own type is kept so it comes back on the next import.
-- **Canvases, sketches, notes, groups and Archi's canvas templates** come through import and export.
-
-### Modelling
-
-- A **palette** of every element type, grouped by layer and limited by the view's viewpoint.
-- **Connecting two elements offers only the relationship kinds the ArchiMate rules allow** between them.
-- **Nesting**: drop an element inside another and choose what the nesting stands for (composition, aggregation,
-  assignment, realization or specialization). It goes into the model, and the nesting shows it.
-- **Add from text**: paste a list of names, from a document or a spreadsheet's columns, and get an element for
-  each.
-- **Copy and paste** within a model and between models, **Duplicate**, and **Generate View For** an element and
-  what it relates to.
-- **Several models open at once**, a tab each. Each has its own file and its own undo history.
-- **Undo for everything**, one history per model as in Archi: edits on any view, and adding, renaming and deleting
-  views. Undoing an edit made on another view takes you to that view.
-- **Specializations**, custom viewpoints, label expressions (`${name}` and `{name}`), and properties with types.
-
-### Editing
-
-- **Align, distribute and match size**, to the shape you selected last, as in Archi; **automatic layouts** (grid,
-  circle, tree, force-directed, by layer) and **Nest its parts inside**, following composition and aggregation.
-- **Properties of several shapes at once**: colours and text style, the elements' type and specialization, and a
-  property to set on all of them.
-- **Format painter**: copy one shape's look onto others.
-- **Colour by** layer, type, specialization or any property (a heat map, e.g. by lifecycle: invest, maintain,
-  retire), with a legend; or a legend of the element types on the view.
-- **Find and replace** across names, documentation and property values, with match case, whole words and
-  regular expressions; **transform names** (Title Case, split camelCase, ...) and **rename a property** across the
-  model; **fit a shape to its text**.
-- **Groups and locks on any view**, multi-select, a marquee, Archi's tabbed groups, bendpoints, pinned line ends,
-  and orthogonal, curved or straight routing.
-
-### Understanding and sharing a model
-
-- A **model tree** by layer, by type or flat, each element with its type's picture, with search and filters (type, specialization, property) that dim the
-  diagram too.
-- An **analysis panel** for the selected element: which views it's on, and its incoming and outgoing relationships.
-- **Check model** finds relationships the rules don't allow, likely duplicates (by their words, so "System of
-  Order Management" matches "order-management system"), unused elements and relationships, empty views, nested
-  elements with no relationship, and unprintable characters. **Merge…** makes duplicates one element, keeping
-  every relationship and appearance on a view.
-- **Derived relationships**, worked out and shown as dotted lines.
-- **Roadmaps**: generate a view of work packages, plateaus or anything with start and end dates, as bars under
-  the months.
-- An **HTML report** in one self-contained file: a tree, search, a page per element, relationship and view, and
-  diagrams whose elements link to their pages.
-- **Images**: export PNG or SVG, or copy to the clipboard.
-
-### Beyond Archi
-
-It runs in the browser and embeds in other apps. It also has:
-
-- ArchiMate 4 types alongside 3.2;
-- derived relationships without scripts;
-- invalid relationships shown as you choose (highlight, tooltip or hidden), per view, per model or for the tool;
-- groups and locks on every view, not just canvases;
-- a canvas template library shared across models;
-- two drawing backends, SVG and Canvas.
-
----
-
-## The data modeller
-
-One model at the conceptual, logical and physical levels, read by each stakeholder in their own terms, governed and
-checked the way DAMA-DMBOK2 has a model reviewed, and ready for banking at BIAN level. It shares the modeller's
-workbench: several models open at once, undo for everything, files saved back where they came from, models kept in
-the browser and backed up.
-
-![The data modeller: a retail bank's logical model through its Sensitive data lens, each entity coloured by how sensitive its data is](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/data-model.png)
-
-### One model, three levels
-
-- **Concepts, logical entities and tables, each a schema of its own**, mapped any number either way: a concept
-  realised by several entities, several entities kept in one table. The **mapping view** draws the three side by
-  side, and **lineage** shows, for anything, what it realises and what realises it, all the way up and down.
-- **Perspectives and contexts**: each stakeholder's names for the concepts, and what they mean in a context, with
-  its rules and where and when it applies, drawn as notes.
-- **Chen, Crow's Foot and UML**, switched with one choice; **subtypes** (complete or not, disjoint or overlapping);
-  **data types and enumerations** defined once, code sets with their source and version, each linked to its
-  **ISO 20022** type.
-- **Edit it on the diagram**: double-click an entity for its attribute table (keys, required, what a foreign key
-  references, classification), right-click for new and delete. Every edit is one undo step.
-
-### Banking at BIAN level
-
-- **Service domains** with their control records, behaviour qualifiers and the service operations BIAN names for
-  them, and the **service landscape**: every service domain in its business area and domain.
-- **Read BIAN's own published material instead of retyping it.** A BIAN semantic API (all 258 of release 14 read,
-  plain or with ISO 20022) becomes a control record, its qualifiers, their data types and ISO 20022 types in one
-  step. BIAN's service landscape in ArchiMate becomes service domains in their areas and domains: run over BIAN's
-  v14 model, every one it places lands in the right place.
-- BIAN's vocabulary, its functional patterns, artifacts, action terms and qualifier types, is as BIAN's v14 model
-  has it.
-
-### Governed and checked (DAMA-DMBOK2)
-
-- **Stewardship**: a business owner, data steward and technical custodian on each entity.
-- **Classification**: sensitivity levels and categories (PII, PCI...) on entities and attributes, **checked down
-  the mappings**, so a table is never classified lower than the data it holds; a **lens** colours the model by it.
-- **Check model** reviews it as DMBOK2 would: missing keys, foreign keys without relationships, repeating groups,
-  unresolved many-to-many, one name with two types, concepts without definitions, names against the **naming
-  standard** you set, **data quality rules** by dimension (completeness, validity, uniqueness...), and BIAN's one
-  owner per business object.
-- **The Data Model Scorecard**: Steve Hoberman's ten categories, six measured from the model and four for a
-  reviewer, weighted as you choose, in one total.
-
-### To the database and the warehouse
-
-- **SQL DDL**: the physical level as `CREATE TABLE`s with keys, foreign keys and comments.
-- **A dimensional scheme**: facts at their grain with additive, semi-additive or non-additive measures, dimensions
-  with their history type and conformed across facts, a **star schema** view, and checks as Kimball would make
-  them.
-- **Bases** to start a model from, and **compare and merge** between models.
-
----
-
-## Requirements and story maps
-
-Bring in the epics and stories your team already has: a Jira, Azure DevOps or GitLab export (CSV or JSON), or a
-storyboard written in Markdown. Read as a **story map** (epics across, releases down) or a **backlog**, each story
-carries its "As a … I want … so that …", its acceptance criteria (read from their own field, a list in the
-description, or Given/When/Then) and its **traces** to what realises it: an ArchiMate requirement or work package,
-a component, a data concept, entity or table. Drag a card to another epic or release; the toolbar says how many
-stories say when they are done and how many trace to something, and **Check model** lists what is missing.
-
-- **Every model alike**: kept, compared and merged as the ArchiMate and data models are, a newer tracker export
-  brought in over the old one replacing what changed.
-- **Yours to extend**: a host says what can be traced to, what a story's live state is, and how to find stories by
-  meaning (a vector store) — the SqlEngine workbench does all three.
-
-## The diagram editor
-
-The ArchiMate modeller is built on a general diagram editor, and every editing feature above is part of that
-editor rather than of ArchiMate: align, format painter, properties of several shapes, find and replace, copy and
-paste, groups and locks, undo, HTML report, image export, several models at once. The diagram editor gives you all
-of them for its own palettes:
-
-- basic shapes
-- flowchart and BPMN
-- swimlanes
-- UML: class, sequence, activity, state machine, use case, component, deployment
-- entity-relationship
-- data flow
-- network
-- org chart
-
-Each model can hold several diagrams, with line styles inherited from the diagram, then the model, then the tool.
-
-![The diagram editor: an order flowchart with a decision, labelled lines and the flowchart palette](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/diagram-editor.png)
-
-## The chart library
-
-The engine underneath draws 24 chart types: bar, line, area, pie, scatter, bubble, heatmap, candlestick,
-waterfall, histogram, box plot, violin, radar, funnel, treemap, sunburst, gauges (dial, arc, ring, rings, bar, bullet, thermometer, KPI tiles; with bands, targets, LED segments and animation), 3D bars and others. It also
-draws graph and railroad diagrams, all to SVG or Canvas. `test/charts.html` is the full gallery, and
-`examples/csv-chart-builder.html` builds a chart from a CSV file without code.
-
-![Gauges: a dial with bands, ticks and a target; arc, ring and nested rings; a two-needle dial; LED segments; a thermometer; KPI tiles with sparklines](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/gauges.png)
-
-![Charts from the gallery: bar, grouped bar, line and line with error bars](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/charts.png)
-
----
-
-## Put it in your own page
-
-```bash
-npm install @peter.prib/renderengine
-```
-
-The package is the ES module source and the built bundles:
-
-- `import { ArchimateModel, RenderEngine } from '@peter.prib/renderengine'`: the engine and the models (this also
-  imports under Node).
-- `import '@peter.prib/renderengine/workspace'`: the workbench shell and its parts (browser only).
-- `@peter.prib/renderengine/dist/renderengine-workspace.umd.js`: the same as one classic script.
-
-The modeller is a part in the `<ws-workspace>` workbench shell. One script tag and a JSON spec is the whole host
-page:
-
-```html
-<script src="dist/renderengine-workspace.umd.js"></script>
-<script>
-    const { el, registerBuiltinParts, ArchimateCanvasPart } = RenderEngineWorkspace;
-    const ws = document.body.appendChild(el({ tag: 'ws-workspace', attrs: { label: 'Modeller' } }));
-    registerBuiltinParts(ws);
-    ws.registerPart(ArchimateCanvasPart);
-    ws.addParts([{
-        id: 'models', type: 'models', editorType: 'archimate-canvas',   // several models, a tab each
-        models: [{
-            id: 'shop', name: 'Shop',
-            elements: [{ id: 'c', name: 'Customer', type: 'business-actor' }, { id: 'o', name: 'Order', type: 'business-process' }],
-            relationships: [{ source: 'c', target: 'o', kind: 'assignment' }],
-            views: [{ id: 'v', name: 'Main', elementIds: ['c', 'o'], layout: { c: { x: 100, y: 100 }, o: { x: 320, y: 100 } } }],
-        }],
-    }]);
-    ws.registerPerspective('Modelling', { type: 'part', partId: 'models' });
-    ws.activatePerspective('Modelling');
-</script>
-```
-
-Use `editorType: 'diagram-editor'` with `DiagramEditorPart` for the diagram editor, `editorType: 'data-model'`
-with `DataModelPart` for the data modeller, and `editorType: 'requirements'` with `RequirementsPart` for the story
-map. The models are plain JavaScript APIs (`ArchimateModel`, `DiagramModel`, `DataModel`, `RequirementsModel`), so
-you can also build, check, score, search, report on and export models with no UI at all, under Node too:
-`new DataModel(spec).checker().run()`, `new SqlDdl().write(model)`,
-`new RequirementsModel(new TrackerExport().read(csv)).coverage()`.
-
----
-
-## Documentation
-
-| Document | What it covers |
-|---|---|
-| `documentation/REFERENCE.md` | The API: chart methods and options, the editors, models, commands, file formats |
-| `documentation/WORKSPACE_ARCHITECTURE.md` | The `<ws-workspace>` shell: parts, layout, context bus, work area |
-| `documentation/ARCHITECTURE.md` | Module layout and the chart-type catalog |
-| `documentation/LLM_ARCHIMATE_GUIDE.md` | Generating valid ArchiMate model data, with a prompt template |
-| `documentation/tasks.md` | What's still to come |
-
-## Build and test
-
-```bash
-npm install        # dev tools only: Rollup
-npm run build      # dist/renderengine.umd.js and dist/renderengine-workspace.umd.js
-npm test           # Node's built-in test runner, no browser needed
-npm run screenshots  # this README's pictures, from the example pages (headless Chrome)
-```
-
-There are two bundles:
-
-- `renderengine.umd.js` (global `RenderEngine`): the engine.
-- `renderengine-workspace.umd.js` (global `RenderEngineWorkspace`): the workbench shell and its parts, plus the
-  whole engine API, so one script tag is enough.
-
-The source is plain ES modules. `src/index.js` imports under Node, which is how the tests run.
-
-## Licence
-
-MIT. Contributions welcome.
+# RenderEngine
+
+**An ArchiMate modeller, a data modeller, a requirements story map, a diagram editor, an OLAP cube & predictive query studio, and a 50+ type chart library that run in any browser tab, with no install and no runtime dependencies.**
+
+Open an Archi model, edit it the way you would in Archi, and save it back to the same `.archimate` file, without
+installing anything. Model data at three levels, governed and checked the way DAMA-DMBOK2 has it, with banking at
+BIAN level built in. Slice, drill and chart multi-dimensional OLAP cubes across 35+ linked chart types with live
+statistical aggregations, derived formula measures and predictive natural-language querying. Or put the same editors
+and charts in your own app: it's a JavaScript library and a set of workbench parts, not a desktop application.
+
+![The ArchiMate modeller: a layered view of an online shop, the model tree by layer with each type's picture, and the palette](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/archimate-modeller.png)
+
+---
+
+## Try it
+
+```bash
+git clone https://gitlab.com/peter.prib/renderengine.git
+cd renderengine
+npm start          # the ArchiMate modeller
+npm run diagram    # the diagram editor (flowcharts, UML and more)
+npm run datamodel  # the data modeller: an enterprise ontology and a retail bank at BIAN level
+```
+
+No server is needed, and nothing needs installing first, because the built bundles are in `dist/`. You can also
+open the pages directly:
+
+- `examples/archimate.html`: the ArchiMate modeller
+- `examples/diagram-editor.html`: the diagram editor
+- `examples/data-model.html`: the data modeller
+- `test/charts.html`: the full interactive chart, diagram and OLAP cubing gallery
+
+The models they open come from [modelengine](https://gitlab.com/peter.prib/modelengine), kept apart from the tools: with
+it checked out beside renderengine, the pages open on its models (File ▸ Open from ▸ modelengine lists them by domain);
+without it they start empty.
+
+All work in current Chrome, Edge, Firefox and Safari. Save writes back to the file you opened in Chrome and Edge;
+other browsers download the file instead.
+
+---
+
+## The ArchiMate modeller
+
+### Works with Archi
+
+- **Open and save Archi files** (`.archimate`, and archives that include pictures). Save writes back to the file you
+  opened; Save As picks a new one. Unsaved changes are marked (•), and the page asks before you leave or close them.
+- **Kept in the browser, backed up to the cloud.** Every model is kept in the browser as you edit it, and those open
+  reopen next time, with no file saved. File ▸ This browser backs them all up to a folder you choose once: pick one
+  that OneDrive, Google Drive or Dropbox syncs and the backups reach the cloud (each named with the date and time,
+  plus one under its plain name); browsers that cannot write to a folder download them instead.
+- **Import a spreadsheet** into the open model: choose the name, type, id, documentation, property and "related
+  to" columns; existing elements are updated, not duplicated (Archi's elements.csv imports as it is).
+- **draw.io export**: every view a page, in draw.io's ArchiMate shapes, with the elements' properties.
+- **Open Group Exchange Format** import and export, the standard way into Sparx, BiZZdesign and other tools.
+  Checked with Archi 5.8, which validates and opens the files.
+- **Every element Archi receives is a valid ArchiMate 3.2 class.** A type Archi lacks is written as its nearest
+  3.2 type, and its own type is kept so it comes back on the next import.
+- **Canvases, sketches, notes, groups and Archi's canvas templates** come through import and export.
+
+### Modelling
+
+- A **palette** of every element type, grouped by layer and limited by the view's viewpoint.
+- **Connecting two elements offers only the relationship kinds the ArchiMate rules allow** between them.
+- **Nesting**: drop an element inside another and choose what the nesting stands for (composition, aggregation,
+  assignment, realization or specialization). It goes into the model, and the nesting shows it.
+- **Add from text**: paste a list of names, from a document or a spreadsheet's columns, and get an element for
+  each.
+- **Copy and paste** within a model and between models, **Duplicate**, and **Generate View For** an element and
+  what it relates to.
+- **Delete from model**, from the tree or a diagram, after a dialog listing the views it is on and the
+  relationships that go with it; one undo puts it all back.
+- **Several models open at once**, a tab each. Each has its own file and its own undo history.
+- **Undo for everything**, one history per model as in Archi: edits on any view, and adding, renaming and deleting
+  views. Undoing an edit made on another view takes you to that view.
+- **Specializations**, custom viewpoints, label expressions (`${name}` and `{name}`), and properties with types.
+
+### Editing
+
+- **Align, distribute and match size**, to the shape you selected last, as in Archi; **automatic layouts** (grid,
+  circle, tree, force-directed, by layer) and **Nest its parts inside**, following composition and aggregation.
+- **Properties of several shapes at once**: colours and text style, the elements' type and specialization, and a
+  property to set on all of them.
+- **Format painter**: copy one shape's look onto others.
+- **Colour by** layer, type, specialization or any property (a heat map, e.g. by lifecycle: invest, maintain,
+  retire), by a formula over properties (`Cost * Risk_score`), or by the change work packages and deliverables make
+  (create, update, delete, boxes and lines), with a legend boxed round its entries; or a legend of the element types
+  on the view.
+- **Find and replace** across names, documentation and property values, with match case, whole words and
+  regular expressions; **transform names** (Title Case, split camelCase, ...) and **rename a property** across the
+  model; **fit a shape to its text**.
+- **Groups and locks on any view**, multi-select, a marquee, Archi's tabbed groups, bendpoints, pinned line ends,
+  and orthogonal, curved or straight routing. Lines take the shortest way between shapes, as in Archi.
+
+### Understanding and sharing a model
+
+- A **model tree** by layer, by type or flat, each element with its type's picture, with search and filters (type, specialization, property) that dim the
+  diagram too.
+- An **analysis panel** for the selected element: which views it's on, and its incoming and outgoing relationships.
+- **Check model** finds relationships the rules don't allow, likely duplicates (by their words, so "System of
+  Order Management" matches "order-management system"), unused elements and relationships, empty views, nested
+  elements with no relationship, and unprintable characters. **Merge…** makes duplicates one element, keeping
+  every relationship and appearance on a view.
+- **Derived relationships**, worked out and shown as dotted lines.
+- **Roadmaps**: generate a view of work packages, plateaus or anything with start and end dates, as bars under
+  the months.
+- An **HTML report** in one self-contained file: a tree, search, a page per element, relationship and view, and
+  diagrams whose elements link to their pages.
+- **Images**: export PNG or SVG, or copy to the clipboard: the whole drawing with a margin (or only what is in
+  sight), at a scale you choose, from the File menu or a right-click; a setting shows where the picture will end.
+
+### Beyond Archi
+
+It runs in the browser and embeds in other apps. It also has:
+
+- ArchiMate 4 types alongside 3.2;
+- derived relationships without scripts;
+- invalid relationships shown as you choose (highlight, tooltip or hidden), per view, per model or for the tool;
+- groups and locks on every view, not just canvases;
+- a canvas template library shared across models;
+- two drawing backends, SVG and Canvas.
+
+---
+
+## The data modeller
+
+One model at the conceptual, logical and physical levels, read by each stakeholder in their own terms, governed and
+checked the way DAMA-DMBOK2 has a model reviewed, and ready for banking at BIAN level. A model can be imported
+from pasted JSON, an assistant's answer included. It shares the modeller's
+workbench: several models open at once, undo for everything, files saved back where they came from, models kept in
+the browser and backed up.
+
+![The data modeller: a retail bank's logical model through its Sensitive data lens, each entity coloured by how sensitive its data is](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/data-model.png)
+
+### One model, three levels
+
+- **Concepts, logical entities and tables, each a schema of its own**, mapped any number either way: a concept
+  realised by several entities, several entities kept in one table. The **mapping view** draws the three side by
+  side, and **lineage** shows, for anything, what it realises and what realises it, all the way up and down.
+- **Perspectives and contexts**: each stakeholder's names for the concepts, and what they mean in a context, with
+  its rules and where and when it applies, drawn as notes.
+- **Chen, Crow's Foot and UML**, switched with one choice; **subtypes** (complete or not, disjoint or overlapping);
+  **data types and enumerations** defined once, code sets with their source and version, each linked to its
+  **ISO 20022** type.
+- **Edit it on the diagram**: double-click an entity for its attribute table (keys, required, what a foreign key
+  references, classification), right-click for new and delete. Every edit is one undo step.
+
+### Banking at BIAN level
+
+- **Service domains** with their control records, behaviour qualifiers and the service operations BIAN names for
+  them, and the **service landscape**: every service domain in its business area and domain.
+- **Read BIAN's own published material instead of retyping it.** A BIAN semantic API (all 258 of release 14 read,
+  plain or with ISO 20022) becomes a control record, its qualifiers, their data types and ISO 20022 types in one
+  step. BIAN's service landscape in ArchiMate becomes service domains in their areas and domains: run over BIAN's
+  v14 model, every one it places lands in the right place.
+- BIAN's vocabulary, its functional patterns, artifacts, action terms and qualifier types, is as BIAN's v14 model
+  has it.
+
+### Governed and checked (DAMA-DMBOK2)
+
+- **Stewardship**: a business owner, data steward and technical custodian on each entity.
+- **Classification**: sensitivity levels and categories (PII, PCI...) on entities and attributes, **checked down
+  the mappings**, so a table is never classified lower than the data it holds; a **lens** colours the model by it.
+- **Check model** reviews it as DMBOK2 would: missing keys, foreign keys without relationships, repeating groups,
+  unresolved many-to-many, one name with two types, concepts without definitions, names against the **naming
+  standard** you set, **data quality rules** by dimension (completeness, validity, uniqueness...), and BIAN's one
+  owner per business object.
+- **The Data Model Scorecard**: Steve Hoberman's ten categories, six measured from the model and four for a
+  reviewer, weighted as you choose, in one total.
+
+### To the database and the warehouse
+
+- **SQL DDL**: the physical level as `CREATE TABLE`s with keys, foreign keys and comments.
+- **A dimensional scheme**: facts at their grain with additive, semi-additive or non-additive measures, dimensions
+  with their history type and conformed across facts, a **star schema** view, and checks as Kimball would make
+  them.
+- **Bases** to start a model from, and **compare and merge** between models.
+
+---
+
+## Requirements and story maps
+
+Bring in the epics and stories your team already has: a Jira, Azure DevOps or GitLab export (CSV or JSON), or a
+storyboard written in Markdown. Read as a **story map** (epics across, releases down) or a **backlog**, each story
+carries its "As a … I want … so that …", its acceptance criteria (read from their own field, a list in the
+description, or Given/When/Then) and its **traces** to what realises it: an ArchiMate requirement or work package,
+a component, a data concept, entity or table. Drag a card to another epic or release; the toolbar says how many
+stories say when they are done and how many trace to something, and **Check model** lists what is missing.
+
+- **Every model alike**: kept, compared and merged as the ArchiMate and data models are, a newer tracker export
+  brought in over the old one replacing what changed.
+- **Yours to extend**: a host says what can be traced to, what a story's live state is, and how to find stories by
+  meaning (a vector store) — the SqlEngine workbench does all three.
+
+---
+
+## The diagram editor
+
+The ArchiMate modeller is built on a general diagram editor, and every editing feature above is part of that
+editor rather than of ArchiMate: align, format painter, properties of several shapes, find and replace, copy and
+paste, groups and locks, undo, HTML report, image export, several models at once. The diagram editor gives you all
+of them for its own palettes:
+
+- basic shapes
+- flowchart and BPMN
+- swimlanes
+- UML: class, sequence, activity, state machine, use case, component, deployment
+- entity-relationship
+- data flow
+- network
+- org chart
+
+Each model can hold several diagrams, with line styles inherited from the diagram, then the model, then the tool.
+
+![The diagram editor: an order flowchart with a decision, labelled lines and the flowchart palette](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/diagram-editor.png)
+
+---
+
+## OLAP cubing, linked pivot charts and predictive Ask
+
+Turn any rolled-up query result (`GROUP BY ROLLUP`, `semanticengine`, or in-memory facts) into an interactive
+multi-dimensional cube grid (`pivotView`), a synchronized chart (`pivotChart`) that follows every slice and drill,
+and a natural-language query bar (`askBar`) with configurable next-word prediction.
+
+### The cube grid (`pivotView` / `PivotView`)
+
+- **Interactive layout chips (PresentBlox)**: move dimensions between rows, columns, and page filters, reorder
+  hierarchy levels outward or inward, or keep/exclude individual dimension members.
+- **Expand, collapse and drill**: collapse any member onto its rolled-up subtotal (`▾` / `▸`), expand to a chosen
+  hierarchy depth (`expandToLevel`), drill down/up along a dimension hierarchy (`onDrill`), or **drill through** any
+  cell to inspect the underlying fact rows (`onDrillThrough`).
+- **7 summary & statistical aggregations**: switch cell and subtotal calculations on the fly between **Sum**,
+  **Count**, **Average**, **Median**, **Minimum**, **Maximum**, and **Std Dev** (`setAggregation`), computed over raw
+  underlying facts (`options.facts`) or descendant leaf cells.
+- **Post-rollup derived formula measures**: define calculated measures such as `Margin = Revenue - Cost` or
+  `MarginPct = Revenue > 0 ? Margin / Revenue * 100 : 0` (`derivedMeasures`, `addDerivedMeasure`). Formulas are
+  compiled once via `compileFormula` and evaluated *after* rollup so ratios and percentages are always accurate on
+  subtotals and grand totals.
+- **Ranking, sorting and % of total**: hierarchical sibling sorting (`sortRowsBy`), **Top-N / Bottom-N** filtering
+  within each parent group (`setTopN`), multi-measure visibility toggles (`toggleMeasure`), percentage-of-total
+  display modes (`% of row total`, `% of column total`, `% of grand total`), traffic-light conditional formatting
+  (custom rules or automatic tertiles), and serializable saved views (`displayState` / `restoreDisplay`).
+
+### Linked pivot chart (`pivotChart` / `PivotChart`)
+
+- **Follows the cube automatically**: charts the finest visible members on both axes without double-counting totals,
+  and redraws whenever the cube is pivoted, collapsed, sorted, filtered, or switched to a different aggregation.
+- **35+ cube chart types, switchable in one click**: switch live between **Line**, **Grouped Bar**, **Stacked Bar**,
+  **Bar**, **Area**, **Stacked Area**, **Stream Graph**, **Pareto (80/20)**, **Waterfall**, **Marimekko / Mekko Mosaic**,
+  **Lollipop**, **Pipe**, **Heatmap**, **Radar / Spider**, **Parallel Coordinates**, **Population Pyramid**,
+  **Sankey**, **Chord**, **Treemap**, **Sunburst**, **Icicle**, **Outline**, **Pie**, **Donut**, **Funnel**,
+  **Radial Bar**, **Word Cloud**, **Box Plot**, **Density**, **Violin**, **Histogram**, **Q-Q Plot**, **Lag Plot**,
+  **Scatter**, **Bubble**, **3D Bar**, **3D Orbit Bar**, and **3D Orbit Stacked Bar** (`pivotChartLink.setType(key)`
+  or via the in-chart `▾` menu).
+- **Click any chart mark to drill the cube**: clicking a bar, line vertex, slice, treemap/icicle cell, Sankey node,
+  or scatter point expands a collapsed member or drills the cube down to the next hierarchy level on either axis.
+
+### Predictive natural-language query bar (`askBar` / `AskView`)
+
+- **Scrollable next-word prediction**: predicts the next keyword, measure, dimension, operator, or member value as
+  you type, with semantic badges (`sense`, `reference`, `relation`, `kind`) and full keyboard navigation (`↑`/`↓`,
+  `Tab`/`Enter` to accept, `Escape` to dismiss).
+- **User-configurable viewport**: built-in `Predict next word` selector lets users choose how many predicted items
+  are visible at once in the scrollable list (`Off`, `3`, `5`, `8`, `10`, or custom `predictChoices`).
+- **Pluggable intelligence**: pass declarative `measures`, `dimensions`, `members` and `sampleQuestions` for the
+  built-in grammar/bigram predictor, or wire `suggest: (text) => semanticEngine.suggest(text)` to connect directly
+  to `semanticengine`.
+
+```javascript
+import { RenderEngine } from '@peter.prib/renderengine';
+
+// 1. Render an interactive OLAP cube with derived formulas and statistical aggregations
+const gridEngine = new RenderEngine('#pivot-grid', { rendererType: 'svg' });
+const cube = gridEngine.pivotView(rolledUpRows, {
+    title: 'Revenue & Margin Cube',
+    rowAxis: ['Year', 'Quarter'],
+    columnAxis: ['Region'],
+    facts: rawFactRows,                      // enables exact Count, Avg, Median, Min, Max, Std Dev
+    measures: ['Revenue', 'Cost'],
+    derivedMeasures: [
+        { name: 'Margin', formula: 'Revenue - Cost' },
+        { name: 'MarginPct', formula: 'Revenue > 0 ? Margin / Revenue * 100 : 0' },
+    ],
+    columns: [
+        { name: 'Year', kind: 'time' },
+        { name: 'Quarter', kind: 'time' },
+        { name: 'Region', kind: 'categorical' },
+        { name: 'Revenue', kind: 'numeric', measure: 'additive' },
+        { name: 'Cost', kind: 'numeric', measure: 'additive' },
+        { name: '_rollup', kind: 'rollup', dimensions: ['Year', 'Quarter', 'Region'] },
+    ],
+});
+
+// Switch aggregations, Top-N ranking, or % of total programmatically (or from the ⋮ menu):
+cube.setAggregation('median');               // 'sum' | 'count' | 'avg' | 'median' | 'min' | 'max' | 'stddev'
+cube.setTopN(5, { measure: 'Margin', direction: 'desc' });
+cube.setValueMode('rowPercent');             // 'value' | 'rowPercent' | 'columnPercent' | 'totalPercent'
+
+// 2. Link a Pivot Chart that follows the cube and drills on click across 35+ chart types
+const chartEngine = new RenderEngine('#pivot-chart', { rendererType: 'svg' });
+const pivotChart = chartEngine.pivotChart(cube, { type: 'stackedBar' });
+pivotChart.setType('marimekko');             // switch to 'line', 'pareto', 'icicle', 'sankey', 'radar', ...
+
+// 3. Mount a Predictive Ask Bar with configurable next-word prediction in a scrollable list
+const askEngine = new RenderEngine('#ask-results', { rendererType: 'svg' });
+askEngine.askBar('#ask-bar', {
+    value: 'Average Revenue by Quarter and Region',
+    measures: ['Revenue', 'Cost', 'Margin', 'MarginPct'],
+    dimensions: ['Region', 'Year', 'Quarter', 'Month'],
+    members: (dim) => [...new Set(rawFactRows.map(r => r[dim]))],
+    visibleCount: 5,
+    predictChoices: [0, 3, 5, 8, 10],        // 0 = Off, or N visible items in scrollable list
+    onAsk: (question) => { /* update cube & resultView */ },
+});
+```
+
+---
+
+## The chart library (50+ chart & diagram types)
+
+The engine underneath draws **over 50 chart and diagram types** to interchangeable **SVG** or **Canvas** backends,
+all driven by a single declarative catalog (`CHART_CATALOG`):
+
+- **Categorical & comparison**: `barChart`, `groupedBarChart`, `stackedBarChart` (standard and **100% normalized**
+  `normalize: true`), `lollipopChart` (single and multi-series dot-and-stem), `marimekkoChart` (variable-width
+  2D mosaic), `paretoChart` (descending bars + cumulative 0–100% curve + 80% threshold line), `waterfallChart`,
+  `funnelChart`, `populationPyramid`, `radialBarChart`, `wordCloud`.
+- **Time-series & trend**: `lineChart` (with error bars and multi-series), `areaChart` (standard, stacked, and
+  **100% normalized** `normalize: true`), `stackedAreaChart`, `streamGraph`, `pipeChart` (variable-width flow),
+  `candlestick` (OHLC with event markers), `ganttChart`, `calendarHeatmap`.
+- **Hierarchical & part-of-whole**: `pieChart`, `donutChart`, `treemapChart`, `sunburstChart`, `icicleChart`
+  (top-down rectangular partition), `outlineChart` (collapsible indented tree with inline proportion bars).
+- **Statistical & distribution**: `histogram`, `boxPlot`, `densityChart` (kernel density estimation), `violinChart`,
+  `qqPlot` (normal quantile-quantile), `lagPlot` (lag-1 autocorrelation), `scatterChart` (with 2D error bars and
+  reference lines), `bubbleChart`.
+- **Multivariate, matrix & network flow**: `heatmap` (correlation and matrix grids), `radarChart` (spider),
+  `parallelCoordinates`, `sankeyDiagram`, `chordDiagram`, `geoMap` (choropleth polygons + city bubbles).
+- **3D & orbit camera**: `barChart3D`, `bubbleChart3D`, `lineChart3D`, and true 4×4 matrix perspective orbit charts
+  (`scatter3DOrbit`, `barChart3DOrbit`, `stackedBarChart3DOrbit`, `graphDiagram3D`).
+- **Gauges & KPIs**: `dialGauge` (bands, ticks, multi-needle, targets), `arcGauge`, `ringGauge`, `ringsGauge`,
+  `barGauge`, `bulletChart`, `thermometerGauge`, `ledGauge`, `kpiTile` (with sparklines and delta badges).
+- **Interactive tables & diagrams**: `tableChart`, `resultView` (sortable, filterable, pageable table that switches
+  into any fitting chart via `ChartChooser`), `pivotView`, `pivotChart`, `graphDiagram` (force-directed, hierarchical
+  Dewey keys, table nodes, orthogonal/polyline obstacle-avoiding routing, manual bendpoints), `archimateDiagram`,
+  `dataModelDiagram`, `railroadDiagram`, `ebnfDiagram`.
+
+`test/charts.html` is the full interactive workspace gallery (including the dedicated **Cubing** perspective), and
+`examples/csv-chart-builder.html` builds charts from a CSV file without code.
+
+```javascript
+const engine = new RenderEngine('#chart', { rendererType: 'svg' });
+
+// 80/20 Pareto Chart with cumulative percentage curve and 80% reference line
+engine.paretoChart([
+    { label: 'Timeout', value: 142 },
+    { label: 'Auth Expired', value: 88 },
+    { label: 'Rate Limit', value: 54 },
+    { label: 'Schema Mismatch', value: 26 },
+], { title: 'Incident Root Causes', cutoffPercent: 80 });
+
+// 100% Normalized Stacked Bar Chart
+engine.stackedBarChart([
+    { label: 'Q1', values: [180, 90, 50] },
+    { label: 'Q2', values: [210, 110, 60] },
+], { title: 'Segment Share by Quarter', normalize: true, seriesLabels: ['Enterprise', 'Mid-Market', 'SMB'], legend: true });
+
+// Marimekko (Mekko Mosaic), Lollipop, and Icicle Partition Charts
+engine.marimekkoChart(seriesData, { title: 'Market Share by Region & Segment' });
+engine.lollipopChart(seriesData, { title: 'Regional Revenue Comparison', legend: true });
+engine.icicleChart(hierarchyTree, { title: 'Disk Usage by Category' });
+```
+
+![Gauges: a dial with bands, ticks and a target; arc, ring and nested rings; a two-needle dial; LED segments; a thermometer; KPI tiles with sparklines](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/gauges.png)
+
+![Charts from the gallery: bar, grouped bar, line and line with error bars](https://gitlab.com/peter.prib/renderengine/-/raw/master/docs/images/charts.png)
+
+---
+
+## Put it in your own page
+
+```bash
+npm install @peter.prib/renderengine
+```
+
+The package is the ES module source and the built bundles:
+
+- `import { ArchimateModel, RenderEngine } from '@peter.prib/renderengine'`: the engine and the models (this also
+  imports under Node).
+- `import '@peter.prib/renderengine/workspace'`: the workbench shell and its parts (browser only).
+- `@peter.prib/renderengine/dist/renderengine-workspace.umd.js`: the same as one classic script.
+
+The modeller is a part in the `<ws-workspace>` workbench shell. One script tag and a JSON spec is the whole host
+page:
+
+```html
+<script src="dist/renderengine-workspace.umd.js"></script>
+<script>
+    const { el, registerBuiltinParts, ArchimateCanvasPart } = RenderEngineWorkspace;
+    const ws = document.body.appendChild(el({ tag: 'ws-workspace', attrs: { label: 'Modeller' } }));
+    registerBuiltinParts(ws);
+    ws.registerPart(ArchimateCanvasPart);
+    ws.addParts([{
+        id: 'models', type: 'models', editorType: 'archimate-canvas',   // several models, a tab each
+        models: [{
+            id: 'shop', name: 'Shop',
+            elements: [{ id: 'c', name: 'Customer', type: 'business-actor' }, { id: 'o', name: 'Order', type: 'business-process' }],
+            relationships: [{ source: 'c', target: 'o', kind: 'assignment' }],
+            views: [{ id: 'v', name: 'Main', elementIds: ['c', 'o'], layout: { c: { x: 100, y: 100 }, o: { x: 320, y: 100 } } }],
+        }],
+    }]);
+    ws.registerPerspective('Modelling', { type: 'part', partId: 'models' });
+    ws.activatePerspective('Modelling');
+</script>
+```
+
+Use `editorType: 'diagram-editor'` with `DiagramEditorPart` for the diagram editor, `editorType: 'data-model'`
+with `DataModelPart` for the data modeller, and `editorType: 'requirements'` with `RequirementsPart` for the story
+map. The models are plain JavaScript APIs (`ArchimateModel`, `DiagramModel`, `DataModel`, `RequirementsModel`), so
+you can also build, check, score, search, report on and export models with no UI at all, under Node too:
+`new DataModel(spec).checker().run()`, `new SqlDdl().write(model)`,
+`new RequirementsModel(new TrackerExport().read(csv)).coverage()`.
+
+---
+
+## Documentation
+
+| Document | What it covers |
+|---|---|
+| `documentation/REFERENCE.md` | The API: chart methods and options, the editors, models, commands, file formats |
+| `documentation/WORKSPACE_ARCHITECTURE.md` | The `<ws-workspace>` shell: parts, layout, context bus, work area |
+| `documentation/ARCHITECTURE.md` | Module layout and the chart-type catalog |
+| `documentation/LLM_ARCHIMATE_GUIDE.md` | Generating valid ArchiMate model data, with a prompt template |
+| `documentation/tasks.md` | What's still to come |
+
+## Build and test
+
+```bash
+npm install        # dev tools only: Rollup
+npm run build      # dist/renderengine.umd.js and dist/renderengine-workspace.umd.js
+npm test           # Node's built-in test runner, no browser needed
+npm run screenshots  # this README's pictures, from the example pages (headless Chrome)
+```
+
+There are two bundles:
+
+- `renderengine.umd.js` (global `RenderEngine`): the engine.
+- `renderengine-workspace.umd.js` (global `RenderEngineWorkspace`): the workbench shell and its parts, plus the
+  whole engine API, so one script tag is enough.
+
+The source is plain ES modules. `src/index.js` imports under Node, which is how the tests run.
+
+## Licence
+
+MIT. Contributions welcome.