meadow-integration 1.0.20 → 1.0.21

package/example-applications/mapping-demo/.quackage.json

@@ -0,0 +1,10 @@
+{
+	"GulpfileConfiguration": {
+		"EntrypointInputSourceFile": "{~Data:AppData.CWD~}/source/MappingDemoApp.js",
+		"LibraryObjectName": "MappingDemoApplication",
+		"LibraryOutputFolder": "{~Data:AppData.CWD~}/web/",
+		"LibraryUniminifiedFileName": "mapping-demo-editor.js",
+		"LibraryMinifiedFileName": "mapping-demo-editor.min.js",
+		"BrowserifyIgnore": []
+	}
+}

package/example-applications/mapping-demo/README.md

@@ -0,0 +1,99 @@
+# Mapping Demo
+
+A self-contained example application that demonstrates the full
+`meadow-integration` data pipeline:
+
+```
+CSV file  →  Parse  →  Map  →  TabularTransform  →  Comprehension  →  IntegrationAdapter  →  Meadow DB
+```
+
+## What it demonstrates
+
+| Stage | Component | Description |
+|---|---|---|
+| Parse | CSV reader | Reads `data/books-sample.csv` into raw record objects |
+| Map | Mapping JSON | Declares how CSV columns become entity fields via Pict templates |
+| Transform | `TabularTransform` | Applies the mapping to every record, produces a GUID-keyed comprehension |
+| Load | `IntegrationAdapter` | Upserts comprehension records into a Meadow REST API (idempotent) |
+| Verify | `meadow-endpoints` | Reads records back through the auto-generated Book REST API |
+
+The demo uses an **in-memory SQLite database** so nothing is written to disk.
+
+## Running
+
+From the `meadow-integration` module root, install dependencies once:
+
+```sh
+npm install
+```
+
+Then start the demo server:
+
+```sh
+cd example-applications/mapping-demo
+node server.js
+```
+
+Open **http://localhost:8092/** in your browser and click through each pipeline step.
+
+## Endpoints
+
+| Method | Path | Description |
+|---|---|---|
+| `GET` | `/` | Pipeline demo web UI |
+| `GET` | `/1.0/Demo/Status` | Server status and endpoint list |
+| `GET` | `/1.0/Demo/SampleData` | Raw parsed CSV records |
+| `GET` | `/1.0/Demo/Mapping` | Current mapping configuration |
+| `POST` | `/1.0/Demo/Transform` | Run mapping → comprehension |
+| `POST` | `/1.0/Demo/Load` | Push comprehension via IntegrationAdapter |
+| `GET` | `/1.0/Demo/Books` | Read loaded books from database |
+| `GET` | `/1.0/Books/0/20` | Meadow-Endpoints live Book list |
+| `GET` | `/1.0/Books/Count` | Record count |
+
+## Sample data
+
+`data/books-sample.csv` contains 20 well-known books with these columns:
+
+```
+id, title, original_publication_year, isbn, language_code
+```
+
+## Mapping configuration
+
+`mappings/books-to-book.json` maps the CSV columns to the `Book` entity
+(from the retold-harness bookstore schema):
+
+```json
+{
+    "Entity": "Book",
+    "GUIDTemplate": "DemoBook-{~D:Record.id~}",
+    "Mappings": {
+        "Title":           "{~D:Record.title~}",
+        "Language":        "{~D:Record.language_code~}",
+        "PublicationYear": "{~D:Record.original_publication_year~}",
+        "ISBN":            "{~D:Record.isbn~}",
+        "Genre":           "Classic",
+        "Type":            "Fiction",
+        "ImageURL":        ""
+    }
+}
+```
+
+The `GUIDTemplate` uses `Record.id` from the source CSV to generate a stable
+external GUID (`DemoBook-1`, `DemoBook-2`, …). The `IntegrationAdapter`
+tracks these in a `GUIDMap` so repeat loads are safe upserts.
+
+## Architecture
+
+```
+server.js
+├── SQLite (in-memory)          ← meadow-connection-sqlite
+├── Meadow DAL (Book entity)    ← meadow + inline schema
+├── meadow-endpoints (Book)     ← auto-generates /1.0/Book(s) REST routes
+├── Demo pipeline endpoints     ← /1.0/Demo/*
+│   ├── TabularTransform        ← parse + map = comprehension
+│   └── IntegrationAdapter      ← comprehension → upsert via REST
+└── Static web UI               ← web/index.html
+```
+
+All components run on a single Orator (Restify) server at port 8092.

package/example-applications/mapping-demo/data/books-sample.csv

@@ -0,0 +1,21 @@
+id,title,original_publication_year,isbn,language_code
+1,The Hunger Games,2008,439023483,eng
+2,Harry Potter and the Sorcerers Stone,1997,439554934,eng
+3,Twilight,2005,316015849,en-US
+4,To Kill a Mockingbird,1960,61120081,eng
+5,The Great Gatsby,1925,743273567,eng
+6,The Fault in Our Stars,2012,525478817,eng
+7,The Hobbit,1937,547928227,eng
+8,The Hitchhikers Guide to the Galaxy,1979,345391802,eng
+9,Pride and Prejudice,1813,679783261,eng
+10,Brave New World,1932,60929871,eng
+11,1984,1949,451524934,eng
+12,One Hundred Years of Solitude,1967,060883287,spa
+13,Animal Farm,1945,452284244,eng
+14,Fahrenheit 451,1953,1451673319,eng
+15,The Catcher in the Rye,1951,316769177,eng
+16,Lord of the Flies,1954,571056865,eng
+17,Dune,1965,441013597,eng
+18,A Brief History of Time,1988,553380163,eng
+19,Don Quixote,1605,60934344,spa
+20,Crime and Punishment,1866,14044913,rus

package/example-applications/mapping-demo/generate-build-config.js

@@ -0,0 +1,44 @@
+'use strict';
+
+/**
+ * Copies pict.min.js from node_modules into web/ so the server can serve it.
+ * The quackage build config comes from .quackage.json — no manual generation needed.
+ *
+ * Run automatically via `npm run build` before starting the demo.
+ */
+
+const libPath = require('path');
+const libFS = require('fs');
+
+// ── Copy pict.min.js from node_modules into web/ ──────────────────────────────
+
+const tmpPictDistCandidates =
+[
+	libPath.resolve(__dirname, 'node_modules/pict/dist/pict.min.js'),
+	libPath.resolve(__dirname, '../../node_modules/pict/dist/pict.min.js')
+];
+
+let tmpPictMinSrc = null;
+for (let i = 0; i < tmpPictDistCandidates.length; i++)
+{
+	if (libFS.existsSync(tmpPictDistCandidates[i]))
+	{
+		tmpPictMinSrc = tmpPictDistCandidates[i];
+		break;
+	}
+}
+
+if (tmpPictMinSrc)
+{
+	const tmpWebDir = libPath.resolve(__dirname, 'web');
+	if (!libFS.existsSync(tmpWebDir))
+	{
+		libFS.mkdirSync(tmpWebDir, { recursive: true });
+	}
+	libFS.copyFileSync(tmpPictMinSrc, libPath.join(tmpWebDir, 'pict.min.js'));
+	console.log('Copied pict.min.js to web/');
+}
+else
+{
+	console.warn('WARNING: Could not find pict/dist/pict.min.js — run npm install in meadow-integration first.');
+}

package/example-applications/mapping-demo/mappings/books-to-book.json

@@ -0,0 +1,14 @@
+{
+	"Entity": "Book",
+	"GUIDTemplate": "DemoBook-{~D:Record.id~}",
+	"Mappings":
+	{
+		"Title": "{~D:Record.title~}",
+		"Language": "{~D:Record.language_code~}",
+		"PublicationYear": "{~D:Record.original_publication_year~}",
+		"ISBN": "{~D:Record.isbn~}",
+		"Genre": "Classic",
+		"Type": "Fiction",
+		"ImageURL": ""
+	}
+}

package/example-applications/mapping-demo/package.json

@@ -0,0 +1,14 @@
+{
+	"name": "mapping-demo",
+	"version": "1.0.0",
+	"description": "Meadow-Integration pipeline demo: parse → map → visual map → comprehension → load",
+	"main": "server.js",
+	"scripts": {
+		"start": "node server.js",
+		"build": "node generate-build-config.js && ../../node_modules/.bin/quack build",
+		"demo": "npm run build && node server.js"
+	},
+	"author": "steven velozo <steven@velozo.com>",
+	"license": "MIT",
+	"dependencies": {}
+}