retold 4.0.1 → 4.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.local.json +38 -1
- package/README.md +92 -2
- package/docs/README.md +7 -6
- package/docs/_sidebar.md +36 -21
- package/docs/_topbar.md +2 -2
- package/docs/architecture/comprehensions.md +282 -0
- package/docs/architecture/fluid-models.md +355 -0
- package/docs/architecture/module-architecture.md +234 -0
- package/docs/{modules.md → architecture/modules.md} +25 -22
- package/docs/cover.md +2 -2
- package/docs/css/docuserve.css +6 -6
- package/docs/examples/examples.md +71 -0
- package/docs/examples/todolist/todo-list-cli-client.md +178 -0
- package/docs/examples/todolist/todo-list-console-client.md +152 -0
- package/docs/examples/todolist/todo-list-model.md +114 -0
- package/docs/examples/todolist/todo-list-server.md +128 -0
- package/docs/examples/todolist/todo-list-web-client.md +177 -0
- package/docs/examples/todolist/todo-list.md +162 -0
- package/docs/getting-started.md +8 -7
- package/docs/index.html +4 -4
- package/docs/{meadow.md → modules/meadow.md} +4 -6
- package/docs/{orator.md → modules/orator.md} +1 -0
- package/docs/{pict.md → modules/pict.md} +30 -8
- package/docs/{utility.md → modules/utility.md} +0 -9
- package/docs/retold-catalog.json +1792 -231
- package/docs/retold-keyword-index.json +136439 -64616
- package/examples/todo-list/Dockerfile +45 -0
- package/examples/todo-list/README.md +394 -0
- package/examples/todo-list/cli-client/package-lock.json +418 -0
- package/examples/todo-list/cli-client/package.json +19 -0
- package/examples/todo-list/cli-client/source/TodoCLI-CLIProgram.js +30 -0
- package/examples/todo-list/cli-client/source/TodoCLI-Run.js +3 -0
- package/examples/todo-list/cli-client/source/commands/add/TodoCLI-Command-Add.js +74 -0
- package/examples/todo-list/cli-client/source/commands/complete/TodoCLI-Command-Complete.js +84 -0
- package/examples/todo-list/cli-client/source/commands/list/TodoCLI-Command-List.js +110 -0
- package/examples/todo-list/cli-client/source/commands/remove/TodoCLI-Command-Remove.js +49 -0
- package/examples/todo-list/cli-client/source/services/TodoCLI-Service-API.js +92 -0
- package/examples/todo-list/console-client/console-client.cjs +913 -0
- package/examples/todo-list/console-client/package-lock.json +426 -0
- package/examples/todo-list/console-client/package.json +19 -0
- package/examples/todo-list/console-client/views/PictView-TUI-Header.cjs +43 -0
- package/examples/todo-list/console-client/views/PictView-TUI-Layout.cjs +58 -0
- package/examples/todo-list/console-client/views/PictView-TUI-StatusBar.cjs +41 -0
- package/examples/todo-list/console-client/views/PictView-TUI-TaskList.cjs +104 -0
- package/examples/todo-list/docker-motd.sh +36 -0
- package/examples/todo-list/docker-run.sh +2 -0
- package/examples/todo-list/docker-shell.sh +2 -0
- package/examples/todo-list/model/MeadowSchema-Task.json +152 -0
- package/examples/todo-list/model/Task-Compiled.json +25 -0
- package/examples/todo-list/model/Task.mddl +15 -0
- package/examples/todo-list/model/data/seeded_todo_events.csv +1001 -0
- package/examples/todo-list/server/database-initialization-service.cjs +273 -0
- package/examples/todo-list/server/package-lock.json +6113 -0
- package/examples/todo-list/server/package.json +19 -0
- package/examples/todo-list/server/server.cjs +138 -0
- package/examples/todo-list/web-client/css/todolist-theme.css +235 -0
- package/examples/todo-list/web-client/generate-build-config.cjs +18 -0
- package/examples/todo-list/web-client/html/index.html +18 -0
- package/examples/todo-list/web-client/package-lock.json +12030 -0
- package/examples/todo-list/web-client/package.json +43 -0
- package/examples/todo-list/web-client/source/TodoList-Application-Config.json +12 -0
- package/examples/todo-list/web-client/source/TodoList-Application.cjs +383 -0
- package/examples/todo-list/web-client/source/providers/Provider-TaskData.cjs +243 -0
- package/examples/todo-list/web-client/source/providers/Router-Config.json +32 -0
- package/examples/todo-list/web-client/source/views/View-Layout.cjs +75 -0
- package/examples/todo-list/web-client/source/views/View-TaskForm.cjs +87 -0
- package/examples/todo-list/web-client/source/views/View-TaskList.cjs +127 -0
- package/examples/todo-list/web-client/source/views/calendar/View-MonthView.cjs +293 -0
- package/examples/todo-list/web-client/source/views/calendar/View-WeekView.cjs +149 -0
- package/examples/todo-list/web-client/source/views/calendar/View-YearView.cjs +226 -0
- package/modules/Include-Retold-Module-List.sh +2 -2
- package/package.json +5 -5
- package/docs/js/pict.min.js +0 -12
- package/docs/js/pict.min.js.map +0 -1
- package/docs/pict-docuserve.min.js +0 -58
- package/docs/pict-docuserve.min.js.map +0 -1
- /package/docs/{architecture.md → architecture/architecture.md} +0 -0
- /package/docs/{fable.md → modules/fable.md} +0 -0
|
@@ -0,0 +1,355 @@
|
|
|
1
|
+
# Fluid Models
|
|
2
|
+
|
|
3
|
+
In most application frameworks, a data model is a rigid artifact. You define it once in a migration file or a class, and it becomes the fixed skeleton of your application. Changing it means writing migration scripts, updating API contracts, rebuilding forms, and hoping nothing breaks.
|
|
4
|
+
|
|
5
|
+
Retold takes a different position: **the model is a living thing**. Data models are easy to describe, easy to mutate, and designed to flow through every layer of the stack — from a terse text definition, through database tables and API endpoints, all the way to browser form fields — adapting their shape at each layer without losing their identity.
|
|
6
|
+
|
|
7
|
+
## Describing a Model
|
|
8
|
+
|
|
9
|
+
The fastest way to describe a data model in Retold is Stricture's MicroDDL — a compact text notation where each line defines a column using a single-character sigil.
|
|
10
|
+
|
|
11
|
+
A classic order management system (think Northwind) might start like this:
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
!Customer
|
|
15
|
+
@IDCustomer
|
|
16
|
+
%GUIDCustomer
|
|
17
|
+
$CompanyName 128
|
|
18
|
+
$ContactName 128
|
|
19
|
+
$ContactTitle 64
|
|
20
|
+
$Phone 24
|
|
21
|
+
$City 64
|
|
22
|
+
$Country 64
|
|
23
|
+
&CreateDate
|
|
24
|
+
#CreatingIDUser
|
|
25
|
+
&UpdateDate
|
|
26
|
+
#UpdatingIDUser
|
|
27
|
+
^Deleted
|
|
28
|
+
&DeleteDate
|
|
29
|
+
#DeletingIDUser
|
|
30
|
+
|
|
31
|
+
!Product
|
|
32
|
+
@IDProduct
|
|
33
|
+
%GUIDProduct
|
|
34
|
+
$ProductName 128
|
|
35
|
+
.UnitPrice 10,2
|
|
36
|
+
#UnitsInStock
|
|
37
|
+
~IDSupplier -> IDSupplier
|
|
38
|
+
&CreateDate
|
|
39
|
+
#CreatingIDUser
|
|
40
|
+
&UpdateDate
|
|
41
|
+
#UpdatingIDUser
|
|
42
|
+
^Deleted
|
|
43
|
+
&DeleteDate
|
|
44
|
+
#DeletingIDUser
|
|
45
|
+
|
|
46
|
+
!Order
|
|
47
|
+
@IDOrder
|
|
48
|
+
%GUIDOrder
|
|
49
|
+
~IDCustomer -> IDCustomer
|
|
50
|
+
&OrderDate
|
|
51
|
+
$ShipCity 64
|
|
52
|
+
$ShipCountry 64
|
|
53
|
+
&CreateDate
|
|
54
|
+
#CreatingIDUser
|
|
55
|
+
&UpdateDate
|
|
56
|
+
#UpdatingIDUser
|
|
57
|
+
^Deleted
|
|
58
|
+
&DeleteDate
|
|
59
|
+
#DeletingIDUser
|
|
60
|
+
|
|
61
|
+
!OrderDetail
|
|
62
|
+
@IDOrderDetail
|
|
63
|
+
%GUIDOrderDetail
|
|
64
|
+
~IDOrder -> IDOrder
|
|
65
|
+
~IDProduct -> IDProduct
|
|
66
|
+
.UnitPrice 10,2
|
|
67
|
+
#Quantity
|
|
68
|
+
.Discount 5,2
|
|
69
|
+
&CreateDate
|
|
70
|
+
#CreatingIDUser
|
|
71
|
+
&UpdateDate
|
|
72
|
+
#UpdatingIDUser
|
|
73
|
+
^Deleted
|
|
74
|
+
&DeleteDate
|
|
75
|
+
#DeletingIDUser
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
The sigils are deliberate shorthand:
|
|
79
|
+
|
|
80
|
+
| Sigil | Type | Example |
|
|
81
|
+
|-------|------|---------|
|
|
82
|
+
| `!` | Table definition | `!Customer` |
|
|
83
|
+
| `@` | Auto-incrementing ID | `@IDCustomer` |
|
|
84
|
+
| `%` | GUID | `%GUIDCustomer` |
|
|
85
|
+
| `$` | String (with size) | `$CompanyName 128` |
|
|
86
|
+
| `#` | Integer | `#UnitsInStock` |
|
|
87
|
+
| `.` | Decimal (precision,scale) | `.UnitPrice 10,2` |
|
|
88
|
+
| `&` | DateTime | `&OrderDate` |
|
|
89
|
+
| `^` | Boolean | `^Deleted` |
|
|
90
|
+
| `~` | Foreign key | `~IDCustomer -> IDCustomer` |
|
|
91
|
+
| `*` | Text (unbounded) | `*Description` |
|
|
92
|
+
|
|
93
|
+
This is the entire data model for four related entities. No XML, no decorators, no class hierarchies. The definition is compact enough to hold in your head and quick enough to change on a whiteboard.
|
|
94
|
+
|
|
95
|
+
## One Definition, Many Shapes
|
|
96
|
+
|
|
97
|
+
From this single MicroDDL source, Stricture generates multiple representations:
|
|
98
|
+
|
|
99
|
+
```mermaid
|
|
100
|
+
graph LR
|
|
101
|
+
mddl["MicroDDL<br/><i>Source of truth</i>"]
|
|
102
|
+
meadow["Meadow Schema<br/><i>JSON column defs,<br/>defaults, validation</i>"]
|
|
103
|
+
mysql["MySQL Queries<br/><i>CREATE TABLE<br/>statements</i>"]
|
|
104
|
+
pict["Pict Config<br/><i>Form layouts,<br/>UI metadata</i>"]
|
|
105
|
+
auth["Authorization<br/><i>Role-based<br/>access rules</i>"]
|
|
106
|
+
|
|
107
|
+
mddl --> meadow
|
|
108
|
+
mddl --> mysql
|
|
109
|
+
mddl --> pict
|
|
110
|
+
mddl --> auth
|
|
111
|
+
|
|
112
|
+
style mddl fill:#f5f5f5,stroke:#bdbdbd,color:#333
|
|
113
|
+
style meadow fill:#fff3e0,stroke:#ffa726,color:#333
|
|
114
|
+
style mysql fill:#fff3e0,stroke:#ffa726,color:#333
|
|
115
|
+
style pict fill:#f3e5f5,stroke:#ab47bc,color:#333
|
|
116
|
+
style auth fill:#e3f2fd,stroke:#42a5f5,color:#333
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Each output is a different shape of the same logical model:
|
|
120
|
+
|
|
121
|
+
**Meadow Schema** — JSON column definitions that drive the ORM, query generation, default objects, and JSON Schema validation:
|
|
122
|
+
|
|
123
|
+
```json
|
|
124
|
+
{
|
|
125
|
+
"Scope": "Customer",
|
|
126
|
+
"DefaultIdentifier": "IDCustomer",
|
|
127
|
+
"Schema": [
|
|
128
|
+
{ "Column": "IDCustomer", "Type": "AutoIdentity" },
|
|
129
|
+
{ "Column": "GUIDCustomer", "Type": "AutoGUID" },
|
|
130
|
+
{ "Column": "CompanyName", "Type": "String", "Size": "128" },
|
|
131
|
+
{ "Column": "ContactName", "Type": "String", "Size": "128" },
|
|
132
|
+
{ "Column": "City", "Type": "String", "Size": "64" },
|
|
133
|
+
{ "Column": "CreateDate", "Type": "CreateDate" },
|
|
134
|
+
{ "Column": "UpdateDate", "Type": "UpdateDate" },
|
|
135
|
+
{ "Column": "Deleted", "Type": "Deleted" }
|
|
136
|
+
],
|
|
137
|
+
"DefaultObject": {
|
|
138
|
+
"IDCustomer": 0,
|
|
139
|
+
"GUIDCustomer": "",
|
|
140
|
+
"CompanyName": "",
|
|
141
|
+
"ContactName": "",
|
|
142
|
+
"City": ""
|
|
143
|
+
}
|
|
144
|
+
}
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
**MySQL DDL** — CREATE TABLE statements ready to run against a database.
|
|
148
|
+
|
|
149
|
+
**Pict Configuration** — UI definitions for form views, list views, and record views — which fields to show, how to group them, what input types to use.
|
|
150
|
+
|
|
151
|
+
## The Model Changes Shape at Every Layer
|
|
152
|
+
|
|
153
|
+
Consider a modern social platform with Users, Posts, and Comments. Watch how the model's shape changes as it flows through the stack — and how Retold makes that natural rather than painful.
|
|
154
|
+
|
|
155
|
+
### At the Schema Layer
|
|
156
|
+
|
|
157
|
+
The MicroDDL is terse and structural:
|
|
158
|
+
|
|
159
|
+
```
|
|
160
|
+
!User
|
|
161
|
+
@IDUser
|
|
162
|
+
%GUIDUser
|
|
163
|
+
$UserName 64
|
|
164
|
+
$Email 256
|
|
165
|
+
$DisplayName 128
|
|
166
|
+
&CreateDate
|
|
167
|
+
#CreatingIDUser
|
|
168
|
+
&UpdateDate
|
|
169
|
+
#UpdatingIDUser
|
|
170
|
+
^Deleted
|
|
171
|
+
&DeleteDate
|
|
172
|
+
#DeletingIDUser
|
|
173
|
+
|
|
174
|
+
!Post
|
|
175
|
+
@IDPost
|
|
176
|
+
%GUIDPost
|
|
177
|
+
~IDUser -> IDUser
|
|
178
|
+
$Title 256
|
|
179
|
+
*Body
|
|
180
|
+
&PublishedDate
|
|
181
|
+
&CreateDate
|
|
182
|
+
#CreatingIDUser
|
|
183
|
+
&UpdateDate
|
|
184
|
+
#UpdatingIDUser
|
|
185
|
+
^Deleted
|
|
186
|
+
&DeleteDate
|
|
187
|
+
#DeletingIDUser
|
|
188
|
+
|
|
189
|
+
!Comment
|
|
190
|
+
@IDComment
|
|
191
|
+
%GUIDComment
|
|
192
|
+
~IDPost -> IDPost
|
|
193
|
+
~IDUser -> IDUser
|
|
194
|
+
*Body
|
|
195
|
+
&CreateDate
|
|
196
|
+
#CreatingIDUser
|
|
197
|
+
&UpdateDate
|
|
198
|
+
#UpdatingIDUser
|
|
199
|
+
^Deleted
|
|
200
|
+
&DeleteDate
|
|
201
|
+
#DeletingIDUser
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### At the Database Layer
|
|
205
|
+
|
|
206
|
+
Meadow consumes the schema and automatically handles audit columns. When you create a Post, you send:
|
|
207
|
+
|
|
208
|
+
```javascript
|
|
209
|
+
{ Title: 'Hello World', Body: 'First post!', IDUser: 42 }
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Meadow merges your data with the schema's default object and fills in the audit columns:
|
|
213
|
+
|
|
214
|
+
```javascript
|
|
215
|
+
{
|
|
216
|
+
IDPost: 0, // from defaults
|
|
217
|
+
GUIDPost: '8f14e45f-...', // auto-generated
|
|
218
|
+
Title: 'Hello World', // from your data
|
|
219
|
+
Body: 'First post!', // from your data
|
|
220
|
+
IDUser: 42, // from your data
|
|
221
|
+
PublishedDate: null, // from defaults
|
|
222
|
+
CreateDate: '2025-02-17 14:30:00', // auto-stamped
|
|
223
|
+
CreatingIDUser: 42, // auto-stamped from session
|
|
224
|
+
UpdateDate: '2025-02-17 14:30:00', // auto-stamped
|
|
225
|
+
UpdatingIDUser: 42, // auto-stamped from session
|
|
226
|
+
Deleted: 0 // from defaults
|
|
227
|
+
}
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
The schema knows which columns are audit columns by their type (`CreateDate`, `UpdateDate`, `Deleted`). Meadow inspects the schema on every operation — not just at startup — so changing the schema changes behavior immediately.
|
|
231
|
+
|
|
232
|
+
### At the Query Layer
|
|
233
|
+
|
|
234
|
+
FoxHound reads the schema to generate intelligent queries. If the schema includes a `Deleted` column with type `Deleted`, every read query automatically adds `WHERE Deleted = 0`. You never write that filter. The schema declares the intent; the query layer implements it.
|
|
235
|
+
|
|
236
|
+
```javascript
|
|
237
|
+
// You write:
|
|
238
|
+
_PostMeadow.doReads(_PostMeadow.query.addFilter('IDUser', 42), fCallback);
|
|
239
|
+
|
|
240
|
+
// FoxHound generates (simplified):
|
|
241
|
+
// SELECT * FROM Post WHERE IDUser = 42 AND Deleted = 0
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
### At the API Layer
|
|
245
|
+
|
|
246
|
+
Meadow-Endpoints reads the same schema and generates a full REST API. The schema's authorization rules control who can do what:
|
|
247
|
+
|
|
248
|
+
```
|
|
249
|
+
GET /Posts → list (filtered by Deleted automatically)
|
|
250
|
+
GET /Post/42 → read single
|
|
251
|
+
POST /Post → create (audit columns auto-stamped)
|
|
252
|
+
PUT /Post → update (UpdateDate auto-stamped)
|
|
253
|
+
DEL /Post/42 → soft delete (sets Deleted = 1)
|
|
254
|
+
GET /Post/Schema → returns the live schema JSON
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
The `GET /Post/Schema` endpoint is important — it exposes the current schema to any client. This means the model is not hidden inside server code. It is a queryable, inspectable resource.
|
|
258
|
+
|
|
259
|
+
### At the UI Layer
|
|
260
|
+
|
|
261
|
+
Pict reads the schema through Manyfest descriptors and generates form fields. Each column gains UI metadata:
|
|
262
|
+
|
|
263
|
+
```javascript
|
|
264
|
+
{
|
|
265
|
+
Hash: 'Title',
|
|
266
|
+
Name: 'Post Title',
|
|
267
|
+
DataType: 'String',
|
|
268
|
+
PictForm: {
|
|
269
|
+
InputType: 'TextInput',
|
|
270
|
+
Section: 'PostDetails',
|
|
271
|
+
Row: 0
|
|
272
|
+
}
|
|
273
|
+
}
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
The same address notation — `Record.Title`, `AppData.Posts[0].Body` — works in templates, in code, and in form bindings. Manyfest provides safe navigation that never throws on missing paths, so the UI does not break when the model evolves.
|
|
277
|
+
|
|
278
|
+
## Runtime Mutation
|
|
279
|
+
|
|
280
|
+
The model is not frozen after compilation. Meadow schemas are live objects with independent, replaceable parts:
|
|
281
|
+
|
|
282
|
+
```javascript
|
|
283
|
+
// Load initial schema
|
|
284
|
+
_PostMeadow.loadFromPackage('./Post-Schema.json');
|
|
285
|
+
|
|
286
|
+
// Later, update just the column definitions
|
|
287
|
+
_PostMeadow.setSchema(updatedColumnArray);
|
|
288
|
+
|
|
289
|
+
// Or just the validation rules
|
|
290
|
+
_PostMeadow.setJsonSchema(updatedValidationSchema);
|
|
291
|
+
|
|
292
|
+
// Or just the default values
|
|
293
|
+
_PostMeadow.setDefault({ ...existingDefaults, NewField: 'default' });
|
|
294
|
+
|
|
295
|
+
// Or just the authorization policies
|
|
296
|
+
_PostMeadow.setAuthorizer(updatedAuthRules);
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
Each part — column definitions, JSON Schema validation, default objects, authorization — can be modified independently. When you call `setSchema()`, the change propagates immediately to the database provider. Subsequent queries, creates, and updates use the new definition. There is no restart, no cache flush, no migration ceremony.
|
|
300
|
+
|
|
301
|
+
This matters for real applications. A bookstore that adds a `Price` column to its `Book` entity:
|
|
302
|
+
|
|
303
|
+
```javascript
|
|
304
|
+
// Existing schema
|
|
305
|
+
let schema = _BookMeadow.schema;
|
|
306
|
+
|
|
307
|
+
// Add the new column
|
|
308
|
+
schema.push({ Column: 'Price', Type: 'Decimal', Size: '10,2' });
|
|
309
|
+
|
|
310
|
+
// Update live
|
|
311
|
+
_BookMeadow.setSchema(schema);
|
|
312
|
+
_BookMeadow.setDefault({ ...existingDefaults, Price: 0.0 });
|
|
313
|
+
|
|
314
|
+
// All subsequent operations include Price
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
The database table still needs an `ALTER TABLE` — Retold does not run migrations for you. But the application layer adapts instantly. You can evolve the schema in code, test the new shape, and apply the database change when ready.
|
|
318
|
+
|
|
319
|
+
## Address-Based Access Across Layers
|
|
320
|
+
|
|
321
|
+
Manyfest provides the glue that makes model fluidity practical. Every piece of data in a Retold application is reachable through an address — a dot-notation string like `Record.Contact.Email` or `AppData.Posts[0].Title`.
|
|
322
|
+
|
|
323
|
+
```javascript
|
|
324
|
+
// Safe read — returns undefined if any part of the path is missing
|
|
325
|
+
let title = _Manyfest.getValueAtAddress(pRecord, 'Posts[0].Title');
|
|
326
|
+
|
|
327
|
+
// Safe write — creates intermediate objects as needed
|
|
328
|
+
_Manyfest.setValueAtAddress(pRecord, 'Posts[0].Title', 'Updated Title');
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
The same address notation works in Pict templates:
|
|
332
|
+
|
|
333
|
+
```
|
|
334
|
+
{~Data:Record.Title~}
|
|
335
|
+
{~Data:AppData.Posts[0].Title~}
|
|
336
|
+
{~Data:Bundle.Authors.ByID.42.Name~}
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
And in form bindings, where the template address maps directly to a Manyfest descriptor, which maps to a Meadow column, which maps to a database field. One notation from top to bottom.
|
|
340
|
+
|
|
341
|
+
When the model changes shape — a field is renamed, a nested object is restructured, a new relationship is added — the address is the single point of update. Change the address in the Manyfest descriptor and every layer that references it adapts.
|
|
342
|
+
|
|
343
|
+
## Why This Matters
|
|
344
|
+
|
|
345
|
+
In a traditional framework, changing a data model means touching every layer: migration files, ORM classes, API serializers, validation schemas, form definitions, and template bindings. Each layer has its own notation for describing the same thing, and keeping them synchronized is the source of a large class of bugs.
|
|
346
|
+
|
|
347
|
+
Retold's approach is different:
|
|
348
|
+
|
|
349
|
+
- **One source** (MicroDDL) generates all layer-specific representations
|
|
350
|
+
- **One notation** (Manyfest addresses) accesses data everywhere
|
|
351
|
+
- **One schema object** (Meadow) drives queries, validation, defaults, and authorization
|
|
352
|
+
- **Live mutation** means the model evolves without restarts or redeployment ceremonies
|
|
353
|
+
- **Schema introspection** at operation time means changes take effect immediately
|
|
354
|
+
|
|
355
|
+
The model is not a static artifact that you deploy and forget. It is a fluid description that adapts to each layer's needs and evolves with your application.
|
|
@@ -0,0 +1,234 @@
|
|
|
1
|
+
# Ecosystem Architecture - Module Philosophy
|
|
2
|
+
|
|
3
|
+
Retold modules each embody a specific design philosophy. Together they form a coherent system for building applications — from the stateless service core, through data access and API generation, up to state-driven UI rendering. At the same time, each module is designed to be used (*and useful*) independently. You can adopt Fable's configuration and logging without using Pict. You can use Meadow for data access without Orator. These modules are decoupled by design, but also fit together in a complementary way.
|
|
4
|
+
|
|
5
|
+
```mermaid
|
|
6
|
+
graph TB
|
|
7
|
+
subgraph Clients["Client Applications"]
|
|
8
|
+
direction LR
|
|
9
|
+
browser["Browser App<br/><i>Pict + Pict-Application<br/>+ Pict-Views</i>"]
|
|
10
|
+
console["Console App<br/><i>Pict + Terminal UI</i>"]
|
|
11
|
+
cli["CLI Tool<br/><i>CommandLineUtility</i>"]
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
internet(("The Internet<br/>☁"))
|
|
15
|
+
|
|
16
|
+
subgraph Server["Server / Mid-Tier"]
|
|
17
|
+
direction TB
|
|
18
|
+
orator["Orator<br/><i>HTTP Server Abstraction</i>"]
|
|
19
|
+
endpoints["Meadow-Endpoints<br/><i>Auto-generated REST API</i>"]
|
|
20
|
+
meadow["Meadow<br/><i>Data Access Abstraction</i>"]
|
|
21
|
+
|
|
22
|
+
subgraph DataTools["Schema & Query Tools"]
|
|
23
|
+
direction LR
|
|
24
|
+
foxhound["FoxHound<br/><i>Query Generation</i>"]
|
|
25
|
+
stricture["Stricture<br/><i>MicroDDL → Schema</i>"]
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
orator --> endpoints
|
|
29
|
+
endpoints --> meadow
|
|
30
|
+
meadow --> DataTools
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
subgraph Foundation["Foundation (Stateless)"]
|
|
34
|
+
direction LR
|
|
35
|
+
fable["Fable<br/><i>DI, Config, Logging</i>"]
|
|
36
|
+
manyfest["Manyfest<br/><i>Address-Based<br/>Object Access</i>"]
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
Clients -- "HTTP requests" --> internet
|
|
40
|
+
internet -- "HTTP requests" --> orator
|
|
41
|
+
|
|
42
|
+
browser -. "Fable services" .-> Foundation
|
|
43
|
+
Server --> Foundation
|
|
44
|
+
|
|
45
|
+
style Clients fill:#e8f5e9,stroke:#43a047,color:#333
|
|
46
|
+
style browser fill:#fff,stroke:#66bb6a,color:#333
|
|
47
|
+
style console fill:#fff,stroke:#66bb6a,color:#333
|
|
48
|
+
style cli fill:#fff,stroke:#66bb6a,color:#333
|
|
49
|
+
style internet fill:#e1f5fe,stroke:#03a9f4,color:#333,stroke-width:2px
|
|
50
|
+
style Server fill:#e3f2fd,stroke:#42a5f5,color:#333
|
|
51
|
+
style orator fill:#fff,stroke:#64b5f6,color:#333
|
|
52
|
+
style endpoints fill:#fff,stroke:#64b5f6,color:#333
|
|
53
|
+
style meadow fill:#fff3e0,stroke:#ffa726,color:#333
|
|
54
|
+
style DataTools fill:#fff8e1,stroke:#ffcc80,color:#333
|
|
55
|
+
style foxhound fill:#fff,stroke:#ffcc80,color:#333
|
|
56
|
+
style stricture fill:#fff,stroke:#ffcc80,color:#333
|
|
57
|
+
style Foundation fill:#fce4ec,stroke:#ef5350,color:#333
|
|
58
|
+
style fable fill:#fff,stroke:#ef9a9a,color:#333
|
|
59
|
+
style manyfest fill:#fff,stroke:#ef9a9a,color:#333
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Clients — browsers, terminals, CLI tools — communicate with the server over HTTP. Orator receives those requests, Meadow-Endpoints maps them to data operations, and Meadow executes them against the database. Fable provides the stateless service container that every module on both sides of the network depends on. Manyfest provides the consistent data-addressing language used everywhere.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## Fable Is Stateless
|
|
67
|
+
|
|
68
|
+
Fable draws a hard line between **configuration** and **state**.
|
|
69
|
+
|
|
70
|
+
In Retold's vocabulary, *state* is **application data** — the records in a database, the items in a shopping cart, the text a user has typed into a form. State changes as users interact with the system. It is the core subject matter of the application.
|
|
71
|
+
|
|
72
|
+
Configuration is everything else: the address of the database server, the port the API listens on, the log level, the product name. Configuration describes how the application runs in its environment, not what the application is doing. Fable manages configuration. It does not manage state.
|
|
73
|
+
|
|
74
|
+
This distinction matters because the "Out of the Tar Pit" problem — the paper Fable's design references — identifies uncontrolled state as the primary source of software complexity. By keeping the service container stateless, Fable ensures that the foundation layer has no opinion about your data. Services register with Fable, receive configuration and logging, and operate on whatever data flows through them without Fable holding onto it.
|
|
75
|
+
|
|
76
|
+
Every Retold module extends `fable-serviceproviderbase` and registers with a Fable instance. That instance provides dependency injection, logging, UUID generation, and shared settings. But the Fable instance itself stores no application data. It is the wiring, not the warehouse.
|
|
77
|
+
|
|
78
|
+
## Pict Is for State and Transformation
|
|
79
|
+
|
|
80
|
+
If Fable is deliberately stateless, Pict is deliberately stateful. Pict's job is to hold application state and transform it consistently into output using a templating language.
|
|
81
|
+
|
|
82
|
+
Pict provides three dedicated state containers:
|
|
83
|
+
|
|
84
|
+
| Container | Purpose | Example Contents |
|
|
85
|
+
|-----------|---------|------------------|
|
|
86
|
+
| **AppData** | Primary application state | User records, form values, loaded entities |
|
|
87
|
+
| **Bundle** | Supporting reference data | Lookup tables, dropdown options, translations |
|
|
88
|
+
| **TempData** | Ephemeral intermediate data | Chart caches, calculated summaries, UI flags |
|
|
89
|
+
|
|
90
|
+
All state lives in known locations. Templates reference state through address expressions like `AppData.Tasks[0].Name`. Code accesses the same state through the same addresses. There is one source of truth and one notation for reaching it.
|
|
91
|
+
|
|
92
|
+
The key insight is that **state and lifecycle are coupled**. Pict's lifecycle phases — Solve, Render, Marshal — all operate on these state containers:
|
|
93
|
+
|
|
94
|
+
1. **Solve** reads state and writes derived values back into state
|
|
95
|
+
2. **Render** reads state and produces output through templates
|
|
96
|
+
3. **Marshal** collects input and writes it back into state
|
|
97
|
+
|
|
98
|
+
This cycle is predictable and debuggable. At any point in the lifecycle you can inspect the state containers and know exactly what data the application is working with.
|
|
99
|
+
|
|
100
|
+
Pict intentionally separates logic from templates. Templates describe *what* to render; code (in views and providers) describes *when* and *how* data moves. You can override templates without altering behavior, and you can alter behavior without touching templates.
|
|
101
|
+
|
|
102
|
+
## Pict-Application Manages the Lifecycle
|
|
103
|
+
|
|
104
|
+
Pict-Application is the orchestration layer. It coordinates the startup, data loading, solving, rendering, and marshaling of an entire application composed of multiple views and providers.
|
|
105
|
+
|
|
106
|
+
The lifecycle follows a defined sequence:
|
|
107
|
+
|
|
108
|
+
```mermaid
|
|
109
|
+
graph LR
|
|
110
|
+
init["Initialize<br/><i>Register views,<br/>providers, templates</i>"]
|
|
111
|
+
data["Load Data<br/><i>Fetch from APIs,<br/>populate AppData</i>"]
|
|
112
|
+
solve["Solve<br/><i>Calculate derived<br/>values</i>"]
|
|
113
|
+
render["Render<br/><i>Transform state<br/>into output</i>"]
|
|
114
|
+
marshal["Marshal<br/><i>Collect input<br/>back into state</i>"]
|
|
115
|
+
|
|
116
|
+
init --> data --> solve --> render --> marshal
|
|
117
|
+
marshal -. "user interaction" .-> solve
|
|
118
|
+
|
|
119
|
+
style init fill:#fce4ec,stroke:#ef5350,color:#333
|
|
120
|
+
style data fill:#fff3e0,stroke:#ffa726,color:#333
|
|
121
|
+
style solve fill:#e8f5e9,stroke:#43a047,color:#333
|
|
122
|
+
style render fill:#e3f2fd,stroke:#42a5f5,color:#333
|
|
123
|
+
style marshal fill:#f3e5f5,stroke:#ab47bc,color:#333
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Each view and provider within the application participates in these phases. Pict-Application ensures they execute in the correct order (controlled by configurable ordinals) and provides auto-behaviors to reduce boilerplate — for example, automatically solving and rendering after initialization completes.
|
|
127
|
+
|
|
128
|
+
The application also manages authentication flows, data loading sequences, and the coordination between views that need to share state. It is the conductor; views and providers are the musicians.
|
|
129
|
+
|
|
130
|
+
## Pict-View Is Not Just a Screen
|
|
131
|
+
|
|
132
|
+
A Pict view is any representation of information. It is not a page. It is not a screen. It is not a route.
|
|
133
|
+
|
|
134
|
+
A single screen might contain dozens of views: a header view, a navigation view, a list view, a detail panel view, a status bar view. Or a single view might render to a log file, a terminal widget, or a test harness instead of a browser DOM. The view is the unit of *rendering*, not the unit of *navigation*.
|
|
135
|
+
|
|
136
|
+
Each view contains **renderables** — individual render instructions that specify:
|
|
137
|
+
|
|
138
|
+
- Which **template** to use
|
|
139
|
+
- Which **data address** to read from state
|
|
140
|
+
- Which **destination** to render into
|
|
141
|
+
- Which **method** to use (replace, append, prepend)
|
|
142
|
+
|
|
143
|
+
This makes views composable. Small views handle small concerns. Larger patterns emerge from combining them. A form section view renders input fields. A recordset view renders a list of records. A content view renders static markup. You assemble an application from these building blocks.
|
|
144
|
+
|
|
145
|
+
Because views render through a content assignment abstraction, they are not bound to the browser DOM. The same view code can render to a blessed terminal widget, a log stream, or a mock environment for testing — by swapping the content assignment functions. The view does not know or care where its output ends up.
|
|
146
|
+
|
|
147
|
+
## Orator Abstracts Web Servers
|
|
148
|
+
|
|
149
|
+
Orator provides a thin abstraction over HTTP servers. Your application code interacts with Orator's interface — defining routes, middleware, and lifecycle hooks — without coupling to a specific server implementation.
|
|
150
|
+
|
|
151
|
+
The default implementation uses Restify (`orator-serviceserver-restify`), but the abstraction means you could swap in a different HTTP library or use IPC mode for testing without changing application routes or middleware. Orator handles the HTTP lifecycle: receiving requests, running middleware, dispatching to handlers, and sending responses.
|
|
152
|
+
|
|
153
|
+
Orator also provides static file serving (`orator-static-server`) and reverse proxy capabilities (`orator-http-proxy`), making it a complete server-side HTTP toolkit without being a heavyweight framework.
|
|
154
|
+
|
|
155
|
+
The philosophy is deliberate thinness. Orator does not dictate application structure. It provides the plumbing for getting HTTP requests to your code and responses back to the client.
|
|
156
|
+
|
|
157
|
+
## Meadow Abstracts Data Access
|
|
158
|
+
|
|
159
|
+
Meadow is a provider-agnostic data broker. You define entities once — their fields, types, and relationships — and Meadow handles CRUD operations against whatever database is connected.
|
|
160
|
+
|
|
161
|
+
The abstraction has real teeth: the same Meadow entity definition works with MySQL, MSSQL, SQLite, and ALASQL (for in-browser use). Switch databases by swapping a connection module. Your application code, entity definitions, and endpoint configurations do not change.
|
|
162
|
+
|
|
163
|
+
Meadow automatically manages audit columns (who created a record, who last updated it, and when), soft deletes (marking records as deleted without removing them), GUID-based uniqueness, and data marshalling between JavaScript objects and database rows.
|
|
164
|
+
|
|
165
|
+
The data access pattern is deliberately simple: Create, Read, Reads (list), Update, Delete, Count, and Undelete. These seven operations cover the vast majority of data access needs. When they do not, behavior hooks let you inject custom logic at any point in the operation lifecycle.
|
|
166
|
+
|
|
167
|
+
## Meadow-Endpoints Connects Data Access to a Consistent API
|
|
168
|
+
|
|
169
|
+
Meadow-Endpoints takes a Meadow entity and automatically generates a full REST API for it. Define an entity called `Book` and you immediately get `GET /Books`, `GET /Book/:id`, `POST /Book`, `PUT /Book`, `DEL /Book/:id`, and more — with filtering, pagination, sorting, and schema introspection built in.
|
|
170
|
+
|
|
171
|
+
This is not code generation that produces files you maintain. The endpoints are generated at runtime from the entity definition. Change the schema, restart the server, and the API updates. Add a new entity, wire it to Meadow-Endpoints, and the routes appear.
|
|
172
|
+
|
|
173
|
+
Behavior hooks provide the extension points. Need authentication? Add a `before` hook. Need to transform data before it reaches the client? Add an `after` hook. Need custom validation? Hook into the create and update paths. The auto-generated API is the foundation; hooks let you customize without replacing it.
|
|
174
|
+
|
|
175
|
+
## FoxHound Generates Queries
|
|
176
|
+
|
|
177
|
+
FoxHound is the query generation engine inside Meadow. It provides a chainable API for building queries — adding filters, setting sort order, configuring pagination — and then generates the correct SQL dialect for whichever database you are using.
|
|
178
|
+
|
|
179
|
+
```javascript
|
|
180
|
+
_Query.addFilter('Status', 'Complete')
|
|
181
|
+
.setSort('CreatedDate')
|
|
182
|
+
.setBegin(0).setCap(50)
|
|
183
|
+
.buildReadQuery();
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
This same code produces valid MySQL, MSSQL, SQLite, or ALASQL depending on the dialect. Application code never writes raw SQL. FoxHound's job is to be the single translation layer between a database-agnostic query description and dialect-specific SQL.
|
|
187
|
+
|
|
188
|
+
FoxHound also powers the **FilteredTo** URL syntax used by Meadow-Endpoints, which encodes filters, sorts, and pagination into URL paths for GET requests. This means clients can express complex queries through standard HTTP URLs without POST bodies.
|
|
189
|
+
|
|
190
|
+
## Stricture Transforms MicroDDL into Schemas
|
|
191
|
+
|
|
192
|
+
Stricture provides a compact notation — MicroDDL — for defining data models. A MicroDDL file is a human-readable text format where each line defines a column using single-character sigils:
|
|
193
|
+
|
|
194
|
+
```
|
|
195
|
+
!ID
|
|
196
|
+
@GUIDTask
|
|
197
|
+
#Name
|
|
198
|
+
#Description
|
|
199
|
+
#Status
|
|
200
|
+
&Hours
|
|
201
|
+
%Due
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
From this terse input, Stricture generates:
|
|
205
|
+
|
|
206
|
+
- **JSON schemas** for Meadow entity definitions
|
|
207
|
+
- **CREATE TABLE statements** for MySQL, MSSQL, or SQLite
|
|
208
|
+
- **Documentation** describing the data model
|
|
209
|
+
- **Seed data templates** for testing
|
|
210
|
+
|
|
211
|
+
The philosophy is *define once, generate everywhere*. The MicroDDL is the single source of truth for a data model. Every downstream artifact — database tables, API schemas, documentation — derives from it. Change the MicroDDL, regenerate, and all representations stay in sync.
|
|
212
|
+
|
|
213
|
+
## Manyfest Provides Address-Based Object Access
|
|
214
|
+
|
|
215
|
+
Manyfest solves a recurring problem: the same data structure is described and accessed differently at every layer of an application. The database has column names. The API has JSON keys. The frontend has display labels. Business logic has domain terms.
|
|
216
|
+
|
|
217
|
+
Manyfest unifies this through **address-based access**. An address is a string like `Record.Contact.Email` that describes a location in a nested JavaScript object. Manyfest provides safe accessors that navigate the address without throwing exceptions on missing intermediate objects:
|
|
218
|
+
|
|
219
|
+
```javascript
|
|
220
|
+
// Safe read — returns undefined if any part of the path is missing
|
|
221
|
+
let email = _Manyfest.getValueAtAddress(pRecord, 'Contact.Email');
|
|
222
|
+
|
|
223
|
+
// Safe write — creates intermediate objects as needed
|
|
224
|
+
_Manyfest.setValueAtAddress(pRecord, 'Contact.Email', 'new@example.com');
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Beyond safe access, Manyfest provides **descriptors** — metadata that maps addresses to human-readable names, short names, descriptions, data types, and hashes. This means a single Manyfest definition can drive:
|
|
228
|
+
|
|
229
|
+
- API field documentation
|
|
230
|
+
- Form field labels and validation
|
|
231
|
+
- Report column headers
|
|
232
|
+
- Data transformation between layers
|
|
233
|
+
|
|
234
|
+
The address notation is the same notation Pict uses for template expressions and state access. This consistency means a Manyfest schema defined for the data layer can flow all the way through to form labels in the browser without translation.
|