@objectstack/metadata 4.0.4 → 4.1.0

package/README.md

@@ -2,7 +2,7 @@
 
 > **Metadata Loading, Persistence & Customization Layer for ObjectStack.**
 
-`@objectstack/metadata` is the central service responsible for loading, validating, persisting and watching all metadata definitions (Objects, Views, Flows, Apps, Agents, etc.) in the ObjectStack platform.
+`@objectstack/metadata` is the central service responsible for loading, validating, optionally persisting, and watching all metadata definitions (Objects, Views, Flows, Apps, Agents, etc.) in the ObjectStack platform.
 
 It implements the **`IMetadataService`** contract from `@objectstack/spec` and acts as the single source of truth that all other packages depend on.
 
@@ -40,17 +40,22 @@ It implements the **`IMetadataService`** contract from `@objectstack/spec` and a
 
 ## Core Concepts
 
-### 1. Metadata Sources (Three-Scope Model)
+### 1. Metadata Sources (Runtime Boundary)
 
-ObjectStack adopts a three-scope layered model for metadata:
+ObjectStack separates ObjectOS runtime reads from control-plane metadata
+persistence:
 
-| Scope      | Storage      | Mutability   | Description                                |
-|:-----------|:-------------|:-------------|:-------------------------------------------|
-| `system`   | Filesystem   | Read-only    | Defined in code, shipped with packages      |
-| `platform` | Database     | Admin-editable | Created/modified by admins via UI          |
-| `user`     | Database     | User-editable  | Personal customizations per user           |
+| Runtime context | Storage | Mutability | Description |
+|:----------------|:--------|:-----------|:------------|
+| ObjectOS local/dev | Filesystem or local artifact | Read-only at boot | `MetadataPlugin` scans files or hydrates from `dist/objectstack.json`. |
+| ObjectOS production | Artifact API response | Read-only at boot | Metadata is immutable for a `commitId` / `checksum`; project DB stores business rows only. |
+| Control plane / tooling | `DatabaseLoader` when explicitly configured | Writable | Stores project metadata revisions, overlays, and history outside the ObjectOS project DB. |
 
-Resolution order: **system** ← merge(**platform**) ← merge(**user**).
+`MetadataPlugin` does **not** automatically bridge ObjectQL to
+`DatabaseLoader`, and it does not register `sys_metadata` /
+`sys_metadata_history` into the ObjectOS manifest. Database-backed metadata
+persistence remains available through `MetadataManager.setDatabaseDriver()` or
+`setDataEngine()` for control-plane services that opt in explicitly.
 
 ### 2. Loaders
 
@@ -61,7 +66,7 @@ Loaders are pluggable data sources that know how to read/write metadata from dif
 | `FilesystemLoader`  | `file:`        | ✅   | ✅    | ✅    | Implemented  |
 | `MemoryLoader`      | `memory:`      | ✅   | ✅    | ❌    | Implemented  |
 | `RemoteLoader`      | `http:`        | ✅   | ✅    | ❌    | Implemented  |
-| `DatabaseLoader`    | `datasource:`  | ✅   | ✅    | ❌    | Implemented  |
+| `DatabaseLoader`    | `datasource:`  | ✅   | ✅    | ❌    | Implemented (read-through LRU cache) |
 
 ### 3. Serializers
 
@@ -69,7 +74,7 @@ Serializers convert metadata objects to/from different file formats:
 
 - **JSONSerializer** — `.json` files with optional key sorting
 - **YAMLSerializer** — `.yaml`/`.yml` files (JSON_SCHEMA for security)
-- **TypeScriptSerializer** — `.ts`/`.js` module exports (for `defineObject()`, `defineView()`, etc.)
+- **TypeScriptSerializer** — `.ts`/`.js` module exports (for `ObjectSchema.create()`, `defineView()`, etc.)
 
 ### 4. Overlay / Customization System
 
@@ -110,7 +115,76 @@ Integrates with the ObjectStack kernel plugin system:
 
 - Registers as the primary `IMetadataService` provider
 - Auto-loads all metadata types from the filesystem on startup (sorted by `loadOrder`)
+- Can hydrate runtime metadata from a local project artifact (`dist/objectstack.json`)
 - Supports YAML, JSON, TypeScript, and JavaScript metadata formats
+- Keeps ObjectOS metadata read-only; database persistence is not auto-enabled
+
+#### Bootstrap Modes
+
+`MetadataPluginConfigSchema.bootstrap` controls how the plugin primes metadata
+on `start()`. Pick the mode that matches the deployment target:
+
+| Mode | When to use | Behavior |
+|:-----|:------------|:---------|
+| `eager` (default) | Local dev, traditional servers | Scans filesystem (or hydrates the artifact source if set) at boot. |
+| `lazy` | Cold-start sensitive runtimes, on-demand workloads | Skips the filesystem priming pass — reads flow through `MetadataManager.load*` / `list*` and registered loaders (incl. the DatabaseLoader read-through cache). An `artifactSource` is still honored if provided. |
+| `artifact-only` | **Edge / serverless / read-only production** | Refuses to touch the filesystem. Requires `artifactSource.mode = 'local-file'`. Throws if no artifact source is configured. |
+
+```typescript
+// Edge / serverless: fail-fast if no artifact is wired
+new MetadataPlugin({
+  config: { bootstrap: 'artifact-only' },
+  artifactSource: { mode: 'local-file', path: './dist/objectstack.json' },
+});
+
+// On-demand reads via DatabaseLoader, no FS scan
+new MetadataPlugin({
+  config: { bootstrap: 'lazy' },
+});
+```
+
+#### Persistence Write Gates
+
+`MetadataManagerConfigSchema.persistence` is a two-axis runtime gate that lets
+you freeze metadata mutation in sealed deployments while leaving reads open.
+Both flags default to `true` so dev / Studio flows are unaffected.
+
+| Flag | Effect when `false` |
+|:-----|:--------------------|
+| `persistence.writable` | `MetadataManager.register()` becomes a no-op (or throws when `validation.throwOnError` is set). Use for read-only project kernels booted from a compiled artifact. |
+| `persistence.overlayWritable` | `MetadataManager.saveOverlay()` is rejected. Use to disable Studio overlays in fully-frozen production deployments. |
+
+```typescript
+new MetadataManager({
+  persistence: { writable: false, overlayWritable: false },
+});
+```
+
+#### DatabaseLoader Read-Through Cache
+
+`DatabaseLoader` wraps `load` / `loadMany` / `list` / `stat` results in a
+generic LRU cache (see `src/utils/lru-cache.ts`). Writes invalidate the
+affected entries, so reads always observe writes made through the same loader
+instance; out-of-band SQL writes are honored within `ttl` milliseconds.
+
+Configuration lives under `cache.databaseLoader`:
+
+```typescript
+new MetadataManager({
+  datasource: 'default',
+  cache: {
+    enabled: true,
+    databaseLoader: {
+      enabled: true,
+      maxSize: 500,        // Max cached (type, name) entries
+      ttl: 60_000,          // Cache TTL in milliseconds
+    },
+  },
+});
+```
+
+The cache exposes diagnostic counters (`size` / `hits` / `misses` / `hitRate`)
+through `LRUCache.stats()` for `metrics` endpoints.
 
 ## Metadata Types
 
@@ -194,7 +268,7 @@ manager.watchService('object', (event) => {
 ```typescript
 import { MetadataPlugin } from '@objectstack/metadata/node';
 
-const plugin = MetadataPlugin({
+const plugin = new MetadataPlugin({
   rootDir: './src',
   watch: process.env.NODE_ENV === 'development',
 });
@@ -202,6 +276,19 @@ const plugin = MetadataPlugin({
 kernel.use(plugin);
 ```
 
+### With Local Artifact Boot
+
+```typescript
+import { MetadataPlugin } from '@objectstack/metadata/node';
+
+const plugin = new MetadataPlugin({
+  watch: false,
+  artifactSource: { mode: 'local-file', path: './dist/objectstack.json' },
+});
+
+kernel.use(plugin);
+```
+
 ## Package Publishing
 
 ObjectStack supports **package-level metadata publishing** — all metadata items within a package are published atomically.