@homedev/objector 1.3.8 → 1.3.10

package/README.md

@@ -0,0 +1,608 @@
+# @homedev/objector
+
+Object processing library for YAML/JSON with support for includes, field processors, conditional logic, and extensibility.
+
+## Overview
+
+Objector is a comprehensive tool for loading, processing, and transforming YAML/JSON documents with advanced features like:
+
+- 🔄 Document includes and composition
+- 🔌 Extensible field processors and data sources
+- 🎯 Conditional logic and flow control
+- 📦 Module system with custom functions
+- 🔍 Variable substitution and selectors
+- 📝 Output generation and file writing
+- 🎨 Template inheritance and extensions
+
+## Installation
+
+```bash
+bun add @homedev/objector
+# or
+npm install @homedev/objector
+# or
+pnpm add @homedev/objector
+```
+
+## Quick Start
+
+```typescript
+import { Objector } from '@homedev/objector'
+
+const objector = new Objector()
+
+// Load and process a YAML/JSON file
+const result = await objector.load('config.yaml')
+```
+
+```typescript
+import { Objector } from '@homedev/objector'
+
+const objector = new Objector()
+      .variables({ env: 'production', version: '1.0.0' }) // Add custom variables
+      .include('settings.yaml') // Include additional files
+
+// Process the document
+const processed = await objector.load('main.yaml')
+```
+
+## Core Features
+
+### 1. Document Loading and Includes
+
+Load YAML/JSON files with automatic format detection and includes:
+
+```typescript
+const objector = new Objector()
+
+// Set include directories
+objector.includeDirectory('./config', './templates')
+
+// Load a file (automatically resolves from include directories)
+const config = await objector.load('app-config.yaml')
+```
+
+**In your YAML/JSON:**
+```yaml
+includes:
+  - base-config.yaml
+  - settings/*.yaml
+  - ../shared/defaults.yaml
+
+myConfig:
+  setting: value
+```
+
+### 2. Variables and Substitution
+
+Inject and reference variables throughout your documents:
+
+```typescript
+objector.variables({
+  env: 'production',
+  apiUrl: 'https://api.example.com',
+  version: '2.1.0'
+})
+```
+
+**Usage in documents:**
+```yaml
+app:
+  environment: $(env)
+  api: $(apiUrl)
+  version: $(version)
+```
+
+### 3. Conditional Logic
+
+Control document structure with conditions:
+
+```yaml
+features:
+  - name: cache
+    .if: $(eq:env, "production")
+    .then:
+      enabled: true
+      ttl: 3600
+    .else:
+      enabled: false
+
+  - name: debug
+    .skip: $(eq:env, "production")
+    logging: verbose
+```
+
+### 4. Loops and Iteration
+
+Generate repeated structures with `each`:
+
+```yaml
+environments:
+  .each:
+    .from:
+      - dev
+      - staging
+      - prod
+    .model:
+      name: $(item)
+      url: https://$(item).example.com
+      replicas: $(switch:
+        from: $(item)
+        cases:
+          prod: 5
+          staging: 2
+        default: 1)
+```
+
+### 5. Template Inheritance
+
+Extend and merge templates:
+
+```yaml
+baseTemplate:
+  server:
+    port: 8080
+    timeout: 30
+
+production:
+  extends: .baseTemplate
+  server:
+    port: 443
+    ssl: true
+    # Merged result: port=443, timeout=30, ssl=true
+```
+
+### 6. File Operations
+
+Read files and scan directories:
+
+```yaml
+certificate: $(file:./certs/server.crt)
+
+migrations:
+  .from: $(scan:"*.sql", "./migrations")
+  .model:
+    name: $(item)
+    content: $(file:$(item))
+```
+
+### 7. Custom Modules
+
+Add JavaScript functions as data sources:
+
+```yaml
+modules:
+  utils: |
+    export function formatDate(ctx, date) {
+      return new Date(date).toISOString()
+    }
+    export function generateId(ctx, prefix) {
+      return `${prefix}-${Math.random().toString(36).substr(2, 9)}`
+    }
+
+records:
+  - id: $(utils.generateId:"rec")
+    created: $(utils.formatDate:"2024-01-01")
+```
+
+### 8. Output Generation
+
+Generate multiple output files from one source:
+
+```yaml
+output:
+  - file: dist/config.json
+    content:
+      version: $(version)
+      settings: $(config)
+  
+  - file: dist/README.md
+    content: |
+      # Configuration
+      Version: $(version)
+```
+
+```typescript
+// Write all outputs to disk
+await objector.writeAll()
+```
+
+## Built-in Keys
+
+Keys are object properties prefixed with `.` and are used for structure/flow control.
+
+- `.output` - Register output payload(s)
+- `.load` - Load one or more additional files
+- `.each` - Repeat a model over a source list
+- `.map` - Map a source list into a transformed list
+- `.from` - Resolve a selector/reference into current node
+- `.concat` - Concatenate arrays from selectors
+- `.extends` - Merge from one or many templates
+- `.group` - Expand grouped items into parent list
+- `.if` - Conditional branch using `.then` / `.else`
+- `.switch` - Case-based branching with `cases` / `default`
+- `.skip` - Remove parent when condition is true
+- `.metadata` - Attach metadata for a path
+- `.modules` - Register source functions from inline JS modules
+- `.try` - Guard expression with optional `.catch`
+- `.let` - Introduce temporary variables into context
+- `.tap` - Debug current path/value to console
+
+## Built-in Data Sources
+
+Sources are used with `$()` syntax.
+
+### Core Sources
+- Files: `include`, `exists`, `file`, `scan`
+- String/format: `substring`, `repeat`, `pad`, `formatAs`
+- List: `merge`, `join`, `range`, `filter`
+- Conditions: `if`, `check`, `and`, `or`, `xor`, `true`, `false`, `eq`, `ne`, `lt`, `le`, `gt`, `ge`, `in`, `notin`, `contains`, `notcontains`, `containsi`, `notcontainsi`, `null`, `notnull`, `empty`, `notempty`, `nullorempty`, `notnullorempty`
+- Types: `convert`, `isType`, `typeOf`
+- Flow helpers: `each`, `switch`, `default`, `section`
+
+### Built-in Macro Sources
+These are also available as `$()` sources by default.
+
+- String/text: `env`, `uuid`, `pascal`, `camel`, `capital`, `snake`, `kebab`, `len`, `reverse`, `concat`, `trim`, `ltrim`, `rtrim`, `lower`, `upper`, `encode`, `decode`
+- Collection/object: `list`, `object`, `unique`, `groupBy`, `chunk`, `flatten`, `compact`, `head`, `tail`
+- Path/filepath: `pathJoin`, `resolve`, `dirname`, `basename`, `normalize`, `extname`, `relative`, `isAbsolute`, `segments`
+- Hash/encoding: `md5`, `sha1`, `sha256`, `sha512`, `hash`, `hmac`, `base64Encode`, `base64Decode`, `hexEncode`, `hexDecode`
+- Math: `add`, `subtract`, `multiply`, `divide`, `modulo`, `power`, `sqrt`, `abs`, `floor`, `ceil`, `round`, `min`, `max`, `clamp`, `random`
+- Date/time: `timestamp`, `iso`, `utc`, `formatDate`, `parseDate`, `year`, `month`, `day`, `dayOfWeek`, `hours`, `minutes`, `seconds`
+- Regex: `regexTest`, `regexMatch`, `regexMatchAll`, `regexReplace`, `regexReplaceAll`, `regexSplit`, `regexExec`, `regexSearch`
+- Validation/predicates: `isEmail`, `isUrl`, `isJson`, `isUuid`, `minLength`, `maxLength`, `lengthEquals`, `isNumber`, `isString`, `isBoolean`, `isArray`, `isObject`, `isEmpty`, `isTruthy`, `isFalsy`, `inRange`, `matchesPattern`, `includes`, `startsWith`, `endsWith`
+- Debug: `log`
+
+## API Reference
+
+### Objector Class
+
+#### Constructor
+```typescript
+const objector = new Objector()
+```
+
+#### Configuration Methods
+
+- **`use(handler: (o: Objector) => void): this`**  
+  Apply a configuration handler
+
+- **`includeDirectory(...dirs: string[]): this`**  
+  Add directories to search for includes
+
+- **`include(...files: string[]): this`**  
+  Add include files directly
+
+- **`keys(keys: FieldProcessorMap): this`**  
+  Add or override key processors
+
+- **`sources(sources: FieldProcessorMap): this`**  
+  Add or override data sources
+
+- **`sections(sections: Record<string, FieldProcessorSectionFunc>): this`**  
+  Register section handlers by type
+
+- **`variables(vars: Record<string, any>): this`**  
+  Set global variables
+
+- **`functions(funcs: Record<string, AsyncFunction>): this`**  
+  Register callable functions for expressions and script sections
+
+- **`fieldOptions(options: ProcessFieldsOptions): this`**  
+  Customize field processing behavior
+
+- **`filter(f: RegExp): this`**  
+  Add a filter for field processing
+
+- **`output(o: Output): this`**  
+  Add an output definition
+
+- **`metadata(path: string, value: unknown): this`**  
+  Store metadata for a path
+
+- **`section(section: FieldProcessorSection): this`**  
+  Register a named section
+
+- **`encoders(encoders: Record<string, Encoder>): this`**  
+  Add output encoders (used by `formatAs`)
+
+- **`decoders(decoders: Record<string, LoadFormatParser>): this`**  
+  Add input decoders for `load()`
+
+- **`setCacheMaxSize(size: number): this`**  
+  Set include/content cache size
+
+- **`clearCache(): this`**  
+  Clear cached loaded content
+
+#### Processing Methods
+
+- **`async load<T>(fileName: string, cwd?: string, container?: any, noProcess?: boolean): Promise<T | null>`**  
+  Load and process a file
+
+- **`async loadObject<T>(data: T, container?: any, options?: LoadObjectOptions): Promise<T>`**  
+  Process an object with field processors
+
+- **`async writeAll(options?: WriteOutputsOption): Promise<void>`**  
+  Write all registered outputs to disk
+
+#### Query Methods
+
+- **`getIncludeDirectories(): string[]`**  
+  Get all include directories
+
+- **`getFunctions(): Record<string, AsyncFunction>`**  
+  Get registered functions
+
+- **`getOutputs(): Output[]`**  
+  Get all registered outputs
+
+- **`getMetadata(path: string): any`**  
+  Get metadata for a path
+
+- **`getAllMetadata(): Record<string, any>`**  
+  Get all metadata
+
+- **`getCurrentContext(): LoadContext | null`**  
+  Get current processing context
+
+- **`getSection(name: string): FieldProcessorSection | undefined`**  
+  Get a stored section by name
+
+- **`getEncoder(name: string): Encoder | undefined`**  
+  Get an encoder by name
+
+- **`getDecoder(name: string): LoadFormatParser | undefined`**  
+  Get a decoder by name
+
+## Advanced Usage
+
+### Custom Data Sources
+
+```typescript
+import { Objector } from '@homedev/objector'
+
+const objector = new Objector()
+
+objector.sources({
+  // Simple value source
+  timestamp: () => Date.now(),
+  
+  // Source with arguments
+  add: (ctx) => {
+    const [a, b] = ctx.args
+    return a + b
+  },
+  
+  // Async source
+  fetchData: async (ctx) => {
+    const url = ctx.args[0]
+    const response = await fetch(url)
+    return response.json()
+  }
+})
+```
+
+**Usage:**
+```yaml
+data:
+  created: $(timestamp)
+  sum: $(add:10, 20)
+  remote: $(fetchData:"https://api.example.com/data")
+```
+
+### Custom Key Processors
+
+```typescript
+objector.keys({
+  // Custom transformation key
+  uppercase: (ctx) => {
+    return ctx.value.toUpperCase()
+  },
+  
+  // Conditional key
+  when: (ctx) => {
+    const condition = ctx.value
+    if (!condition) {
+      return NavigateResult.DeleteParent()
+    }
+    return NavigateResult.DeleteItem()
+  }
+})
+```
+
+**Usage:**
+```yaml
+settings:
+  name:
+    .uppercase: hello world
+    # Result: "HELLO WORLD"
+
+feature:
+  .when: $(eq:env, "production")
+  enabled: true
+  # Only included when condition is true
+```
+
+### Metadata Collection
+
+```typescript
+const objector = new Objector()
+
+// Metadata is collected during processing
+await objector.load('config.yaml')
+
+// Retrieve metadata
+const metadata = objector.getMetadata('some.path')
+const allMetadata = objector.getAllMetadata()
+```
+
+**In documents:**
+```yaml
+service:
+  .metadata:
+    version: 1.0
+    author: team-a
+  name: api
+  port: 8080
+```
+
+### Section Blocks (`<@ section ... @>`) 
+
+Objector can process inline section blocks inside strings. This is useful for generated text files (Markdown, scripts, templates).
+
+### Basic Syntax
+
+```txt
+<@ section[:type] @>
+# optional YAML config
+---
+# section body
+<@ /section @>
+```
+
+- `:type` is optional. Example: `section:script`.
+- `---` separates YAML config from section content.
+- If no `---` is present, the block is treated as content-only.
+
+### Script Sections
+
+`script` is the default built-in section type.
+
+```yaml
+doc: |
+  <@ section:script @>
+  using: [shout, project]
+  condition: context.enabled
+  each: context.items
+  ---
+  section.writeLine(`# ${project}`)
+  return shout(String(item))
+  <@ /section @>
+```
+
+For `section:script`, the body runs as async JavaScript with access to:
+
+- `section`: helper object (`write`, `writeLine`, `prepend`, `clear`, `setOptions`, `getLines`)
+- `context`: current root/context object
+- `config`: section YAML config
+- `system`: current `Objector` instance
+- `item`: current item when `each` is used
+- values listed in `using` (from registered `functions()` or root variables)
+
+`condition` can skip rendering, and `each` can iterate an array expression. If the script returns a string, that value becomes the rendered section output.
+
+### Reusing Named Sections
+
+You can store a section by name and render it later with `$(section:name)`:
+
+```yaml
+doc: |
+  <@ section @>
+  name: greeting
+  show: true
+  ---
+  Hello $(user)
+  <@ /section @>
+
+footer: $(section:greeting)
+```
+
+## Examples
+
+### Example 1: Multi-Environment Configuration
+
+```yaml
+# base.yaml
+includes:
+  - vars-$(env).yaml
+
+application:
+  name: myapp
+  version: 1.0.0
+  
+database:
+  host: $(db.host)
+  port: $(db.port)
+  name: $(db.name)
+
+features:
+  - name: caching
+    .if: $(eq:env, "production")
+    .then:
+      enabled: true
+      ttl: 3600
+    .else:
+      enabled: false
+
+  - name: logging
+    level: $(switch:
+      from: $(env)
+      cases:
+        production: warn
+        staging: info
+      default: debug)
+```
+
+### Example 2: Service Generation
+
+```yaml
+services:
+  .each:
+    .from:
+      - name: api
+        port: 8080
+        replicas: 3
+      - name: worker
+        port: 8081
+        replicas: 5
+      - name: web
+        port: 3000
+        replicas: 2
+    .model:
+      name: $(item.name)
+      type: deployment
+      spec:
+        replicas: $(item.replicas)
+        template:
+          containers:
+            - name: $(item.name)
+              image: myapp/$(item.name):$(version)
+              ports:
+                - containerPort: $(item.port)
+              env:
+                - name: NODE_ENV
+                  value: $(env)
+```
+
+### Example 3: Dynamic Output Generation
+
+```yaml
+outputs:
+  .each:
+    .from: $(scan:"*.template.yaml", "./templates")
+    .model:
+      .output:
+        file: dist/$(replace:$(item), ".template.yaml", ".yaml")
+        content: $(file:$(item))
+```
+
+## FAQ
+
+**Q: How do I debug field processing?**  
+A: Use the `$(log:message)` data source to print values during processing.
+
+**Q: Can I use async operations in custom sources?**  
+A: Yes! All field processors and data sources support async/await.
+
+**Q: How do I handle circular includes?**  
+A: The system tracks included files to prevent circular dependencies.
+
+**Q: Can I extend the built-in processors?**  
+A: Yes, use `objector.keys()` and `objector.sources()` to add or override processors.
+
+**Q: How do I access nested values?**  
+A: Use the selector syntax: `$(some.nested.value)` or `$(@root.some.value)` for root access.
+
+**Q: What's the difference between keys and sources?**  
+A: Keys are prefixed with `.` and control structure/flow (`.if`, `.each`). Sources use `$()` syntax and provide values (`$(env:VAR)`, `$(file:path)`).