@caracal-lynx/sluice 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/CLAUDE.md +1822 -1822
  2. package/README.md +12 -21
  3. package/package.json +1 -1
package/README.md CHANGED
@@ -301,8 +301,8 @@ sluice/
301
301
  │ ├── unit/ ← unit tests (all I/O mocked)
302
302
  │ └── integration/ ← real DuckDB :memory: + CSV fixtures
303
303
  └── clients/ ← 🙈 gitignored — each client has their own repo
304
- ├── cochran/ ← Cochran Group (Annan) pipelines
305
- └── eribe/ ← Eribé Knitwear pipelines
304
+ ├── acme-corp/ ← Acme Corp pipelines
305
+ └── style-co/ ← Style Co pipelines
306
306
  ```
307
307
 
308
308
  ---
@@ -313,7 +313,7 @@ Connection strings and credentials live in `.env` (never in YAML files, never in
313
313
 
314
314
  ```bash
315
315
  # .env
316
- COCHRAN_MSSQL=mssql://user:password@server.cochran.local/LegacyDB
316
+ SOURCE_MSSQL=mssql://user:password@serverlegacy.example.local/LegacyDB
317
317
  BC_BASE_URL=https://api.businesscentral.dynamics.com/v2.0
318
318
  BC_TENANT_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
319
319
  BC_CLIENT_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
@@ -326,7 +326,7 @@ Reference them in YAML with `${ENV_VAR}` — resolved at runtime, never stored i
326
326
  ```yaml
327
327
  source:
328
328
  adapter: mssql
329
- connection: ${COCHRAN_MSSQL}
329
+ connection: ${SOURCE_MSSQL}
330
330
  ```
331
331
 
332
332
  ---
@@ -342,7 +342,7 @@ Name a bundle of checks in a shared rules file and reference them like built-ins
342
342
  ```yaml
343
343
  # shared/rules.yaml
344
344
  rules:
345
- - id: eribeStyleNo
345
+ - id: style-coStyleNo
346
346
  checks:
347
347
  - { type: notNull, severity: critical }
348
348
  - { type: pattern, value: "^[A-Z]{2}[0-9]{4}$", severity: critical }
@@ -356,7 +356,7 @@ dq:
356
356
  rules:
357
357
  - field: STYLE_NO
358
358
  checks:
359
- - { type: eribeStyleNo } # expands to the three checks above ✨
359
+ - { type: style-coStyleNo } # expands to the three checks above ✨
360
360
  ```
361
361
 
362
362
  ### Tier 2 — Plugin Files (TypeScript) 🔌
@@ -432,7 +432,7 @@ Detailed guide: **[PLUGINS.md](./PLUGINS.md)**
432
432
 
433
433
  ## 🔀 Multi-Source Merge
434
434
 
435
- Phase 3 lets a single pipeline extract from **2+ sources** and merge them on a key column before DQ and transform. Useful when the master record for an entity is scattered across systems — master data in SQL Server, pricing enrichment in an Excel sheet, product descriptions in Odoo, and so on.
435
+ Phase 3 lets a single pipeline extract from **2+ sources** and merge them on a key column before DQ and transform. Useful when the master record for an entity is scattered across systems — master data in SQL Server, pricing enrichment in an Excel sheet, product descriptions in a REST API, and so on.
436
436
 
437
437
  ### Built-in merge strategies
438
438
 
@@ -449,8 +449,8 @@ Custom strategies can be dropped in as `*.merge.ts` plugins or shipped as npm pa
449
449
 
450
450
  ```yaml
451
451
  pipeline:
452
- name: eribe-products-merged
453
- client: eribe-knitwear
452
+ name: style-co-products-merged
453
+ client: style-co
454
454
  version: "1.0"
455
455
  entity: Style
456
456
 
@@ -458,7 +458,7 @@ sources:
458
458
  - id: sql-server # staging table: stg_raw_sql-server
459
459
  priority: 1 # lower = higher precedence
460
460
  adapter: mssql
461
- connection: ${ERIBE_MSSQL}
461
+ connection: ${SOURCE_2_MSSQL}
462
462
  query: "SELECT STYLE_NO, STYLE_DESC, COST_PRICE FROM dbo.Styles WHERE Active = 1"
463
463
 
464
464
  - id: excel
@@ -478,7 +478,7 @@ merge:
478
478
  fieldStrategies: # per-field overrides
479
479
  - { field: FIBRE_CONTENT, source: excel } # pin to one source
480
480
  - { field: COST_PRICE, strategy: priority-override }
481
- conflictLog: ./output/eribe-products-conflicts.csv # optional CSV of field disagreements
481
+ conflictLog: ./output/style-co-products-conflicts.csv # optional CSV of field disagreements
482
482
 
483
483
  dq:
484
484
  stopOnCritical: true
@@ -514,16 +514,7 @@ sluice merge list-strategies # ids + descriptions for all registered stra
514
514
  sluice merge info coalesce # details for one strategy
515
515
  ```
516
516
 
517
- A full working example lives at [tests/fixtures/eribe-products-merged.pipeline.yaml](tests/fixtures/eribe-products-merged.pipeline.yaml).
518
-
519
- ---
520
-
521
- ## 🤝 Known Clients
522
-
523
- | Client | Source | Target | Adapter |
524
- |--------|--------|--------|---------|
525
- | Cochran Group (Annan) | MSSQL legacy DB | IFS ERP | `ifs` |
526
- | Eribé Knitwear | MSSQL / CSV exports | BlueCherry ERP | `bluecherry` |
517
+ A full working example lives at [tests/fixtures/style-co-products-merged.pipeline.yaml](tests/fixtures/style-co-products-merged.pipeline.yaml).
527
518
 
528
519
  ---
529
520
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@caracal-lynx/sluice",
3
- "version": "0.1.0",
3
+ "version": "0.1.1",
4
4
  "description": "Config-driven ETL toolkit for ERP data migrations",
5
5
  "license": "Elastic-2.0",
6
6
  "author": "Caracal Lynx Ltd. <michael.scott@caracallynx.com> (https://caracallynx.com)",