modscape 2.0.0 → 2.0.2

package/README.ja.md

@@ -92,6 +92,7 @@ YAMLのルートレベル構造は以下の通りです：
 domains      – 関連テーブルをまとめるビジュアルコンテナ
 tables       – 3階層メタデータを持つエンティティ定義
 relationships – テーブル間のERカーディナリティ
+lineage      – データの流れ / 変換パス
 annotations  – キャンバス上のスティッキーノート・吹き出し
 layout       – 全座標データ（tables/domains の中に x/y を書いてはいけない）
 ```
@@ -104,7 +105,7 @@ domains:
     name: "主要売上"
     description: "営業チームのトランザクションデータ。"  # 任意
     color: "rgba(59, 130, 246, 0.1)"  # 背景色
-    tables: [orders, dim_customers]
+    tables: [orders, dim_customers]   # 論理的な所属リスト
     isLocked: false  # true にするとキャンバスでの誤ドラッグを防止
 ```
 
@@ -130,11 +131,6 @@ tables:
       businessDefinitions:
         revenue: "割引・返品後の純売上"
 
-    lineage:  # mart/集計テーブルのみに定義
-      upstream:
-        - fct_sales
-        - dim_dates
-
     implementation:  # 任意 – AIコード生成へのヒント
       materialization: incremental  # table | view | incremental | ephemeral
       incremental_strategy: merge   # merge | append | delete+insert
@@ -165,23 +161,22 @@ tables:
           type: "BIGINT"
           constraints: [NOT NULL]
 
-    sampleData:  # 2次元配列。先頭行 = カラムID
-      - [order_id, amount, status]
+    sampleData:  # 実数値の2次元配列
       - [1001, 50.0, "COMPLETED"]
       - [1002, 120.5, "PENDING"]
 ```
 
-**テーブルタイプと `appearance.type` の使い分け：**
+### Data Lineage（データリネージ）
+
+ルートレベルの `lineage` セクションでテーブル間のデータの流れ（どのソースからどの集計テーブルが作られるか）を定義します。リネージモードではアニメーション付きの点線矢印として表示されます。
 
-| type | 用途 |
-|------|------|
-| `fact` | 取引・イベント・測定値 |
-| `dimension` | エンティティ・マスタ・参照リスト |
-| `mart` | 集計・消費者向けテーブル（`lineage.upstream` を必ず定義） |
-| `hub` | Data Vault のビジネスキー |
-| `link` | Data Vault のハブ間結合・トランザクション |
-| `satellite` | Data Vault のハブに紐づく履歴属性 |
-| `table` | 汎用 |
+```yaml
+lineage:
+  - from: fct_orders    # ソーステーブル ID
+    to: mart_revenue    # 派生テーブル ID
+  - from: dim_dates
+    to: mart_revenue
+```
 
 ### Relationships（リレーションシップ）
 
@@ -196,7 +191,7 @@ relationships:
     type: one-to-many  # one-to-one | one-to-many | many-to-one | many-to-many
 ```
 
-> **データリネージ**は `lineage.upstream` で定義し、リネージモードでアニメーション矢印として表示されます。`relationships` に重複して記載しないでください。
+> **ER関係** vs **リネージ**: 構造的な結合（外部キーなど）には `relationships` を、データの加工・変換の流れには `lineage` を使用してください。両方に同じ接続を記述しないでください。
 
 ### Annotations（アノテーション）
 

package/README.md

@@ -95,6 +95,7 @@ Modscape uses a schema designed for data analysis contexts. The full YAML struct
 domains      – visual containers grouping related tables
 tables       – entity definitions with tri-layer metadata
 relationships – ER cardinality between tables
+lineage      – data flow / transformation paths
 annotations  – sticky notes / callouts on the canvas
 layout       – ALL coordinate data (never put x/y inside tables or domains)
 ```
@@ -107,7 +108,7 @@ domains:
     name: "Core Sales"
     description: "Transactional data for the sales team."  # optional
     color: "rgba(59, 130, 246, 0.1)"  # background fill
-    tables: [orders, dim_customers]
+    tables: [orders, dim_customers]   # logical membership
     isLocked: false  # prevent accidental drag when true
 ```
 
@@ -133,11 +134,6 @@ tables:
       businessDefinitions:
         revenue: "Net revenue after discounts"
 
-    lineage:  # for mart/aggregated tables only
-      upstream:
-        - fct_sales
-        - dim_dates
-
     implementation:  # optional – hints for AI code generation
       materialization: incremental  # table | view | incremental | ephemeral
       incremental_strategy: merge   # merge | append | delete+insert
@@ -168,12 +164,23 @@ tables:
           type: "BIGINT"
           constraints: [NOT NULL]
 
-    sampleData:  # 2D array; first row = column IDs
-      - [order_id, amount, status]
+    sampleData:  # 2D array of realistic values
       - [1001, 50.0, "COMPLETED"]
       - [1002, 120.5, "PENDING"]
 ```
 
+### Data Lineage
+
+Top-level `lineage` section declares data flow between tables (which source tables feed which derived tables). This is rendered as dashed arrows in **Lineage Mode**.
+
+```yaml
+lineage:
+  - from: fct_orders    # source table ID
+    to: mart_revenue    # derived table ID
+  - from: dim_dates
+    to: mart_revenue
+```
+
 ### Relationships
 
 ```yaml
@@ -187,7 +194,7 @@ relationships:
     type: one-to-many  # one-to-one | one-to-many | many-to-one | many-to-many
 ```
 
-> **Data Lineage** connections are driven by `lineage.upstream` and rendered as animated arrows in Lineage Mode. Do **not** duplicate them as `relationships` entries.
+> **ER Relationships** vs **Lineage**: Use `relationships` for structural joins (FKs) and `lineage` for data flow (transformations). Do not duplicate them.
 
 ### Annotations
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "modscape",
-  "version": "2.0.0",
+  "version": "2.0.2",
   "description": "Modscape: A YAML-driven data modeling visualizer CLI",
   "repository": {
     "type": "git",
@@ -29,6 +29,7 @@
     "@inquirer/prompts": "^8.3.0",
     "chokidar": "^5.0.0",
     "commander": "^14.0.3",
+    "dagre": "^0.8.5",
     "express": "^5.2.1",
     "js-yaml": "^4.1.1",
     "open": "^11.0.0",

package/src/index.js

@@ -10,6 +10,7 @@ import { exportModel } from './export.js';
 import { createModel } from './create.js';
 import { importDbt } from './import-dbt.js';
 import { syncDbt } from './sync-dbt.js';
+import { applyLayout } from './layout.js';
 import { createRequire } from 'module';
   import { mergeModels } from './merge.js';
   
@@ -104,4 +105,13 @@ program
     mergeModels(paths, options);
   });
 
+program
+  .command('layout')
+  .description('Perform automatic layout calculation and update the YAML file')
+  .argument('<path>', 'path to the YAML model file')
+  .option('-o, --output <path>', 'output file path (defaults to overwriting input)')
+  .action((path, options) => {
+    applyLayout(path, options);
+  });
+
 program.parse();

package/src/layout.js

@@ -0,0 +1,151 @@
+import fs from 'fs';
+import path from 'path';
+import yaml from 'js-yaml';
+import dagre from 'dagre';
+
+/**
+ * CLI-based layout engine for Modscape.
+ * Replicates the logic from the visualizer to ensure consistency.
+ */
+export function applyLayout(inputPath, options = {}) {
+  const absolutePath = path.resolve(process.cwd(), inputPath);
+  if (!fs.existsSync(absolutePath)) {
+    console.error(`  ❌ File not found: ${inputPath}`);
+    return;
+  }
+
+  const raw = fs.readFileSync(absolutePath, 'utf8');
+  let schema;
+  try {
+    schema = yaml.load(raw);
+  } catch (e) {
+    console.error(`  ❌ Failed to parse YAML: ${e.message}`);
+    return;
+  }
+
+  if (!schema || !schema.tables) {
+    console.error('  ❌ Invalid schema: No tables found.');
+    return;
+  }
+
+  console.log(`  🏗️  Calculating layout for ${schema.tables.length} tables...`);
+
+  // Initialize Dagre Graph
+  const g = new dagre.graphlib.Graph({ compound: true });
+  g.setGraph({
+    rankdir: 'LR',
+    nodesep: 80,
+    ranksep: 200,
+    marginx: 80,
+    marginy: 80,
+  });
+  g.setDefaultEdgeLabel(() => ({}));
+
+  // 1. Add Tables
+  schema.tables.forEach((table) => {
+    // Standard table size in canvas units
+    g.setNode(table.id, { width: 280, height: 160 });
+  });
+
+  // 2. Add Lineage Edges
+  if (schema.lineage) {
+    schema.lineage.forEach((edge) => {
+      if (g.hasNode(edge.from) && g.hasNode(edge.to)) {
+        g.setEdge(edge.from, edge.to);
+      }
+    });
+  }
+
+  // 3. Add ER Relationships
+  if (schema.relationships) {
+    schema.relationships.forEach((rel) => {
+      if (g.hasNode(rel.from.table) && g.hasNode(rel.to.table)) {
+        g.setEdge(rel.from.table, rel.to.table);
+      }
+    });
+  }
+
+  // 4. Setup Domains
+  const domainTableMap = new Map();
+  if (schema.domains) {
+    schema.domains.forEach((domain) => {
+      g.setNode(domain.id, { label: domain.name, cluster: true });
+      domain.tables.forEach((tableId) => {
+        if (g.hasNode(tableId)) {
+          g.setParent(tableId, domain.id);
+        }
+      });
+      domainTableMap.set(domain.id, domain.tables);
+    });
+  }
+
+  // Execute Layout Calculation
+  dagre.layout(g);
+
+  // 5. Post-process: Convert Dagre results to Modscape Layout format
+  const newLayout = {};
+  const PAD = 48;
+  const TABLE_W = 280;
+  const TABLE_H = 160; // Default height for layout estimation
+  const GAP = 40;
+
+  // Process Domains (Grid packing)
+  if (schema.domains) {
+    schema.domains.forEach(domain => {
+      const members = domain.tables.filter(tid => g.hasNode(tid));
+      if (members.length === 0) return;
+
+      // Sort by dagre rank (left -> right)
+      members.sort((a, b) => g.node(a).x - g.node(b).x);
+
+      const cols = Math.min(3, Math.ceil(Math.sqrt(members.length)));
+      const rowCount = Math.ceil(members.length / cols);
+
+      const gridW = cols * (TABLE_W + GAP) - GAP;
+      const gridH = rowCount * (TABLE_H + GAP) - GAP;
+
+      // Anchor grid to dagre centroid
+      const cx = members.reduce((s, tid) => s + g.node(tid).x, 0) / members.length;
+      const cy = members.reduce((s, tid) => s + g.node(tid).y, 0) / members.length;
+      const originX = cx - gridW / 2;
+      const originY = cy - gridH / 2;
+
+      members.forEach((tid, idx) => {
+        const col = idx % cols;
+        const row = Math.floor(idx / cols);
+        const xOff = col * (TABLE_W + GAP) + TABLE_W / 2;
+        const yOff = row * (TABLE_H + GAP) + TABLE_H / 2;
+        newLayout[tid] = {
+          x: Math.round(originX + xOff),
+          y: Math.round(originY + yOff),
+          parentId: domain.id
+        };
+      });
+
+      // Domain bounding box
+      const HEADER = 28;
+      newLayout[domain.id] = {
+        x: Math.round(originX - PAD),
+        y: Math.round(originY - PAD - HEADER),
+        width: Math.round(gridW + PAD * 2),
+        height: Math.round(gridH + PAD * 2 + HEADER)
+      };
+    });
+  }
+
+  // Standalone Tables
+  const domainTableIds = new Set(schema.domains?.flatMap(d => d.tables) ?? []);
+  schema.tables.forEach(t => {
+    if (!domainTableIds.has(t.id)) {
+      const pos = g.node(t.id);
+      newLayout[t.id] = { x: Math.round(pos.x), y: Math.round(pos.y) };
+    }
+  });
+
+  // 6. Save back to YAML
+  schema.layout = newLayout;
+  const outputPath = options.output ? path.resolve(process.cwd(), options.output) : absolutePath;
+  
+  fs.writeFileSync(outputPath, yaml.dump(schema, { indent: 2, lineWidth: -1, noRefs: true }), 'utf8');
+  console.log(`  ✅ Layout complete! Saved to: ${path.relative(process.cwd(), outputPath)}`);
+}

package/src/templates/claude/modeling.md

@@ -1,7 +1,7 @@
 Start an interactive data modeling session.
 
 ## Instructions
-1. FIRST, read `.modscape/rules.md` to understand the project strategy, naming conventions, and YAML schema.
+1. FIRST, read `.modscape/rules.md` to understand the project strategy, naming conventions, and YAML schema. If `.modscape/rules.custom.md` exists, read it too — custom rules take priority over the base rules.
 2. SECOND, analyze the existing `model.yaml` if it exists.
 3. Listen to the user's requirements and propose/apply changes to `model.yaml` strictly following the rules.
 

package/src/templates/codex/modscape-modeling/SKILL.md

@@ -9,7 +9,7 @@ You are a professional Data Modeler. Your primary directive is to manage `model.
 
 ## COMMAND: /modscape:modeling
 When the user issues this command:
-1. READ `.modscape/rules.md` to understand project strategy and conventions.
+1. READ `.modscape/rules.md` to understand project strategy and conventions. If `.modscape/rules.custom.md` exists, read it too — custom rules take priority over the base rules.
 2. ANALYZE `model.yaml` (if present).
 3. INTERACT with the user to gather requirements and update the model strictly following the rules.
 
@@ -17,7 +17,7 @@ When the user issues this command:
 - **Appearance**: When creating new tables, include the `appearance` block with an appropriate `type`.
 - **Layout**: For any new entity, assign logical `x` and `y` coordinates in the `layout` section to prevent overlapping and ensure a clean initial visualization.
 
-ALWAYS follow the rules defined in `.modscape/rules.md` for any modeling tasks.
+ALWAYS follow the rules defined in `.modscape/rules.md` (and `.modscape/rules.custom.md` if present) for any modeling tasks.
 
 ## COMMAND: /modscape:codegen
 When the user issues this command:

package/src/templates/gemini/modscape-modeling/SKILL.md

@@ -7,7 +7,7 @@ description: Create the data model defined in `model.yaml` according to project
 
 You are a professional Data Modeler. Your primary directive is to manage `model.yaml`.
 
-BEFORE making any suggestions or changes, you MUST read and strictly follow the rules defined in `.modscape/rules.md`.
+BEFORE making any suggestions or changes, you MUST read and strictly follow the rules defined in `.modscape/rules.md`. If `.modscape/rules.custom.md` exists, read it too — custom rules take priority over the base rules.
 
 If a requested change violates these rules, warn the user.
 
@@ -24,4 +24,4 @@ If a requested change violates these rules, warn the user.
 ## Interactive Modeling
 When the user wants to perform modeling tasks, ensure you are utilizing the strategy and conventions defined in the project rules.
 
-ALWAYS follow the rules defined in `.modscape/rules.md` for any modeling tasks.
+ALWAYS follow the rules defined in `.modscape/rules.md` (and `.modscape/rules.custom.md` if present) for any modeling tasks.

package/src/templates/rules.md

@@ -3,6 +3,9 @@
 > **Purpose**: This file teaches AI agents how to write valid `model.yaml` files for Modscape.
 > Read this file completely before generating or editing any YAML.
 
+> **Extension**: If `.modscape/rules.custom.md` exists in this project, read it **in addition** to this file.
+> Rules in `rules.custom.md` take **priority** over this file when they conflict.
+
 ---
 
 ## QUICK REFERENCE (read this first)
@@ -589,7 +592,41 @@ modscape merge ./sales ./marketing -o combined.yaml
 
 ---
 
-## 13. Complete Example
+## 13. Project-Specific Rule Extensions
+
+A project MAY place a `.modscape/rules.custom.md` file to define rules that extend or override this base file.
+
+**How to use it:**
+
+- Create `.modscape/rules.custom.md` in the project root (alongside `rules.md`)
+- Write any project-specific rules, naming conventions, or overrides in Markdown
+- AI agents reading this file will also check for `rules.custom.md` and apply it
+
+**Priority:** Rules in `rules.custom.md` take priority over this file when they conflict.
+
+**What to put in `rules.custom.md`** (examples):
+
+```markdown
+## Naming Conventions
+- All fact table IDs must use the prefix `fct_` followed by the domain: e.g., `fct_sales_orders`
+- All dimension table IDs must end in `_dim`: e.g., `customers_dim`
+
+## Allowed Table Types
+- This project only uses `fact`, `dimension`, and `mart`. Do NOT use `hub`, `link`, or `satellite`.
+
+## Domain Topology
+- This project has three domains: `sales`, `marketing`, `finance`
+- Every new table MUST be assigned to one of these domains
+
+## SCD Policy
+- Dimension tables use SCD Type 1 only. Do NOT apply `scd: type2` or higher.
+```
+
+`rules.custom.md` is NOT generated by `modscape init`. Create it manually when your project needs it.
+
+---
+
+## 14. Complete Example
 
 ```yaml
 domains:

package/visualizer/package.json

@@ -1,7 +1,7 @@
 {
   "name": "visualizer",
   "private": true,
-  "version": "2.0.0",
+  "version": "2.0.2",
   "type": "module",
   "scripts": {
     "dev": "vite",