modscape 1.0.2 → 1.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ja.md +28 -21
- package/README.md +28 -22
- package/package.json +1 -1
- package/src/index.js +1 -1
- package/src/templates/rules.md +77 -49
- package/visualizer/package.json +1 -1
- package/visualizer-dist/assets/{index-B2sFswTF.js → index-BFMiG42J.js} +41 -36
- package/visualizer-dist/assets/index-B_0PaiqL.css +1 -0
- package/visualizer-dist/index.html +2 -2
- package/visualizer-dist/assets/index-CgNe-MGv.css +0 -1
package/README.ja.md
CHANGED
|
@@ -21,20 +21,19 @@
|
|
|
21
21
|
## 主な機能
|
|
22
22
|
|
|
23
23
|
- **YAML-as-Code**: データアーキテクチャ全体を単一のYAMLファイルで定義。Gitによる変更管理が可能。
|
|
24
|
-
-
|
|
25
|
-
-
|
|
26
|
-
|
|
27
|
-
- **オートセーブ**をONにすれば、キャンバス上の変更が即座にローカルファイルに反映されます。
|
|
28
|
-
- **ダーク/ライトモード対応**: 利用環境やドキュメント作成の用途に合わせて、ワンクリックでテーマを切り替え可能。
|
|
29
|
-
- **データ分析特化のモデリング**: `fact`, `dimension`, `mart`, `hub`, `link`, `satellite` などのエンティティタイプを標準サポート。
|
|
24
|
+
- **3階層ネーミングシステム**: エンティティを **概念名**(ビジュアル)、**論理名**(ビジネス定義)、**物理名**(実際のテーブル名)の3段階でドキュメント化。
|
|
25
|
+
- **自動レイアウト調整**: インテリジェントな階層型レイアウトエンジンにより、リレーションに基づいてテーブルとドメインを自動的に整列(※モデルの複雑さによっては手動での微調整が必要な場合があります)。
|
|
26
|
+
- **刷新されたモデリング・ノード**: 左上に突き出した「インデックス・タブ」で種類(FACT, DIM, HUB等)を明示。長い物理名は自動省略され、プロフェッショナルな外観を維持。
|
|
30
27
|
- **インタラクティブなビジュアルキャンバス**:
|
|
31
|
-
- **ドラッグで接続**:
|
|
28
|
+
- **ドラッグで接続**: カラム間のリレーションを直感的に作成。吸着機能で快適な操作感。
|
|
29
|
+
- **意味的なエッジバッジ**: 接続点に `( 1 )` や `[ M ]` バッジを表示し、カーディナリティ(多重度)を視覚化。
|
|
32
30
|
- **データリネージ・モード**: データの流れをアニメーション付きの点線矢印で可視化。
|
|
33
31
|
- **ドメイン階層ナビゲーション**: テーブルをビジネスドメインごとに整理し、構造化されたサイドバーから素早くアクセス。
|
|
34
|
-
-
|
|
35
|
-
-
|
|
36
|
-
-
|
|
37
|
-
|
|
32
|
+
- **統合 Undo/Redo & オートセーブ**:
|
|
33
|
+
- ドラッグや自動整列、編集などの操作が内蔵エディタの履歴と同期。
|
|
34
|
+
- オートセーブにより、ローカルのYAMLを常に最新の状態に維持。
|
|
35
|
+
- **ダーク/ライトモード対応**: 利用環境やドキュメント作成の用途に合わせて、ワンクリックでテーマを切り替え可能。
|
|
36
|
+
- **データ分析特化のモデリング**: `fact`, `dimension`, `mart`, `hub`, `link`, `satellite` に加え、汎用的な `table` タイプを標準サポート。
|
|
38
37
|
- **AIエージェント対応**: **Gemini, Claude, Codex** 用の雛形を内蔵。LLMを活用してモデリング作業を劇的に加速。
|
|
39
38
|
|
|
40
39
|
## インストール
|
|
@@ -74,25 +73,34 @@ npm install -g modscape
|
|
|
74
73
|
domains:
|
|
75
74
|
- id: core_sales
|
|
76
75
|
name: 主要売上
|
|
77
|
-
color: "
|
|
78
|
-
tables: [orders
|
|
76
|
+
color: "rgba(59, 130, 246, 0.1)"
|
|
77
|
+
tables: [orders]
|
|
79
78
|
|
|
80
79
|
# 2. Tables: エンティティ定義
|
|
81
80
|
tables:
|
|
82
81
|
- id: orders
|
|
83
|
-
name:
|
|
82
|
+
name: 注文 # 概念名(大)
|
|
83
|
+
logical_name: "顧客注文履歴" # 論理名(中)
|
|
84
|
+
physical_name: "fct_retail_sales" # 物理名(小)
|
|
84
85
|
appearance:
|
|
85
|
-
type: fact # fact | dimension | mart | hub | link | satellite
|
|
86
|
+
type: fact # fact | dimension | mart | hub | link | satellite | table
|
|
86
87
|
sub_type: transaction
|
|
87
|
-
|
|
88
|
-
icon: 📦
|
|
88
|
+
icon: 💰
|
|
89
89
|
columns:
|
|
90
90
|
- id: order_id
|
|
91
91
|
logical:
|
|
92
92
|
name: ORDER_ID
|
|
93
93
|
isPrimaryKey: true
|
|
94
|
-
additivity: fully
|
|
95
|
-
|
|
94
|
+
additivity: fully
|
|
95
|
+
sampleData:
|
|
96
|
+
- [order_id, amount, status]
|
|
97
|
+
- [1001, 50.0, "COMPLETED"]
|
|
98
|
+
|
|
99
|
+
# 3. Relationships: カーディナリティの定義
|
|
100
|
+
relationships:
|
|
101
|
+
- from: { table: customers, column: customer_id }
|
|
102
|
+
to: { table: orders, column: customer_id }
|
|
103
|
+
type: one-to-many
|
|
96
104
|
```
|
|
97
105
|
|
|
98
106
|
---
|
|
@@ -121,11 +129,10 @@ Modscape は以下の素晴らしいオープンソースプロジェクトに
|
|
|
121
129
|
|
|
122
130
|
- [React Flow](https://reactflow.dev/) - インタラクティブなグラフ UI フレームワーク。
|
|
123
131
|
- [CodeMirror 6](https://codemirror.net/) - 次世代のウェブベース・コードエディタ。
|
|
132
|
+
- [Dagre](https://github.com/dagrejs/dagre) - 階層型グラフ・レイアウトエンジン。
|
|
124
133
|
- [Lucide React](https://lucide.dev/) - シンプルで美しいアイコンセット。
|
|
125
134
|
- [Zustand](https://github.com/pmndrs/zustand) - React 用の状態管理ライブラリ。
|
|
126
|
-
- [Express](https://expressjs.com/) - Node.js 用のウェブフレームワーク。
|
|
127
135
|
- [js-yaml](https://github.com/nodeca/js-yaml) - JavaScript 用 YAML パーサー。
|
|
128
|
-
- [Commander.js](https://github.com/tj/commander.js) - CLI フレームワーク。
|
|
129
136
|
|
|
130
137
|
## ライセンス
|
|
131
138
|
MIT
|
package/README.md
CHANGED
|
@@ -21,21 +21,19 @@ In modern data analysis platforms, data modeling is no longer just about drawing
|
|
|
21
21
|
## Key Features
|
|
22
22
|
|
|
23
23
|
- **YAML-as-Code**: Define your entire data architecture in a single, human-readable YAML file. Track changes via Git.
|
|
24
|
-
- **
|
|
25
|
-
- **
|
|
26
|
-
- **
|
|
27
|
-
- Visual actions (dragging, resizing, metadata edits) are synchronized with the editor's history. Undo your last action with `Ctrl+Z`.
|
|
28
|
-
- Optional **Auto-save** ensures your local YAML is always up-to-date with your visual changes.
|
|
29
|
-
- **Dark/Light Mode Support**: Switch between themes seamlessly for better eye comfort or documentation exports.
|
|
30
|
-
- **Specialized Modeling Types**: Native support for entity types like `fact`, `dimension`, `mart`, `hub`, `link`, and `satellite`.
|
|
24
|
+
- **Tri-Layer Naming System**: Document entities across three levels of abstraction: **Conceptual** (Visual name), **Logical** (Formal business name), and **Physical** (Actual database table name).
|
|
25
|
+
- **Auto-Format Layout**: Automatically arrange tables and domains based on their relationships using an intelligent hierarchical layout engine (Note: manual adjustment may be required for complex models).
|
|
26
|
+
- **Redesigned Modeling Nodes**: Protruding "Index Tabs" for entity types (FACT, DIM, HUB, LINK, etc.) and auto-truncating physical names for a professional look.
|
|
31
27
|
- **Interactive Visual Canvas**:
|
|
32
28
|
- **Drag-to-Connect**: Create relationships between columns intuitively with "Magnetic Snapping".
|
|
29
|
+
- **Semantic Edge Badges**: Visually identify cardinality with `( 1 )` and `[ M ]` badges at the connection points.
|
|
33
30
|
- **Data Lineage Mode**: Visualize data flow with animated dashed arrows.
|
|
34
31
|
- **Domain-Grouped Navigation**: Organize tables into visual business domains and navigate them via a structured sidebar.
|
|
35
|
-
- **
|
|
36
|
-
-
|
|
37
|
-
- **
|
|
38
|
-
|
|
32
|
+
- **Unified Undo/Redo & Auto-save**:
|
|
33
|
+
- Visual actions (dragging, formatting, editing) are synchronized with the built-in CodeMirror editor's history.
|
|
34
|
+
- Optional **Auto-save** ensures your local YAML is always up-to-date.
|
|
35
|
+
- **Dark/Light Mode Support**: Switch between themes seamlessly for better eye comfort or documentation exports.
|
|
36
|
+
- **Specialized Modeling Types**: Native support for entity types like `fact`, `dimension`, `mart`, `hub`, `link`, `satellite`, and generic `table`.
|
|
39
37
|
- **AI-Agent Ready**: Built-in scaffolding for **Gemini, Claude, and Codex** to accelerate your modeling workflow using LLMs.
|
|
40
38
|
|
|
41
39
|
## Installation
|
|
@@ -90,26 +88,35 @@ Modscape uses a schema designed for data analysis contexts.
|
|
|
90
88
|
domains:
|
|
91
89
|
- id: core_sales
|
|
92
90
|
name: Core Sales
|
|
93
|
-
color: "
|
|
94
|
-
tables: [orders
|
|
91
|
+
color: "rgba(59, 130, 246, 0.1)"
|
|
92
|
+
tables: [orders]
|
|
95
93
|
|
|
96
|
-
# 2. Tables: Entity definitions with
|
|
94
|
+
# 2. Tables: Entity definitions with tri-layer metadata
|
|
97
95
|
tables:
|
|
98
96
|
- id: orders
|
|
99
|
-
name:
|
|
97
|
+
name: Orders # Conceptual (Big)
|
|
98
|
+
logical_name: "Customer Purchase Record" # Logical (Medium)
|
|
99
|
+
physical_name: "fct_retail_sales" # Physical (Small)
|
|
100
100
|
appearance:
|
|
101
|
-
type: fact # fact | dimension | mart | hub | link | satellite
|
|
101
|
+
type: fact # fact | dimension | mart | hub | link | satellite | table
|
|
102
102
|
sub_type: transaction
|
|
103
|
-
|
|
104
|
-
icon: 📦
|
|
103
|
+
icon: 💰
|
|
105
104
|
columns:
|
|
106
105
|
- id: order_id
|
|
107
106
|
logical:
|
|
108
107
|
name: ORDER_ID
|
|
109
108
|
type: Int
|
|
110
109
|
isPrimaryKey: true
|
|
111
|
-
additivity: fully
|
|
112
|
-
|
|
110
|
+
additivity: fully
|
|
111
|
+
sampleData:
|
|
112
|
+
- [order_id, amount, status]
|
|
113
|
+
- [1001, 50.0, "COMPLETED"]
|
|
114
|
+
|
|
115
|
+
# 3. Relationships: Define ER cardinality
|
|
116
|
+
relationships:
|
|
117
|
+
- from: { table: customers, column: customer_id }
|
|
118
|
+
to: { table: orders, column: customer_id }
|
|
119
|
+
type: one-to-many
|
|
113
120
|
```
|
|
114
121
|
|
|
115
122
|
---
|
|
@@ -138,11 +145,10 @@ Modscape is made possible by these incredible open-source projects:
|
|
|
138
145
|
|
|
139
146
|
- [React Flow](https://reactflow.dev/) - Interactive node-based UI framework.
|
|
140
147
|
- [CodeMirror 6](https://codemirror.net/) - Next-generation code editor for the web.
|
|
148
|
+
- [Dagre](https://github.com/dagrejs/dagre) - Directed graph layout engine.
|
|
141
149
|
- [Lucide React](https://lucide.dev/) - Beautifully simple pixel-perfect icons.
|
|
142
150
|
- [Zustand](https://github.com/pmndrs/zustand) - Bear necessities for state management.
|
|
143
|
-
- [Express](https://expressjs.com/) - Fast, unopinionated web framework for Node.js.
|
|
144
151
|
- [js-yaml](https://github.com/nodeca/js-yaml) - JavaScript YAML parser and dumper.
|
|
145
|
-
- [Commander.js](https://github.com/tj/commander.js) - CLI framework.
|
|
146
152
|
|
|
147
153
|
## License
|
|
148
154
|
MIT
|
package/package.json
CHANGED
package/src/index.js
CHANGED
package/src/templates/rules.md
CHANGED
|
@@ -1,92 +1,120 @@
|
|
|
1
1
|
# Data Modeling Rules for this Project
|
|
2
2
|
|
|
3
3
|
## 1. Modeling Strategy
|
|
4
|
-
<!-- Define your modeling methodology here. Example: Data Vault 2.0, Star Schema, 3NF -->
|
|
5
4
|
- **Dimensional Modeling (Star Schema)**: Recommended for most analytical use cases.
|
|
6
5
|
- Use `appearance.type: fact` for central measurement tables.
|
|
7
6
|
- Use `appearance.type: dimension` for descriptive attribute tables.
|
|
8
7
|
- Use `appearance.type: mart` for downstream reporting-ready tables (Data Marts).
|
|
9
8
|
- **Data Vault 2.0**: For highly scalable enterprise data warehouses.
|
|
10
9
|
- Use `appearance.type: hub`, `link`, or `satellite`.
|
|
10
|
+
- **Generic Modeling**: For all other physical tables.
|
|
11
|
+
- Use `appearance.type: table` for raw mirror tables, simple RDB exports, or utility tables.
|
|
11
12
|
|
|
12
|
-
## 2.
|
|
13
|
-
Modscape supports
|
|
13
|
+
## 2. Table Naming Hierarchy
|
|
14
|
+
Modscape supports three levels of naming to bridge the gap between business and technology.
|
|
14
15
|
|
|
15
|
-
|
|
16
|
-
|
|
16
|
+
1. **Conceptual Name (`name`)**: The primary title on the canvas (e.g., "Customers"). Largest font.
|
|
17
|
+
2. **Logical Name (`logical_name`)**: Formal business name (e.g., "Customer Master"). Hidden on canvas if identical to Conceptual Name.
|
|
18
|
+
3. **Physical Name (`physical_name`)**: Actual database table name (e.g., `dim_customers`). Defaults to the technical `id` if empty.
|
|
17
19
|
|
|
20
|
+
## 3. Analytics & Visual Metadata
|
|
21
|
+
Modscape supports attributes to communicate the "Story" and "Grain" of your data.
|
|
22
|
+
|
|
23
|
+
### Grain & History (`appearance.sub_type` and `appearance.scd`)
|
|
18
24
|
- **`sub_type` (The "What")**:
|
|
19
25
|
- **For Fact Tables**: `transaction` (atomic event), `periodic` (state interval), `accumulating` (milestones), `factless` (occurrence only).
|
|
20
26
|
- **For Dimension Tables**: `conformed` (shared), `junk` (flags), `degenerate` (in-fact).
|
|
21
27
|
- **`scd` (The "How it changes")**:
|
|
22
28
|
`type0` (fixed), `type1` (overwrite), `type2` (history row), `type3` (history col), `type4` (history table), `type5` (1+4), `type6` (1+2+3), `type7` (1+2).
|
|
23
29
|
|
|
30
|
+
### Visual Semantics (`appearance.icon` and `appearance.color`)
|
|
31
|
+
Categorize entities visually to make the canvas intuitive:
|
|
32
|
+
- **`icon`**: Use a single Emoji (e.g., 🛒, 👥, 📋) to represent the business concept.
|
|
33
|
+
- **`color`**: Use Hex or RGBA codes for specific table highlighting (optional).
|
|
34
|
+
|
|
24
35
|
### Data Lineage (`lineage.upstream`)
|
|
25
|
-
|
|
36
|
+
Define dependencies to communicate data flow:
|
|
26
37
|
- Use `lineage.upstream: [table_id1, table_id2]` to list source tables.
|
|
27
38
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
-
|
|
31
|
-
-
|
|
39
|
+
## 4. Sample Data Stories
|
|
40
|
+
**Every table must include realistic sample data** to explain the context without requiring SQL queries.
|
|
41
|
+
- **Format**: A 2D array where the first row is headers.
|
|
42
|
+
- **Rule**: Provide at least 3 rows of high-quality, representative data.
|
|
43
|
+
```yaml
|
|
44
|
+
sampleData:
|
|
45
|
+
- [header1, header2]
|
|
46
|
+
- [value1, value2]
|
|
47
|
+
```
|
|
32
48
|
|
|
33
|
-
|
|
34
|
-
|
|
49
|
+
## 5. Physical Modeling Rules
|
|
50
|
+
Provide physical mapping metadata to ensure the model is implementation-ready.
|
|
35
51
|
|
|
36
|
-
|
|
37
|
-
- **
|
|
38
|
-
- **
|
|
39
|
-
- **Suffixes**: (e.g., `_id` for primary keys, `_h` for history tables)
|
|
52
|
+
- **Physical Naming**: All names (tables, schemas, columns) must use `snake_case`.
|
|
53
|
+
- **Schema Standards**: `raw`, `staging`, `analytics`, or `mart`.
|
|
54
|
+
- **Data Types**: Use standard SQL types (e.g., `VARCHAR`, `BIGINT`, `NUMBER(38,2)`, `DATE`, `TIMESTAMP_NTZ`).
|
|
40
55
|
|
|
41
|
-
##
|
|
42
|
-
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
56
|
+
## 6. Logical Column Rules
|
|
57
|
+
- **Key Flags**: Mark `isPrimaryKey: true`, `isForeignKey: true`, or `isPartitionKey: true` (for performance).
|
|
58
|
+
- **Metadata Columns**: Mark technical columns (e.g., `updated_at`) with `isMetadata: true` (🕒 icon).
|
|
59
|
+
- **Additivity**:
|
|
60
|
+
- `fully`: Can be summed across all dimensions (Σ icon).
|
|
61
|
+
- `semi`: Summing is restricted (e.g., bank balance) (Σ~ icon).
|
|
62
|
+
- `non`: Summing is never valid (e.g., unit price) (⊘ icon).
|
|
45
63
|
|
|
46
|
-
##
|
|
64
|
+
## 7. Relationship Cardinality
|
|
65
|
+
Explicitly define the nature of ER connections using `type`:
|
|
66
|
+
- Options: `one-to-one`, `one-to-many`, `many-to-one`, `many-to-many`.
|
|
67
|
+
|
|
68
|
+
## 8. Domain Organization
|
|
69
|
+
Group tables into business domains to manage complexity.
|
|
70
|
+
- Assign a unique `color` to each domain container.
|
|
71
|
+
- Use `description` to explain the domain's business purpose.
|
|
72
|
+
|
|
73
|
+
## 9. YAML Schema Reference (Hero Example)
|
|
47
74
|
```yaml
|
|
75
|
+
domains:
|
|
76
|
+
- id: sales
|
|
77
|
+
name: Sales Intelligence
|
|
78
|
+
color: "rgba(59, 130, 246, 0.1)"
|
|
79
|
+
tables: [fct_orders]
|
|
80
|
+
|
|
48
81
|
tables:
|
|
49
82
|
- id: fct_orders
|
|
50
83
|
name: Orders Fact
|
|
84
|
+
logical_name: "Sales Transaction Record"
|
|
85
|
+
physical_name: "fct_orders"
|
|
51
86
|
appearance:
|
|
52
87
|
type: fact
|
|
53
88
|
sub_type: transaction
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
89
|
+
icon: "🛒"
|
|
90
|
+
physical:
|
|
91
|
+
name: fct_orders
|
|
92
|
+
schema: analytics
|
|
57
93
|
columns:
|
|
58
94
|
- id: order_id
|
|
59
95
|
logical: { name: ID, type: Int, isPrimaryKey: true }
|
|
96
|
+
physical: { name: order_id, type: BIGINT }
|
|
60
97
|
- id: amount
|
|
61
|
-
logical: { name: Amount, type: Decimal, additivity: fully }
|
|
98
|
+
logical: { name: Amount, type: Decimal, additivity: fully }
|
|
99
|
+
physical: { name: total_amount, type: NUMBER(18,2) }
|
|
100
|
+
sampleData:
|
|
101
|
+
- [order_id, amount, status]
|
|
102
|
+
- [1001, 50.0, "COMPLETED"]
|
|
103
|
+
- [1002, 120.5, "PENDING"]
|
|
62
104
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
sub_type: conformed
|
|
68
|
-
scd: type2
|
|
69
|
-
columns:
|
|
70
|
-
- id: customer_id
|
|
71
|
-
logical: { name: ID, type: Int, isPrimaryKey: true }
|
|
72
|
-
|
|
73
|
-
- id: mart_daily_revenue
|
|
74
|
-
name: Daily Revenue Mart
|
|
75
|
-
appearance:
|
|
76
|
-
type: mart
|
|
77
|
-
sub_type: periodic
|
|
78
|
-
columns:
|
|
79
|
-
- id: report_date
|
|
80
|
-
logical: { name: Date, type: Date, isPrimaryKey: true }
|
|
81
|
-
- id: daily_amount
|
|
82
|
-
logical: { name: Revenue, type: Decimal, additivity: fully }
|
|
105
|
+
relationships:
|
|
106
|
+
- from: { table: dim_customers, column: customer_id }
|
|
107
|
+
to: { table: fct_orders, column: customer_id }
|
|
108
|
+
type: one-to-many
|
|
83
109
|
```
|
|
84
110
|
|
|
85
|
-
##
|
|
111
|
+
## 10. Layout Management
|
|
86
112
|
- **Visualizer Priority**: The GUI handles layout through drag-and-drop.
|
|
87
113
|
- **AI Agent Responsibility**: When creating new entities, assign initial (x, y) coordinates to place them near related tables.
|
|
88
114
|
|
|
89
|
-
##
|
|
115
|
+
## 11. The Golden Rules
|
|
90
116
|
- When updating `model.yaml`, always self-audit against these rules.
|
|
91
|
-
-
|
|
92
|
-
-
|
|
117
|
+
- **Utilize the 3-layer naming (Conceptual, Logical, Physical) for clarity.**
|
|
118
|
+
- **Generate realistic `sampleData` for every new entity.**
|
|
119
|
+
- **Provide `physical` mappings to make the model implementation-ready.**
|
|
120
|
+
- Use `isPartitionKey` for large tables to inform performance design.
|