modscape 0.9.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ja.md +36 -100
- package/README.md +30 -77
- package/package.json +1 -1
- package/src/dev.js +3 -19
- package/src/index.js +2 -2
- package/src/templates/rules.md +30 -21
- package/visualizer/package.json +5 -1
- package/visualizer-dist/assets/index-BO9IScoI.js +51 -0
- package/visualizer-dist/assets/index-CZOjoDrj.css +1 -0
- package/visualizer-dist/index.html +2 -2
- package/visualizer-dist/assets/index-BQzAkOWB.js +0 -40
- package/visualizer-dist/assets/index-p_KeFJzm.css +0 -1
package/README.ja.md
CHANGED
|
@@ -20,26 +20,25 @@
|
|
|
20
20
|
|
|
21
21
|
## 主な機能
|
|
22
22
|
|
|
23
|
-
- **YAML-as-Code**:
|
|
24
|
-
-
|
|
25
|
-
-
|
|
26
|
-
-
|
|
23
|
+
- **YAML-as-Code**: データアーキテクチャ全体を単一のYAMLファイルで定義。Gitによる変更管理が可能。
|
|
24
|
+
- **統合プロフェッショナル・エディタ**: **CodeMirror 6** を内蔵。シンタックスハイライト対応のエディタをサイドバーで直接利用可能。
|
|
25
|
+
- **統合 Undo/Redo & オートセーブ**:
|
|
26
|
+
- ビジュアル操作(ドラッグ、リサイズ、編集)がエディタの履歴と同期。 `Ctrl+Z` で直前の操作を元に戻せます。
|
|
27
|
+
- **オートセーブ**をONにすれば、キャンバス上の変更が即座にローカルファイルに反映されます。
|
|
28
|
+
- **ダーク/ライトモード対応**: 利用環境やドキュメント作成の用途に合わせて、ワンクリックでテーマを切り替え可能。
|
|
29
|
+
- **データ分析特化のモデリング**: `fact`, `dimension`, `mart`, `hub`, `link`, `satellite` などのエンティティタイプを標準サポート。
|
|
27
30
|
- **インタラクティブなビジュアルキャンバス**:
|
|
28
|
-
- **ドラッグで接続**:
|
|
29
|
-
-
|
|
30
|
-
-
|
|
31
|
+
- **ドラッグで接続**: カラム間のリレーションを直感的に作成。吸着(Snapping)機能で快適な操作感。
|
|
32
|
+
- **データリネージ・モード**: データの流れをアニメーション付きの点線矢印で可視化。
|
|
33
|
+
- **ドメイン階層ナビゲーション**: テーブルをビジネスドメインごとに整理し、構造化されたサイドバーから素早くアクセス。
|
|
31
34
|
- **分析メタデータ**:
|
|
32
|
-
-
|
|
33
|
-
- **SCD
|
|
34
|
-
- **加算規則(Additivity)**: カラムが合計可能か(`fully`, `semi`, `non
|
|
35
|
-
- **メタデータ/監査追跡**: 監査用カラムを専用アイコンで識別。
|
|
35
|
+
- **ファクトテーブル・タイプ**: `transaction`, `periodic`, `accumulating`, `factless` といったデータの性質を定義。
|
|
36
|
+
- **SCD管理**: `SCD Type 2` などの履歴管理方式を可視化。
|
|
37
|
+
- **加算規則(Additivity)**: カラムが合計可能か(`fully`, `semi`, `non`)を明示 (Σ アイコン)。
|
|
36
38
|
- **AIエージェント対応**: **Gemini, Claude, Codex** 用の雛形を内蔵。LLMを活用してモデリング作業を劇的に加速。
|
|
37
|
-
- **ドキュメント・エクスポート**: 社内WikiやGitHub/GitLab Pagesで利用可能な、Mermaid図入りのMarkdownを生成。
|
|
38
39
|
|
|
39
40
|
## インストール
|
|
40
41
|
|
|
41
|
-
npm経由でグローバルにインストールします:
|
|
42
|
-
|
|
43
42
|
```bash
|
|
44
43
|
npm install -g modscape
|
|
45
44
|
```
|
|
@@ -49,107 +48,51 @@ npm install -g modscape
|
|
|
49
48
|
## はじめに
|
|
50
49
|
|
|
51
50
|
### A: AI駆動のモデリング(推奨)
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
1. **初期化**: 使用するAIエージェントに合わせてモデリングルールと手順書を生成します。
|
|
51
|
+
1. **初期化**: 使用するAIエージェントに合わせてモデリングルールを生成します。
|
|
55
52
|
```bash
|
|
56
|
-
#
|
|
57
|
-
modscape init --gemini
|
|
58
|
-
|
|
59
|
-
# Claude Code の場合
|
|
60
|
-
modscape init --claude
|
|
61
|
-
|
|
62
|
-
# Codex の場合
|
|
63
|
-
modscape init --codex
|
|
53
|
+
modscape init --gemini # または --claude, --codex
|
|
64
54
|
```
|
|
65
55
|
2. **起動**: ビジュアライザーを起動します。
|
|
66
56
|
```bash
|
|
67
57
|
modscape dev model.yaml
|
|
68
58
|
```
|
|
69
|
-
3. **AIに指示**: AI
|
|
59
|
+
3. **AIに指示**: AIにこう伝えてください: *" .modscape/rules.md のルールに従って、model.yaml に新しい 'Marketing' ドメインを追加して。"*
|
|
70
60
|
|
|
71
61
|
### B: 手動モデリング
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
1. **YAML作成**: `model.yaml` ファイルを作成します([YAMLリファレンス](#モデルの定義-yaml)を参照)。
|
|
62
|
+
1. **YAML作成**: `model.yaml` ファイルを作成します。
|
|
75
63
|
2. **起動**: ビジュアライザーを起動します。
|
|
76
64
|
```bash
|
|
77
65
|
modscape dev model.yaml
|
|
78
66
|
```
|
|
79
|
-
3. **サンプルを試す**: 組み込みのパターンを確認できます:
|
|
80
|
-
```bash
|
|
81
|
-
modscape dev samples/
|
|
82
|
-
```
|
|
83
67
|
|
|
84
68
|
---
|
|
85
69
|
|
|
86
70
|
## モデルの定義 (YAML)
|
|
87
71
|
|
|
88
|
-
Modscapeは、データ分析のコンテキストに最適化されたスキーマを採用しています。この単一のファイルが、論理および物理データアーキテクチャの「信頼できる唯一の情報源(SSOT)」として機能します。
|
|
89
|
-
|
|
90
72
|
```yaml
|
|
91
|
-
# 1. Domains:
|
|
73
|
+
# 1. Domains: 関連するテーブルをグループ化するコンテナ
|
|
92
74
|
domains:
|
|
93
75
|
- id: core_sales
|
|
94
76
|
name: 主要売上
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
tables: [orders, products] # 所属するテーブルIDのリスト
|
|
77
|
+
color: "#3b82f6"
|
|
78
|
+
tables: [orders, products]
|
|
98
79
|
|
|
99
|
-
# 2. Tables:
|
|
80
|
+
# 2. Tables: エンティティ定義
|
|
100
81
|
tables:
|
|
101
82
|
- id: orders
|
|
102
83
|
name: ORDERS
|
|
103
84
|
appearance:
|
|
104
|
-
type: fact # fact | dimension | hub | link | satellite
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
icon: 📦 # カスタム絵文字や文字
|
|
109
|
-
color: "#f87171" # エンティティのテーマカラー
|
|
110
|
-
|
|
111
|
-
conceptual:
|
|
112
|
-
description: "販売取引記録"
|
|
113
|
-
tags: ["WHO", "WHEN", "HOW MUCH"] # 粒度の識別子(BEAM*等)
|
|
114
|
-
|
|
115
|
-
physical:
|
|
116
|
-
name: STG_ORDERS # 実際のDBテーブル名
|
|
117
|
-
schema: ANALYTICS_DB # DBスキーマ/名前空間
|
|
118
|
-
|
|
85
|
+
type: fact # fact | dimension | mart | hub | link | satellite
|
|
86
|
+
sub_type: transaction
|
|
87
|
+
scd: type2
|
|
88
|
+
icon: 📦
|
|
119
89
|
columns:
|
|
120
90
|
- id: order_id
|
|
121
91
|
logical:
|
|
122
92
|
name: ORDER_ID
|
|
123
|
-
type: Int
|
|
124
|
-
description: "ユニークな識別子"
|
|
125
93
|
isPrimaryKey: true
|
|
126
|
-
isForeignKey: false
|
|
127
|
-
isPartitionKey: false
|
|
128
|
-
# --- 分析メタデータ ---
|
|
129
|
-
isMetadata: false # 監査用カラム等に true を設定 (🕒 アイコン)
|
|
130
94
|
additivity: fully # fully | semi | non (Σ アイコン)
|
|
131
|
-
|
|
132
|
-
name: O_ID
|
|
133
|
-
type: NUMBER(38,0)
|
|
134
|
-
constraints: ["NOT NULL"]
|
|
135
|
-
|
|
136
|
-
- id: customer_id
|
|
137
|
-
logical: { name: CUSTOMER_ID, type: Int, isForeignKey: true }
|
|
138
|
-
|
|
139
|
-
# 3. Sample Data: データを通じたストーリーの提示
|
|
140
|
-
sampleData:
|
|
141
|
-
- [1001, 501]
|
|
142
|
-
- [1002, 502]
|
|
143
|
-
|
|
144
|
-
# 4. Relationships: カラムレベルの接続
|
|
145
|
-
relationships:
|
|
146
|
-
- from: { table: orders, column: customer_id }
|
|
147
|
-
to: { table: customers, column: id }
|
|
148
|
-
type: many-to-one # one-to-one | one-to-many | many-to-many
|
|
149
|
-
|
|
150
|
-
# 5. Layout: 座標情報 (自動管理)
|
|
151
|
-
layout:
|
|
152
|
-
orders: { x: 100, y: 100, width: 320, height: 400 }
|
|
95
|
+
isMetadata: false # 監査用カラム等 (🕒 アイコン)
|
|
153
96
|
```
|
|
154
97
|
|
|
155
98
|
---
|
|
@@ -157,39 +100,32 @@ layout:
|
|
|
157
100
|
## 使い方
|
|
158
101
|
|
|
159
102
|
### 開発モード (インタラクティブ)
|
|
160
|
-
YAMLを編集し、エンティティを視覚的に配置します。
|
|
161
|
-
|
|
162
103
|
```bash
|
|
163
104
|
modscape dev ./models
|
|
164
105
|
```
|
|
165
|
-
-
|
|
166
|
-
- **永続化**: レイアウトやメタデータの変更は、直接ファイルに書き戻されます。
|
|
106
|
+
- **永続化**: レイアウトやメタデータの変更は、直接ファイルに書き戻されます(オートセーブ対応)。
|
|
167
107
|
|
|
168
108
|
### ビルドモード (静的サイト)
|
|
169
|
-
チーム共有用のスタンドアロンなドキュメントサイトを生成します。
|
|
170
|
-
|
|
171
109
|
```bash
|
|
172
110
|
modscape build ./models -o docs-site
|
|
173
111
|
```
|
|
174
112
|
|
|
175
113
|
### エクスポートモード (Markdown)
|
|
176
|
-
Mermaid図を含むMarkdownドキュメントを生成します。
|
|
177
|
-
|
|
178
114
|
```bash
|
|
179
115
|
modscape export ./models -o docs/ARCHITECTURE.md
|
|
180
116
|
```
|
|
181
117
|
|
|
182
|
-
## AIエージェントとの統合
|
|
183
|
-
|
|
184
|
-
Modscapeは **AI-First** を掲げて設計されています。 `modscape init` を実行することで、 `.modscape/rules.md` を通じてAIエージェントと「契約」を結び、プロジェクト全体で一貫したモデリングパターン(命名規則、データ型など)を維持できます。
|
|
185
|
-
|
|
186
118
|
## クレジット
|
|
187
119
|
|
|
188
|
-
Modscape
|
|
189
|
-
|
|
190
|
-
-
|
|
191
|
-
-
|
|
192
|
-
-
|
|
120
|
+
Modscape は以下の素晴らしいオープンソースプロジェクトによって支えられています:
|
|
121
|
+
|
|
122
|
+
- [React Flow](https://reactflow.dev/) - インタラクティブなグラフ UI フレームワーク。
|
|
123
|
+
- [CodeMirror 6](https://codemirror.net/) - 次世代のウェブベース・コードエディタ。
|
|
124
|
+
- [Lucide React](https://lucide.dev/) - シンプルで美しいアイコンセット。
|
|
125
|
+
- [Zustand](https://github.com/pmndrs/zustand) - React 用の状態管理ライブラリ。
|
|
126
|
+
- [Express](https://expressjs.com/) - Node.js 用のウェブフレームワーク。
|
|
127
|
+
- [js-yaml](https://github.com/nodeca/js-yaml) - JavaScript 用 YAML パーサー。
|
|
128
|
+
- [Commander.js](https://github.com/tj/commander.js) - CLI フレームワーク。
|
|
193
129
|
|
|
194
130
|
## ライセンス
|
|
195
131
|
MIT
|
package/README.md
CHANGED
|
@@ -21,20 +21,22 @@ In modern data analysis platforms, data modeling is no longer just about drawing
|
|
|
21
21
|
## Key Features
|
|
22
22
|
|
|
23
23
|
- **YAML-as-Code**: Define your entire data architecture in a single, human-readable YAML file. Track changes via Git.
|
|
24
|
-
- **Instant Local Visualization**: Visualize your YAML models
|
|
25
|
-
- **
|
|
26
|
-
- **
|
|
24
|
+
- **Instant Local Visualization**: Visualize your YAML models instantly on your machine. No database connections or cloud infrastructure required—just point to your file and start exploring.
|
|
25
|
+
- **Integrated Professional Editor**: Powered by **CodeMirror 6**, providing syntax highlighting and a rich YAML editing experience directly in the sidebar.
|
|
26
|
+
- **Unified Undo/Redo & Auto-save**:
|
|
27
|
+
- Visual actions (dragging, resizing, metadata edits) are synchronized with the editor's history. Undo your last action with `Ctrl+Z`.
|
|
28
|
+
- Optional **Auto-save** ensures your local YAML is always up-to-date with your visual changes.
|
|
29
|
+
- **Dark/Light Mode Support**: Switch between themes seamlessly for better eye comfort or documentation exports.
|
|
30
|
+
- **Specialized Modeling Types**: Native support for entity types like `fact`, `dimension`, `mart`, `hub`, `link`, and `satellite`.
|
|
27
31
|
- **Interactive Visual Canvas**:
|
|
28
|
-
- **Drag-to-Connect**: Create relationships between columns intuitively.
|
|
29
|
-
- **
|
|
30
|
-
- **Domain
|
|
32
|
+
- **Drag-to-Connect**: Create relationships between columns intuitively with "Magnetic Snapping".
|
|
33
|
+
- **Data Lineage Mode**: Visualize data flow with animated dashed arrows.
|
|
34
|
+
- **Domain-Grouped Navigation**: Organize tables into visual business domains and navigate them via a structured sidebar.
|
|
31
35
|
- **Analytics Metadata**:
|
|
32
|
-
- **Fact
|
|
36
|
+
- **Fact Table Types**: Define `transaction`, `periodic`, `accumulating`, or `factless` grains.
|
|
33
37
|
- **SCD Management**: Visualize `SCD Type 2` and other history-tracking dimensions.
|
|
34
|
-
- **Additivity Rules**: Mark columns as `fully`, `semi`, or `non-additive`
|
|
35
|
-
- **Metadata/Audit Tracking**: Identify audit columns with specialized visual cues.
|
|
38
|
+
- **Additivity Rules**: Mark columns as `fully`, `semi`, or `non-additive` (Σ icon).
|
|
36
39
|
- **AI-Agent Ready**: Built-in scaffolding for **Gemini, Claude, and Codex** to accelerate your modeling workflow using LLMs.
|
|
37
|
-
- **Documentation Export**: Generate Mermaid-compatible Markdown for your internal wikis or GitHub/GitLab pages.
|
|
38
40
|
|
|
39
41
|
## Installation
|
|
40
42
|
|
|
@@ -76,80 +78,38 @@ Best for direct architectural control.
|
|
|
76
78
|
```bash
|
|
77
79
|
modscape dev model.yaml
|
|
78
80
|
```
|
|
79
|
-
3. **Explore Samples**: Try the built-in patterns:
|
|
80
|
-
```bash
|
|
81
|
-
modscape dev samples/
|
|
82
|
-
```
|
|
83
81
|
|
|
84
82
|
---
|
|
85
83
|
|
|
86
84
|
## Defining Your Model (YAML)
|
|
87
85
|
|
|
88
|
-
Modscape uses a schema designed for data analysis contexts.
|
|
86
|
+
Modscape uses a schema designed for data analysis contexts.
|
|
89
87
|
|
|
90
88
|
```yaml
|
|
91
89
|
# 1. Domains: Visual containers for grouping business logic
|
|
92
90
|
domains:
|
|
93
91
|
- id: core_sales
|
|
94
92
|
name: Core Sales
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
tables: [orders, products] # List of table IDs
|
|
93
|
+
color: "#3b82f6"
|
|
94
|
+
tables: [orders, products]
|
|
98
95
|
|
|
99
96
|
# 2. Tables: Entity definitions with multi-layer metadata
|
|
100
97
|
tables:
|
|
101
98
|
- id: orders
|
|
102
99
|
name: ORDERS
|
|
103
100
|
appearance:
|
|
104
|
-
type: fact # fact | dimension | hub | link | satellite
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
icon: 📦 # Custom emoji or character
|
|
109
|
-
color: "#f87171" # Custom theme color for this entity
|
|
110
|
-
|
|
111
|
-
conceptual:
|
|
112
|
-
description: "Sales transaction records"
|
|
113
|
-
tags: ["WHO", "WHEN", "HOW MUCH"] # BEAM* or Grain identifiers
|
|
114
|
-
|
|
115
|
-
physical:
|
|
116
|
-
name: STG_ORDERS # Actual table name in DB
|
|
117
|
-
schema: ANALYTICS_DB # DB schema/namespace
|
|
118
|
-
|
|
101
|
+
type: fact # fact | dimension | mart | hub | link | satellite
|
|
102
|
+
sub_type: transaction
|
|
103
|
+
scd: type2
|
|
104
|
+
icon: 📦
|
|
119
105
|
columns:
|
|
120
106
|
- id: order_id
|
|
121
107
|
logical:
|
|
122
108
|
name: ORDER_ID
|
|
123
109
|
type: Int
|
|
124
|
-
description: "Unique identifier"
|
|
125
110
|
isPrimaryKey: true
|
|
126
|
-
isForeignKey: false
|
|
127
|
-
isPartitionKey: false
|
|
128
|
-
# --- Analytics Metadata ---
|
|
129
|
-
isMetadata: false # Set true for audit columns (🕒 icon)
|
|
130
111
|
additivity: fully # fully | semi | non (Σ icon)
|
|
131
|
-
|
|
132
|
-
name: O_ID
|
|
133
|
-
type: NUMBER(38,0)
|
|
134
|
-
constraints: ["NOT NULL"]
|
|
135
|
-
|
|
136
|
-
- id: customer_id
|
|
137
|
-
logical: { name: CUSTOMER_ID, type: Int, isForeignKey: true }
|
|
138
|
-
|
|
139
|
-
# 3. Sample Data: Storytelling through data
|
|
140
|
-
sampleData:
|
|
141
|
-
- [1001, 501]
|
|
142
|
-
- [1002, 502]
|
|
143
|
-
|
|
144
|
-
# 4. Relationships: Column-level connectivity
|
|
145
|
-
relationships:
|
|
146
|
-
- from: { table: orders, column: customer_id }
|
|
147
|
-
to: { table: customers, column: id }
|
|
148
|
-
type: many-to-one # one-to-one | one-to-many | many-to-many
|
|
149
|
-
|
|
150
|
-
# 5. Layout: Visual coordinates (Auto-managed)
|
|
151
|
-
layout:
|
|
152
|
-
orders: { x: 100, y: 100, width: 320, height: 400 }
|
|
112
|
+
isMetadata: false # audit column (🕒 icon)
|
|
153
113
|
```
|
|
154
114
|
|
|
155
115
|
---
|
|
@@ -157,39 +117,32 @@ layout:
|
|
|
157
117
|
## Usage
|
|
158
118
|
|
|
159
119
|
### Development Mode (Interactive)
|
|
160
|
-
Edit your YAML and arrange entities visually.
|
|
161
|
-
|
|
162
120
|
```bash
|
|
163
121
|
modscape dev ./models
|
|
164
122
|
```
|
|
165
|
-
-
|
|
166
|
-
- **Persistence**: Layout and metadata changes are saved directly to your files.
|
|
123
|
+
- **Persistence**: Layout and metadata changes are saved directly to your files (supports Auto-save).
|
|
167
124
|
|
|
168
125
|
### Build Mode (Static Site)
|
|
169
|
-
Generate a standalone documentation site for your team.
|
|
170
|
-
|
|
171
126
|
```bash
|
|
172
127
|
modscape build ./models -o docs-site
|
|
173
128
|
```
|
|
174
129
|
|
|
175
130
|
### Export Mode (Markdown)
|
|
176
|
-
Generate documentation with Mermaid diagrams.
|
|
177
|
-
|
|
178
131
|
```bash
|
|
179
132
|
modscape export ./models -o docs/ARCHITECTURE.md
|
|
180
133
|
```
|
|
181
134
|
|
|
182
|
-
## AI Agent Integration
|
|
183
|
-
|
|
184
|
-
Modscape is designed to be **AI-First**. By running `modscape init`, you establish a contract with your AI agent via `.modscape/rules.md`, ensuring consistent modeling patterns (naming conventions, data types, etc.) across your entire project.
|
|
185
|
-
|
|
186
135
|
## Credits
|
|
187
136
|
|
|
188
|
-
Modscape is
|
|
189
|
-
|
|
190
|
-
-
|
|
191
|
-
-
|
|
192
|
-
-
|
|
137
|
+
Modscape is made possible by these incredible open-source projects:
|
|
138
|
+
|
|
139
|
+
- [React Flow](https://reactflow.dev/) - Interactive node-based UI framework.
|
|
140
|
+
- [CodeMirror 6](https://codemirror.net/) - Next-generation code editor for the web.
|
|
141
|
+
- [Lucide React](https://lucide.dev/) - Beautifully simple pixel-perfect icons.
|
|
142
|
+
- [Zustand](https://github.com/pmndrs/zustand) - Bear necessities for state management.
|
|
143
|
+
- [Express](https://expressjs.com/) - Fast, unopinionated web framework for Node.js.
|
|
144
|
+
- [js-yaml](https://github.com/nodeca/js-yaml) - JavaScript YAML parser and dumper.
|
|
145
|
+
- [Commander.js](https://github.com/tj/commander.js) - CLI framework.
|
|
193
146
|
|
|
194
147
|
## License
|
|
195
148
|
MIT
|
package/package.json
CHANGED
package/src/dev.js
CHANGED
|
@@ -43,7 +43,7 @@ function scanFiles(inputPaths) {
|
|
|
43
43
|
|
|
44
44
|
export async function startDevServer(paths, visualizerPath) {
|
|
45
45
|
const app = express();
|
|
46
|
-
app.use(express.json());
|
|
46
|
+
app.use(express.json({ limit: '10mb' }));
|
|
47
47
|
|
|
48
48
|
const inputPaths = Array.isArray(paths) ? paths : [paths];
|
|
49
49
|
let modelMap = scanFiles(inputPaths);
|
|
@@ -93,30 +93,14 @@ export async function startDevServer(paths, visualizerPath) {
|
|
|
93
93
|
}
|
|
94
94
|
});
|
|
95
95
|
|
|
96
|
-
// API to save layout changes
|
|
97
|
-
app.post('/api/layout', (req, res) => {
|
|
98
|
-
const modelPath = getModelPath(req.query.model);
|
|
99
|
-
if (!modelPath) return res.status(404).json({ error: 'Model not found' });
|
|
100
|
-
|
|
101
|
-
try {
|
|
102
|
-
const layout = req.body;
|
|
103
|
-
const content = fs.readFileSync(modelPath, 'utf8');
|
|
104
|
-
const data = yaml.load(content);
|
|
105
|
-
data.layout = layout;
|
|
106
|
-
fs.writeFileSync(modelPath, yaml.dump(data, { indent: 2, lineWidth: -1 }), 'utf8');
|
|
107
|
-
res.json({ success: true });
|
|
108
|
-
} catch (e) {
|
|
109
|
-
res.status(500).json({ error: e.message });
|
|
110
|
-
}
|
|
111
|
-
});
|
|
112
|
-
|
|
113
96
|
// API to save entire YAML content
|
|
114
|
-
app.post('/api/save
|
|
97
|
+
app.post('/api/save', (req, res) => {
|
|
115
98
|
const modelPath = getModelPath(req.query.model);
|
|
116
99
|
if (!modelPath) return res.status(404).json({ error: 'Model not found' });
|
|
117
100
|
|
|
118
101
|
try {
|
|
119
102
|
const { yaml: yamlContent } = req.body;
|
|
103
|
+
// Basic validation
|
|
120
104
|
yaml.load(yamlContent);
|
|
121
105
|
fs.writeFileSync(modelPath, yamlContent, 'utf8');
|
|
122
106
|
res.json({ success: true });
|
package/src/index.js
CHANGED
|
@@ -14,8 +14,8 @@ const program = new Command();
|
|
|
14
14
|
|
|
15
15
|
program
|
|
16
16
|
.name('modscape')
|
|
17
|
-
.description('A YAML-driven data modeling visualizer CLI')
|
|
18
|
-
.version('0.
|
|
17
|
+
.description('Modscape: A YAML-driven data modeling visualizer CLI')
|
|
18
|
+
.version('1.0.0');
|
|
19
19
|
|
|
20
20
|
program
|
|
21
21
|
.command('init')
|
package/src/templates/rules.md
CHANGED
|
@@ -5,24 +5,25 @@
|
|
|
5
5
|
- **Dimensional Modeling (Star Schema)**: Recommended for most analytical use cases.
|
|
6
6
|
- Use `appearance.type: fact` for central measurement tables.
|
|
7
7
|
- Use `appearance.type: dimension` for descriptive attribute tables.
|
|
8
|
+
- Use `appearance.type: mart` for downstream reporting-ready tables (Data Marts).
|
|
8
9
|
- **Data Vault 2.0**: For highly scalable enterprise data warehouses.
|
|
9
10
|
- Use `appearance.type: hub`, `link`, or `satellite`.
|
|
10
11
|
|
|
11
12
|
## 2. Analytics Metadata
|
|
12
13
|
Modscape supports attributes to communicate the "Story" and "Grain" of your data to humans and AI agents.
|
|
13
14
|
|
|
14
|
-
### Fact Table
|
|
15
|
-
|
|
16
|
-
- `periodic`: Snapshot fact table capturing state at set intervals (e.g., monthly inventory).
|
|
17
|
-
- `accumulating`: Fact table updated as a process moves through milestones (e.g., order → shipping → delivery).
|
|
18
|
-
- `factless`: Tables that record the occurrence of an event without numeric measures (e.g., event attendance).
|
|
15
|
+
### Fact Table Types / Dimension History (`appearance.sub_type` and `appearance.scd`)
|
|
16
|
+
Modscape separates a table's core nature from its history tracking method.
|
|
19
17
|
|
|
20
|
-
|
|
21
|
-
- `
|
|
22
|
-
- `
|
|
23
|
-
- `
|
|
24
|
-
|
|
25
|
-
|
|
18
|
+
- **`sub_type` (The "What")**:
|
|
19
|
+
- **For Fact Tables**: `transaction` (atomic event), `periodic` (state interval), `accumulating` (milestones), `factless` (occurrence only).
|
|
20
|
+
- **For Dimension Tables**: `conformed` (shared), `junk` (flags), `degenerate` (in-fact).
|
|
21
|
+
- **`scd` (The "How it changes")**:
|
|
22
|
+
`type0` (fixed), `type1` (overwrite), `type2` (history row), `type3` (history col), `type4` (history table), `type5` (1+4), `type6` (1+2+3), `type7` (1+2).
|
|
23
|
+
|
|
24
|
+
### Data Lineage (`lineage.upstream`)
|
|
25
|
+
Explicitly define dependencies to communicate the data flow:
|
|
26
|
+
- Use `lineage.upstream: [table_id1, table_id2]` to list source tables.
|
|
26
27
|
|
|
27
28
|
### Column Additivity (`logical.additivity`)
|
|
28
29
|
- `fully`: Can be summed across all dimensions (e.g., sales amount). Displays as `Σ`.
|
|
@@ -34,7 +35,7 @@ Mark technical or audit columns (like `created_at`, `dbt_updated_at`) with `isMe
|
|
|
34
35
|
|
|
35
36
|
## 3. Naming Conventions
|
|
36
37
|
- **Casing**: [Select one: snake_case / UPPER_SNAKE_CASE / camelCase]
|
|
37
|
-
- **Prefixes**: (e.g., `f_` for facts, `d_` for dimensions)
|
|
38
|
+
- **Prefixes**: (e.g., `f_` for facts, `d_` for dimensions, `m_` for marts)
|
|
38
39
|
- **Suffixes**: (e.g., `_id` for primary keys, `_h` for history tables)
|
|
39
40
|
|
|
40
41
|
## 4. Standard Data Types
|
|
@@ -49,28 +50,36 @@ tables:
|
|
|
49
50
|
name: Orders Fact
|
|
50
51
|
appearance:
|
|
51
52
|
type: fact
|
|
52
|
-
|
|
53
|
+
sub_type: transaction
|
|
54
|
+
scd: type2
|
|
53
55
|
conceptual:
|
|
54
56
|
description: "Records of customer purchases"
|
|
55
|
-
tags: ["WHO", "WHEN", "HOW MUCH"]
|
|
56
57
|
columns:
|
|
57
58
|
- id: order_id
|
|
58
59
|
logical: { name: ID, type: Int, isPrimaryKey: true }
|
|
59
60
|
- id: amount
|
|
60
|
-
logical: { name: Amount, type: Decimal, additivity: fully }
|
|
61
|
-
- id: updated_at
|
|
62
|
-
logical: { name: Updated At, type: Timestamp, isMetadata: true }
|
|
61
|
+
logical: { name: Amount, type: Decimal, additivity: fully }
|
|
63
62
|
|
|
64
63
|
- id: dim_customers
|
|
65
64
|
name: Customers Dim
|
|
66
65
|
appearance:
|
|
67
66
|
type: dimension
|
|
68
|
-
|
|
67
|
+
sub_type: conformed
|
|
68
|
+
scd: type2
|
|
69
69
|
columns:
|
|
70
70
|
- id: customer_id
|
|
71
71
|
logical: { name: ID, type: Int, isPrimaryKey: true }
|
|
72
|
-
|
|
73
|
-
|
|
72
|
+
|
|
73
|
+
- id: mart_daily_revenue
|
|
74
|
+
name: Daily Revenue Mart
|
|
75
|
+
appearance:
|
|
76
|
+
type: mart
|
|
77
|
+
sub_type: periodic
|
|
78
|
+
columns:
|
|
79
|
+
- id: report_date
|
|
80
|
+
logical: { name: Date, type: Date, isPrimaryKey: true }
|
|
81
|
+
- id: daily_amount
|
|
82
|
+
logical: { name: Revenue, type: Decimal, additivity: fully }
|
|
74
83
|
```
|
|
75
84
|
|
|
76
85
|
## 6. Layout Management
|
|
@@ -79,5 +88,5 @@ tables:
|
|
|
79
88
|
|
|
80
89
|
## 7. The Golden Rules
|
|
81
90
|
- When updating `model.yaml`, always self-audit against these rules.
|
|
82
|
-
- Set appropriate `appearance.type` and `
|
|
91
|
+
- Set appropriate `appearance.type`, `sub_type`, and `scd` for new tables.
|
|
83
92
|
- Use `additivity` for numeric measures to inform BI tools and analysts.
|
package/visualizer/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "visualizer",
|
|
3
3
|
"private": true,
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "1.0.0",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"scripts": {
|
|
7
7
|
"dev": "vite",
|
|
@@ -10,11 +10,15 @@
|
|
|
10
10
|
"preview": "vite preview"
|
|
11
11
|
},
|
|
12
12
|
"dependencies": {
|
|
13
|
+
"@codemirror/lang-yaml": "^6.1.2",
|
|
14
|
+
"@codemirror/theme-one-dark": "^6.1.3",
|
|
13
15
|
"@radix-ui/react-scroll-area": "^1.2.10",
|
|
14
16
|
"@radix-ui/react-slot": "^1.2.4",
|
|
15
17
|
"@radix-ui/react-tabs": "^1.1.13",
|
|
18
|
+
"@uiw/react-codemirror": "^4.25.7",
|
|
16
19
|
"class-variance-authority": "^0.7.1",
|
|
17
20
|
"clsx": "^2.1.1",
|
|
21
|
+
"codemirror": "^6.0.2",
|
|
18
22
|
"js-yaml": "^4.1.1",
|
|
19
23
|
"lucide-react": "^0.575.0",
|
|
20
24
|
"react": "^19.2.0",
|