notion2pg-bulk 0.1.0__tar.gz

notion2pg_bulk-0.1.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.3
+Name: notion2pg-bulk
+Version: 0.1.0
+Summary: A Python package that automatically finds and migrates to PostgreSQL all databases in a Notion workspace.
+Keywords: notion,postgresql,migration,database,export,workspace,notion export,notion migrate
+Author: benvigano
+License: GPL-3.0-or-later
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Dist: notion-client>=2.2.1
+Requires-Dist: psycopg2-binary>=2.9.5
+Requires-Dist: sqlalchemy>=2.0.0
+Requires-Dist: tabulate>=0.9.0
+Requires-Dist: tqdm>=4.64.0
+Requires-Python: >=3.8
+Project-URL: Homepage, https://github.com/benvigano/notion2pg-bulk
+Project-URL: Issues, https://github.com/benvigano/notion2pg-bulk/issues
+Project-URL: Repository, https://github.com/benvigano/notion2pg-bulk
+Description-Content-Type: text/markdown
+
+# notion2pg_bulk
+
+- Automatically discovers and migrates **all databases in a workspace** regardless of their views or position in the page structure
+- Preserves **database descriptions and property descriptions**
+- Supports **relation properties** using PostgreSQL arrays
+- **Select field support**: Single and multi-select options stored in separate lookup tables with foreign key constraints
+- Complies with Notion API limits (3 requests/second average)
+
+## Installation
+```bash
+pip install notion2pg-bulk
+```
+
+## Setup and Usage
+
+### Requirements
+- a **Notion API key** connected to all the databased that you wish to migrate (note: Notion now allows to connect databases ditectly from the integration configuration page, under the 'Access' tab. For quickly connecting **all databases** in a workspace, just tick all top level pages.
+- PostgreSQL database connection URL
+
+### Interactive Mode
+**On (Default):**
+- Dispays detected databases and fields and asks for confirmation before running the migration
+- Progress bar
+- Shows post-migration notes
+
+**Off:**
+(`--quiet` flag in CLI or set `interactive_mode=False` in Python API)
+- `.run()` runs migration directly and silently
+
+### CLI
+**Options:**
+- `--notion-token`: Notion integration token (or set `NOTION_TOKEN` env var)
+- `--database-url`: PostgreSQL connection string (or set `DATABASE_URL` env var)
+- `--quiet`: Run in non-interactive mode (skip validation steps and progress bars)
+- `--extract-page-content`: Extract free-form content from page bodies (slower migration)
+
+**Examples:**
+```bash
+# Interactive mode (default)
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..."
+
+# Non-interactive mode (skip validation steps)
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..." --quiet
+
+# Extract page content (slower but includes free-form content)
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..." --extract-page-content
+
+# Non-interactive with page content extraction
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..." --quiet --extract-page-content
+```
+
+### Python API:
+```python
+from notion2pg_bulk import NotionMigrator
+import sqlalchemy as sa
+
+engine = sa.create_engine('postgresql://user:password@localhost/dbname')
+migrator = NotionMigrator(
+    notion_token="your_notion_integration_token",
+    db_connection=engine,
+    interactive_mode=True,
+    extract_page_content=True,
+)
+migrator.run()
+```
+
+## Property Type Mapping
+
+| Notion Property | PostgreSQL Type | Notes |
+|----------------|-----------------|-------|
+| Title | `TEXT` | Primary identifier |
+| Rich Text | `TEXT` | Preserved as markdown with formatting |
+| Number | `NUMERIC` | |
+| Select | `TEXT` | Single option value + lookup table with foreign key |
+| Multi-select | `TEXT[]` | Array of option values + lookup table with check constraints |
+| Date | `TIMESTAMP` | |
+| Checkbox | `BOOLEAN` | |
+| URL | `TEXT` | |
+| Email | `TEXT` | |
+| Phone | `TEXT` | |
+| Relation | `TEXT[]` | Array of related page IDs |
+| People | `TEXT[]` | Array of user IDs |
+| Files | `TEXT[]` | Array of file URLs |
+| Created time | `TIMESTAMP` | |
+| Created by | `TEXT` | User ID |
+| Last edited time | `TIMESTAMP` | |
+| Last edited by | `TEXT` | User ID |
+| Formula | - | ⚠️ Skipped (not supported by Notion API) |
+| Rollup | - | ⚠️ Skipped (not supported by Notion API) |
+
+## Additional Page Content Mode
+
+The migrator can optionally extract free-form content from database pages. Extracted content is stored in the `additional_page_content` column as plain text.
+
+```python
+migrator = NotionMigrator(
+    notion_token=token,
+    db_connection=engine,
+    extract_page_content=True  # Default is false
+)
+```
+
+**Supported blocks:**
+- Paragraph text
+- Headings (all levels)
+- Bulleted and numbered lists
+- Code blocks
+- To-do items
+- Callouts
+- Toggle blocks
+- Dividers
+- Embedded databases (ID reference only)
+
+**Unsupported blocks:**
+- Images and media files
+- Complex block types (equations, embeds, etc.)
+- Block formatting and styling
+- Page hierarchies and relationships
+
+**Post-migration analysis:**
+When page content extraction is enabled, the migrator provides a comprehensive summary of:
+- Unsupported block types found and their locations (grouped by record)
+- Embedded databases that were referenced in page content but not connected to the integration (with full database IDs)

notion2pg_bulk-0.1.0/README.md

@@ -0,0 +1,123 @@
+# notion2pg_bulk
+
+- Automatically discovers and migrates **all databases in a workspace** regardless of their views or position in the page structure
+- Preserves **database descriptions and property descriptions**
+- Supports **relation properties** using PostgreSQL arrays
+- **Select field support**: Single and multi-select options stored in separate lookup tables with foreign key constraints
+- Complies with Notion API limits (3 requests/second average)
+
+## Installation
+```bash
+pip install notion2pg-bulk
+```
+
+## Setup and Usage
+
+### Requirements
+- a **Notion API key** connected to all the databased that you wish to migrate (note: Notion now allows to connect databases ditectly from the integration configuration page, under the 'Access' tab. For quickly connecting **all databases** in a workspace, just tick all top level pages.
+- PostgreSQL database connection URL
+
+### Interactive Mode
+**On (Default):**
+- Dispays detected databases and fields and asks for confirmation before running the migration
+- Progress bar
+- Shows post-migration notes
+
+**Off:**
+(`--quiet` flag in CLI or set `interactive_mode=False` in Python API)
+- `.run()` runs migration directly and silently
+
+### CLI
+**Options:**
+- `--notion-token`: Notion integration token (or set `NOTION_TOKEN` env var)
+- `--database-url`: PostgreSQL connection string (or set `DATABASE_URL` env var)
+- `--quiet`: Run in non-interactive mode (skip validation steps and progress bars)
+- `--extract-page-content`: Extract free-form content from page bodies (slower migration)
+
+**Examples:**
+```bash
+# Interactive mode (default)
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..."
+
+# Non-interactive mode (skip validation steps)
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..." --quiet
+
+# Extract page content (slower but includes free-form content)
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..." --extract-page-content
+
+# Non-interactive with page content extraction
+notion2pg-bulk --notion-token "your_token" --database-url "postgresql://..." --quiet --extract-page-content
+```
+
+### Python API:
+```python
+from notion2pg_bulk import NotionMigrator
+import sqlalchemy as sa
+
+engine = sa.create_engine('postgresql://user:password@localhost/dbname')
+migrator = NotionMigrator(
+    notion_token="your_notion_integration_token",
+    db_connection=engine,
+    interactive_mode=True,
+    extract_page_content=True,
+)
+migrator.run()
+```
+
+## Property Type Mapping
+
+| Notion Property | PostgreSQL Type | Notes |
+|----------------|-----------------|-------|
+| Title | `TEXT` | Primary identifier |
+| Rich Text | `TEXT` | Preserved as markdown with formatting |
+| Number | `NUMERIC` | |
+| Select | `TEXT` | Single option value + lookup table with foreign key |
+| Multi-select | `TEXT[]` | Array of option values + lookup table with check constraints |
+| Date | `TIMESTAMP` | |
+| Checkbox | `BOOLEAN` | |
+| URL | `TEXT` | |
+| Email | `TEXT` | |
+| Phone | `TEXT` | |
+| Relation | `TEXT[]` | Array of related page IDs |
+| People | `TEXT[]` | Array of user IDs |
+| Files | `TEXT[]` | Array of file URLs |
+| Created time | `TIMESTAMP` | |
+| Created by | `TEXT` | User ID |
+| Last edited time | `TIMESTAMP` | |
+| Last edited by | `TEXT` | User ID |
+| Formula | - | ⚠️ Skipped (not supported by Notion API) |
+| Rollup | - | ⚠️ Skipped (not supported by Notion API) |
+
+## Additional Page Content Mode
+
+The migrator can optionally extract free-form content from database pages. Extracted content is stored in the `additional_page_content` column as plain text.
+
+```python
+migrator = NotionMigrator(
+    notion_token=token,
+    db_connection=engine,
+    extract_page_content=True  # Default is false
+)
+```
+
+**Supported blocks:**
+- Paragraph text
+- Headings (all levels)
+- Bulleted and numbered lists
+- Code blocks
+- To-do items
+- Callouts
+- Toggle blocks
+- Dividers
+- Embedded databases (ID reference only)
+
+**Unsupported blocks:**
+- Images and media files
+- Complex block types (equations, embeds, etc.)
+- Block formatting and styling
+- Page hierarchies and relationships
+
+**Post-migration analysis:**
+When page content extraction is enabled, the migrator provides a comprehensive summary of:
+- Unsupported block types found and their locations (grouped by record)
+- Embedded databases that were referenced in page content but not connected to the integration (with full database IDs)

notion2pg_bulk-0.1.0/pyproject.toml

@@ -0,0 +1,43 @@
+[project]
+name = "notion2pg-bulk"
+version = "0.1.0"
+description = "A Python package that automatically finds and migrates to PostgreSQL all databases in a Notion workspace."
+readme = "README.md"
+requires-python = ">=3.8"
+dependencies = [
+    "notion-client>=2.2.1",
+    "psycopg2-binary>=2.9.5",
+    "sqlalchemy>=2.0.0",
+    "tabulate>=0.9.0",
+    "tqdm>=4.64.0",
+]
+authors = [
+    { name = "benvigano" }
+]
+license = { text = "GPL-3.0-or-later" }
+keywords = ["notion", "postgresql", "migration", "database", "export", "workspace", "notion export", "notion migrate"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Database",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+
+[project.urls]
+Homepage = "https://github.com/benvigano/notion2pg-bulk"
+Repository = "https://github.com/benvigano/notion2pg-bulk"
+Issues = "https://github.com/benvigano/notion2pg-bulk/issues"
+
+[project.scripts]
+notion2pg-bulk = "notion2pg_bulk.cli:main"
+
+[build-system]
+requires = ["uv_build>=0.8.13,<0.9.0"]
+build-backend = "uv_build"

notion2pg_bulk-0.1.0/src/notion2pg_bulk/__init__.py

@@ -0,0 +1,6 @@
+"""
+Notion Workspace to PostgreSQL Migration Tool
+"""
+from .migrator import NotionMigrator
+
+__all__ = ["NotionMigrator"]

notion2pg_bulk-0.1.0/src/notion2pg_bulk/cli.py

@@ -0,0 +1,69 @@
+"""
+Command line interface for the Notion to PostgreSQL migration tool.
+"""
+
+import argparse
+import os
+import sys
+import sqlalchemy as sa
+from .migrator import NotionMigrator
+
+
+def main():
+    """Main CLI entry point."""
+    parser = argparse.ArgumentParser(
+        description="Migrate Notion workspace to PostgreSQL database"
+    )
+    
+    parser.add_argument(
+        "--notion-token",
+        help="Notion integration token (or set NOTION_TOKEN env var)"
+    )
+    
+    parser.add_argument(
+        "--database-url",
+        help="PostgreSQL connection string (or set DATABASE_URL env var)"
+    )
+    
+    parser.add_argument(
+        "--extract-page-content",
+        action="store_true",
+        help="Extract free-form content from page bodies (slower migration)"
+    )
+    
+    parser.add_argument(
+        "--quiet",
+        action="store_true",
+        help="Run in non-interactive mode (skip validation steps and progress bars)"
+    )
+    
+    args = parser.parse_args()
+    
+    # Get credentials
+    notion_token = args.notion_token or os.getenv("NOTION_TOKEN")
+    if not notion_token:
+        print("❌ Error: Notion token required. Set --notion-token or NOTION_TOKEN env var")
+        return 1
+    
+    database_url = args.database_url or os.getenv("DATABASE_URL")
+    if not database_url:
+        print("❌ Error: Database URL required. Set --database-url or DATABASE_URL env var")
+        return 1
+    
+    # Create database connection
+    engine = sa.create_engine(database_url)
+    
+    # Initialize and run migrator
+    migrator = NotionMigrator(
+        notion_token=notion_token,
+        db_connection=engine,
+        interactive_mode=not args.quiet,
+        extract_page_content=args.extract_page_content
+    )
+    
+    migrator.run()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())