noteshift 0.1.0__tar.gz

noteshift-0.1.0/PKG-INFO

@@ -0,0 +1,141 @@
+Metadata-Version: 2.3
+Name: noteshift
+Version: 0.1.0
+Summary: Export Notion pages and databases to Obsidian-friendly Markdown.
+Keywords: notion,obsidian,migration,markdown,export
+Author: Fragment256
+License: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Utilities
+Requires-Dist: httpx>=0.28.1,<0.29
+Requires-Dist: python-slugify>=8.0.4,<9
+Requires-Dist: typer>=0.24.1,<1
+Requires-Dist: mypy>=1.18.1,<2 ; extra == 'dev'
+Requires-Dist: pytest>=9.0.2,<10 ; extra == 'dev'
+Requires-Dist: pytest-cov>=6.0.0,<7 ; extra == 'dev'
+Requires-Dist: pytest-vcr>=1.0.2,<2 ; extra == 'dev'
+Requires-Dist: ruff>=0.15.4,<0.16 ; extra == 'dev'
+Requires-Dist: pytest>=9.0.2,<10 ; extra == 'test'
+Requires-Dist: pytest-cov>=6.0.0,<7 ; extra == 'test'
+Requires-Dist: pytest-vcr>=1.0.2,<2 ; extra == 'test'
+Requires-Python: >=3.11
+Project-URL: Homepage, https://github.com/Fragment256/noteshift
+Project-URL: Repository, https://github.com/Fragment256/noteshift
+Project-URL: Issues, https://github.com/Fragment256/noteshift/issues
+Provides-Extra: dev
+Provides-Extra: test
+Description-Content-Type: text/markdown
+
+# NoteShift (`noteshift`)
+
+**NoteShift** exports Notion content to Obsidian-friendly Markdown with predictable filenames, link rewriting, and checkpoint/resume support.
+
+[![CI](https://github.com/Fragment256/noteshift/actions/workflows/ci.yml/badge.svg)](https://github.com/Fragment256/noteshift/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/noteshift.svg)](https://pypi.org/project/noteshift/)
+[![Python Versions](https://img.shields.io/pypi/pyversions/noteshift.svg)](https://pypi.org/project/noteshift/)
+[![codecov](https://codecov.io/gh/Fragment256/noteshift/branch/main/graph/badge.svg)](https://codecov.io/gh/Fragment256/noteshift)
+
+## Why it exists
+
+Teams migrating from Notion to Obsidian consistently report four pains:
+
+1. broken internal links after export
+2. inconsistent filenames and folder layout
+3. long exports failing midway without resume
+4. low confidence in migration correctness
+
+NoteShift is focused on solving those pains first.
+
+## Current capabilities
+
+- Export a Notion page tree to Markdown
+- Export Notion data sources/databases through API layer
+- Rewrite internal links for Obsidian compatibility
+- Preserve and download attachments
+- Resume interrupted runs via checkpoint file
+- Emit migration report (`migration_report.json` + `.md`)
+
+## Documentation
+
+- Docs index: [`docs/`](docs/index.md)
+- Start here: [Getting started](docs/getting-started.md)
+- Library integration: [API contract](docs/api-contract.md)
+
+## Installation
+
+### Install from PyPI
+
+```bash
+uv tool install noteshift
+# or
+pipx install noteshift
+```
+
+### Install from source (development)
+
+```bash
+uv tool install .
+uv sync --extra dev --extra test
+```
+
+## Authentication
+
+Set a Notion integration token in `NOTION_TOKEN`.
+
+```bash
+export NOTION_TOKEN="secret_xxx"
+```
+
+## Basic usage
+
+```bash
+noteshift export \
+  --page-id "<notion-page-id>" \
+  --out ./export \
+  --max-depth 2 \
+  --overwrite
+```
+
+## Output
+
+A successful run writes:
+
+- Markdown files for exported pages
+- downloaded assets in the export tree
+- `.checkpoint.json` for resume
+- `migration_report.json`
+- `migration_report.md`
+
+## Development
+
+```bash
+uv sync --extra dev --extra test
+uv run ruff format .
+uv run ruff check .
+uv run mypy src
+uv run pytest --cov=noteshift --cov-report=term
+```
+
+## Contract tests (`pytest-vcr`)
+
+Contract tests are deterministic and replay HTTP traffic from sanitized cassettes:
+
+```bash
+uv run pytest -m contract
+```
+
+To re-record cassettes intentionally, set a real token in your environment and run:
+
+```bash
+VCR_RECORD_MODE=once uv run pytest -m contract
+```
+
+## License
+
+MIT

noteshift-0.1.0/README.md

@@ -0,0 +1,107 @@
+# NoteShift (`noteshift`)
+
+**NoteShift** exports Notion content to Obsidian-friendly Markdown with predictable filenames, link rewriting, and checkpoint/resume support.
+
+[![CI](https://github.com/Fragment256/noteshift/actions/workflows/ci.yml/badge.svg)](https://github.com/Fragment256/noteshift/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/noteshift.svg)](https://pypi.org/project/noteshift/)
+[![Python Versions](https://img.shields.io/pypi/pyversions/noteshift.svg)](https://pypi.org/project/noteshift/)
+[![codecov](https://codecov.io/gh/Fragment256/noteshift/branch/main/graph/badge.svg)](https://codecov.io/gh/Fragment256/noteshift)
+
+## Why it exists
+
+Teams migrating from Notion to Obsidian consistently report four pains:
+
+1. broken internal links after export
+2. inconsistent filenames and folder layout
+3. long exports failing midway without resume
+4. low confidence in migration correctness
+
+NoteShift is focused on solving those pains first.
+
+## Current capabilities
+
+- Export a Notion page tree to Markdown
+- Export Notion data sources/databases through API layer
+- Rewrite internal links for Obsidian compatibility
+- Preserve and download attachments
+- Resume interrupted runs via checkpoint file
+- Emit migration report (`migration_report.json` + `.md`)
+
+## Documentation
+
+- Docs index: [`docs/`](docs/index.md)
+- Start here: [Getting started](docs/getting-started.md)
+- Library integration: [API contract](docs/api-contract.md)
+
+## Installation
+
+### Install from PyPI
+
+```bash
+uv tool install noteshift
+# or
+pipx install noteshift
+```
+
+### Install from source (development)
+
+```bash
+uv tool install .
+uv sync --extra dev --extra test
+```
+
+## Authentication
+
+Set a Notion integration token in `NOTION_TOKEN`.
+
+```bash
+export NOTION_TOKEN="secret_xxx"
+```
+
+## Basic usage
+
+```bash
+noteshift export \
+  --page-id "<notion-page-id>" \
+  --out ./export \
+  --max-depth 2 \
+  --overwrite
+```
+
+## Output
+
+A successful run writes:
+
+- Markdown files for exported pages
+- downloaded assets in the export tree
+- `.checkpoint.json` for resume
+- `migration_report.json`
+- `migration_report.md`
+
+## Development
+
+```bash
+uv sync --extra dev --extra test
+uv run ruff format .
+uv run ruff check .
+uv run mypy src
+uv run pytest --cov=noteshift --cov-report=term
+```
+
+## Contract tests (`pytest-vcr`)
+
+Contract tests are deterministic and replay HTTP traffic from sanitized cassettes:
+
+```bash
+uv run pytest -m contract
+```
+
+To re-record cassettes intentionally, set a real token in your environment and run:
+
+```bash
+VCR_RECORD_MODE=once uv run pytest -m contract
+```
+
+## License
+
+MIT

noteshift-0.1.0/pyproject.toml

@@ -0,0 +1,78 @@
+[project]
+name = "noteshift"
+version = "0.1.0"
+description = "Export Notion pages and databases to Obsidian-friendly Markdown."
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "MIT" }
+authors = [
+    { name = "Fragment256" }
+]
+keywords = ["notion", "obsidian", "migration", "markdown", "export"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: End Users/Desktop",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Utilities",
+]
+dependencies = [
+    "httpx>=0.28.1,<0.29",
+    "python-slugify>=8.0.4,<9",
+    "typer>=0.24.1,<1",
+]
+
+[project.urls]
+Homepage = "https://github.com/Fragment256/noteshift"
+Repository = "https://github.com/Fragment256/noteshift"
+Issues = "https://github.com/Fragment256/noteshift/issues"
+
+[project.scripts]
+noteshift = "noteshift.cli:main"
+
+[project.optional-dependencies]
+dev = [
+    "mypy>=1.18.1,<2",
+    "pytest>=9.0.2,<10",
+    "pytest-cov>=6.0.0,<7",
+    "pytest-vcr>=1.0.2,<2",
+    "ruff>=0.15.4,<0.16",
+]
+test = [
+    "pytest>=9.0.2,<10",
+    "pytest-cov>=6.0.0,<7",
+    "pytest-vcr>=1.0.2,<2",
+]
+
+[tool.ruff]
+line-length = 88
+target-version = "py312"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B"]
+ignore = ["B008"]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+addopts = "-q"
+testpaths = ["tests"]
+markers = [
+    "contract: deterministic API contract tests (vcr-backed)",
+]
+
+[tool.mypy]
+python_version = "3.12"
+warn_unused_configs = true
+warn_redundant_casts = true
+warn_unused_ignores = true
+check_untyped_defs = true
+no_implicit_optional = true
+strict_equality = true
+mypy_path = "src"
+
+[build-system]
+requires = ["uv_build>=0.10.4,<0.11.0"]
+build-backend = "uv_build"

noteshift-0.1.0/src/noteshift/__init__.py

@@ -0,0 +1,28 @@
+from __future__ import annotations
+
+from importlib import import_module
+from typing import Any
+
+__all__ = [
+    "__version__",
+    "ExportPlan",
+    "ExportResult",
+    "NoteshiftConfig",
+    "PreflightReport",
+    "ProgressEvent",
+    "ProgressSink",
+    "preflight",
+    "run_export",
+]
+
+__version__ = "0.1.0"
+
+
+def __getattr__(name: str) -> Any:
+    if name in {"ExportPlan", "ExportResult", "NoteshiftConfig", "PreflightReport"}:
+        return getattr(import_module("noteshift.types"), name)
+    if name in {"ProgressEvent", "ProgressSink"}:
+        return getattr(import_module("noteshift.events"), name)
+    if name in {"preflight", "run_export"}:
+        return getattr(import_module("noteshift.api"), name)
+    raise AttributeError(f"module 'noteshift' has no attribute {name!r}")

noteshift-0.1.0/src/noteshift/api.py

@@ -0,0 +1,241 @@
+from __future__ import annotations
+
+import json
+import os
+from datetime import UTC, datetime
+from pathlib import Path
+
+from noteshift.checkpoint import Checkpoint
+from noteshift.db_export import export_child_database
+from noteshift.events import ProgressEvent, ProgressSink
+from noteshift.exporter import export_page_tree
+from noteshift.notion import NotionClient
+from noteshift.types import ExportPlan, ExportResult, NoteshiftConfig, PreflightReport
+
+
+def _emit(progress: ProgressSink | None, event: ProgressEvent) -> None:
+    if progress is not None:
+        progress(event)
+
+
+def _write_migration_report(
+    out_dir: Path, checkpoint: Checkpoint
+) -> tuple[Path, list[str]]:
+    report_errors: list[str] = []
+    report_data: dict[str, object] = {
+        "timestamp": datetime.now(UTC).isoformat(),
+        "pages_exported_total": len(checkpoint.page_ids),
+        "databases_exported_total": len(checkpoint.database_ids),
+        "rows_exported_total": checkpoint.rows_exported,
+        "attachments_downloaded_total": checkpoint.attachments_downloaded,
+        "files_written_total": len(checkpoint.files_written),
+        "warnings": checkpoint.warnings,
+    }
+
+    json_report_path = out_dir / "migration_report.json"
+    try:
+        with open(json_report_path, "w", encoding="utf-8") as f:
+            json.dump(report_data, f, indent=2)
+
+        md_report_path = out_dir / "migration_report.md"
+        md_lines = [
+            "# Migration Report",
+            "",
+            "## Summary",
+            "",
+            "| Metric | Value |",
+            "| :----- | :---- |",
+            f"| Timestamp | {report_data['timestamp']} |",
+            f"| Pages Exported | {report_data['pages_exported_total']} |",
+            f"| Databases Exported | {report_data['databases_exported_total']} |",
+            f"| Rows Exported | {report_data['rows_exported_total']} |",
+            (
+                "| Attachments Downloaded | "
+                f"{report_data['attachments_downloaded_total']} |"
+            ),
+            f"| Files Written | {report_data['files_written_total']} |",
+            "",
+            "## Warnings",
+            "",
+        ]
+        warnings = report_data["warnings"]
+        if isinstance(warnings, list) and warnings:
+            for warning in warnings:
+                md_lines.append(f"- {warning}")
+        else:
+            md_lines.append("No warnings.")
+
+        with open(md_report_path, "w", encoding="utf-8") as f:
+            f.write("\n".join(md_lines) + "\n")
+    except OSError as exc:
+        report_errors.append(f"Failed to write migration report files: {exc}")
+
+    return json_report_path, report_errors
+
+
+def _database_title(schema: dict) -> str:
+    title_obj = schema.get("title")
+    if isinstance(title_obj, list) and title_obj:
+        first = title_obj[0]
+        if isinstance(first, dict) and first.get("plain_text"):
+            return str(first["plain_text"])
+    return "Database"
+
+
+def preflight(plan: ExportPlan, config: NoteshiftConfig) -> PreflightReport:
+    errors: list[str] = []
+    warnings: list[str] = []
+
+    token = config.notion_token or os.getenv("NOTION_TOKEN")
+    if not token:
+        errors.append(
+            "Missing Notion token. Provide config.notion_token or set NOTION_TOKEN."
+        )
+
+    if not plan.page_ids and not plan.database_ids:
+        errors.append(
+            "Export plan is empty. Provide at least one page_id or database_id."
+        )
+
+    if config.max_depth < 0:
+        errors.append("max_depth must be >= 0.")
+
+    out_dir = config.out_dir.resolve()
+    if out_dir.exists():
+        if not out_dir.is_dir():
+            errors.append(
+                f"Output path {out_dir} exists and is not a directory. "
+                "Choose a directory path for out_dir."
+            )
+        elif any(out_dir.iterdir()) and not config.overwrite:
+            errors.append(
+                f"Output dir {out_dir} is not empty. "
+                "Use overwrite=True or choose a new out_dir."
+            )
+
+    return PreflightReport(ok=not errors, errors=errors, warnings=warnings)
+
+
+def run_export(
+    plan: ExportPlan,
+    config: NoteshiftConfig,
+    progress: ProgressSink | None = None,
+) -> ExportResult:
+    report = preflight(plan, config)
+    if not report.ok:
+        raise ValueError("; ".join(report.errors))
+
+    token = config.notion_token or os.getenv("NOTION_TOKEN")
+    if token is None:
+        raise ValueError("Missing Notion token.")
+
+    out_dir = config.out_dir.resolve()
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    checkpoint_path = out_dir / ".checkpoint.json"
+    checkpoint = Checkpoint.load(checkpoint_path)
+    if config.force:
+        checkpoint = Checkpoint()
+
+    all_errors: list[str] = []
+
+    _emit(progress, ProgressEvent(type="phase", message="starting_export"))
+
+    for page_id in plan.page_ids:
+        _emit(progress, ProgressEvent(type="item_start", id=page_id, title="page"))
+        try:
+            export_page_tree(
+                token=token,
+                root_page_id=page_id,
+                out_dir=out_dir,
+                checkpoint=checkpoint,
+                force=config.force,
+                max_depth=config.max_depth,
+            )
+            _emit(progress, ProgressEvent(type="item_done", id=page_id, title="page"))
+        except Exception as exc:  # noqa: BLE001
+            msg = f"Failed to export page {page_id}: {exc}"
+            all_errors.append(msg)
+            _emit(
+                progress,
+                ProgressEvent(type="error", id=page_id, title="page", message=msg),
+            )
+            if config.fail_fast:
+                raise RuntimeError(msg) from exc
+
+    if plan.database_ids:
+        client = NotionClient(token)
+        for database_id in plan.database_ids:
+            _emit(
+                progress,
+                ProgressEvent(type="item_start", id=database_id, title="database"),
+            )
+            try:
+                schema = client.get_data_source(database_id)
+                title = _database_title(schema)
+                db_result = export_child_database(
+                    client=client,
+                    data_source_id=database_id,
+                    title=title,
+                    out_dir=out_dir,
+                )
+                checkpoint.add_database(database_id)
+                checkpoint.add_rows(db_result.rows_exported)
+                for warning in db_result.warnings:
+                    checkpoint.add_warning(warning)
+                    _emit(
+                        progress,
+                        ProgressEvent(
+                            type="warning",
+                            id=database_id,
+                            title="database",
+                            message=warning,
+                        ),
+                    )
+                _emit(
+                    progress,
+                    ProgressEvent(type="item_done", id=database_id, title="database"),
+                )
+            except Exception as exc:  # noqa: BLE001
+                msg = f"Failed to export database {database_id}: {exc}"
+                all_errors.append(msg)
+                _emit(
+                    progress,
+                    ProgressEvent(
+                        type="error", id=database_id, title="database", message=msg
+                    ),
+                )
+                if config.fail_fast:
+                    raise RuntimeError(msg) from exc
+
+    checkpoint.save(checkpoint_path)
+    _emit(progress, ProgressEvent(type="checkpoint", message=str(checkpoint_path)))
+
+    report_path, report_errors = _write_migration_report(out_dir, checkpoint)
+
+    summary = ExportResult(
+        out_dir=out_dir,
+        report_path=report_path,
+        checkpoint_path=checkpoint_path,
+        pages_exported=len(checkpoint.page_ids),
+        databases_exported=len(checkpoint.database_ids),
+        rows_exported=checkpoint.rows_exported,
+        attachments_downloaded=checkpoint.attachments_downloaded,
+        warnings=list(checkpoint.warnings),
+        errors=all_errors + report_errors,
+    )
+
+    _emit(
+        progress,
+        ProgressEvent(
+            type="summary",
+            message=(
+                f"pages={summary.pages_exported}, "
+                f"databases={summary.databases_exported}, "
+                f"rows={summary.rows_exported}, "
+                f"attachments={summary.attachments_downloaded}"
+            ),
+        ),
+    )
+
+    return summary

noteshift-0.1.0/src/noteshift/checkpoint.py

@@ -0,0 +1,100 @@
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass, field
+from datetime import UTC
+from pathlib import Path
+from typing import Any
+
+
+@dataclass
+class Checkpoint:
+    """Tracks export progress for resumable exports."""
+
+    version: int = 1
+    timestamp: str = ""
+    # IDs of exported items
+    page_ids: set[str] = field(default_factory=set)
+    database_ids: set[str] = field(default_factory=set)
+    # Files written (relative paths from out_dir)
+    files_written: list[str] = field(default_factory=list)
+    # Statistics
+    attachments_downloaded: int = 0
+    rows_exported: int = 0
+    # Warnings accumulated
+    warnings: list[str] = field(default_factory=list)
+
+    @classmethod
+    def load(cls, path: Path) -> Checkpoint:
+        """Load checkpoint from file, or return empty if not exists."""
+        if not path.exists():
+            return cls()
+        try:
+            with open(path, encoding="utf-8") as f:
+                data = json.load(f)
+            return cls(
+                version=data.get("version", 1),
+                timestamp=data.get("timestamp", ""),
+                page_ids=set(data.get("page_ids", [])),
+                database_ids=set(data.get("database_ids", [])),
+                files_written=data.get("files_written", []),
+                attachments_downloaded=data.get("attachments_downloaded", 0),
+                rows_exported=data.get("rows_exported", 0),
+                warnings=data.get("warnings", []),
+            )
+        except (json.JSONDecodeError, KeyError, TypeError):
+            return cls()
+
+    def save(self, path: Path) -> None:
+        """Save checkpoint to file."""
+        from datetime import datetime
+
+        self.timestamp = datetime.now(UTC).isoformat()
+        data = {
+            "version": self.version,
+            "timestamp": self.timestamp,
+            "page_ids": sorted(self.page_ids),
+            "database_ids": sorted(self.database_ids),
+            "files_written": self.files_written,
+            "attachments_downloaded": self.attachments_downloaded,
+            "rows_exported": self.rows_exported,
+            "warnings": self.warnings,
+        }
+        path.parent.mkdir(parents=True, exist_ok=True)
+        with open(path, "w", encoding="utf-8") as f:
+            json.dump(data, f, indent=2)
+
+    def is_page_exported(self, page_id: str) -> bool:
+        return page_id in self.page_ids
+
+    def is_database_exported(self, db_id: str) -> bool:
+        return db_id in self.database_ids
+
+    def add_page(self, page_id: str) -> None:
+        self.page_ids.add(page_id)
+
+    def add_database(self, db_id: str) -> None:
+        self.database_ids.add(db_id)
+
+    def add_file(self, rel_path: str) -> None:
+        if rel_path not in self.files_written:
+            self.files_written.append(rel_path)
+
+    def add_attachment(self) -> None:
+        self.attachments_downloaded += 1
+
+    def add_rows(self, count: int) -> None:
+        self.rows_exported += count
+
+    def add_warning(self, warning: str) -> None:
+        self.warnings.append(warning)
+
+    def get_stats(self) -> dict[str, Any]:
+        return {
+            "pages_exported": len(self.page_ids),
+            "databases_exported": len(self.database_ids),
+            "files_written": len(self.files_written),
+            "attachments_downloaded": self.attachments_downloaded,
+            "rows_exported": self.rows_exported,
+            "warnings_count": len(self.warnings),
+        }