obsidian-import 0.1.0__tar.gz

obsidian_import-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.pyc
+*.egg-info/
+dist/
+build/
+.pixi/
+.pytest_cache/
+.hypothesis/
+.ruff_cache/
+.coverage

obsidian_import-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Matthias Christenson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

obsidian_import-0.1.0/PKG-INFO

@@ -0,0 +1,175 @@
+Metadata-Version: 2.4
+Name: obsidian-import
+Version: 0.1.0
+Summary: Extract files (PDF, DOCX, PPTX, XLSX) into Obsidian-flavored Markdown
+Project-URL: Documentation, https://neuralsignal.github.io/obsidian-import/
+Project-URL: Repository, https://github.com/neuralsignal/obsidian-import
+Author: Matthias Christenson
+License-Expression: MIT
+License-File: LICENSE
+Keywords: docx,extraction,import,markdown,obsidian,pdf,pptx,xlsx
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Office/Business
+Classifier: Topic :: Text Processing :: Markup :: Markdown
+Requires-Python: >=3.12
+Requires-Dist: click<9,>=8.0
+Requires-Dist: defusedxml<1,>=0.7
+Requires-Dist: openpyxl<4,>=3.1
+Requires-Dist: pdfplumber<1,>=0.11
+Requires-Dist: pypdf<6,>=5.0
+Requires-Dist: python-pptx<2,>=1.0
+Requires-Dist: pyyaml<7,>=6.0
+Provides-Extra: dev
+Requires-Dist: hypothesis<7,>=6.0; extra == 'dev'
+Requires-Dist: pytest-cov<6,>=5.0; extra == 'dev'
+Requires-Dist: pytest<9,>=8.0; extra == 'dev'
+Provides-Extra: docling
+Requires-Dist: docling>=2.0; extra == 'docling'
+Provides-Extra: markitdown
+Requires-Dist: markitdown[all]>=0.1; extra == 'markitdown'
+Description-Content-Type: text/markdown
+
+# obsidian-import
+
+Extract files (PDF, DOCX, PPTX, XLSX) into Obsidian-flavored Markdown.
+
+The mirror of [obsidian-export](https://github.com/neuralsignal/obsidian-export): where obsidian-export converts Obsidian notes to PDF/DOCX, obsidian-import converts external documents into Obsidian-ready markdown with YAML frontmatter.
+
+## Installation
+
+```bash
+pip install obsidian-import
+```
+
+With optional backends:
+
+```bash
+pip install obsidian-import[markitdown]    # fallback for HTML, CSV, etc.
+pip install obsidian-import[docling]       # high-quality ML-based extraction
+```
+
+## Quick Start
+
+### Single file
+
+```bash
+obsidian-import convert report.pdf --output vault/imports/report.md
+```
+
+### Batch extraction
+
+```bash
+obsidian-import batch --config config.yaml
+```
+
+### Check backend availability
+
+```bash
+obsidian-import doctor
+```
+
+## Python API
+
+```python
+from pathlib import Path
+from obsidian_import import extract_file, discover_files
+from obsidian_import.config import load_config
+from obsidian_import.output import format_output
+
+config = load_config(Path("config.yaml"))
+
+# Single file
+doc = extract_file(Path("report.pdf"), config)
+markdown = format_output(doc, config.output)
+
+# Batch discovery
+for file in discover_files(config):
+    print(f"{file.extension}  {file.size_bytes:,} bytes  {file.path}")
+```
+
+## Configuration
+
+Create a `config.yaml`:
+
+```yaml
+input:
+  directories:
+    - path: /path/to/documents
+      extensions: [".pdf", ".docx", ".pptx", ".xlsx"]
+      exclude: ["*.tmp", "~$*"]
+
+output:
+  directory: ./extracted
+  frontmatter: true
+  metadata_fields:
+    - title
+    - source
+    - original_path
+    - file_type
+    - extracted_at
+    - page_count
+
+backends:
+  pdf: native        # pdfplumber + pypdf
+  docx: native       # defusedxml
+  pptx: native       # python-pptx
+  xlsx: native       # openpyxl
+  default: native    # fallback for unknown extensions
+
+extraction:
+  timeout_seconds: 120
+  max_file_size_mb: 100
+  xlsx_max_rows_per_sheet: 500
+```
+
+## Backend Selection
+
+| Backend | Extensions | Dependencies | Quality |
+|---------|-----------|--------------|---------|
+| `native` | .pdf, .docx, .pptx, .xlsx | Core (included) | Good for text-heavy documents |
+| `markitdown` | Any | `[markitdown]` extra | Good fallback for HTML, CSV, etc. |
+| `docling` | Any | `[docling]` extra | Best for complex layouts, tables |
+
+## CLI Reference
+
+| Command | Description |
+|---------|-------------|
+| `obsidian-import convert <path>` | Extract a single file |
+| `obsidian-import discover --config <yaml>` | List matching files |
+| `obsidian-import batch --config <yaml>` | Extract all discovered files |
+| `obsidian-import doctor` | Check backend availability |
+
+## Output Format
+
+Extracted files are written as Obsidian-flavored markdown with YAML frontmatter:
+
+```markdown
+---
+title: Annual Report
+source: obsidian-import
+original_path: /documents/report.pdf
+file_type: pdf
+extracted_at: 2026-03-09T10:30:00Z
+page_count: 12
+---
+
+# Annual Report
+
+## Page 1
+
+Content extracted from the first page...
+```
+
+## Related Packages
+
+- [obsidian-export](https://github.com/neuralsignal/obsidian-export) — Convert Obsidian notes to PDF/DOCX
+- [agentic-brain](https://github.com/neuralsignal/agentic-brain) — Agentic knowledge management (consumes both packages)
+
+## License
+
+MIT

obsidian_import-0.1.0/README.md

@@ -0,0 +1,139 @@
+# obsidian-import
+
+Extract files (PDF, DOCX, PPTX, XLSX) into Obsidian-flavored Markdown.
+
+The mirror of [obsidian-export](https://github.com/neuralsignal/obsidian-export): where obsidian-export converts Obsidian notes to PDF/DOCX, obsidian-import converts external documents into Obsidian-ready markdown with YAML frontmatter.
+
+## Installation
+
+```bash
+pip install obsidian-import
+```
+
+With optional backends:
+
+```bash
+pip install obsidian-import[markitdown]    # fallback for HTML, CSV, etc.
+pip install obsidian-import[docling]       # high-quality ML-based extraction
+```
+
+## Quick Start
+
+### Single file
+
+```bash
+obsidian-import convert report.pdf --output vault/imports/report.md
+```
+
+### Batch extraction
+
+```bash
+obsidian-import batch --config config.yaml
+```
+
+### Check backend availability
+
+```bash
+obsidian-import doctor
+```
+
+## Python API
+
+```python
+from pathlib import Path
+from obsidian_import import extract_file, discover_files
+from obsidian_import.config import load_config
+from obsidian_import.output import format_output
+
+config = load_config(Path("config.yaml"))
+
+# Single file
+doc = extract_file(Path("report.pdf"), config)
+markdown = format_output(doc, config.output)
+
+# Batch discovery
+for file in discover_files(config):
+    print(f"{file.extension}  {file.size_bytes:,} bytes  {file.path}")
+```
+
+## Configuration
+
+Create a `config.yaml`:
+
+```yaml
+input:
+  directories:
+    - path: /path/to/documents
+      extensions: [".pdf", ".docx", ".pptx", ".xlsx"]
+      exclude: ["*.tmp", "~$*"]
+
+output:
+  directory: ./extracted
+  frontmatter: true
+  metadata_fields:
+    - title
+    - source
+    - original_path
+    - file_type
+    - extracted_at
+    - page_count
+
+backends:
+  pdf: native        # pdfplumber + pypdf
+  docx: native       # defusedxml
+  pptx: native       # python-pptx
+  xlsx: native       # openpyxl
+  default: native    # fallback for unknown extensions
+
+extraction:
+  timeout_seconds: 120
+  max_file_size_mb: 100
+  xlsx_max_rows_per_sheet: 500
+```
+
+## Backend Selection
+
+| Backend | Extensions | Dependencies | Quality |
+|---------|-----------|--------------|---------|
+| `native` | .pdf, .docx, .pptx, .xlsx | Core (included) | Good for text-heavy documents |
+| `markitdown` | Any | `[markitdown]` extra | Good fallback for HTML, CSV, etc. |
+| `docling` | Any | `[docling]` extra | Best for complex layouts, tables |
+
+## CLI Reference
+
+| Command | Description |
+|---------|-------------|
+| `obsidian-import convert <path>` | Extract a single file |
+| `obsidian-import discover --config <yaml>` | List matching files |
+| `obsidian-import batch --config <yaml>` | Extract all discovered files |
+| `obsidian-import doctor` | Check backend availability |
+
+## Output Format
+
+Extracted files are written as Obsidian-flavored markdown with YAML frontmatter:
+
+```markdown
+---
+title: Annual Report
+source: obsidian-import
+original_path: /documents/report.pdf
+file_type: pdf
+extracted_at: 2026-03-09T10:30:00Z
+page_count: 12
+---
+
+# Annual Report
+
+## Page 1
+
+Content extracted from the first page...
+```
+
+## Related Packages
+
+- [obsidian-export](https://github.com/neuralsignal/obsidian-export) — Convert Obsidian notes to PDF/DOCX
+- [agentic-brain](https://github.com/neuralsignal/agentic-brain) — Agentic knowledge management (consumes both packages)
+
+## License
+
+MIT

obsidian_import-0.1.0/obsidian_import/__init__.py

@@ -0,0 +1,81 @@
+"""obsidian-import: Extract files into Obsidian-flavored Markdown.
+
+Public API:
+    extract_file(path, config) -> ExtractedDocument
+    extract_text(path, config) -> str
+    discover_files(config) -> Iterator[DiscoveredFile]
+"""
+
+from __future__ import annotations
+
+from collections.abc import Iterator
+from pathlib import Path
+
+from obsidian_import.config import ImportConfig
+from obsidian_import.discovery import DiscoveredFile
+from obsidian_import.discovery import discover_files as _discover_files
+from obsidian_import.output import ExtractedDocument
+from obsidian_import.registry import extract_with_backend
+
+
+def extract_file(path: Path, config: ImportConfig) -> ExtractedDocument:
+    """Extract a single file to Obsidian-flavored markdown.
+
+    Uses the configured backend for the file's extension.
+    Returns an ExtractedDocument with the extracted markdown and metadata.
+    """
+    extension = path.suffix.lower()
+    extra_kwargs: dict[str, object] = {}
+
+    if extension == ".xlsx":
+        extra_kwargs["max_rows_per_sheet"] = config.extraction.xlsx_max_rows_per_sheet
+
+    markdown = extract_with_backend(
+        path,
+        backends=config.backends,
+        timeout_seconds=config.extraction.timeout_seconds,
+        **extra_kwargs,
+    )
+
+    page_count = _estimate_page_count(markdown, extension)
+
+    return ExtractedDocument(
+        source_path=path,
+        markdown=markdown,
+        title=path.stem,
+        file_type=extension.lstrip("."),
+        page_count=page_count,
+    )
+
+
+def discover_files(config: ImportConfig) -> Iterator[DiscoveredFile]:
+    """Discover files matching the configured input directories and extensions."""
+    return _discover_files(config)
+
+
+def extract_text(path: Path, config: ImportConfig) -> str:
+    """Extract raw markdown text from a file. No frontmatter, no metadata wrapping."""
+    extension = path.suffix.lower()
+    extra_kwargs: dict[str, object] = {}
+    if extension == ".xlsx":
+        extra_kwargs["max_rows_per_sheet"] = config.extraction.xlsx_max_rows_per_sheet
+    return extract_with_backend(
+        path,
+        backends=config.backends,
+        timeout_seconds=config.extraction.timeout_seconds,
+        **extra_kwargs,
+    )
+
+
+def _estimate_page_count(markdown: str, extension: str) -> int | None:
+    """Estimate page count from extracted markdown.
+
+    For PDFs, count '## Page N' headings. For other formats, return None.
+    """
+    if extension == ".pdf":
+        count = 0
+        for line in markdown.splitlines():
+            if line.startswith("## Page "):
+                count += 1
+        return count if count > 0 else None
+    return None

obsidian_import-0.1.0/obsidian_import/backends/__init__.py

@@ -0,0 +1 @@
+"""Backend modules for obsidian-import extractors."""

obsidian_import-0.1.0/obsidian_import/backends/docling.py

@@ -0,0 +1,31 @@
+"""High-quality document extraction using docling.
+
+Requires the [docling] extra: pip install obsidian-import[docling]
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from obsidian_import.exceptions import BackendNotAvailableError
+from obsidian_import.timeout import run_with_timeout
+
+
+def extract(path: Path, timeout_seconds: int) -> str:
+    """Extract text using docling for high-quality document conversion."""
+    try:
+        from docling.document_converter import DocumentConverter  # noqa: F811
+    except ImportError as exc:
+        raise BackendNotAvailableError(
+            "docling is not installed. Install with: pip install obsidian-import[docling]"
+        ) from exc
+
+    def _do_extract() -> str:
+        converter = DocumentConverter()
+        doc_result = converter.convert(str(path))
+        text = doc_result.document.export_to_markdown()
+        if not text or not text.strip():
+            return f"*No text content extracted from `{path.name}`.*"
+        return text
+
+    return run_with_timeout(_do_extract, timeout_seconds, "docling", path)

obsidian_import-0.1.0/obsidian_import/backends/markitdown.py

@@ -0,0 +1,31 @@
+"""Fallback extractor using markitdown for unrecognized formats.
+
+Requires the [markitdown] extra: pip install obsidian-import[markitdown]
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from obsidian_import.exceptions import BackendNotAvailableError
+from obsidian_import.timeout import run_with_timeout
+
+
+def extract(path: Path, timeout_seconds: int) -> str:
+    """Extract text using markitdown as a fallback converter."""
+    try:
+        from markitdown import MarkItDown  # noqa: F811
+    except ImportError as exc:
+        raise BackendNotAvailableError(
+            "markitdown is not installed. Install with: pip install obsidian-import[markitdown]"
+        ) from exc
+
+    def _do_extract() -> str:
+        converter = MarkItDown()
+        converted = converter.convert(str(path))
+        text = converted.text_content
+        if not text or not text.strip():
+            return f"*No text content extracted from `{path.name}`.*"
+        return text
+
+    return run_with_timeout(_do_extract, timeout_seconds, "markitdown", path)

obsidian_import-0.1.0/obsidian_import/backends/native_docx.py

@@ -0,0 +1,130 @@
+"""DOCX text extraction using defusedxml + zipfile.
+
+Opens the DOCX as a ZIP archive, parses word/document.xml to extract
+text with structure preservation (headings, paragraphs, tables).
+"""
+
+from __future__ import annotations
+
+import zipfile
+from pathlib import Path
+from xml.etree.ElementTree import Element
+
+from obsidian_import.exceptions import ExtractionError
+from obsidian_import.timeout import run_with_timeout
+
+_NS = {
+    "w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main",
+    "r": "http://schemas.openxmlformats.org/officeDocument/2006/relationships",
+}
+
+
+def extract(path: Path, timeout_seconds: int) -> str:
+    """Extract text from a DOCX file, returning markdown."""
+    return run_with_timeout(lambda: _extract_docx(path), timeout_seconds, "DOCX", path)
+
+
+def _extract_docx(path: Path) -> str:
+    """Internal DOCX extraction logic."""
+    from defusedxml.ElementTree import fromstring
+
+    if not zipfile.is_zipfile(str(path)):
+        raise ExtractionError(f"Not a valid DOCX (ZIP) file: {path}")
+
+    with zipfile.ZipFile(str(path), "r") as zf:
+        if "word/document.xml" not in zf.namelist():
+            raise ExtractionError(f"No word/document.xml found in: {path}")
+
+        doc_xml = zf.read("word/document.xml")
+        root = fromstring(doc_xml)
+
+    sections: list[str] = [f"# {path.stem}"]
+    body = root.find(f"{{{_NS['w']}}}body")
+    if body is None:
+        return f"# {path.stem}\n\n*No body content found.*"
+
+    for element in body:
+        tag = _local_name(element)
+
+        if tag == "p":
+            text = _extract_paragraph(element)
+            if text:
+                sections.append(text)
+
+        elif tag == "tbl":
+            table_md = _extract_table(element)
+            if table_md:
+                sections.append(table_md)
+
+    return "\n\n".join(sections)
+
+
+def _local_name(element: Element) -> str:
+    """Get the local name of an XML element (strip namespace)."""
+    tag = element.tag
+    if "}" in tag:
+        return tag.split("}", 1)[1]
+    return tag
+
+
+def _extract_paragraph(para: Element) -> str:
+    """Extract text from a w:p element, applying heading styles."""
+    ppr = para.find(f"{{{_NS['w']}}}pPr")
+    heading_level = 0
+    if ppr is not None:
+        pstyle = ppr.find(f"{{{_NS['w']}}}pStyle")
+        if pstyle is not None:
+            style_val = pstyle.get(f"{{{_NS['w']}}}val", "")
+            if style_val.startswith("Heading"):
+                try:
+                    heading_level = int(style_val.replace("Heading", ""))
+                except ValueError:
+                    heading_level = 0
+
+    texts: list[str] = []
+    for run in para.iter(f"{{{_NS['w']}}}r"):
+        for t in run.iter(f"{{{_NS['w']}}}t"):
+            if t.text:
+                texts.append(t.text)
+
+    text = "".join(texts).strip()
+    if not text:
+        return ""
+
+    if heading_level > 0:
+        return f"{'#' * (heading_level + 1)} {text}"
+
+    return text
+
+
+def _extract_table(tbl: Element) -> str:
+    """Extract a w:tbl element as a markdown table."""
+    rows: list[list[str]] = []
+
+    for tr in tbl.iter(f"{{{_NS['w']}}}tr"):
+        cells: list[str] = []
+        for tc in tr.iter(f"{{{_NS['w']}}}tc"):
+            cell_texts: list[str] = []
+            for p in tc.iter(f"{{{_NS['w']}}}p"):
+                p_text = _extract_paragraph(p)
+                if p_text:
+                    cell_texts.append(p_text)
+            cells.append(" ".join(cell_texts).replace("\n", " ").strip())
+        if cells:
+            rows.append(cells)
+
+    if not rows:
+        return ""
+
+    max_cols = max(len(row) for row in rows)
+    for row in rows:
+        while len(row) < max_cols:
+            row.append("")
+
+    headers = rows[0]
+    md = ["| " + " | ".join(headers) + " |"]
+    md.append("| " + " | ".join(["---"] * max_cols) + " |")
+    for row in rows[1:]:
+        md.append("| " + " | ".join(row) + " |")
+
+    return "\n".join(md)

obsidian_import-0.1.0/obsidian_import/backends/native_pdf.py

@@ -0,0 +1,74 @@
+"""PDF text extraction using pdfplumber + pypdf.
+
+Extracts text with layout preservation, tables as markdown, and form field metadata.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from obsidian_import.timeout import run_with_timeout
+
+
+def extract(path: Path, timeout_seconds: int) -> str:
+    """Extract text and tables from a PDF file, returning markdown."""
+    return run_with_timeout(lambda: _extract_pdf(path), timeout_seconds, "PDF", path)
+
+
+def _extract_pdf(path: Path) -> str:
+    """Internal PDF extraction logic."""
+    import pdfplumber
+    from pypdf import PdfReader
+
+    sections: list[str] = []
+
+    reader = PdfReader(str(path))
+    meta = reader.metadata
+    if meta:
+        title = meta.title or path.stem
+        if meta.author:
+            sections.append(f"**Author:** {meta.author}")
+        if meta.creation_date:
+            sections.append(f"**Created:** {meta.creation_date}")
+    else:
+        title = path.stem
+
+    sections.insert(0, f"# {title}")
+
+    fields = reader.get_fields()
+    if fields:
+        field_lines = ["", "## Form Fields", ""]
+        for name, field in fields.items():
+            field_type = field.get("/FT", "unknown")
+            value = field.get("/V", "")
+            field_lines.append(f"- **{name}** ({field_type}): {value}")
+        sections.append("\n".join(field_lines))
+
+    with pdfplumber.open(str(path)) as pdf:
+        for i, page in enumerate(pdf.pages, 1):
+            page_sections: list[str] = [f"\n## Page {i}\n"]
+
+            tables = page.extract_tables()
+            if tables:
+                for table in tables:
+                    if not table or not table[0]:
+                        continue
+                    headers = [str(cell or "").strip() for cell in table[0]]
+                    md_table = ["| " + " | ".join(headers) + " |"]
+                    md_table.append("| " + " | ".join(["---"] * len(headers)) + " |")
+                    for row in table[1:]:
+                        cells = [str(cell or "").strip().replace("\n", " ") for cell in row]
+                        while len(cells) < len(headers):
+                            cells.append("")
+                        cells = cells[: len(headers)]
+                        md_table.append("| " + " | ".join(cells) + " |")
+                    page_sections.append("\n".join(md_table))
+
+            text = page.extract_text()
+            if text:
+                page_sections.append(text.strip())
+
+            if len(page_sections) > 1:
+                sections.append("\n".join(page_sections))
+
+    return "\n\n".join(sections)