obsitex 0.0.0__tar.gz

obsitex-0.0.0/PKG-INFO

@@ -0,0 +1,8 @@
+Metadata-Version: 2.1
+Name: obsitex
+Version: 0.0.0
+Author: Rui Reis
+Author-email: ruipedronetoreis12@gmail.com
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8

obsitex-0.0.0/README.md

@@ -0,0 +1,230 @@
+# ObsiTex: Convert Obsidian Notes to LaTeX
+
+ObsiTex is a Python package that automates the conversion of Obsidian Markdown files and folders into structured LaTeX documents. Designed for researchers, students, and technical writers, it ensures a seamless transition from Markdown to a fully formatted LaTeX file, preserving document structure and references.
+
+<p align="center">
+<img src="samples/images/banner.png" alt="ObsiTex" width="70%"/>
+</p>
+
+## Why Use ObsiTex?
+
+- Eliminates **manual LaTeX formatting** for Markdown-based notes.
+- Preserves headings, lists, equations, figures, citations, and more.
+- Supports **entire folders**, making it ideal for research papers and dissertations.
+- Uses **Jinja2 templates**, allowing full customization of the LaTeX output.
+
+
+
+## Quick Start
+
+To install ObsiTex, use pip:
+
+```sh
+pip install obsitex
+```
+
+Convert an Obsidian folder into a LaTeX document:
+
+```sh
+obsitex --input "My Obsidian Folder" --main-tex output.tex
+```
+
+Convert a single Markdown file:
+
+```sh
+obsitex --input "My Note.md" --main-tex output.tex
+```
+
+Use ObsiTex as a Python library:
+
+```python
+from obsitex import ObsidianParser
+
+parser = ObsidianParser()
+parser.add_dir("My Obsidian Folder")
+
+latex_content: str = parser.to_latex()
+```
+
+## Supported Elements
+
+Most of the standard Markdown elements are supported, including: 
+- Equations (inline and block)
+- Figures (with captions and metadata)
+- Tables (with captions and metadata)
+- Citations (using BibTeX references)
+- Headings (up to level 6, but can be customized)
+- Lists (enumerated and bulleted)
+- Blockquotes
+- Standard text formatting (bold, italics, code blocks, etc.)
+
+### Citations
+
+This system works best if used with the Obsidian plugin [obsidian-citation-plugin](https://github.com/hans/obsidian-citation-plugin), which allows for the easy insertion of citations in markdown files. The citations must be in the format `[[@citekey]]`, where `citekey` is the key of the reference in the BibTeX file.
+
+### Callouts
+
+#### Figure
+
+Use the following syntax to create a figure in Obsidian:
+
+```md
+> [!figure] Caption for the figure
+> ![[example-image.png]]
+> %%
+> width: 0.5
+> label: example-image
+> position: H
+> %%
+```
+
+Which may be broken down as follows:
+- `Caption for the figure`: The caption for the figure.
+- `![[example-image.png]]`: The path to the image, if a figure is present, then the graphics folder must be provided.
+- `%%`: This is a Obsidian comment, which allows for additional metadata to be added to the figure, without affecting the markdown rendering in Obsidian. If not present, default values will be used. This content must be YAML formatted.
+
+#### Table
+
+Use the following syntax to create a table in Obsidian:
+
+```md
+> [!table] Random Table
+> | Name  | Age | City     |
+> |-------|-----|----------|
+> | Alice | 25  | New York |
+> | Bob   | 30  | London   |
+> | Eve   | 28  | Tokyo    |
+> %%
+> prop: value
+> %%
+```
+
+This allows for the creation of tables in markdown, thus easily rendered in Obsidian, and then converted to LaTeX.
+
+Similarly to figures, metadata can be added to the table, in order to customize the rendering of the table in LaTeX. This content must be YAML formatted.
+
+#### Styling
+
+These are custom blocks, thus won't have styling in Obsidian unless explictly defined in a CSS snippet. You can define the styling by following the instructions in the [Obsidian documentation](https://help.obsidian.md/Editing+and+formatting/Callouts#Customize+callouts). 
+
+
+## Samples
+
+- [Motivation Letter](#single-file---motivation-letter-for-willy-wonkas-chocolate-factory): Single file motivation letter with no citations or figures, one of the simplest use cases.
+- [Sock Research Paper](#single-file---research-paper-on-socks): Single file research paper on socks with authors, affilitions, abstract, and content defined entirely in markdown.
+- [MSc Dissertation on Ducks](#folder---msc-thesis-on-ducks): Folder containing a MSc thesis on ducks, with multiple markdown files under a common folder, and an `Index.md` file defining the hierarchy of the thesis.
+
+These samples were all converted to PDF using XeLaTeX, and the output files are available in the `output` folder of each sample.
+
+### Single File - Motivation Letter for Willy Wonka's Chocolate Factory
+
+Charlie Beckett is applying for a position at Willy Wonka's Chocolate Factory, and has written a motivation letter in markdown. Of course this isn't the only factory he's applying to, so he wants the letter to be easily customizable for other applications.
+
+
+Unlike the other examples, this example doesn't require a BibTex file or graphics folder, as it doesn't contain any citations or figures. The LaTeX file can be generated by:
+
+```bash
+cd samples/motivation-letter;
+
+obsitex --input "Motivation Letter.md" \
+    --template template.tex \
+    --main-tex output/main.tex ;
+```
+
+#### Output Files
+
+- [main.tex](samples/motivation-letter/output/main.tex)
+- [main.pdf](samples/motivation-letter/output/main.pdf)
+
+
+### Single File - Research Paper on Socks
+
+Made up authors from the International Sock Research Institute, Textile Innovation Center, and Academy of Footwear Sciences have written a research paper on socks that was entirely developed in markdown. The authors now want to convert this document to pdf in order to submit it to a conference.
+
+The LaTeX file and correspondings Bib file can be generated by:
+
+```bash
+cd samples/sock-research-paper;
+
+obsitex --input "The Evolution of Socks.md" \
+    --graphics ../images \
+    --bibtex ../shared-references.bib \
+    --template template.tex \
+    --main-tex output/main.tex \
+    --main-bibtex output/main.bib ;
+```
+
+#### Output Files
+
+- [main.tex](samples/sock-research-paper/output/main.tex)
+- [main.bib](samples/sock-research-paper/output/main.bib)
+- [main.pdf](samples/sock-research-paper/output/main.pdf)
+
+### Folder - MSc Thesis on Ducks
+
+An unknown author has written a MSc thesis on ducks, and has organized the thesis in multiple markdown files under a common folder. The author now wants to convert this thesis to a single LaTeX file.
+
+The folder structure is as follows:
+
+
+```
+obsidian-folder
+├── Findings and Implications
+│   ├── Conclusion
+│   │   └── Conclusion.md
+│   ├── Findings and Discussion
+│   │   ├── Economic Viability.md
+│   │   ├── Findings and Discussion.md
+│   │   ├── Social and Cultural Implications.md
+│   │   └── Urban Planning and Infrastructure Challenges.md
+│   └── Findings and Implications.md
+├── Index.md
+└── Introduction and Background
+    ├── Introduction
+    │   └── Introduction.md
+    ├── Introduction and Background.md
+    ├── Literature Review
+    │   └── Literature Review.md
+    └── Methodology
+        └── Methodology.md
+
+8 directories, 11 files
+```
+
+`Index.md` defines the entry point for `obsitex`, by creating links between the different sections of the thesis, in the target order. In this example, the `Index.md` file is as follows:
+
+```markdown
+[[Introduction and Background]]
+
+[[Findings and Implications]]
+```
+
+Thus, the first part will be the `Introduction and Background` part, followed by the `Findings and Implications` part. The LaTeX file and correspondings Bib file can be generated by:
+
+```bash
+cd samples/msc-dissertation;
+
+obsitex --input obsidian-folder \
+    --graphics ../images \
+    --bibtex ../shared-references.bib \
+    --template template.tex \
+    --main-tex output/main.tex \
+    --main-bibtex output/main.bib ;
+```
+
+#### Output Files
+
+- [main.tex](samples/msc-dissertation/output/main.tex)
+- [main.bib](samples/msc-dissertation/output/main.bib)
+- [main.pdf](samples/msc-dissertation/output/main.pdf)
+
+## Acknowledgments
+
+This work was inspired by:
+- [Obsidian Citation Plugin](https://github.com/hans/obsidian-citation-plugin) – For enabling seamless reference management within Obsidian.
+- [Alejandro Daniel Noel](https://github.com/adanielnoel/Obsidian-to-latex) – His work served as an initial and valuable basis for this project.
+- [dbt](https://github.com/dbt-labs/dbt-core) – For giving me the idea of using Jinja2 templates for LaTeX conversion.
+
+## License
+
+This project is licensed under the MIT License.

obsitex-0.0.0/obsitex/__init__.py

@@ -0,0 +1 @@
+from obsitex.parser import ObsidianParser

obsitex-0.0.0/obsitex/cli.py

@@ -0,0 +1,102 @@
+import argparse
+import logging
+from pathlib import Path
+
+from obsitex import ObsidianParser
+from obsitex.constants import DEFAULT_JINJA2_MAIN_TEMPLATE
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Convert Obsidian notes to LaTeX")
+
+    # Defines the inputs
+    parser.add_argument(
+        "--input",
+        "-i",
+        type=Path,
+        help="Path to the input file or folder containing the Obsidian notes.",
+        required=True,
+    )
+
+    parser.add_argument(
+        "--bibtex",
+        "-b",
+        type=Path,
+        help="Path to the BibTeX database file with all references.",
+    )
+    parser.add_argument(
+        "--graphics",
+        "-g",
+        type=Path,
+        help="Path to the graphics folder, where all images are assumed to be stored.",
+    )
+    parser.add_argument(
+        "--template",
+        "-t",
+        type=Path,
+        help="Path to the Jinja2 LaTeX template, won't use template if not provided.",
+    )
+
+    # Defines the outputs
+    parser.add_argument(
+        "--main-tex",
+        "-mt",
+        type=Path,
+        help="Path to the LaTeX file that will be generated, containing all compiled LaTeX.",
+        required=True,
+    )
+    parser.add_argument(
+        "--main-bibtex",
+        "-mb",
+        type=Path,
+        help="Path to the BibTeX file that will be generated, containing the references - only generated if citations are used.",
+    )
+
+    # Administrative options
+    parser.add_argument(
+        "--debug",
+        "-d",
+        action="store_true",
+        help="Enable debug mode, which will print additional information by enabling logging.",
+    )
+
+    args = parser.parse_args()
+
+    if args.debug:
+        logging.basicConfig(level=logging.DEBUG)
+
+    if not args.input.exists():
+        raise FileNotFoundError(f"Input path {args.input} does not exist.")
+
+    # Read the template if it exists
+    if args.template is not None and args.template.is_file():
+        with open(args.template, "r") as file:
+            template = file.read()
+        logging.info(f"Using template from {args.template}.")
+    else:
+        template = DEFAULT_JINJA2_MAIN_TEMPLATE
+        logging.info("No template provided, using default template.")
+
+    # Create the parser
+    parser = ObsidianParser(
+        graphics_folder=args.graphics,
+        main_template=template,
+        bibtex_database_path=args.bibtex,
+        out_bitex_path=args.main_bibtex,
+    )
+
+    if args.input.is_dir():
+        parser.add_dir(args.input)
+    elif args.input.is_file():
+        parser.add_file(args.input)
+    else:
+        raise ValueError(f"Invalid path: {args.input}")
+
+    with open(args.main_tex, "w") as file:
+        file.write(parser.to_latex())
+
+    print(f"Output written to {args.main_tex}")
+
+
+if __name__ == "__main__":
+    main()

obsitex-0.0.0/obsitex/constants.py

@@ -0,0 +1,41 @@
+DEFAULT_JINJA2_JOB_TEMPLATE = "{{ parsed_latex_content }}"
+DEFAULT_JINJA2_MAIN_TEMPLATE = """
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% This file was automatically generated by obsitex.
+%% A tool to convert Obsidian markdown files to LaTeX.
+%% https://github.com/ruipreis/obsitex
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+{{ parsed_latex_content }}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%% End of generated file
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+"""
+
+DEFAULT_HLEVEL_MAPPING = {
+    -2: "part",
+    -1: "chapter",
+    0: "section",
+    1: "subsection",
+    2: "subsubsection",
+    3: "paragraph",
+}
+
+# How markers are placed in parsed latex
+DEFAULT_APPENDIX_MARKER = """
+\\appendix
+"""
+
+DEFAULT_BIBLIOGRAPHY_MARKER = """
+\\bibliography{main}                         
+"""
+
+SPECIAL_CALLOUTS = [
+    "[!figure]",
+    "[!table]",
+    "[!chart]",
+]
+
+QUOTE_MARKER = "> "
+CALLOUT_CONFIG_MARKER = "%%"

obsitex-0.0.0/obsitex/parser/__init__.py

@@ -0,0 +1,211 @@
+import logging
+from pathlib import Path
+from typing import Optional, Sequence
+
+import bibtexparser
+from jinja2 import Environment
+
+from obsitex.constants import (
+    DEFAULT_APPENDIX_MARKER,
+    DEFAULT_BIBLIOGRAPHY_MARKER,
+    DEFAULT_HLEVEL_MAPPING,
+    DEFAULT_JINJA2_JOB_TEMPLATE,
+    DEFAULT_JINJA2_MAIN_TEMPLATE,
+)
+from obsitex.parser.blocks import (
+    PARSEABLE_BLOCKS,
+    LaTeXBlock,
+    MarkerBlock,
+    Paragraph,
+    Section,
+)
+from obsitex.planner import ExecutionPlan
+from obsitex.planner.jobs import AddBibliography, AddHeader, AddText, PlannedJob
+
+# Increase logging level to bibtexparser - avoid warnings
+logging.getLogger("bibtexparser").setLevel(logging.ERROR)
+
+
+class ObsidianParser:
+    def __init__(
+        self,
+        bibtex_database_path: Optional[Path] = None,
+        implictly_add_bibtex: bool = True,
+        out_bitex_path: Optional[Path] = None,
+        graphics_folder: Optional[Path] = None,
+        job_template: str = DEFAULT_JINJA2_JOB_TEMPLATE,
+        main_template: str = DEFAULT_JINJA2_MAIN_TEMPLATE,
+        hlevel_mapping: dict = DEFAULT_HLEVEL_MAPPING,
+        appendix_marker: str = DEFAULT_APPENDIX_MARKER,
+        bibliography_marker: str = DEFAULT_BIBLIOGRAPHY_MARKER,
+        base_hlevel: int = 0,
+    ):
+        self.job_template = job_template
+        self.main_template = main_template
+        self.hlevel_mapping = hlevel_mapping
+        self.appendix_marker = appendix_marker
+        self.bibliography_marker = bibliography_marker
+        self.out_bitex_path = out_bitex_path
+
+        # Construct an execution plan, which will collect the jobs to run from
+        # the files and pths provided
+        self.execution_plan = ExecutionPlan(
+            bibtex_database_path=bibtex_database_path,
+            implictly_add_bibtex=implictly_add_bibtex,
+        )
+
+        # Extra arguments that should be injected when converting to latex
+        self.extra_args = {
+            "hlevel_mapping": self.hlevel_mapping,
+            "graphics_folder": graphics_folder,
+        }
+
+        # Flag to continuously check if in appendix
+        self.in_appendix = False
+
+        # Set of blocks that will be added to the main tex file
+        self.blocks: Sequence[LaTeXBlock] = []
+
+        # Keep track of the latest header level
+        self.base_hlevel = base_hlevel
+        self.latest_parsed_hlevel = base_hlevel
+
+    def add_file(self, file_path: Path, adjust_hlevel: bool = True):
+        # By default adding a file assumes a single file structure
+        if adjust_hlevel:
+            self.latest_parsed_hlevel = self.base_hlevel - 1
+
+        self.execution_plan.add_file(file_path)
+
+    def add_dir(self, dir_path: Path):
+        self.execution_plan.add_dir(dir_path)
+
+    def apply_jobs(self):
+        for job in self.execution_plan.iter_jobs():
+            self.parse_job(job)
+
+    def to_latex(self) -> str:
+        # Reset the parser blocks and apply
+        self.blocks = []
+        self.apply_jobs()
+
+        # Create template for job level and main
+        job_template = Environment().from_string(self.job_template)
+        main_template = Environment().from_string(self.main_template)
+
+        # Render each block onto the job template
+        rendered_blocks = "\n\n".join(
+            [
+                job_template.render(
+                    parsed_latex_content=block.formatted_text(**self.extra_args),
+                    **block.metadata,
+                )
+                for block in self.blocks
+            ]
+        )
+
+        # Render the main template with the rendered blocks
+        # the global variables are shared by all blocks, we use the first
+        # block for simplicity
+        if len(self.blocks) > 0:
+            global_configs = self.blocks[0].metadata
+        else:
+            global_configs = {}
+
+        return main_template.render(
+            parsed_latex_content=rendered_blocks,
+            **global_configs,
+        )
+
+    def parse_job(self, job: PlannedJob) -> str:
+        if not self.in_appendix:
+            self.in_appendix = job.is_in_appendix
+
+            # If in appendix, add the appendix marker
+            if self.in_appendix:
+                marker_block = MarkerBlock(self.appendix_marker)
+                marker_block.metadata = job.configs
+                self.blocks.append(marker_block)
+                logging.info("Added appendix marker to the parser.")
+
+        # Given a job, returns the corresponding latex code
+        if isinstance(job, AddHeader):
+            self.latest_parsed_hlevel = job.level
+            return self._parse_header(job)
+        elif isinstance(job, AddText):
+            return self._parse_text(job)
+        elif isinstance(job, AddBibliography):
+            return self._parse_bibliography(job)
+        else:
+            raise ValueError(f"Unknown job type {job}")
+
+    def _parse_header(self, job: AddHeader):
+        section_block = Section(job.level, job.header)
+        self.blocks.append(section_block)
+        logging.info(
+            f'Added header "{job.header}" with level {job.level} to the parser.'
+        )
+
+    def _parse_text(self, job: AddText):
+        lines = job.text.split("\n")
+        curr_i = 0
+        initial_block_count = len(self.blocks)
+
+        while curr_i < len(lines):
+            found_block = False
+
+            for block_class in PARSEABLE_BLOCKS:
+                block_instance = block_class.detect_block(lines, curr_i)
+
+                if block_instance is not None:
+                    block, curr_i = block_instance
+
+                    if isinstance(block, Section):
+                        block.hlevel += self.latest_parsed_hlevel
+
+                    found_block = True
+                    block.metadata = job.configs
+                    self.blocks.append(block)
+                    break
+
+            if not found_block:
+                # If remaining, assume it's a paragraph
+                paragraph_block = Paragraph(lines[curr_i])
+                paragraph_block.metadata = job.configs
+                self.blocks.append(paragraph_block)
+
+            curr_i += 1
+
+        logging.info(
+            f"Added {len(self.blocks) - initial_block_count} blocks to the parser, total {len(self.blocks)}."
+        )
+
+    def _parse_bibliography(self, job: AddBibliography):
+        if self.out_bitex_path is None:
+            raise ValueError("Bibliography was added but no output path was set.")
+
+        # Select the keys to be included in the bibliography, and export
+        with open(job.bibtex_path, "r") as file:
+            bib_database = bibtexparser.load(file)
+
+        # Index the bib tex keys and verify if all are present
+        bib_keys = {entry["ID"]: entry for entry in bib_database.entries}
+        missing_keys = [key for key in job.citations if key not in bib_keys]
+
+        if len(missing_keys) > 0:
+            raise ValueError(
+                f"Missing {len(missing_keys)} keys in bibliography: {missing_keys}"
+            )
+
+        # Write the selected entries to a new BibTeX file
+        new_db = bibtexparser.bparser.BibTexParser()  # Get a new BibDatabase instance
+        new_db.entries = [bib_keys[key] for key in job.citations]
+
+        with open(self.out_bitex_path, "w") as file:
+            bibtexparser.dump(new_db, file)
+
+        # Add the proper marker
+        marker_block = MarkerBlock(self.bibliography_marker)
+        marker_block.metadata = job.configs
+        self.blocks.append(marker_block)
+        logging.info("Added bibliography marker to the parser.")