mdengine 0.8.1__tar.gz → 0.10.0__tar.gz

{mdengine-0.8.1/src/mdengine.egg-info → mdengine-0.10.0}/PKG-INFO

@@ -1,12 +1,13 @@
 Metadata-Version: 2.4
 Name: mdengine
-Version: 0.8.1
+Version: 0.10.0
 Summary: Convert PDF, Office, images, audio, video, text/JSON/XML, ZIP archives, web URLs, databases, graphs, and OpenAPI specs to Markdown.
 Author: mdengine contributors
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/vishal7090/md-generator
 Project-URL: Repository, https://github.com/vishal7090/md-generator
 Project-URL: Issues, https://github.com/vishal7090/md-generator/issues
+Project-URL: Documentation, https://vishal7090.github.io/md-generator/
 Keywords: markdown,pdf,docx,pptx,xlsx,ocr,zip,url,html,youtube,playwright,database,openapi,codeflow
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3 :: Only
@@ -43,9 +44,14 @@ Requires-Dist: paddlepaddle>=2.5.0; extra == "image-ocr"
 Requires-Dist: paddleocr>=2.7.0.3; extra == "image-ocr"
 Requires-Dist: easyocr>=1.7.0; extra == "image-ocr"
 Provides-Extra: text
+Requires-Dist: xmltodict>=0.13.0; extra == "text"
+Requires-Dist: lxml>=5.0.0; extra == "text"
 Provides-Extra: archive
 Requires-Dist: Pillow>=10.0.0; extra == "archive"
 Requires-Dist: pytesseract>=0.3.10; extra == "archive"
+Provides-Extra: archive-formats
+Requires-Dist: py7zr>=0.21.0; extra == "archive-formats"
+Requires-Dist: patoolib>=2.0.0; extra == "archive-formats"
 Provides-Extra: url
 Requires-Dist: httpx>=0.27.0; extra == "url"
 Requires-Dist: beautifulsoup4>=4.12.0; extra == "url"
@@ -92,6 +98,7 @@ Requires-Dist: sqlalchemy>=2.0.0; extra == "db"
 Requires-Dist: pyyaml>=6.0.0; extra == "db"
 Requires-Dist: psycopg2-binary>=2.9.9; extra == "db"
 Requires-Dist: pymysql>=1.1.0; extra == "db"
+Requires-Dist: pyodbc>=5.0.0; extra == "db"
 Requires-Dist: oracledb>=2.0.0; extra == "db"
 Requires-Dist: pymongo>=4.6.0; extra == "db"
 Requires-Dist: mermaid-py>=0.8.0; extra == "db"
@@ -120,6 +127,29 @@ Provides-Extra: codeflow-semantic
 Requires-Dist: numpy>=1.24.0; extra == "codeflow-semantic"
 Requires-Dist: scikit-learn>=1.3.0; extra == "codeflow-semantic"
 Requires-Dist: sentence-transformers>=3.0.0; extra == "codeflow-semantic"
+Provides-Extra: log
+Requires-Dist: pandas>=2.0.0; extra == "log"
+Requires-Dist: python-dateutil>=2.8.0; extra == "log"
+Requires-Dist: pyyaml>=6.0.0; extra == "log"
+Provides-Extra: log-cluster
+Requires-Dist: scikit-learn>=1.3.0; extra == "log-cluster"
+Provides-Extra: log-semantic
+Requires-Dist: numpy>=1.24.0; extra == "log-semantic"
+Requires-Dist: scikit-learn>=1.3.0; extra == "log-semantic"
+Requires-Dist: sentence-transformers>=3.0.0; extra == "log-semantic"
+Provides-Extra: log-pretty
+Requires-Dist: loguru>=0.7.0; extra == "log-pretty"
+Provides-Extra: log-otel-proto
+Requires-Dist: opentelemetry-proto>=1.27.0; extra == "log-otel-proto"
+Requires-Dist: protobuf>=4.25.0; extra == "log-otel-proto"
+Provides-Extra: log-export-parquet
+Requires-Dist: pyarrow>=14.0.0; extra == "log-export-parquet"
+Provides-Extra: log-stream-kafka
+Requires-Dist: kafka-python>=2.0.2; extra == "log-stream-kafka"
+Provides-Extra: log-stream-redis
+Requires-Dist: redis>=5.0.0; extra == "log-stream-redis"
+Provides-Extra: log-stream-ws
+Requires-Dist: websockets>=12.0; extra == "log-stream-ws"
 Provides-Extra: api
 Requires-Dist: fastapi>=0.115.0; extra == "api"
 Requires-Dist: uvicorn[standard]>=0.32.0; extra == "api"
@@ -129,6 +159,14 @@ Requires-Dist: pydantic-settings>=2.0.0; extra == "api"
 Provides-Extra: mcp
 Requires-Dist: mcp>=1.2.0; extra == "mcp"
 Requires-Dist: fastmcp>=2.3.0; extra == "mcp"
+Provides-Extra: docs
+Requires-Dist: mkdocs-material; extra == "docs"
+Requires-Dist: mkdocstrings; extra == "docs"
+Requires-Dist: mkdocstrings-python; extra == "docs"
+Requires-Dist: pymdown-extensions; extra == "docs"
+Requires-Dist: mkdocs-mermaid2-plugin; extra == "docs"
+Requires-Dist: mkdocs-git-revision-date-localized-plugin; extra == "docs"
+Requires-Dist: mike; extra == "docs"
 Provides-Extra: skill-openai
 Requires-Dist: openai>=1.30.0; extra == "skill-openai"
 Provides-Extra: skill-rag-chroma
@@ -148,6 +186,8 @@ Requires-Dist: mdengine[graph]; extra == "dev"
 Requires-Dist: mdengine[openapi]; extra == "dev"
 Requires-Dist: mdengine[codeflow]; extra == "dev"
 Requires-Dist: mdengine[codeflow-treesitter]; extra == "dev"
+Requires-Dist: mdengine[log]; extra == "dev"
+Requires-Dist: mdengine[log-cluster]; extra == "dev"
 Provides-Extra: all
 Requires-Dist: pymupdf>=1.24.0; extra == "all"
 Requires-Dist: pdfplumber>=0.11.0; extra == "all"
@@ -185,7 +225,7 @@ Single Python distribution for converting **PDF**, **Word (.docx)**, **PowerPoin
 - **Python:** 3.10+
 - **License:** [MIT](LICENSE)
 
-**Quick links:** [On a new computer](#on-a-new-computer) · [Command-line execution](#command-line-execution) · [Python library](#python-library) · [Audio and video](#audio-and-video-to-markdown) · [HTTP API](#http-api-fastapi) · [MCP](#mcp-model-context-protocol) · [AI assistant CLI](#ai-assistant-cli) · [Development](#development) · [Code of Conduct](CODE_OF_CONDUCT.md)
+**Quick links:** [On a new computer](#on-a-new-computer) · [Command-line execution](#command-line-execution) · [Python library](#python-library) · [Audio and video](#audio-and-video-to-markdown) · [HTTP API](#http-api-fastapi) · [MCP](#mcp-model-context-protocol) · [AI assistant CLI](#ai-assistant-cli) · [Development](#development) · [Published documentation](https://vishal7090.github.io/md-generator/) · [Code of Conduct](CODE_OF_CONDUCT.md)
 
 ---
 
@@ -254,9 +294,23 @@ pip install "mdengine[all]"
 | `url-full` | `url` plus PDF/Word/PPTX/XLSX/archive stack for **post-converting** downloaded linked files to Markdown |
 | `audio` | **Audio → Markdown** via Whisper (`openai-whisper`); ships `imageio-ffmpeg` for a bundled **ffmpeg** when none is on `PATH` |
 | `video` | **Video → Markdown** (ffmpeg extracts mono 16 kHz WAV, then same Whisper stack as `audio`) |
+| `youtube` | **YouTube → Markdown** (captions + page metadata; `youtube-transcript-api`, httpx, BeautifulSoup, lxml) |
+| `playwright` | **Playwright-rendered SPA → Markdown** (`playwright` + same HTML stack as `url`; run `playwright install chromium` after install) |
+| `db` | **Database metadata → Markdown** (SQLAlchemy, drivers, `mermaid-py` for ERD helpers) |
+| `openapi` | **OpenAPI / Swagger → Markdown** (`prance`, `openapi-spec-validator`, PyYAML) |
+| `codeflow` | **Static codeflow / call graphs → Markdown** (`networkx`, `javalang`; core scanners) |
+| `codeflow-worker` | Optional **Celery + Redis** workers for the codeflow API async path |
+| `codeflow-treesitter` | **Tree-sitter** parsers (JS/TS/TSX, C++) for richer codeflow slices |
+| `codeflow-clang` | **libclang** bindings for C/C++ parsing when used by the codeflow pipeline |
+| `codeflow-semantic` | **Semantic clustering** (SentenceTransformers, scikit-learn, numpy; large install) |
+| `log` | **Log files → Markdown** (`pandas`, PyYAML, date parsing) |
+| `log-cluster` | Log pipeline **clustering** (scikit-learn) |
+| `log-semantic` | Log pipeline **semantic** helpers (SentenceTransformers + scikit-learn; large) |
+| `log-pretty` | Optional **loguru** for pretty console diagnostics during log conversion |
 | `api` | FastAPI, uvicorn, httpx, pydantic-settings |
 | `mcp` | MCP servers (`mcp`, `fastmcp` where used) |
 | `graph` | **Graph → Markdown** (Neo4j Bolt + NetworkX GraphML/GML): `networkx`, `neo4j`, `pyyaml` |
+| `docs` | **MkDocs** site tooling (Material, mkdocstrings, Mermaid plugin, mike) for maintainers publishing reference docs |
 | `dev` | pytest + API/MCP test helpers |
 | `all` | Large superset of dependencies (use only if you need everything) |
 | `skill-openai` | Optional helpers for OpenAI-style calls with assembled skill context (`md_generator.tools.assistant`) |
@@ -317,7 +371,17 @@ If the shell reports “command not found”, ensure the Python **Scripts** dire
 | `md-openapi` | `md_generator.openapi.cli.main:main` | `pip install "mdengine[openapi]"` then `md-openapi generate --file openapi.yaml --output ./docs` (or `mdengine openapi-to-md generate …`) |
 | `md-openapi-api` | `md_generator.openapi.api.run:main` | FastAPI on port **8015** (`OPENAPI_TO_MD_PORT`): `POST /openapi-to-md/generate` (OpenAPI upload → ZIP), `/health`, MCP at **`/mcp`** |
 | `md-openapi-mcp` | `md_generator.openapi.api.mcp_server:main` | Standalone MCP: `api_validate_openapi_yaml`, `api_generate_readme_markdown`, `api_run_sync_zip_base64` |
-| `mdengine` | `md_generator.engine_cli:main` | `mdengine ai assist …`, `mdengine ai export …`, `mdengine skill build …`, `mdengine db-to-md …`, `mdengine graph-to-md …`, `mdengine openapi-to-md generate …` |
+| `md-playwright` | `md_generator.playwright.cli:main` | `pip install "mdengine[playwright]"` then `md-playwright https://spa.example/app ./spa-out` (after `playwright install chromium`) |
+| `md-playwright-api` | `md_generator.playwright.api.run:main` | FastAPI + MCP on port **8014** (`PLAYWRIGHT_TO_MD_API_PORT` / settings); see [HTTP API](#http-api-fastapi) |
+| `md-playwright-mcp` | `md_generator.playwright.api.mcp_server:main` | Standalone MCP for Playwright capture tools |
+| `md-codeflow` | `md_generator.codeflow.cli.main:main` | Same as `codeflow` — `md-codeflow scan …` (alias of packaged `codeflow` script) |
+| `codeflow` | `md_generator.codeflow.cli.main:main` | `pip install "mdengine[codeflow]"` then `codeflow scan path/to/src --output ./out --lang java` |
+| `md-codeflow-api` | `md_generator.codeflow.api.run:main` | FastAPI on port **8016** (`CODEFLOW_TO_MD_PORT`); upload/workspace job flow + MCP at **`/mcp`** when MCP extras resolve |
+| `md-codeflow-mcp` | `md_generator.codeflow.api.mcp_server:main` | Standalone **stdio** MCP for codeflow tools |
+| `md-log` | `md_generator.log.cli.main:main` | `pip install "mdengine[log]"` then `md-log --config log.yaml` (or `mdengine log-to-md …`) |
+| `md-log-api` | `md_generator.log.api.run:main` | FastAPI on port **8012** (`LOG_TO_MD_PORT`); set a different port if **`md-graph-api`** or **`md-video-api`** already uses **8012** |
+| `md-log-mcp` | `md_generator.log.api.mcp_server:main` | Standalone MCP (`stdio` \| `sse` \| `streamable-http`) |
+| `mdengine` | `md_generator.engine_cli:main` | `mdengine ai assist …`, `mdengine ai export …`, `mdengine skill build …`, `mdengine db-to-md …`, `mdengine log-to-md …`, `mdengine graph-to-md …`, `mdengine openapi-to-md generate …`, `mdengine codeflow-to-md scan …` |
 
 **openapi-to-md (`md-openapi`):** OpenAPI **3.x** is parsed directly; **Swagger 2.0** (`swagger: "2.0"`) documents are **converted in-process** to OpenAPI 3.0.3 (deterministic, in-repo converter) before `$ref` resolution. Edge-heavy specs (unusual OAuth2 flows, vendor extensions) may still need fixes after conversion.
 
@@ -327,6 +391,14 @@ If the shell reports “command not found”, ensure the Python **Scripts** dire
 
 **db-to-md SQLite:** set **`database.type: sqlite`** and **`database.uri`** to a SQLAlchemy URL such as **`sqlite:///my.db`** or **`sqlite:////C:/data/my.db`**. The default SQLite catalog is **`main`**; packaged YAML that still has **`schema: public`** is normalized to **`main`** when the type is SQLite. CLI: **`--type sqlite`**. Stored routines, sequences, and partitions are not used by SQLite and stay empty; tables, indexes, views, triggers, and ERD (via FK introspection) follow the same export path as other SQL engines.
 
+**db-to-md SQL Server:** set **`database.type: mssql`** (aliases: `sqlserver`) and a **`mssql+pyodbc://`** URI with an ODBC driver query (for example `?driver=ODBC+Driver+18+for+SQL+Server`). Set **`database.schema`** to the SQL Server schema (default **`dbo`**). Optional features: **`synonyms`**, **`dependencies`** (lineage from `sys.sql_expression_dependencies`). Table/column/routine **MS_Description** extended properties are exported when present. Requires **`pyodbc`** (`mdengine[db]`).
+
+**db-to-md Microsoft Access:** set **`database.type: access`** with a file path via **`mssql+pyodbc:///?odbc_connect=...`** (built automatically from uploads) or use **`POST /db-to-md/run/access`** / **`POST /db-to-md/job/access`** to upload **`.mdb` / `.accdb`** files (Windows + Microsoft Access ODBC driver). Saved **Query** objects are exported as views (SELECT) or query procedures (action queries). Requires **`pyodbc`** and an Access-compatible ODBC driver.
+
+**db-to-md CLI validation:** **`md-db --test-connection`** (or **`mdengine db-to-md --test-connection`**) checks connectivity without exporting. **`--list-schemas`** prints schema/catalog names (`--output-format json` optional).
+
+**db-to-md export manifest:** every run can write **`export_manifest.json`** at the output root (counts, file list, ERD paths). Control with **`output.write_manifest`** (default true). **`output.markdown_cross_links`** adds **Related** links between tables via foreign keys when using split files (default true).
+
 **db-to-md API SQLite upload:** **`POST /db-to-md/run/sqlite`** — `multipart/form-data` with field **`file`** (the `.sqlite` / `.db` bytes; must start with the standard `SQLite format 3` header) and optional **`config`** (JSON string with the same shape as a normal run body minus `database`: `schema`, `output`, `features`, `execution`, `limits`, `erd`). Returns the metadata ZIP immediately. **`POST /db-to-md/job/sqlite`** — same form fields; saves the file under the job workspace and runs the existing job pipeline (`GET /db-to-md/job/{id}/download` when complete). Upload size cap: env **`DB_TO_MD_MAX_SQLITE_UPLOAD_MB`** (default **256**). Large ZIPs may still require the async job path if they exceed **`DB_TO_MD_MAX_SYNC_ZIP_MB`** on the sync upload route.
 
 **graph-to-md (Neo4j + NetworkX):** library lives under [`md_generator/graph/`](src/md_generator/graph/). **Sources:** **`networkx`** (GraphML/GML via `graph.graph_file` or `--graph-file`) or **`neo4j`** (`graph.uri`, `graph.user`, `graph.password`, optional `graph.database` for `session(database=…)`). **Output layout:** by default **`output.combine_markdown: true`** writes **`nodes.md`**, **`relationship.md`**, and **`graph_summary.md`** (summary plus embedded nodes and relationships). Set **`combine_markdown: false`** or CLI **`--individual` / `--markdown-layout individual`** for per-entity files under **`nodes/`** and **`relationships/`**. **Diagrams:** **`viz.mermaid: true`** (default) writes **`graph/graph.mmd`** and embeds a fenced Mermaid block in the export **`README.md`** (no Graphviz required). **`viz.enabled: true`** or CLI **`--viz`** also writes **`graph/graph.dot`** and runs **`dot`** for PNG/SVG/PDF when Graphviz is on `PATH` (or **`GRAPHVIZ_DOT`**). CLI **`--no-mermaid`** disables Mermaid; **`--depth`**, **`--start-node`**, **`--max-nodes`**, **`--max-edges`** bound traversal. Packaged defaults: [`src/md_generator/graph/config/default.yaml`](src/md_generator/graph/config/default.yaml). Tests: **`graph-to-md/tests/`**; API image: [`graph-to-md/Dockerfile.api`](graph-to-md/Dockerfile.api).
@@ -350,6 +422,10 @@ md-url https://example.com/page ./page-bundle --artifact-layout
 md-audio ./voice.mp3 ./voice.md --model tiny
 md-video ./screen.mp4 ./screen.md --model base
 pip install "mdengine[graph]" && md-graph --source neo4j --uri neo4j://localhost:7687 --user neo4j --password secret --database neo4j --output ./graph-out --viz
+pip install "mdengine[playwright]" && playwright install chromium && md-playwright https://example.com/app ./spa-out
+pip install "mdengine[openapi]" && md-openapi generate --file openapi.yaml --output ./openapi-md
+pip install "mdengine[log]" && md-log --config log.yaml
+md-codeflow scan ./src --output ./cf-out --lang python
 ```
 
 **Windows PowerShell** (same commands; use backslashes for paths if you prefer)
@@ -362,6 +438,9 @@ md-audio .\voice.mp3 .\voice.md --model tiny
 md-video .\screen.mp4 .\screen.md --model base
 pip install "mdengine[graph]"
 md-graph --source neo4j --uri neo4j://localhost:7687 --user neo4j --password secret --database neo4j --output .\graph-out --viz
+pip install "mdengine[playwright]"
+playwright install chromium
+md-playwright https://example.com/app .\spa-out
 ```
 
 **Windows CMD**
@@ -396,19 +475,19 @@ python pdf-to-md\converter.py input.pdf out.md
 PYTHONPATH=src python pdf-to-md/converter.py input.pdf out.md
 ```
 
-### 6. Convert every file in `docs/` (strictly command-line)
+### 6. Convert every file in `example/` (strictly command-line)
 
-To process **all supported files** under the [`docs/`](docs/) folder using only the installed **`md-*`** tools (no Python snippets), use the batch driver:
+To process **all supported files** under the [`example/`](example/) folder using only the installed **`md-*`** tools (no Python snippets), use the batch driver:
 
 | Platform | Command (run from **repository root** unless noted) |
 |----------|------------------------------------------------------|
 | Windows | `powershell -ExecutionPolicy Bypass -File scripts/run-docs-cli.ps1` |
-| Windows | Or double-click / run [`docs/run-all-cli.cmd`](docs/run-all-cli.cmd) (changes to repo root, then runs the script on `docs\`) |
+| Windows | Or double-click / run [`example/run-all-cli.cmd`](example/run-all-cli.cmd) (changes to repo root, then runs the script on `example\`) |
 | macOS / Linux | `bash scripts/run-docs-cli.sh` |
 
 Optional environment variables for the shell script: `DOCS_DIR`, `OUT_DIR`, `IMAGE_ENGINES` (default `tess`). PowerShell script parameters: `-DocsDir`, `-OutDir`, `-ImageEngines`.
 
-Outputs are written to **`docs/cli-output/<basename>/`** (one subfolder per input file). **`.csv`** files are converted with **`md-xlsx`** (same engine as Excel). **`.md`** files are skipped.
+Outputs are written to **`example/cli-output/<basename>/`** (one subfolder per input file). **`.csv`** files are converted with **`md-xlsx`** (same engine as Excel). **`.md`** files are skipped.
 
 ---
 
@@ -529,6 +608,113 @@ convert_zip(
 
 `repo_root` on `ConvertOptions` is **deprecated and ignored**; converters are loaded in-process from `md_generator`.
 
+### URL (HTML)
+
+```python
+from pathlib import Path
+
+import httpx
+from md_generator.url.convert_impl import run_single
+from md_generator.url.options import ConvertOptions
+
+with httpx.Client(follow_redirects=True, timeout=30.0) as client:
+    run_single(
+        "https://example.com/",
+        Path("web-out"),
+        ConvertOptions(artifact_layout=True, verbose=False),
+        client,
+    )
+```
+
+See [url-to-md/README.md](url-to-md/README.md) for crawl modes, post-convert assets, and API bodies.
+
+### Playwright (SPA)
+
+```python
+import asyncio
+from pathlib import Path
+
+from md_generator.playwright.options import PlaywrightOptions
+from md_generator.playwright.pipeline import convert_url_to_md
+
+async def main() -> None:
+    await convert_url_to_md(
+        "https://example.com/app",
+        Path("spa-out"),
+        PlaywrightOptions(),
+    )
+
+asyncio.run(main())
+```
+
+Install browsers once: `playwright install chromium`.
+
+### Database metadata (db-to-md)
+
+```python
+from pathlib import Path
+
+from md_generator.db.core.extractor import extract_to_markdown
+from md_generator.db.core.run_config import load_run_config
+
+cfg = load_run_config(Path("db-export.yaml"))
+extract_to_markdown(cfg)
+```
+
+YAML shape and ERD options are covered above under **db-to-md** CLI notes.
+
+### Graph (Neo4j / NetworkX)
+
+```python
+from pathlib import Path
+
+from md_generator.graph.core.extractor import extract_to_markdown
+from md_generator.graph.core.run_config import load_graph_run_config
+
+cfg = load_graph_run_config(Path("graph-export.yaml"))
+extract_to_markdown(cfg)
+```
+
+### OpenAPI → Markdown
+
+```python
+from pathlib import Path
+
+from md_generator.openapi.core.extractor import extract_to_markdown
+from md_generator.openapi.core.run_config import ApiRunConfig
+
+cfg = ApiRunConfig(file=Path("openapi.yaml"), output_path=Path("api-md"))
+extract_to_markdown(cfg)
+```
+
+Use `load_api_run_config(Path("openapi-md.yaml"))` when you prefer a single YAML file for input/output/format defaults.
+
+### Log files (log-to-md)
+
+```python
+from pathlib import Path
+
+from md_generator.log.core.extractor import extract_to_markdown
+from md_generator.log.core.run_config import load_run_config as load_log_run_config
+
+cfg = load_log_run_config(Path("log-export.yaml"))
+extract_to_markdown(cfg)
+```
+
+### Codeflow (static call graphs)
+
+```python
+from pathlib import Path
+
+from md_generator.codeflow.core.extractor import run_scan
+from md_generator.codeflow.core.run_config import ScanConfig
+
+cfg = ScanConfig(project_root=Path("my-service/src"), output_path=Path("codeflow-out"), languages="java")
+run_scan(cfg)
+```
+
+More flags and language notes: [Development](#development) (section **Codeflow**) and [`codeflow-to-md/docs/graph-and-outputs.md`](codeflow-to-md/docs/graph-and-outputs.md).
+
 ---
 
 ## Audio and video to Markdown
@@ -645,7 +831,7 @@ Equivalent modules: `python -m md_generator.media.audio.api.mcp_server`, `python
 
 ### Thin shims (repo clone)
 
-[`audio-to-md/converter.py`](audio-to-md/converter.py), [`video-to-md/converter.py`](video-to-md/converter.py), and [`youtube-to-md/converter.py`](youtube-to-md/converter.py) delegate to the same `main` as `md-audio` / `md-video` / `md-youtube`. Tests and `pytest.ini` live under `audio-to-md/tests/`, `video-to-md/tests/`, and `youtube-to-md/tests/`. [`db-to-md/converter.py`](db-to-md/converter.py) delegates to `md-db`; tests live under `db-to-md/tests/`. **graph-to-md** uses `md-graph` / `mdengine graph-to-md` directly (no thin `converter.py` shim); tests live under [`graph-to-md/tests/`](graph-to-md/tests/). [`openapi-to-md/converter.py`](openapi-to-md/converter.py) delegates to `md-openapi`; tests live under [`openapi-to-md/tests/`](openapi-to-md/tests/); example OpenAPI and output notes: [`openapi-to-md/examples/`](openapi-to-md/examples/).
+[`audio-to-md/converter.py`](audio-to-md/converter.py), [`video-to-md/converter.py`](video-to-md/converter.py), and [`youtube-to-md/converter.py`](youtube-to-md/converter.py) delegate to the same `main` as `md-audio` / `md-video` / `md-youtube`. Tests and `pytest.ini` live under `audio-to-md/tests/`, `video-to-md/tests/`, and `youtube-to-md/tests/`. [`db-to-md/converter.py`](db-to-md/converter.py) delegates to `md-db`; tests live under `db-to-md/tests/`. **graph-to-md** uses `md-graph` / `mdengine graph-to-md` directly (no thin `converter.py` shim); tests live under [`graph-to-md/tests/`](graph-to-md/tests/). [`openapi-to-md/converter.py`](openapi-to-md/converter.py) delegates to `md-openapi`; tests live under [`openapi-to-md/tests/`](openapi-to-md/tests/); example OpenAPI and output notes: [`openapi-to-md/examples/`](openapi-to-md/examples/). [`log-to-md/converter.py`](log-to-md/converter.py) delegates to `md-log`; tests under [`log-to-md/tests/`](log-to-md/tests/). [`codeflow-to-md/converter.py`](codeflow-to-md/converter.py) invokes the same CLI entrypoint as `md-codeflow` / `codeflow`; tests under [`codeflow-to-md/tests/`](codeflow-to-md/tests/). [`url-to-md/converter.py`](url-to-md/converter.py) matches `md-url`; tests under [`url-to-md/tests/`](url-to-md/tests/). [`playwright-to-md/tests/`](playwright-to-md/tests/) cover the Playwright pipeline (no root `converter.py` shim required).
 
 ---
 
@@ -681,23 +867,28 @@ Install `mdengine[api]` plus the format extra(s), then run the **`app`** object
 | Audio (Whisper) | `md_generator.media.audio.api.main:create_app` (use **`--factory`**) or `…main:app` | `audio`, `api`, `mcp` |
 | Video (Whisper) | `md_generator.media.video.api.main:create_app` (use **`--factory`**) or `…main:app` | `video`, `api`, `mcp` |
 | YouTube | `md_generator.media.youtube.api.main:create_app` (use **`--factory`**) or `…main:app` | `youtube`, `api`, `mcp` |
+| Codeflow (workspace scans) | `md_generator.codeflow.api.main:app` | `codeflow`, `api`, `mcp` |
+| Log files → Markdown | `md_generator.log.api.main:app` | `log`, `api`, `mcp` |
 
-Examples:
+Examples (ports are **illustrative** so several services can run together; each `md-*-api` / `run` module has its own **default** — set `*_PORT` / `*_API_PORT` env vars to match your layout):
 
 ```bash
 uvicorn md_generator.pdf.api.main:app --host 127.0.0.1 --port 8001
 uvicorn md_generator.word.api.main:app --host 127.0.0.1 --port 8002
-uvicorn md_generator.archive.api.main:app --host 127.0.0.1 --port 8010
-uvicorn md_generator.url.api.main:app --host 127.0.0.1 --port 8011
+uvicorn md_generator.archive.api.main:app --host 127.0.0.1 --port 8008
+uvicorn md_generator.url.api.main:app --host 127.0.0.1 --port 8017
+uvicorn md_generator.db.api.main:app --host 127.0.0.1 --port 8010
 uvicorn md_generator.playwright.api.main:app --host 127.0.0.1 --port 8014
-uvicorn md_generator.graph.api.main:app --host 127.0.0.1 --port 8012
+uvicorn md_generator.graph.api.main:app --host 127.0.0.1 --port 8020
 uvicorn md_generator.openapi.api.main:app --host 127.0.0.1 --port 8015
+uvicorn md_generator.codeflow.api.main:app --host 127.0.0.1 --port 8016
+uvicorn md_generator.log.api.main:app --host 127.0.0.1 --port 8018
 uvicorn md_generator.media.audio.api.main:create_app --factory --host 127.0.0.1 --port 8011
 uvicorn md_generator.media.video.api.main:create_app --factory --host 127.0.0.1 --port 8012
 uvicorn md_generator.media.youtube.api.main:create_app --factory --host 127.0.0.1 --port 8013
 ```
 
-**Port note:** **`md-graph-api`** and **`md-video-api`** both default to **8012**; set **`GRAPH_TO_MD_PORT`** or **`MD_VIDEO_API_PORT`** when you need both on one machine. **`md-openapi-api`** defaults to **8015** (`OPENAPI_TO_MD_PORT`) so it does not collide with **`md-youtube-api`** (**8013**) or **`md-playwright-api`** (**8014**).
+**Port note:** **`md-graph-api`**, **`md-video-api`**, and **`md-log-api`** all default to **8012** in their respective `run.py` modules. Run only one on that port, or set **`GRAPH_TO_MD_PORT`**, **`MD_VIDEO_API_PORT`**, and **`LOG_TO_MD_PORT`** so each process listens on a unique port. **`md-db-api`** defaults to **8010** (not shown colliding with the archive example above, which uses **8008** on purpose). **`md-openapi-api`** defaults to **8015** (`OPENAPI_TO_MD_PORT`). **`md-youtube-api`** defaults to **8013**; **`md-playwright-api`** / **`PLAYWRIGHT_TO_MD_API_PORT`** default to **8014**. **`md-codeflow-api`** defaults to **8016** (`CODEFLOW_TO_MD_PORT`).
 
 ### MCP over HTTP on the same server
 
@@ -720,6 +911,8 @@ Prefixes differ per service (often read from a `.env` file next to the process):
 | Database metadata | `DB_TO_MD_` | `DB_TO_MD_JOB_SQLITE_PATH`, `DB_TO_MD_JOB_WORKSPACE_ROOT`, `DB_TO_MD_CORS_ORIGINS`, `DB_TO_MD_MAX_SYNC_ZIP_MB`, `DB_TO_MD_HOST`, `DB_TO_MD_PORT` (default **8010**) |
 | Graph metadata | `GRAPH_TO_MD_` | `GRAPH_TO_MD_JOB_SQLITE_PATH`, `GRAPH_TO_MD_JOB_WORKSPACE_ROOT`, `GRAPH_TO_MD_CORS_ORIGINS`, `GRAPH_TO_MD_MAX_SYNC_ZIP_MB`, `GRAPH_TO_MD_HOST`, `GRAPH_TO_MD_PORT` (default **8012**) |
 | OpenAPI → Markdown | `OPENAPI_TO_MD_` | `OPENAPI_TO_MD_CORS_ORIGINS`, `OPENAPI_TO_MD_MAX_SYNC_ZIP_MB`, `OPENAPI_TO_MD_HOST`, `OPENAPI_TO_MD_PORT` (default **8015**) |
+| Codeflow API | `CODEFLOW_TO_MD_` / `CODEFLOW_` | `CODEFLOW_TO_MD_HOST`, `CODEFLOW_TO_MD_PORT` (default **8016**); in-app limits: `CODEFLOW_MAX_UPLOAD_ZIP_MB`, `CODEFLOW_MAX_SYNC_ZIP_MB`, `CODEFLOW_JOB_WORKSPACE_ROOT`, `CODEFLOW_SQLITE_PATH`, CORS via `CODEFLOW_CORS` |
+| Log API | `LOG_TO_MD_` | `LOG_TO_MD_HOST`, `LOG_TO_MD_PORT` (default **8012** — change if **graph** or **video** API uses that port), `LOG_TO_MD_CORS_ORIGINS`, `LOG_TO_MD_MAX_SYNC_ZIP_MB`, `LOG_TO_MD_MAX_LOG_UPLOAD_MB`, workspace / SQLite paths |
 | Audio API | `MD_AUDIO_` | `MD_AUDIO_MAX_UPLOAD_MB`, `MD_AUDIO_MAX_SYNC_UPLOAD_MB`, `MD_AUDIO_JOB_TTL_SECONDS`, `MD_AUDIO_TEMP_DIR`, `MD_AUDIO_CORS_ORIGINS`, `MD_AUDIO_API_HOST`, `MD_AUDIO_API_PORT` |
 | Video API | `MD_VIDEO_` | Same pattern as audio with `MD_VIDEO_*` (defaults: larger upload/sync caps, port **8012**) |
 | YouTube API | `MD_YOUTUBE_` | `MD_YOUTUBE_JOB_TTL_SECONDS`, `MD_YOUTUBE_TEMP_DIR`, `MD_YOUTUBE_CORS_ORIGINS`, `MD_YOUTUBE_API_HOST`, `MD_YOUTUBE_API_PORT` (default **8013**); optional `MD_YOUTUBE_YTDLP` path for audio fallback |
@@ -746,13 +939,15 @@ Two usage patterns:
 | PPTX | `python -m md_generator.ppt.api.mcp_server` (see module docstring for flags) |
 | Image | `python -m md_generator.image.api.mcp_server` (see module for CLI) |
 | URL / HTML | `python -m md_generator.url.api.mcp_server` / `--transport sse` / `--transport streamable-http` |
-| Playwright / SPA | `md-playwright-mcp` or `python -m md_generator.playwright.api.mcp_server` / `--transport sse` / `--transport streamable-http` |
+| Playwright / SPA | `md-playwright`, `md-playwright-api`, `md-playwright-mcp`, or `python -m md_generator.playwright.api.mcp_server` / `--transport sse` / `--transport streamable-http` |
 | Audio | `md-audio-mcp` or `python -m md_generator.media.audio.api.mcp_server` — `--transport stdio` (default), `sse`, `streamable-http` |
 | Video | `md-video-mcp` or `python -m md_generator.media.video.api.mcp_server` — same transports |
 | YouTube | `md-youtube-mcp` or `python -m md_generator.media.youtube.api.mcp_server` — same transports |
 | Database metadata | `md-db-mcp` or `python -m md_generator.db.api.mcp_server` — `--transport stdio` (default), `sse`, `streamable-http` |
 | Graph (Neo4j / NetworkX) | `md-graph-mcp` or `python -m md_generator.graph.api.mcp_server` — `--transport stdio` (default), `sse`, `streamable-http` |
 | OpenAPI → Markdown | `md-openapi-mcp` or `python -m md_generator.openapi.api.mcp_server` — `--transport stdio` (default), `sse`, `streamable-http` |
+| Codeflow | `md-codeflow-mcp` or `python -m md_generator.codeflow.api.mcp_server` — **stdio** MCP (see module for details) |
+| Log files | `md-log-mcp` or `python -m md_generator.log.api.mcp_server` — `--transport stdio` (default), `sse`, `streamable-http` |
 
 **Word** and **XLSX** also ship a small runner script in the repo:
 
@@ -881,7 +1076,9 @@ pip install -e ".[dev,all]"   # or a smaller subset of extras
 python -m pytest
 ```
 
-Tests live under each legacy folder’s `tests/` directory (e.g. `pdf-to-md/tests/`), **`graph-to-md/tests/`**, **`openapi-to-md/tests/`**, and **[`tool-assistant/tests/`](tool-assistant/tests/)** for the skill SDK; `pyproject.toml` sets `pythonpath = ["src"]` so **`md_generator`** (including **`md_generator.tools.assistant`**) resolves without a manual `PYTHONPATH` when you use `pytest` from the config.
+Tests live under each legacy folder’s `tests/` directory (e.g. `pdf-to-md/tests/`), plus **`url-to-md/tests/`**, **`playwright-to-md/tests/`**, **`youtube-to-md/tests/`**, **`graph-to-md/tests/`**, **`openapi-to-md/tests/`**, **`codeflow-to-md/tests/`**, **`log-to-md/tests/`**, and **[`tool-assistant/tests/`](tool-assistant/tests/)** for the skill SDK; `pyproject.toml` sets `pythonpath = ["src"]` so **`md_generator`** (including **`md_generator.tools.assistant`**) resolves without a manual `PYTHONPATH` when you use `pytest` from the config.
+
+**Docs site (maintainers):** install **`mdengine[docs]`**, then build or serve the MkDocs project from the repo root (`mkdocs.yml`). The published site URL is listed under **Documentation** in `[project.urls]` on PyPI ([GitHub Pages build](https://vishal7090.github.io/md-generator/)).
 
 ### Codeflow (`md_generator.codeflow`)
 
@@ -913,6 +1110,8 @@ python -m md_generator.codeflow.cli.main scan path/to/src --output ./codeflow-ou
 | Path | Role |
 |------|------|
 | `LICENSE` | MIT license text |
+| `docs/` | **MkDocs** source (module guides, `index.md`); build with **`mdengine[docs]`** |
+| `mkdocs.yml` | MkDocs configuration |
 | `CODE_OF_CONDUCT.md` | [Contributor Covenant](https://www.contributor-covenant.org/) 2.1 |
 | `src/md_generator/` | **Library source** (all formats + `api` subpackages); **audio/video** under [`media/audio/`](src/md_generator/media/audio/) and [`media/video/`](src/md_generator/media/video/); **graph-to-md** under [`graph/`](src/md_generator/graph/) |
 | `pyproject.toml` | Packaging, extras, CLI entry points, pytest |