okf-cli 0.1.1__tar.gz → 0.3.1__tar.gz

okf_cli-0.3.1/.github/workflows/test.yml

@@ -0,0 +1,17 @@
+name: Test
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.11"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v3
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: uv sync
+      - run: uv run pytest -q

okf_cli-0.3.1/AGENTS.md

@@ -0,0 +1,163 @@
+# AGENTS.md — okf-cli contributor context
+
+This file is for AI agents (and humans) picking up work on `okf-cli`.
+
+## Project overview
+
+`okf-cli` is a Python CLI that converts plain markdown directories into
+[Open Knowledge Format (OKF)](OKF_SPEC.md) bundles and provides tools to
+inspect/validate them.
+
+- **Input:** plain `.md` files starting with `# Title` and a `>` description block.
+- **Output:** OKF-conformant markdown with YAML frontmatter, per-directory `index.md` files.
+- **Operations:** `bundle`, `list`, `show`, `validate`.
+
+## Tech stack
+
+- Python 3.11+
+- Package/venv manager: **uv**
+- CLI framework: **typer**
+- YAML parsing: **pyyaml**
+- Tests: **pytest**
+- Lint/format: **ruff** (via `uvx ruff check .` / `uvx ruff format .`)
+- Build: hatchling (configured in `pyproject.toml`)
+
+Always use `uv` for dependency sync and running commands:
+
+```bash
+uv sync                  # install/sync deps
+uv run pytest -q           # run tests
+uv run okf --help          # run CLI
+uv run okf bundle example bundled --default-type reference --force
+```
+
+## Project layout
+
+```
+okf-cli
+├── .github/workflows/test.yml # CI (pytest on 3.11)
+├── OKF_SPEC.md              # OKF v0.1 specification
+├── README.md                # user-facing docs
+├── AGENTS.md                # this file
+├── pyproject.toml           # project metadata + deps
+├── uv.lock                  # uv lockfile
+├── src/okf/
+│   ├── cli.py               # Typer entrypoint, registers commands
+│   ├── core.py              # shared parsing/formatting/conformance
+│   └── commands/
+│       ├── bundle.py        # plain markdown → OKF bundle
+│       ├── list.py          # list concept IDs
+│       ├── show.py          # print concept by ID
+│       └── validate.py      # conformance check
+└── tests/test_cli.py        # pytest suite
+```
+
+## Architecture
+
+### `src/okf/cli.py`
+
+- Creates the `typer.Typer` app named `okf`.
+- Registers `bundle`, `list`, `show`, `validate`.
+- No business logic here.
+
+### `src/okf/core.py`
+
+- `RESERVED` — filenames `bundle` skips: `index.md`, `log.md`, `README.md`.
+- `SPEC_RESERVED` — filenames reserved by the OKF spec: `index.md`, `log.md`.
+- `build_frontmatter(...)` — generates YAML frontmatter using JSON-escaped strings.
+- `parse_md(...)` — extracts title/description/body from plain markdown. Tries strict format first (line 1 `# Title`, `>` block); falls back leniently (title from line 0 only, description from first 80 chars of body). Never raises.
+- `parse_frontmatter(...)` — parses YAML frontmatter with `yaml.safe_load`.
+- `check_conformance(...)` — validates a directory against OKF §9, returns `(errors, warnings)`. Now enforces:
+  - §9.1: Non-reserved `.md` must have parseable frontmatter.
+  - §9.2: Frontmatter must have non-empty `type`.
+  - §9.3: `index.md` must not contain frontmatter (§6), except root `index.md` may have only `okf_version` (§11). `log.md` must not contain frontmatter (§7).
+  - Non-UTF-8 files are flagged as errors.
+
+### Commands
+
+| Command | Input expected | Behavior |
+|---|---|---|
+| `bundle` | non-conformant plain markdown dir | generates OKF bundle; skips `index.md`, `log.md`, `README.md` with warnings; root files need `--default-type`; requires `--force` to overwrite existing output |
+| `list` | OKF-conformant bundle | prints concept IDs; exits 1 if dir is not conformant |
+| `show` | OKF-conformant bundle | prints concept file by ID; exits 1 if dir is not conformant |
+| `validate` | any directory | prints conformance errors and summary per §9 |
+
+## Key conventions
+
+### Reserved filenames
+
+- `bundle` treats `index.md`, `log.md`, `README.md` as reserved because it
+  often runs against raw repo directories that may contain these files.
+- `list`, `show`, `validate` operate on already-conformant OKF bundles, so
+  they only treat `index.md` and `log.md` as reserved per the spec.
+- `README.md` can be a valid concept in an OKF bundle.
+
+### OKF conformance (§9)
+
+1. Every non-reserved `.md` must have parseable YAML frontmatter.
+2. Every frontmatter must contain a non-empty `type` string.
+3. Reserved filenames (`index.md`, `log.md`) follow structure per §6/§7 when present.
+   - `index.md` must not contain frontmatter (§6), except root `index.md` may contain only `okf_version` (§11).
+   - `log.md` must not contain frontmatter (§7).
+4. All files must be valid UTF-8.
+
+`check_conformance()` is the source of truth for this logic.
+
+### Frontmatter
+
+`build_frontmatter()` outputs JSON-escaped YAML values, e.g.:
+
+```yaml
+---
+type: "tables"
+title: "Customer Orders"
+---
+```
+
+`parse_frontmatter()` uses `yaml.safe_load()` and returns `None` for missing,
+malformed, or non-dict frontmatter.
+
+## Testing
+
+Run the full suite before finishing:
+
+```bash
+uv run pytest -q
+```
+
+Add tests for new behavior in `tests/test_cli.py`. Existing patterns:
+
+- Unit tests for `core.py` helpers.
+- CLI integration tests using `typer.testing.CliRunner` and `tmp_path`.
+- Conformance tests for `validate`, `list`, `show`.
+
+## Common tasks
+
+### Regenerate the example bundle
+
+```bash
+uv run okf bundle example bundled --default-type reference --force
+uv run okf validate bundled
+uv run okf list bundled
+```
+
+`bundled/` is gitignored; it is only a local artifact. Use `--force` for re-runs.
+
+### Add a new CLI command
+
+1. Create `src/okf/commands/<name>.py` with a function taking `typer` arguments.
+2. Import and register it in `src/okf/cli.py` with `app.command()(fn)` or
+   `app.command("alias")(fn)`.
+3. Add tests in `tests/test_cli.py`.
+
+### Modify conformance behavior
+
+Update `check_conformance()` in `src/okf/core.py`, then ensure
+`validate`, `list`, and `show` all behave consistently.
+
+## Style guidelines
+
+- Keep changes minimal. Prefer stdlib/installed dependencies over new ones.
+- Do not add abstractions "for later".
+- Use `Path` from `pathlib`, not string path manipulation.
+- Use `uv` for all package/runtime commands.

{okf_cli-0.1.1 → okf_cli-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: okf-cli
-Version: 0.1.1
+Version: 0.3.1
 Summary: Open Knowledge Format tooling
 License: MIT License
         
@@ -24,13 +24,16 @@ License: MIT License
         OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
         SOFTWARE.
 License-File: LICENSE
-Requires-Python: >=3.13
+Requires-Python: >=3.11
+Requires-Dist: pyyaml>=6
 Requires-Dist: typer>=0.15
 Description-Content-Type: text/markdown
 
 # okf-cli — Open Knowledge Format tooling
 
-Converts plain markdown into [OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)-conformant knowledge bundles. Domain experts write `# Title` then `> description` — `okf bundle` generates frontmatter, type, timestamps, and index files.
+Converts plain markdown into [OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)-conformant knowledge bundles. Domain experts write `# Title` then `> description` — `okf bundle` generates frontmatter, type, timestamps, and index files. Files without strict formatting are parsed leniently: title from line 0 if present, description from body text.
+
+Also validates bundles and lists concept IDs for consumption by other tools.
 
 ## Install
 
@@ -38,25 +41,19 @@ Converts plain markdown into [OKF](https://github.com/GoogleCloudPlatform/knowle
 uv tool install okf-cli
 ```
 
-Then run:
-
-```bash
-okf bundle example              # output → bundled/
-cat bundled/index.md
-cat bundled/tables/orders.md
-```
-
 ### Dev quickstart
 
 ```bash
 uv sync
-uv run okf example
+uv run okf --help
 ```
 
-## Usage
+## Commands
+
+### `okf bundle` — convert plain markdown to OKF bundle
 
 ```
-okf <input-dir> [output-dir] [--default-type <name>]
+okf bundle <input-dir> [output-dir] [--default-type <name>] [--force]
 ```
 
 | Argument | Description |
@@ -64,10 +61,86 @@ okf <input-dir> [output-dir] [--default-type <name>]
 | `input-dir` | Directory of plain `.md` files |
 | `output-dir` | Target directory (default: `bundled`) |
 | `--default-type` | Type for root-level files (skip root files if omitted) |
+| `--force`, `-f` | Overwrite output directory if it exists |
+
+```bash
+okf bundle example --default-type reference  # first run → bundled/
+okf bundle example --default-type reference --force  # re-run
+cat bundled/tables/orders.md
+```
+
+### `okf list` — list concept IDs in a bundle
+
+```
+okf list <directory>
+```
+
+Prints concept IDs (bundle-relative path with `.md` stripped) for every
+concept file in the bundle. Reserved filenames (`index.md`, `log.md`)
+are excluded. `README.md` is not reserved by the OKF spec and is listed
+as a concept if present.
+
+Requires the directory to be a valid OKF bundle; fails if any
+non-reserved `.md` is missing frontmatter or a non-empty `type`.
+
+```bash
+okf list bundled/
+# datasets/sales
+# playbooks/incident-response
+# tables/orders
+# …
+```
+
+Useful for piping OKF bundles into other tooling:
+
+```bash
+okf list bundled/ | xargs -I{} okf show bundled/ {}
+```
+
+### `okf show` — read a concept by ID
+
+```
+okf show <directory> <concept-id>
+```
 
-## Writing input files
+Prints the full contents (frontmatter + body) of a concept file.
+Concept IDs are the bundle-relative path with `.md` stripped — exactly
+as printed by `okf list`.
 
-Every `.md` file must start with:
+Requires the directory to be a valid OKF bundle.
+
+```bash
+okf show bundled/ tables/orders
+# ---
+# type: "tables"
+# title: "Customer Orders"
+# ...
+```
+
+Guards against path traversal and rejects reserved filenames.
+
+### `okf validate` — check OKF conformance
+
+```
+okf validate <directory>
+```
+
+Checks OKF v0.1 §9 conformance:
+
+- Every non-reserved `.md` has parseable YAML frontmatter with non-empty `type`.
+- Reserved filenames (`index.md`, `log.md`) follow spec structure (§6/§7).
+- Non-UTF-8 files are flagged.
+
+```bash
+okf validate bundled/
+# 16 files: 16 ok
+```
+
+## Writing input files (for `bundle`)
+
+### Strict format (recommended)
+
+Every `.md` file starts with:
 
 ```markdown
 # Title
@@ -76,37 +149,36 @@ Every `.md` file must start with:
 > Second optional description line.
 ```
 
-Everything after the description block is free-form — preserved unchanged.
+Everything after the description block is preserved unchanged.
+
+### Lenient fallback
+
+Files that don't follow the strict format are still bundled best-effort:
+
+| Condition | Behavior |
+|---|---|
+| No `# Title` on line 1 | Title omitted from frontmatter |
+| No `>` description block | Description derived from first 80 chars of body ("…" if truncated) |
+| Root-level file without `--default-type` | Skip file, warn, continue |
 
 **Rules:**
 
 | Rule | Why |
 |---|---|
-| First line must be `# Title` | Tool reads title here |
-| Followed by `> Description` | Tool reads description here |
 | Folder name = concept type | `tables/orders.md` → `type: "tables"` |
 | Only `.md` files processed | Non-`.md` files ignored |
-| Reserved names skipped: `index.md`, `log.md`, `README.md` | OKF spec reserves these |
-
-**Violations:**
-
-| Condition | Behavior |
-|---|---|
-| Line 1 not `# Title` | Error, stop |
-| Empty title | Error, stop |
-| No `> description` block | Error, stop |
-| Root-level file without `--default-type` | Skip file, warn, continue |
+| `index.md`, `log.md`, `README.md` skipped in `bundle` | Input may contain repo artifacts; these are not concepts. `bundle` warns when it skips them. Other commands (`list`, `show`, `validate`) operate on conformant OKF bundles where only `index.md` and `log.md` are reserved. |
 
 Root files need `--default-type`. Otherwise put files in named folders.
 
 See the [`example/`](example/) directory for a sample of how to structure files.
 
-## How it works
+## How `bundle` works
 
 1. Walk `input-dir` for `.md` files (skip reserved names)
-2. Extract `title` from `#`, `description` from `>`
+2. Extract `title` from `#` on line 1, `description` from `>` block. If strict format not met, falls back: title omitted if absent, description from first 80 chars of body
 3. Set `type` from parent dir name, `timestamp` from file mtime
-4. Write concept files with YAML frontmatter
+4. Write concept files with YAML frontmatter (title field omitted if empty)
 5. Generate `index.md` per directory — `# Contents` for files, `# Directories` for subdirs (recursive)
 
 ## Output
@@ -148,9 +220,13 @@ Generated bundles conform to [OKF v0.1](OKF_SPEC.md) (§9):
 
 ```
 okf-cli
-├── OKF_SPEC.md           # OKF specification
-├── pyproject.toml        # uv-managed Python project
-├── src/okf/cli.py        # Single-file CLI
+├── .github/workflows/test.yml  # CI
+├── OKF_SPEC.md                 # OKF specification
+├── pyproject.toml              # uv-managed Python project
+├── src/okf/
+│   ├── cli.py            # Typer entrypoint
+│   ├── core.py           # Shared parsing/formatting
+│   └── commands/         # bundle, list, show, validate
 ├── tests/test_cli.py     # Tests
 └── example/              # Sample input markdown
 ```

okf_cli-0.3.1/README.md

@@ -0,0 +1,201 @@
+# okf-cli — Open Knowledge Format tooling
+
+Converts plain markdown into [OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)-conformant knowledge bundles. Domain experts write `# Title` then `> description` — `okf bundle` generates frontmatter, type, timestamps, and index files. Files without strict formatting are parsed leniently: title from line 0 if present, description from body text.
+
+Also validates bundles and lists concept IDs for consumption by other tools.
+
+## Install
+
+```bash
+uv tool install okf-cli
+```
+
+### Dev quickstart
+
+```bash
+uv sync
+uv run okf --help
+```
+
+## Commands
+
+### `okf bundle` — convert plain markdown to OKF bundle
+
+```
+okf bundle <input-dir> [output-dir] [--default-type <name>] [--force]
+```
+
+| Argument | Description |
+|---|---|
+| `input-dir` | Directory of plain `.md` files |
+| `output-dir` | Target directory (default: `bundled`) |
+| `--default-type` | Type for root-level files (skip root files if omitted) |
+| `--force`, `-f` | Overwrite output directory if it exists |
+
+```bash
+okf bundle example --default-type reference  # first run → bundled/
+okf bundle example --default-type reference --force  # re-run
+cat bundled/tables/orders.md
+```
+
+### `okf list` — list concept IDs in a bundle
+
+```
+okf list <directory>
+```
+
+Prints concept IDs (bundle-relative path with `.md` stripped) for every
+concept file in the bundle. Reserved filenames (`index.md`, `log.md`)
+are excluded. `README.md` is not reserved by the OKF spec and is listed
+as a concept if present.
+
+Requires the directory to be a valid OKF bundle; fails if any
+non-reserved `.md` is missing frontmatter or a non-empty `type`.
+
+```bash
+okf list bundled/
+# datasets/sales
+# playbooks/incident-response
+# tables/orders
+# …
+```
+
+Useful for piping OKF bundles into other tooling:
+
+```bash
+okf list bundled/ | xargs -I{} okf show bundled/ {}
+```
+
+### `okf show` — read a concept by ID
+
+```
+okf show <directory> <concept-id>
+```
+
+Prints the full contents (frontmatter + body) of a concept file.
+Concept IDs are the bundle-relative path with `.md` stripped — exactly
+as printed by `okf list`.
+
+Requires the directory to be a valid OKF bundle.
+
+```bash
+okf show bundled/ tables/orders
+# ---
+# type: "tables"
+# title: "Customer Orders"
+# ...
+```
+
+Guards against path traversal and rejects reserved filenames.
+
+### `okf validate` — check OKF conformance
+
+```
+okf validate <directory>
+```
+
+Checks OKF v0.1 §9 conformance:
+
+- Every non-reserved `.md` has parseable YAML frontmatter with non-empty `type`.
+- Reserved filenames (`index.md`, `log.md`) follow spec structure (§6/§7).
+- Non-UTF-8 files are flagged.
+
+```bash
+okf validate bundled/
+# 16 files: 16 ok
+```
+
+## Writing input files (for `bundle`)
+
+### Strict format (recommended)
+
+Every `.md` file starts with:
+
+```markdown
+# Title
+
+> Description
+> Second optional description line.
+```
+
+Everything after the description block is preserved unchanged.
+
+### Lenient fallback
+
+Files that don't follow the strict format are still bundled best-effort:
+
+| Condition | Behavior |
+|---|---|
+| No `# Title` on line 1 | Title omitted from frontmatter |
+| No `>` description block | Description derived from first 80 chars of body ("…" if truncated) |
+| Root-level file without `--default-type` | Skip file, warn, continue |
+
+**Rules:**
+
+| Rule | Why |
+|---|---|
+| Folder name = concept type | `tables/orders.md` → `type: "tables"` |
+| Only `.md` files processed | Non-`.md` files ignored |
+| `index.md`, `log.md`, `README.md` skipped in `bundle` | Input may contain repo artifacts; these are not concepts. `bundle` warns when it skips them. Other commands (`list`, `show`, `validate`) operate on conformant OKF bundles where only `index.md` and `log.md` are reserved. |
+
+Root files need `--default-type`. Otherwise put files in named folders.
+
+See the [`example/`](example/) directory for a sample of how to structure files.
+
+## How `bundle` works
+
+1. Walk `input-dir` for `.md` files (skip reserved names)
+2. Extract `title` from `#` on line 1, `description` from `>` block. If strict format not met, falls back: title omitted if absent, description from first 80 chars of body
+3. Set `type` from parent dir name, `timestamp` from file mtime
+4. Write concept files with YAML frontmatter (title field omitted if empty)
+5. Generate `index.md` per directory — `# Contents` for files, `# Directories` for subdirs (recursive)
+
+## Output
+
+Each concept becomes a markdown file with YAML frontmatter:
+
+```yaml
+---
+type: "tables"
+title: "Customer Orders"
+description: "One row per completed customer order across all channels."
+timestamp: "2026-07-04T15:06:51+00:00"
+---
+
+Original body preserved as-is.
+```
+
+Every directory gets its own `index.md`:
+
+```markdown
+# Contents
+
+* [Customer Orders](orders.md) - One row per completed customer order across all channels.
+
+# Directories
+
+* [partitions](partitions/)
+```
+
+## OKF Conformance
+
+Generated bundles conform to [OKF v0.1](OKF_SPEC.md) (§9):
+
+- Every non-reserved `.md` has parseable YAML frontmatter with non-empty `type` ✓
+- Reserved filenames follow spec structure ✓
+- Consumers MUST tolerate missing optional fields, unknown types, broken links ✓
+
+## Project layout
+
+```
+okf-cli
+├── .github/workflows/test.yml  # CI
+├── OKF_SPEC.md                 # OKF specification
+├── pyproject.toml              # uv-managed Python project
+├── src/okf/
+│   ├── cli.py            # Typer entrypoint
+│   ├── core.py           # Shared parsing/formatting
+│   └── commands/         # bundle, list, show, validate
+├── tests/test_cli.py     # Tests
+└── example/              # Sample input markdown
+```

{okf_cli-0.1.1 → okf_cli-0.3.1}/example/README.md

@@ -5,8 +5,8 @@ Run `okf bundle` to convert them into an OKF-conformant knowledge bundle.
 
 ```bash
 # From repo root:
-uv run okf example output-bundle
+uv run okf bundle example output-bundle
 
 # With a default type for root-level files (if any):
-uv run okf example output-bundle --default-type reference
+uv run okf bundle example output-bundle --default-type reference
 ```

okf_cli-0.3.1/example/loose/deep/no-heading.md

@@ -0,0 +1,4 @@
+I am a file deep in a subdirectory with no heading at all.
+
+Type comes from the directory name, title is omitted, and description is
+derived from this body text.

okf_cli-0.3.1/example/loose/no-desc.md

@@ -0,0 +1,6 @@
+# Has Title But No Desc
+
+This file has a proper `# Title` but no `>` blockquote description.
+
+It should still be bundled: title extracted normally, description taken
+from the first 80 characters of the body as a fallback.

okf_cli-0.3.1/example/loose/no-title.md

@@ -0,0 +1,5 @@
+No heading at all — this file has no `# Title` line and no `>` description block.
+
+It should still be bundled: title omitted from frontmatter, description
+derived from the first 80 characters of this body text verbatim, with
+newlines collapsed to spaces.

{okf_cli-0.1.1 → okf_cli-0.3.1}/pyproject.toml

@@ -1,11 +1,11 @@
 [project]
 name = "okf-cli"
-version = "0.1.1"
+version = "0.3.1"
 description = "Open Knowledge Format tooling"
 readme = "README.md"
 license = { file = "LICENSE" }
-requires-python = ">=3.13"
-dependencies = ["typer>=0.15"]
+requires-python = ">=3.11"
+dependencies = ["typer>=0.15", "pyyaml>=6"]
 
 [project.scripts]
 okf = "okf.cli:app"
@@ -18,4 +18,10 @@ build-backend = "hatchling.build"
 packages = ["src/okf"]
 
 [dependency-groups]
-dev = ["pytest>=8"]
+dev = ["pytest>=8", "ruff>=0.14"]
+
+[tool.ruff]
+line-length = 88
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP"]