okf-cli 0.1.0__tar.gz

okf_cli-0.1.0/.gitignore

@@ -0,0 +1,7 @@
+.venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+bundled/
+dist/
+*.egg-info/

okf_cli-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Dheerapat Tookkane
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

okf_cli-0.1.0/OKF_SPEC.md

@@ -0,0 +1,451 @@
+# Open Knowledge Format (OKF)
+
+**Version 0.1 — Draft**
+
+OKF is an open, human- and agent-friendly format for representing
+*knowledge* — the metadata, context, and curated insight that surrounds
+data and systems. It is designed to be authored by people, generated by
+agents, exchanged across organizations, and consumed by both.
+
+The format is intentionally minimal: a directory of markdown files with
+YAML frontmatter. There is no schema registry, no central authority, and
+no required tooling. If you can `cat` a file, you can read OKF; if you
+can `git clone` a repo, you can ship it.
+
+---
+
+## 1. Motivation
+
+The space of knowledge representation for AI agents is evolving quickly,
+and many incompatible conventions are emerging. OKF takes the position
+that knowledge is best represented in commonly accessible, established
+formats that are:
+
+- **Readable** by humans without tooling.
+- **Parseable** by agents without bespoke SDKs.
+- **Diffable** in version control.
+- **Portable** across tools, organizations, and time.
+
+The format is minimally opinionated. It standardizes only the small set
+of structural conventions needed to make a knowledge corpus
+*self-describing* — anything beyond that is left to the producer.
+
+### Goals
+
+1. Define a universal format that **enrichment agents** can write into.
+2. Inform how **consumption agents** should read and traverse it.
+3. Facilitate **exchange** of knowledge across systems and organizations.
+4. Standardize the small number of **required** fields that must be
+   present for content to be meaningfully consumed.
+
+### Non-goals
+
+- Defining a fixed taxonomy of concept types.
+- Prescribing storage, serving, or query infrastructure.
+- Replacing domain-specific schemas (Avro, Protobuf, OpenAPI, etc.) —
+  OKF *references* them; it does not subsume them.
+
+---
+
+## 2. Terminology
+
+- **Knowledge Bundle** — A self-contained, hierarchical collection of
+  knowledge documents. The unit of distribution.
+- **Concept** — A single unit of knowledge within a bundle. Represented
+  as one markdown document. May describe a tangible asset (a table, an
+  API), an abstract idea (a metric, a business process), or anything in
+  between.
+- **Concept ID** — The path of the concept's file within the bundle,
+  with the `.md` suffix removed. For example, `tables/users.md` has
+  concept ID `tables/users`.
+- **Frontmatter** — YAML metadata block delimited by `---` at the top of
+  a markdown file.
+- **Body** — Everything in the file after the frontmatter.
+- **Link** — A standard markdown link from one concept to another, used
+  to express relationships beyond the implicit parent/child hierarchy.
+- **Citation** — A link from a concept to an external source that
+  supports a claim in the body.
+
+---
+
+## 3. Bundle Structure
+
+A bundle is a directory tree of markdown files. The directory structure
+is independent of the domain — producers organize concepts however makes
+sense for the knowledge being captured.
+
+```
+path/to/bundle/
+├── index.md                      # Optional. Directory listing for progressive disclosure.
+├── log.md                        # Optional. Chronological history of updates.
+├── <concept>.md                  # A concept at the bundle root.
+└── <subdirectory>/               # Subdirectories organize concepts into groups.
+    ├── index.md
+    ├── <concept>.md
+    └── <subdirectory>/
+        └── …
+```
+
+A bundle MAY be distributed as:
+
+- A git repository (recommended — provides history, attribution, diffs).
+- A tarball or zip archive of the directory.
+- A subdirectory within a larger repository.
+
+### 3.1 Reserved filenames
+
+The following filenames have defined meaning at any level of the
+hierarchy and MUST NOT be used for concept documents:
+
+| Filename     | Purpose                                                |
+|--------------|--------------------------------------------------------|
+| `index.md`   | Directory listing. See §6.                             |
+| `log.md`     | Update history. See §7.                                |
+
+All other `.md` files are concept documents.
+
+Tags themselves remain a first-class concept — see the `tags`
+frontmatter field in §4.1. OKF does not specify a separate file format
+for aggregating documents by tag; producers that want a tag-browsing
+view can synthesize one at consumption time by scanning frontmatter.
+
+---
+
+## 4. Concept Documents
+
+Every concept is a UTF-8 markdown file. It has two parts:
+
+1. A **YAML frontmatter block**, delimited by `---` on its own line at
+   the start of the file and a closing `---` on its own line.
+2. A **markdown body**, containing free-form content.
+
+### 4.1 Frontmatter
+
+```yaml
+---
+type: <Type name>                  # REQUIRED
+title: <Optional display name>
+description: <Optional one-line summary>
+resource: <Optional canonical URI for the underlying asset>
+tags: [<tag>, <tag>, …]            # Optional
+timestamp: <ISO 8601 datetime>     # Optional last-modified time
+# … other producer-defined key/value pairs
+---
+```
+
+**Required:**
+
+- `type` — A short string identifying the kind of concept. Consumers
+  use this for routing, filtering, and presentation. Example values:
+  `BigQuery Table`, `BigQuery Dataset`, `API Endpoint`, `Metric`,
+  `Playbook`, `Reference`.
+
+  Type values are **not** registered centrally. Producers SHOULD pick
+  values that are descriptive and self-explanatory; consumers MUST
+  tolerate unknown types gracefully (typically by treating them as
+  generic concepts).
+
+**Recommended (in priority order):**
+
+- `title` — Human-readable display name. If omitted, consumers MAY
+  derive a title from the filename.
+- `description` — A single sentence summarizing the concept. Used by
+  `index.md` generators, search snippets, and previews.
+- `resource` — A URI that uniquely identifies the underlying asset the
+  concept describes. Absent for concepts that describe abstract ideas
+  rather than physical resources.
+- `tags` — A YAML list of short strings for cross-cutting categorization.
+- `timestamp` — ISO 8601 datetime of last meaningful change.
+
+**Extensions:** Producers MAY include any additional keys. Consumers
+SHOULD preserve unknown keys when round-tripping and SHOULD NOT reject
+documents with unrecognized fields.
+
+### 4.2 Body
+
+The body is standard markdown. Producers SHOULD favor structural
+markdown — headings, lists, tables, fenced code blocks — over freeform
+prose, since structure aids both human reading and agent retrieval.
+
+There are no required body sections. The following section headings have
+**conventional** meaning and SHOULD be used when applicable:
+
+| Heading        | Purpose                                                |
+|----------------|--------------------------------------------------------|
+| `# Schema`     | Structured description of an asset's columns/fields.   |
+| `# Examples`   | Concrete usage examples, often as fenced code blocks.  |
+| `# Citations`  | External sources backing claims in the body. See §8.   |
+
+### 4.3 Example: a concept bound to a resource
+
+```markdown
+---
+type: BigQuery Table
+title: Customer Orders
+description: One row per completed customer order across all channels.
+resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders
+tags: [sales, orders, revenue]
+timestamp: 2026-05-28T14:30:00Z
+---
+
+# Schema
+
+| Column        | Type      | Description                              |
+|---------------|-----------|------------------------------------------|
+| `order_id`    | STRING    | Globally unique order identifier.        |
+| `customer_id` | STRING    | Foreign key into [customers](/tables/customers.md). |
+| `total_usd`   | NUMERIC   | Order total in US dollars.               |
+| `placed_at`   | TIMESTAMP | When the customer submitted the order.   |
+
+# Joins
+
+Joined with [customers](/tables/customers.md) on `customer_id`.
+
+# Citations
+
+[1] [BigQuery table schema](https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders)
+```
+
+### 4.4 Example: a concept not bound to a resource
+
+```markdown
+---
+type: Playbook
+title: Incident response — data freshness alert
+description: Steps to triage a freshness alert on the orders pipeline.
+tags: [oncall, incident]
+timestamp: 2026-04-12T09:00:00Z
+---
+
+# Trigger
+
+A freshness alert fires when `orders` lags more than 30 minutes behind
+its expected SLA. See the [orders table](/tables/orders.md).
+
+# Steps
+
+1. Check the [ingestion job dashboard](https://example.com/dash).
+2. …
+```
+
+---
+
+## 5. Cross-linking
+
+Concepts MAY link to other concepts using standard markdown links. Two
+forms are supported:
+
+### 5.1 Absolute (bundle-relative) links
+
+Begin with `/`, interpreted relative to the bundle root.
+
+```markdown
+See the [customers table](/tables/customers.md) for the join key.
+```
+
+This is the **recommended** form because it is stable when documents are
+moved within their subdirectory.
+
+### 5.2 Relative links
+
+Standard markdown relative paths.
+
+```markdown
+See the [neighboring concept](./other.md).
+```
+
+### 5.3 Link semantics
+
+A link from concept A to concept B asserts a *relationship*. The
+specific kind of relationship (parent/child, references, joins-with,
+depends-on, etc.) is conveyed by the surrounding prose, not by the link
+itself. Consumers that build a graph view typically treat all links as
+directed edges of an untyped relationship.
+
+Consumers MUST tolerate broken links — a link whose target does not
+exist in the bundle is not malformed; it may simply represent
+not-yet-written knowledge.
+
+---
+
+## 6. Index Files
+
+An `index.md` file MAY appear in any directory, including the bundle
+root. It enumerates the directory's contents to support **progressive
+disclosure** — letting a human or agent see what is available before
+opening individual documents.
+
+Index files contain no frontmatter. The body uses one or more sections,
+each grouping concepts under a heading:
+
+```markdown
+# Section / Group Heading
+
+* [Title 1](relative-url-1) - short description of item 1
+* [Title 2](relative-url-2) - short description of item 2
+
+# Another Section
+
+* [Subdirectory](subdir/) - short description of the subdirectory
+```
+
+Entries SHOULD include the description from the linked concept's
+frontmatter. Producers MAY generate `index.md` automatically; consumers
+MAY synthesize one on the fly when none is present.
+
+---
+
+## 7. Log Files (optional)
+
+A `log.md` file MAY appear at any level of the hierarchy to record the
+history of changes to that scope. The format is a flat list of
+date-grouped entries, newest first:
+
+```markdown
+# Directory Update Log
+
+## 2026-05-22
+* **Update**: Added new BigQuery table reference for [Customer Metrics](/tables/customer-metrics.md).
+* **Creation**: Established the [Dataplex Playbook](/playbooks/dataplex.md).
+
+## 2026-05-15
+* **Initialization**: Created foundational directory structure.
+* **Update**: Added progressive-disclosure guidelines to the root [index](/index.md).
+```
+
+Date headings MUST use ISO 8601 `YYYY-MM-DD` form. Log entries are
+prose; the leading bold word (`**Update**`, `**Creation**`,
+`**Deprecation**`, etc.) is a convention, not a requirement.
+
+---
+
+## 8. Citations
+
+When a concept's body makes claims sourced from external material,
+those sources SHOULD be listed under a `# Citations` heading at the
+bottom of the document, numbered:
+
+```markdown
+# Citations
+
+[1] [BigQuery public dataset announcement](https://cloud.google.com/blog/products/data-analytics/...)
+[2] [Internal data quality runbook](https://wiki.acme.internal/data/quality)
+```
+
+Citation links MAY be absolute URLs, bundle-relative paths, or paths
+into a `references/` subdirectory that mirrors external material as
+first-class OKF concepts.
+
+---
+
+## 9. Conformance
+
+A bundle is **conformant** with OKF v0.1 if:
+
+1. Every non-reserved `.md` file in the tree contains a parseable YAML
+   frontmatter block.
+2. Every frontmatter block contains a non-empty `type` field.
+3. Every reserved filename (`index.md`, `log.md`) follows the structure
+   described in §6 and §7 respectively when present.
+
+Consumers SHOULD treat all other constraints as soft guidance. In
+particular, consumers MUST NOT reject a bundle because of:
+
+- Missing optional frontmatter fields.
+- Unknown `type` values.
+- Unknown additional frontmatter keys.
+- Broken cross-links.
+- Missing `index.md` files.
+
+This permissive consumption model is intentional: OKF is meant to
+remain useful as bundles grow, get refactored, and are partially
+generated by agents.
+
+---
+
+## 10. Relationship to other formats
+
+OKF is intentionally close to several established patterns:
+
+- **LLM "wiki" repositories** that use markdown + frontmatter as
+  agent-readable knowledge bases.
+- **Personal knowledge tools** like Obsidian and Notion, which use
+  hierarchical markdown with cross-links.
+- **"Metadata as code"** approaches that store catalog metadata
+  alongside source code rather than in a separate registry.
+
+OKF differs primarily in being **specified** — pinning down the small
+set of rules needed for interoperability without dictating tooling.
+
+---
+
+## 11. Versioning
+
+This document specifies OKF version **0.1**. Future revisions will be
+versioned in the form `<major>.<minor>`:
+
+- A **minor** version bump introduces backward-compatible additions
+  (new optional fields, new conventional section headings).
+- A **major** version bump may make breaking changes (renaming required
+  fields, changing reserved filenames).
+
+Bundles MAY declare the OKF version they target by including
+`okf_version: "0.1"` in a bundle-root `index.md` frontmatter block (the
+only place frontmatter is permitted in an `index.md`). Consumers that
+do not understand the declared version SHOULD attempt best-effort
+consumption rather than refusing the bundle.
+
+---
+
+## Appendix A — Minimal example bundle
+
+```
+my_bundle/
+├── index.md
+├── datasets/
+│   ├── index.md
+│   └── sales.md
+└── tables/
+    ├── index.md
+    ├── orders.md
+    └── customers.md
+```
+
+`datasets/sales.md`:
+
+```markdown
+---
+type: BigQuery Dataset
+title: Sales
+description: All sales-related tables for the retail business.
+resource: https://console.cloud.google.com/bigquery?p=acme&d=sales
+tags: [sales]
+timestamp: 2026-05-28T00:00:00Z
+---
+
+The sales dataset contains transactional tables, including
+[orders](/tables/orders.md) and [customers](/tables/customers.md).
+```
+
+`tables/orders.md`:
+
+```markdown
+---
+type: BigQuery Table
+title: Orders
+description: One row per completed customer order.
+resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=orders
+tags: [sales, orders]
+timestamp: 2026-05-28T00:00:00Z
+---
+
+# Schema
+
+| Column        | Type      | Description                  |
+|---------------|-----------|------------------------------|
+| `order_id`    | STRING    | Unique order identifier.     |
+| `customer_id` | STRING    | FK to [customers](/tables/customers.md). |
+| `total_usd`   | NUMERIC   | Order total in USD.          |
+
+Part of the [sales dataset](/datasets/sales.md).
+```

okf_cli-0.1.0/PKG-INFO

@@ -0,0 +1,144 @@
+Metadata-Version: 2.4
+Name: okf-cli
+Version: 0.1.0
+Summary: Open Knowledge Format tooling
+License: MIT License
+        
+        Copyright (c) 2026 Dheerapat Tookkane
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Requires-Python: >=3.13
+Requires-Dist: typer>=0.15
+Description-Content-Type: text/markdown
+
+# okf-cli — Open Knowledge Format tooling
+
+Converts plain markdown into [OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)-conformant knowledge bundles. Domain experts write `# Title` then `> description` — `okf enrich` generates frontmatter, type, timestamps, and index files.
+
+## Quickstart
+
+```bash
+uv sync
+uv run okf example              # output → bundled/
+cat bundled/index.md
+cat bundled/tables/orders.md
+```
+
+## Usage
+
+```
+okf <input-dir> [output-dir] [--default-type <name>]
+```
+
+| Argument | Description |
+|---|---|
+| `input-dir` | Directory of plain `.md` files |
+| `output-dir` | Target directory (default: `bundled`) |
+| `--default-type` | Type for root-level files (skip root files if omitted) |
+
+## Writing input files
+
+Every `.md` file must start with:
+
+```markdown
+# Title
+
+> Description
+> Second optional description line.
+```
+
+Everything after the description block is free-form — preserved unchanged.
+
+**Rules:**
+
+| Rule | Why |
+|---|---|
+| First line must be `# Title` | Tool reads title here |
+| Followed by `> Description` | Tool reads description here |
+| Folder name = concept type | `tables/orders.md` → `type: "tables"` |
+| Only `.md` files processed | Non-`.md` files ignored |
+| Reserved names skipped: `index.md`, `log.md`, `README.md` | OKF spec reserves these |
+
+**Violations:**
+
+| Condition | Behavior |
+|---|---|
+| Line 1 not `# Title` | Error, stop |
+| Empty title | Error, stop |
+| No `> description` block | Error, stop |
+| Root-level file without `--default-type` | Skip file, warn, continue |
+
+Root files need `--default-type`. Otherwise put files in named folders.
+
+See the [`example/`](example/) directory for a sample of how to structure files.
+
+## How it works
+
+1. Walk `input-dir` for `.md` files (skip reserved names)
+2. Extract `title` from `#`, `description` from `>`
+3. Set `type` from parent dir name, `timestamp` from file mtime
+4. Write concept files with YAML frontmatter
+5. Generate `index.md` per directory — `# Contents` for files, `# Directories` for subdirs (recursive)
+
+## Output
+
+Each concept becomes a markdown file with YAML frontmatter:
+
+```yaml
+---
+type: "tables"
+title: "Customer Orders"
+description: "One row per completed customer order across all channels."
+timestamp: "2026-07-04T15:06:51+00:00"
+---
+
+Original body preserved as-is.
+```
+
+Every directory gets its own `index.md`:
+
+```markdown
+# Contents
+
+* [Customer Orders](orders.md) - One row per completed customer order across all channels.
+
+# Directories
+
+* [partitions](partitions/)
+```
+
+## OKF Conformance
+
+Generated bundles conform to [OKF v0.1](OKF_SPEC.md) (§9):
+
+- Every non-reserved `.md` has parseable YAML frontmatter with non-empty `type` ✓
+- Reserved filenames follow spec structure ✓
+- Consumers MUST tolerate missing optional fields, unknown types, broken links ✓
+
+## Project layout
+
+```
+okf-cli
+├── OKF_SPEC.md           # OKF specification
+├── pyproject.toml        # uv-managed Python project
+├── src/okf/cli.py        # Single-file CLI
+├── tests/test_cli.py     # Tests
+└── example/              # Sample input markdown
+```