okf-schema 0.2.0__tar.gz

okf_schema-0.2.0/.gitignore

@@ -0,0 +1,220 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[codz]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#   Usually these files are written by a python script from a template
+#   before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py.cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+# Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+# uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+# poetry.lock
+# poetry.toml
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#   pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
+#   https://pdm-project.org/en/latest/usage/project/#working-with-version-control
+# pdm.lock
+# pdm.toml
+.pdm-python
+.pdm-build/
+
+# pixi
+#   Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
+# pixi.lock
+#   Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
+#   in the .venv directory. It is recommended not to include this directory in version control.
+.pixi
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# Redis
+*.rdb
+*.aof
+*.pid
+
+# RabbitMQ
+mnesia/
+rabbitmq/
+rabbitmq-data/
+
+# ActiveMQ
+activemq-data/
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.envrc
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#   JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#   be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#   and can be added to the global gitignore or merged into this file.  For a more nuclear
+#   option (not recommended) you can uncomment the following to ignore the entire idea folder.
+# .idea/
+
+# Abstra
+#   Abstra is an AI-powered process automation framework.
+#   Ignore directories containing user credentials, local state, and settings.
+#   Learn more at https://abstra.io/docs
+.abstra/
+
+# Visual Studio Code
+#   Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
+#   that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
+#   and can be added to the global gitignore or merged into this file. However, if you prefer,
+#   you could uncomment the following to ignore the entire vscode folder
+# .vscode/
+# Temporary file for partial code execution
+tempCodeRunnerFile.py
+
+# Ruff stuff:
+.ruff_cache/
+
+# PyPI configuration file
+.pypirc
+
+# Marimo
+marimo/_static/
+marimo/_lsp/
+__marimo__/
+
+# Streamlit
+.streamlit/secrets.toml
+
+.agents/changes/

okf_schema-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gaetan Semet
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

okf_schema-0.2.0/PKG-INFO

@@ -0,0 +1,281 @@
+Metadata-Version: 2.4
+Name: okf-schema
+Version: 0.2.0
+Summary: CLI tool and Python library for working with OKF (Open Knowledge Format) bundles
+Project-URL: Homepage, https://github.com/gsemet/okf-schema
+Project-URL: Documentation, https://okf-schema.readthedocs.io
+Project-URL: Repository, https://github.com/gsemet/okf-schema
+Project-URL: Issues, https://github.com/gsemet/okf-schema/issues
+Author-email: Gaetan Semet <gaetan@xeberon.net>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing :: Markup
+Requires-Python: >=3.10
+Requires-Dist: click>=8.0
+Requires-Dist: jsonschema>=4.23.0
+Requires-Dist: pyjson5>=1.6.0
+Requires-Dist: ruamel-yaml>=0.18.0
+Description-Content-Type: text/markdown
+
+# okf-schema
+
+[![CI](https://github.com/gsemet/okf-schema/actions/workflows/ci.yml/badge.svg)](https://github.com/gsemet/okf-schema/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/okf-schema)](https://pypi.org/project/okf-schema/)
+[![Python Versions](https://img.shields.io/pypi/pyversions/okf-schema)](https://pypi.org/project/okf-schema/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**okf-schema** is a CLI tool and Python library for working with **OKF (Open Knowledge Format)** bundles
+with JSONSchema validation of the frontmatter metadata, and formatting capabilities while preserving comments.
+
+OKF is a markdown-based knowledge format where each concept is a markdown file with YAML frontmatter.
+
+> [!IMPORTANT]
+> OKF-schema is opinionated. It delivers a valid OKF bundle but is adds a structure on the frontmatter that
+> is not allowed in OKF specification:
+>
+> ```raw
+> Type values are **not** registered centrally. Producers SHOULD pick
+> values that are descriptive and self-explanatory; consumers MUST
+> tolerate unknown types gracefully (typically by treating them as
+> generic concepts).
+> ```
+>
+> In a strict OKF bundle, the `type` field is mandatory but can take any value and the validator needs
+> to allow any field in the frontmatter.
+>
+> OKF-schema **requires** the type to be one of the registered types in the `_schema/` directory
+> and validates the frontmatter against the corresponding schema.
+> Additional properties may or may not be allowed depending on the schema definition.
+
+## What `okf-schema` adds to OKF
+
+Plain OKF only defines a folder of markdown files. `okf-schema` turns those files into a **validated, queryable knowledge base** by adding:
+
+| Capability | What it does |
+|-----------|--------------|
+| **Schema-driven frontmatter validation** | Every concept's YAML frontmatter is checked against a JSONSchema. Invalid fields, missing required keys, or wrong types are reported as structured errors. |
+| **Auto-discovered schemas** | Schemas live inside the bundle under `_schema/` (e.g. `_schema/concept.schema.yaml`). The `type` field in a concept's frontmatter tells `okf-schema` which schema file to load. A concept with `type: concept` is validated against `_schema/concept.schema.yaml`. Schemas can be written in **YAML**, **JSON**, or **JSON5** (JSON with comments and trailing commas). |
+| **Bundle integrity checks** | Detects broken internal links, missing `index.md` files, malformed `log.md` entries, and reserved-file violations. |
+| **Safe linting** | Normalizes YAML frontmatter by flattening nested lists and converting block-style to inline notation while preserving comments and custom quotes via `ruamel.yaml`. |
+| **Analytics** | Bundle statistics. |
+
+See a real schema definition in [`examples/ai-llm-knowledge-base/_schema/concept.schema.yaml`](examples/ai-llm-knowledge-base/_schema/concept.schema.yaml).
+
+Example of structure
+
+```raw
+my-bundle/
+├── _schema/
+│   ├── concept.schema.yaml
+│   ├── tool.schema.json
+│   └── paper.schema.json5
+├── concepts/
+│   ├── rag.md
+│   └── chain-of-thought.md
+├── tools/
+│   ├── langchain.md
+│   └── llamaindex.md
+├── papers/
+│   ├── rag-paper.md
+│   └── chain-of-thought-paper.md
+├── index.md
+└── log.md
+```
+
+The `type` field in each entity frontmatter determines which schema is used for validation.
+For example, `type: concept` uses `_schema/concept.schema.yaml`, while `type: tool` uses `_schema/tool.schema.json`.
+
+Schema extensions supported:
+
+- `.schema.yaml` — YAML (human-friendly, supports comments and anchors)
+- `.schema.json` — JSON (strict syntax, widely supported by editors)
+- `.schema.json5` — JSON5 (JSON with comments, trailing commas, and unquoted keys)
+
+## Installation
+
+```bash
+uv tool install okf-schema
+```
+
+## Quickstart
+
+```bash
+# Initialize a new OKF bundle
+okf-schema init my-bundle
+
+# Update index.md files for all directories
+okf-schema index --path my-bundle/bundle
+
+# Lint frontmatter (flatten nested lists and convert block-style to inline)
+okf-schema lint --path my-bundle/bundle
+
+# Validate a bundle
+okf-schema validate --path my-bundle/bundle
+# or enforce strict validation (fail on warnings)
+okf-schema validate --path my-bundle/bundle --strict
+
+# List all concepts
+okf-schema list --path my-bundle/bundle
+```
+
+## CLI Reference
+
+| Subcommand | Description |
+|-----------|-------------|
+| `init <name>` | Create a new OKF bundle directory structure |
+| `new --path <dir> --name <name>` | Create a new concept file with frontmatter template |
+| `validate --path <bundle>` | Validate bundle structure and frontmatter |
+| `validate --path <bundle> --strict` | Validate and fail on warnings |
+| `lint --path <bundle>` | Lint frontmatter: flatten nested lists and convert block-style to inline |
+| `list --path <bundle>` | List all concepts in a bundle |
+| `show --path <bundle> <concept>` | Show a single concept's frontmatter and body |
+| `index --path <bundle>` | Regenerate all `index.md` files |
+| `stats --path <bundle>` | Show bundle statistics |
+
+## Recommended Workflow
+
+Before packaging or distributing a bundle, run these three commands in order and fix all warnings:
+
+```bash
+okf-schema index --path my-bundle/bundle    # regenerate index.md files
+okf-schema lint --path my-bundle/bundle     # flatten nested lists and convert block lists to inline
+okf-schema validate --path my-bundle/bundle --strict # check structure, schema, and links; fail on warnings
+```
+
+Only zip or ship the bundle once `validate --strict` reports **zero errors and zero warnings**. Warnings such as missing `index.md` (W4), block-style lists (W7), or broken cross-links (W2) signal issues that will degrade the experience for downstream consumers.
+
+## Example: AI & LLM Knowledge Base
+
+The [`examples/ai-llm-knowledge-base/`](examples/ai-llm-knowledge-base/) directory contains a realistic knowledge base with **three concept types** — `concept`, `tool`, and `paper` — each validated by its own schema in `_schema/`.
+
+### How `type` selects the schema
+
+The `type` field in a concept's frontmatter determines which schema file is loaded. A file with `type: concept` is validated against `_schema/concept.schema.yaml`; `type: tool` against `_schema/tool.schema.json`; and `type: paper` against `_schema/paper.schema.json5`.
+
+### Schema format support
+
+`okf-schema` accepts schemas in three formats:
+
+| Extension | Format | Notes |
+|-----------|--------|-------|
+| `.schema.yaml` | YAML | Human-friendly, supports comments and anchors |
+| `.schema.json` | JSON | Strict syntax, widely supported by editors |
+| `.schema.json5` | JSON5 | JSON with comments, trailing commas, and unquoted keys |
+
+### Schema highlights
+
+**`concept.schema.yaml`** — AI concepts with enums, email validation, and kebab-case regex:
+
+```yaml
+properties:
+  category:
+    enum: [LLM, AI Agent, Coding Agent, Prompt Engineering, Tooling, Evaluation]
+  maturity:
+    enum: [experimental, beta, production, deprecated]
+  author_email:
+    type: string
+    format: email
+  tags:
+    type: array
+    items:
+      pattern: "^[a-z0-9-]+$"   # kebab-case only
+```
+
+**`tool.schema.json`** — Developer tools with URI validation and language enums:
+
+```json
+{
+  "properties": {
+    "license": {
+      "enum": ["MIT", "Apache-2.0", "GPL-3.0", "Proprietary", "Other"]
+    },
+    "language": {
+      "enum": ["Python", "JavaScript", "TypeScript", "Rust", "Go", "Java", "Multi-language"]
+    },
+    "url": { "type": "string", "format": "uri" }
+  }
+}
+```
+
+**`paper.schema.json5`** — Research papers with year bounds and venue enums:
+
+```javascript
+// JSON5 allows comments, trailing commas, and unquoted keys
+{
+  properties: {
+    year: { type: "integer", minimum: 1950, maximum: 2030 },
+    venue: {
+      enum: ["NeurIPS", "ICML", "ICLR", "ACL", "EMNLP", "arXiv", "Other"]
+    },
+    bibtex_key: { pattern: "^[A-Za-z0-9_-]+$" },
+  },
+}
+```
+
+### Concept file example (`concepts/rag.md`)
+
+```markdown
+---
+type: concept
+title: Retrieval-Augmented Generation
+description: >
+  A technique that enhances LLM outputs by retrieving relevant documents
+  from an external knowledge store and injecting them into the prompt.
+category: LLM
+maturity: production
+author_email: bob@example.com
+complexity: intermediate
+tags: [rag, retrieval, llm, knowledge-base]
+related_tools: [LangChain, LlamaIndex, OpenAI-API]
+---
+
+# Retrieval-Augmented Generation
+
+RAG combines parametric knowledge (the model's weights) with non-parametric
+knowledge (external documents) to reduce hallucinations...
+```
+
+### Validation in action
+
+```bash
+# Validates all concepts, tools, and papers against their respective schemas
+okf-schema validate --path examples/ai-llm-knowledge-base
+
+# Show bundle statistics
+okf-schema stats --path examples/ai-llm-knowledge-base
+```
+
+## Python API
+
+```python
+from okf_schema.api import validate_bundle
+
+report = validate_bundle("path/to/bundle")
+for finding in report.findings:
+    print(finding.level, finding.message)
+
+# The _schema/ directory inside the bundle is auto-discovered.
+# You can also pass an explicit schema_db path:
+# report = validate_bundle("path/to/bundle", schema_db="path/to/schemas")
+```
+
+## Documentation
+
+Full documentation is available at [https://okf-schema.readthedocs.io](https://okf-schema.readthedocs.io).
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and guidelines.
+
+## License
+
+MIT License — see [LICENSE](LICENSE) for details.