github-dkg 0.1.0__tar.gz

github_dkg-0.1.0/.gitignore

@@ -0,0 +1,5 @@
+*.pyc
+__pycache__/
+.pytest_cache/
+dist/
+*.egg-info/

github_dkg-0.1.0/DESIGN_BRIEF.md

@@ -0,0 +1,192 @@
+# github-dkg Design Brief
+
+**Package:** `github-dkg`  
+**Bounty tag:** `cfi-dkgv10-r1`  
+**Tier target:** Flagship (8,000–10,000 TRAC)
+
+---
+
+## 1. Problem
+
+Software teams produce tacit knowledge continuously — in issue threads, PR descriptions, code review comments, post-mortem discussions. This knowledge evaporates the moment an issue is closed or a PR merges. It lives in GitHub, locked away from agents.
+
+Existing retrieval approaches (GitHub search, grep, CI logs) can surface raw text, but they provide:
+- No provenance — who decided what, and when
+- No trust gradient — a passing remark and an architecture decision look identical
+- No agent-native interface — agents cannot treat GitHub as a queryable knowledge substrate
+
+DKG v10 Working Memory solves all three. This package ingests GitHub's knowledge stream into DKG v10, where it becomes attributable, queryable, and promotable toward on-chain verification.
+
+---
+
+## 2. Target users
+
+- **Platform and DevOps teams** who want their engineering knowledge base to accumulate passively, without manual curation
+- **Research-engineering teams** running long-horizon agentic workflows (code analysis, architecture review, dependency audits) that need to query what the team previously decided
+- **Multi-agent systems** that coordinate across a repository: an agent can write a PR review into Shared Working Memory and a downstream agent can query it to inform its next action
+
+---
+
+## 3. Architecture
+
+```
+GitHub Repository
+        │
+        ├─ Issues (+ comments)
+        ├─ Pull Requests (+ reviews + inline comments)
+        │
+        ▼
+GitHubDKGIngestor
+        │
+        ├─ GitHubClient ──────────────► GitHub REST API v3
+        │   (issues, pulls, reviews,     (unauthenticated rate-limited
+        │    inline comments)             or authenticated via GITHUB_TOKEN)
+        │
+        ├─ MarkdownFormatter ─────────► Structured Markdown Knowledge Asset
+        │   (one KA per issue/PR,         per item, with code-aware tagging
+        │    comments embedded)           and provenance metadata)
+        │
+        └─ DKGClient ─────────────────► POST /api/memory/turn
+                                         (one Knowledge Asset per item,
+                                          scoped to a repo Context Graph,
+                                          sessionUri = github.com/owner/repo)
+
+GitHub Action (Docker)
+        │
+        └─ Triggered on: issues, pull_request, pull_request_review events
+           Reads: GITHUB_EVENT_PATH payload → item number → ingest
+           Writes: GITHUB_OUTPUT turn-uri, layer
+```
+
+### API surface used
+
+All communication is over the public DKG v10 HTTP API — no internal packages.
+
+| Endpoint | Purpose |
+|---|---|
+| `GET /api/agents` | Health check / token validation |
+| `POST /api/memory/turn` | Write Knowledge Asset (one per issue/PR) |
+| `POST /api/memory/search` | Tri-modal search across ingested knowledge |
+| `POST /api/assertion/:name/promote` | SHARE to Shared Working Memory |
+
+---
+
+## 4. Memory layer mapping and LLM-Wiki alignment
+
+Karpathy's LLM-Wiki frames knowledge substrates by who can read and write them. GitHub is where engineering teams produce knowledge that currently has no agent-native substrate. This package maps GitHub's knowledge types to the v10 trust gradient:
+
+| GitHub artifact | DKG v10 layer | Default | Promotion trigger |
+|---|---|---|---|
+| Open issue | Working Memory | `wm` | Team label e.g. `architecture-decision` |
+| Closed issue | Working Memory | `wm` | Post-mortem label or manual promote |
+| Draft PR | Working Memory | `wm` | — |
+| Merged PR | Working Memory | `wm` | Label `architecture-decision`, or manual |
+| Review comment (APPROVED) | Working Memory | `wm` | PR merge + label |
+
+The **sessionUri** for every Knowledge Asset is set to `https://github.com/owner/repo`, linking all assets for a repository into a coherent session in the Context Graph. This allows an agent to retrieve all knowledge about a repository in a single search.
+
+---
+
+## 5. Trust gradient and promotion path
+
+### Working Memory → Shared Working Memory (SHARE)
+
+An engineering team's GitHub repo is a natural unit of Shared Memory. When a significant PR merges — one labelled `architecture-decision`, or one identified by an agent as high-signal — the workflow promotes its Knowledge Asset:
+
+```bash
+github-dkg promote dkg://wm/turn/abc123 --context-graph my-project
+```
+
+This calls `POST /api/assertion/:name/promote` — a Curator-authorized operation. Nothing is promoted automatically; the agent or CI pipeline must explicitly decide.
+
+The example workflow in `examples/workflow.yml` shows a concrete trigger: PRs with the `architecture-decision` label are automatically promoted on merge.
+
+### Shared Working Memory → Verified Memory (PUBLISH)
+
+Round 2 surface. Once an architecture decision or post-mortem is in Shared Working Memory, it can be published to Verified Memory via `POST /api/shared-memory/publish`. The UAL chain is preserved through all promotions: the on-chain record traces back to the original GitHub issue or PR, preserving full provenance.
+
+### Oracle-readiness
+
+Every Knowledge Asset written by this package:
+- Has a stable UAL (the `turnUri` returned by `/api/memory/turn`)
+- Is scoped to a Context Graph, making it consumable by a context oracle querying that graph
+- Uses `sessionUri` to link assets for a repository, enabling oracle queries like "all architecture decisions for repo X"
+- Structured Markdown with explicit field headers (`**Author:**`, `**Labels:**`, `**State:**`) produces consistent RDF triples, making semantic queries predictable
+
+---
+
+## 6. Knowledge Asset format
+
+Each GitHub item is encoded as structured Markdown before ingestion. The DKG node runs structural + semantic extraction on this Markdown, building RDF triples from the field headers and free-text content.
+
+**Issue example:**
+
+```markdown
+**GitHub Issue #42:** Fix null pointer in auth flow
+**Repository:** acme/api
+**Author:** alice  |  **Labels:** bug, priority-high  |  **State:** closed (completed)
+**Created:** 2024-03-01  |  **Closed:** 2024-03-05
+**URL:** https://github.com/acme/api/issues/42
+
+**Description:**
+The login endpoint returns 500 when password contains `$`.
+
+**Comments:**
+- **bob** (2024-03-02): Reproduced on 2.3.1.
+- **alice** (2024-03-04): Fixed in commit abc123.
+```
+
+**PR example:**
+
+```markdown
+**GitHub PR #99:** Add PKCE support for mobile OAuth
+**Repository:** acme/api
+**Author:** carol  |  **Labels:** feature  |  **State:** merged
+**Requested reviewers:** dave
+**Branch:** feature/pkce → main
+**Created:** 2024-03-10  |  **Merged:** 2024-03-15
+**URL:** https://github.com/acme/api/pull/99
+
+**Description:**
+Implements RFC 7636 PKCE for public clients.
+
+**Reviews:**
+- **dave** APPROVED (2024-03-14): LGTM, clean implementation.
+
+**Inline review comments:**
+- `src/auth/pkce.py`:
+  - dave: Consider caching the verifier.
+```
+
+---
+
+## 7. v10 vocabulary compliance
+
+All code and documentation uses the exact DKG v10 terminology:
+
+- **Context Graph** — one per repository, scoping all Knowledge Assets for that repo
+- **Knowledge Asset** — one per GitHub issue or PR
+- **Working Memory** / **Shared Working Memory** / **Verified Memory** — never "private/public/chain"
+- **SHARE** — for promotion to Shared Working Memory
+- **PUBLISH** — for promotion to Verified Memory (Round 2)
+- **Curator** — the authority required for SHARE/PUBLISH operations
+
+One intentional deviation: the CLI uses `layer` as shorthand for `--layer wm|swm` as a usability affordance for operators. Internal code and documentation always expands this to the full v10 term.
+
+---
+
+## 8. Security notes
+
+- All credentials (`DKG_TOKEN`, `GITHUB_TOKEN`) are read from environment variables — never hardcoded or logged
+- No Curator operations (SHARE/PUBLISH) are performed automatically; all promotion is explicit and operator-initiated
+- The Docker action image has no `postinstall` or `preinstall` scripts
+- Network egress: GitHub REST API (`api.github.com`) and the configured DKG node endpoint — no other external domains
+- Write authority: only `POST /api/memory/turn` (write Working Memory) and `POST /api/assertion/:name/promote` (SHARE, Curator-authorized). No chain-write operations.
+- No dynamic code loading, no `eval` on external input
+- The `GITHUB_TOKEN` used in the Action has the minimum required permissions: `contents: read`, `issues: read`, `pull-requests: read`
+
+---
+
+## 9. Maintenance commitment
+
+Six-month support window from submission date. Issues and pull requests will be reviewed within 5 business days. The package follows semantic versioning; breaking changes will be major version bumps with migration notes.

github_dkg-0.1.0/Dockerfile

@@ -0,0 +1,12 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+COPY pyproject.toml LICENSE README.md ./
+COPY src/ ./src/
+
+RUN pip install --no-cache-dir -e .
+
+COPY entrypoint.py ./
+
+ENTRYPOINT ["python", "/app/entrypoint.py"]

github_dkg-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

github_dkg-0.1.0/PKG-INFO

@@ -0,0 +1,143 @@
+Metadata-Version: 2.4
+Name: github-dkg
+Version: 0.1.0
+Summary: Ingest GitHub issues, PRs, and review comments into DKG v10 Working Memory
+Project-URL: Repository, https://github.com/haroldboom/github-dkg
+License: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: click>=8.1
+Requires-Dist: httpx>=0.27
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: respx>=0.21; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# github-dkg
+
+Ingest GitHub issues, pull requests, and review comments into [DKG v10](https://docs.origintrail.io) Working Memory as Knowledge Assets.
+
+Every issue and PR becomes a queryable, attributable Knowledge Asset in your DKG v10 node. Key decisions can be promoted to Shared Working Memory — making your team's engineering knowledge accessible to agents.
+
+## Demo
+
+- **Walkthrough notebook:** [`demo.ipynb`](demo.ipynb) — runs end-to-end against a built-in mock of GitHub and the DKG node, no tokens required. Open in [Colab](https://colab.research.google.com/github/haroldboom/github-dkg/blob/master/demo.ipynb).
+- **Live recording script:** [`examples/demo_video.py`](examples/demo_video.py) — drives all three demos against a real DKG node and the GitHub API; this is the script behind the bounty walkthrough video.
+
+## Install
+
+```bash
+pip install github-dkg
+```
+
+## Quickstart
+
+```bash
+export DKG_TOKEN=your-dkg-token
+export DKG_BASE_URL=http://localhost:9200
+export DKG_CONTEXT_GRAPH=your-context-graph-id
+export GITHUB_TOKEN=your-github-token
+
+# Bulk-ingest all issues and PRs from a repository
+github-dkg ingest owner/repo --context-graph $DKG_CONTEXT_GRAPH
+
+# Ingest a single issue
+github-dkg ingest-one owner/repo 42 --type issue --context-graph $DKG_CONTEXT_GRAPH
+
+# Ingest a single PR
+github-dkg ingest-one owner/repo 99 --type pr --context-graph $DKG_CONTEXT_GRAPH
+
+# Search ingested knowledge
+github-dkg search "authentication bug" --context-graph $DKG_CONTEXT_GRAPH
+
+# Promote a Working Memory asset to Shared Working Memory (SHARE)
+github-dkg promote dkg://wm/turn/abc123 --context-graph $DKG_CONTEXT_GRAPH
+```
+
+## GitHub Action
+
+Automatically ingest issues and PRs as they are created or updated. Add to `.github/workflows/dkg-ingest.yml`:
+
+```yaml
+on:
+  issues:
+    types: [opened, edited, closed]
+  pull_request:
+    types: [opened, edited, closed]
+  pull_request_review:
+    types: [submitted]
+
+jobs:
+  ingest:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: haroldboom/github-dkg@v0.1.0
+        id: ingest
+        with:
+          dkg-token: ${{ secrets.DKG_TOKEN }}
+          dkg-base-url: ${{ secrets.DKG_BASE_URL }}
+          dkg-context-graph: ${{ secrets.DKG_CONTEXT_GRAPH }}
+```
+
+See `examples/workflow.yml` for a complete example including automatic promotion of architecture-decision PRs to Shared Working Memory.
+
+## Python API
+
+```python
+import asyncio
+from github_dkg import DKGClient, GitHubClient, GitHubDKGIngestor
+
+async def main():
+    dkg = DKGClient(base_url="http://localhost:9200", token="your-token")
+    gh = GitHubClient(token="your-github-token")
+    ingestor = GitHubDKGIngestor(dkg=dkg, github=gh, context_graph_id="cg-123")
+
+    # Bulk ingest
+    result = await ingestor.ingest_repo("owner", "repo", since="2024-01-01")
+    print(f"Ingested {result.total} items ({len(result.errors)} errors)")
+
+    # Single item
+    resp = await ingestor.ingest_issue("owner", "repo", 42)
+    print(f"Turn URI: {resp['turnUri']}")
+
+    # Promote to Shared Working Memory
+    await ingestor.promote(resp["turnUri"])
+
+asyncio.run(main())
+```
+
+## `--since` filtering
+
+`--since` accepts an ISO 8601 timestamp and limits ingest to items updated after that point.
+
+- **Issues:** filtered server-side by GitHub via the `since` parameter on `/issues`.
+- **Pull requests:** GitHub's `/pulls` endpoint has no `since` filter, so the package requests `sort=updated&direction=desc` and stops paginating once results fall below the cutoff. Net result: only PRs touched after `--since` are fetched and ingested.
+
+Comment-only updates (a new comment without an issue/PR body edit) still bump `updated_at`, so they're included.
+
+## Rate limiting
+
+`GitHubClient` raises `github_dkg.github_client.GitHubRateLimitError` when GitHub returns `403`/`429` with `X-RateLimit-Remaining: 0`. The exception carries `reset_at` (unix timestamp) so callers can decide whether to back off, sleep, or fail. Authenticated tokens get 5,000 requests/hour; bulk-ingesting a large repo with many comment-heavy PRs can approach this limit.
+
+```python
+from github_dkg.github_client import GitHubRateLimitError
+
+try:
+    result = await ingestor.ingest_repo("OriginTrail", "dkg-v9")
+except GitHubRateLimitError as e:
+    print(f"Rate limited; resets at unix={e.reset_at}")
+```
+
+## Memory layers
+
+| Layer | Flag | Visibility |
+|---|---|---|
+| Working Memory | `--layer wm` (default) | Private to your node |
+| Shared Working Memory | `--layer swm` | Gossiped across the paranet |
+
+Promotion from Working Memory to Shared Working Memory is always explicit — nothing is shared automatically.
+
+## License
+
+MIT

github_dkg-0.1.0/README.md

@@ -0,0 +1,127 @@
+# github-dkg
+
+Ingest GitHub issues, pull requests, and review comments into [DKG v10](https://docs.origintrail.io) Working Memory as Knowledge Assets.
+
+Every issue and PR becomes a queryable, attributable Knowledge Asset in your DKG v10 node. Key decisions can be promoted to Shared Working Memory — making your team's engineering knowledge accessible to agents.
+
+## Demo
+
+- **Walkthrough notebook:** [`demo.ipynb`](demo.ipynb) — runs end-to-end against a built-in mock of GitHub and the DKG node, no tokens required. Open in [Colab](https://colab.research.google.com/github/haroldboom/github-dkg/blob/master/demo.ipynb).
+- **Live recording script:** [`examples/demo_video.py`](examples/demo_video.py) — drives all three demos against a real DKG node and the GitHub API; this is the script behind the bounty walkthrough video.
+
+## Install
+
+```bash
+pip install github-dkg
+```
+
+## Quickstart
+
+```bash
+export DKG_TOKEN=your-dkg-token
+export DKG_BASE_URL=http://localhost:9200
+export DKG_CONTEXT_GRAPH=your-context-graph-id
+export GITHUB_TOKEN=your-github-token
+
+# Bulk-ingest all issues and PRs from a repository
+github-dkg ingest owner/repo --context-graph $DKG_CONTEXT_GRAPH
+
+# Ingest a single issue
+github-dkg ingest-one owner/repo 42 --type issue --context-graph $DKG_CONTEXT_GRAPH
+
+# Ingest a single PR
+github-dkg ingest-one owner/repo 99 --type pr --context-graph $DKG_CONTEXT_GRAPH
+
+# Search ingested knowledge
+github-dkg search "authentication bug" --context-graph $DKG_CONTEXT_GRAPH
+
+# Promote a Working Memory asset to Shared Working Memory (SHARE)
+github-dkg promote dkg://wm/turn/abc123 --context-graph $DKG_CONTEXT_GRAPH
+```
+
+## GitHub Action
+
+Automatically ingest issues and PRs as they are created or updated. Add to `.github/workflows/dkg-ingest.yml`:
+
+```yaml
+on:
+  issues:
+    types: [opened, edited, closed]
+  pull_request:
+    types: [opened, edited, closed]
+  pull_request_review:
+    types: [submitted]
+
+jobs:
+  ingest:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: haroldboom/github-dkg@v0.1.0
+        id: ingest
+        with:
+          dkg-token: ${{ secrets.DKG_TOKEN }}
+          dkg-base-url: ${{ secrets.DKG_BASE_URL }}
+          dkg-context-graph: ${{ secrets.DKG_CONTEXT_GRAPH }}
+```
+
+See `examples/workflow.yml` for a complete example including automatic promotion of architecture-decision PRs to Shared Working Memory.
+
+## Python API
+
+```python
+import asyncio
+from github_dkg import DKGClient, GitHubClient, GitHubDKGIngestor
+
+async def main():
+    dkg = DKGClient(base_url="http://localhost:9200", token="your-token")
+    gh = GitHubClient(token="your-github-token")
+    ingestor = GitHubDKGIngestor(dkg=dkg, github=gh, context_graph_id="cg-123")
+
+    # Bulk ingest
+    result = await ingestor.ingest_repo("owner", "repo", since="2024-01-01")
+    print(f"Ingested {result.total} items ({len(result.errors)} errors)")
+
+    # Single item
+    resp = await ingestor.ingest_issue("owner", "repo", 42)
+    print(f"Turn URI: {resp['turnUri']}")
+
+    # Promote to Shared Working Memory
+    await ingestor.promote(resp["turnUri"])
+
+asyncio.run(main())
+```
+
+## `--since` filtering
+
+`--since` accepts an ISO 8601 timestamp and limits ingest to items updated after that point.
+
+- **Issues:** filtered server-side by GitHub via the `since` parameter on `/issues`.
+- **Pull requests:** GitHub's `/pulls` endpoint has no `since` filter, so the package requests `sort=updated&direction=desc` and stops paginating once results fall below the cutoff. Net result: only PRs touched after `--since` are fetched and ingested.
+
+Comment-only updates (a new comment without an issue/PR body edit) still bump `updated_at`, so they're included.
+
+## Rate limiting
+
+`GitHubClient` raises `github_dkg.github_client.GitHubRateLimitError` when GitHub returns `403`/`429` with `X-RateLimit-Remaining: 0`. The exception carries `reset_at` (unix timestamp) so callers can decide whether to back off, sleep, or fail. Authenticated tokens get 5,000 requests/hour; bulk-ingesting a large repo with many comment-heavy PRs can approach this limit.
+
+```python
+from github_dkg.github_client import GitHubRateLimitError
+
+try:
+    result = await ingestor.ingest_repo("OriginTrail", "dkg-v9")
+except GitHubRateLimitError as e:
+    print(f"Rate limited; resets at unix={e.reset_at}")
+```
+
+## Memory layers
+
+| Layer | Flag | Visibility |
+|---|---|---|
+| Working Memory | `--layer wm` (default) | Private to your node |
+| Shared Working Memory | `--layer swm` | Gossiped across the paranet |
+
+Promotion from Working Memory to Shared Working Memory is always explicit — nothing is shared automatically.
+
+## License
+
+MIT

github_dkg-0.1.0/action.yml

@@ -0,0 +1,57 @@
+name: 'DKG Working Memory Ingest'
+description: 'Ingest GitHub issues, PRs, and review comments into DKG v10 Working Memory as Knowledge Assets'
+author: 'github-dkg'
+
+branding:
+  icon: 'database'
+  color: 'purple'
+
+inputs:
+  dkg-token:
+    description: 'DKG v10 node bearer token'
+    required: true
+  dkg-base-url:
+    description: 'DKG v10 node base URL (e.g. http://my-node:9200)'
+    required: true
+  dkg-context-graph:
+    description: 'Context Graph ID to write into'
+    required: true
+  github-token:
+    description: 'GitHub token for API access'
+    required: false
+    default: ${{ github.token }}
+  layer:
+    description: 'Memory layer — "wm" (Working Memory, private) or "swm" (Shared Working Memory, gossiped)'
+    required: false
+    default: 'wm'
+  event-type:
+    description: 'Which GitHub event triggered this action (issues, pull_request, pull_request_review). Auto-detected from GITHUB_EVENT_NAME if omitted.'
+    required: false
+    default: ''
+  item-number:
+    description: 'Issue or PR number to ingest. Auto-detected from the event payload if omitted.'
+    required: false
+    default: ''
+
+outputs:
+  turn-uri:
+    description: 'UAL of the Knowledge Asset written to Working Memory'
+    value: ${{ steps.ingest.outputs.turn-uri }}
+  layer:
+    description: 'Memory layer the asset was written to'
+    value: ${{ steps.ingest.outputs.layer }}
+
+runs:
+  using: 'docker'
+  image: 'Dockerfile'
+  env:
+    DKG_TOKEN: ${{ inputs.dkg-token }}
+    DKG_BASE_URL: ${{ inputs.dkg-base-url }}
+    DKG_CONTEXT_GRAPH: ${{ inputs.dkg-context-graph }}
+    GITHUB_TOKEN: ${{ inputs.github-token }}
+    INPUT_LAYER: ${{ inputs.layer }}
+    INPUT_EVENT_TYPE: ${{ inputs.event-type }}
+    INPUT_ITEM_NUMBER: ${{ inputs.item-number }}
+    GITHUB_REPOSITORY: ${{ github.repository }}
+    GITHUB_EVENT_NAME: ${{ github.event_name }}
+    GITHUB_EVENT_PATH: ${{ github.event_path }}