@ltm-blueverse/alpha-semantic-hub 0.7.1 → 0.7.3

package/README.md

@@ -18,6 +18,8 @@
 
 ---
 
+> **Install guide:** see [INSTALL.md](INSTALL.md) for step-by-step install instructions for every platform.
+
 Alpha Semantic Hub is a graph-native context platform rebuilt end-to-end under the
 **LTM Blueverse** identity: red brand system, futuristic "Alpha" logo (a red **A**
 inside a red circle), a redesigned futuristic Studio UI/UX, scoped npm packages,
@@ -44,13 +46,13 @@ The Python distribution (`alphasemantichub`; CLI: `alphasemantichub`,
 
 ```bash
 # latest release wheel
-pip install https://github.com/DeejayAI/alpha-semantic-hub/releases/latest/download/alphasemantichub-0.7.0-py3-none-any.whl
+pip install https://github.com/DeejayAI/alpha-semantic-hub/releases/latest/download/alphasemantichub-0.7.1-py3-none-any.whl
 
 # or from a specific release tag
-pip install https://github.com/DeejayAI/alpha-semantic-hub/releases/download/v0.7.0/alphasemantichub-0.7.0-py3-none-any.whl
+pip install https://github.com/DeejayAI/alpha-semantic-hub/releases/download/v0.7.1/alphasemantichub-0.7.1-py3-none-any.whl
 
 # with optional extras
-pip install "https://github.com/DeejayAI/alpha-semantic-hub/releases/latest/download/alphasemantichub-0.7.0-py3-none-any.whl[explorer]"
+pip install "https://github.com/DeejayAI/alpha-semantic-hub/releases/latest/download/alphasemantichub-0.7.1-py3-none-any.whl[explorer]"
 ```
 
 Verify:
@@ -105,7 +107,7 @@ Environment variables (all prefixed `ASHUB_`):
 |---|---|
 | `ASHUB_API_KEY` | API key for protected Studio routes (generate: `openssl rand -hex 32`) |
 | `ASHUB_ALLOW_ANONYMOUS` | `true` bypasses the API key (local-only setups) |
-| `ASHUB_KG_PATH` | Knowledge-graph storage path |
+| `ASHUB_KG_PATH` | Graph persistence file — auto-loaded at startup, auto-saved on every import (set to `/data/graph.json` in the compose stack) |
 | `ASHUB_LOG_LEVEL` | Logging verbosity |
 | `ASHUB_CORS_ORIGINS` | Allowed CORS origins |
 
@@ -117,6 +119,25 @@ The repo ships ready-to-install plugins under `plugins/`. Install the npm metapa
 (`npm install @ltm-blueverse/alpha-semantic-hub`) or point your tool at the checked-out
 `plugins/` directory.
 
+### How the harness integration works
+
+Every integration follows the same shape:
+
+1. The harness (Claude Code, Copilot, Codex, …) launches the **MCP server as a
+   child process** (`python -m alphasemantichub.mcp_server`, stdio transport)
+2. The plugin manifest tells the harness which **skills** (markdown playbooks in
+   `plugins/skills/`) and **agents** (persona definitions in `plugins/agents/`)
+   to load alongside its native tools
+3. The agent can then call knowledge-graph tools (`extract_entities`,
+   `record_decision`, `find_precedents`, `query_graph`, …) directly from its
+   chat/tool loop — reading from and writing to the same graph the Studio shows
+4. Optional **hooks** (`plugins/hooks/hooks.json`) run small guards on
+   Write/Edit/Bash events (e.g. Python syntax checks on graph files)
+
+Because the transport is stdio JSON-RPC, there is no server to manage: install
+the Python package in the interpreter the harness uses, register the command,
+and the tools appear.
+
 ### Claude (Claude Code / Claude Desktop)
 
 ```bash
@@ -173,12 +194,13 @@ Each has a ready manifest: `plugins/.windsurf-plugin`, `plugins/.cline-plugin`,
 
 ---
 
-## 5 · The Studio UI (red / futuristic redesign)
+## 5 · The Studio UI
 
-The bundled Knowledge Explorer has been fully rethemed:
+The bundled Knowledge Explorer ships in the Alpha red brand system with a light,
+readable theme:
 
-- **Alpha red** accent system (`#E11D2E` → `#FF4B55` gradients) on near-black HUD surfaces
-- Sharp futuristic geometry (10px/6px radii), HUD grid texture, red glow accents
+- **White background with near-black text** and light red highlights/captions
+- **Alpha red** accent system (`#E11D2E` → `#FF4B55` gradients), sharp futuristic geometry
 - Red **A-in-circle** brand mark in the navigation rail, favicon and app icons
 - All workspaces (Graph, Analyze, Decisions, Enrich, Manage, Ontology Hub) restyled
 
@@ -188,17 +210,159 @@ Serve it three ways:
 2. **Docker:** `docker compose up -d` → http://localhost:8000
 3. **Static:** grab `@ltm-blueverse/alpha-hub-studio` from npm and serve `static/`
 
-## 6 · MCP server
+## 6 · How the MCP server works
+
+Alpha Semantic Hub exposes its knowledge graph to AI agents over the
+**Model Context Protocol (MCP)**. The server speaks JSON-RPC 2.0 over **stdio** —
+an agent harness launches it as a child process and exchanges newline-delimited
+JSON-RPC messages (`initialize`, `tools/list`, `tools/call`, `resources/list`,
+`resources/read`). No network port is opened, so it works identically in Claude
+Code, VS Code, Codex, Cursor, or any MCP-capable client.
+
+Two builds are included:
+
+| Server | Launch | Tools |
+|---|---|---|
+| **Installed server** (`alphasemantichub.mcp_server`) | `alphasemantichub-mcp` or `python -m alphasemantichub.mcp_server` | 16 tools |
+| **Extended in-repo server** (`alphasemantichub_mcp.mcp`) | `python -m alphasemantichub_mcp.mcp` | 22 tools (adds retrieval, provenance, impact analysis) |
+
+What the tools cover:
+
+- **Extraction** — `extract_entities` (NER over text), `extract_relations`
+  (relations + subject-predicate-object triplets)
+- **Decision intelligence** — `record_decision`, `query_decisions`,
+  `find_precedents` (similarity search over past decisions), `get_causal_chain`
+  (trace what a decision influenced, upstream or downstream), `link_decisions`
+  (typed causal edges: CAUSED / INFLUENCED / PRECEDENT_FOR), `analyze_decision_impact`
+- **Graph** — `add_entity`, `add_relationship`, `query_graph` (node lookup,
+  1–5-hop neighborhoods, keyword search), `get_graph_analytics` (PageRank,
+  community detection), `get_graph_summary`
+- **Reasoning** — `run_reasoning` (forward-chaining IF/THEN rules over facts),
+  `abductive_reasoning` (hypotheses that explain observed facts)
+- **Persistence & I/O** — `export_graph` (Turtle, JSON-LD, RDF/XML, GraphML,
+  CSV, Parquet, N-Triples), `update_node`, `delete_node` (soft archive)
+- **Retrieval (extended server)** — `store_document` (chunk + embed),
+  `retrieve_context` (semantic search + related graph relationships)
+
+State: point `ASHUB_KG_PATH` at a graph file and every mutation (decisions,
+entities, relationships) is persisted and reloaded across sessions — the graph
+your agent edits is the graph the Studio renders.
+
+## 7 · How the ontology is built
+
+Ontologies are produced by `OntologyEngine`, which composes a six-stage
+generator pipeline:
+
+1. **Parse** — normalize `{"entities": [...], "relationships": [...]}` produced
+   by the semantic-extraction pipeline (`NERExtractor`, `RelationExtractor`,
+   `TripletExtractor`)
+2. **Define** — map concepts to candidate class definitions
+3. **Type & property inference** — `PropertyGenerator.infer_properties` stamps
+   classes/properties with URIs from `NamespaceManager`
+   (base namespace `https://alphahub.ltmb.io/ontology/`)
+4. **Hierarchy** — `ClassInferrer.build_class_hierarchy` assembles the class
+   tree (frequency-gated: concepts need `min_occurrences=2` to survive)
+5. **Serialize** — `OWLGenerator` emits Turtle / RDF-XML / JSON-LD / N3
+6. **Validate** — `OntologyValidator` checks consistency and satisfiability;
+   results land in `ontology["validation"]`
+
+Then:
+
+- **SHACL shapes** are derived with `OntologyEngine.to_shacl(...)` (quality
+  tiers basic/standard/strict) and enforced at runtime with
+  `validate_graph(...)` (pyshacl) — reports come back as plain-English
+  violations
+- **Quality gate** — `quality_check(...)` returns a graded report of issues
+- **Versioning** — `VersionManager.create_version` gives each ontology a
+  versioned IRI, and `compare_versions` / `diff_ontologies` produce change
+  reports between versions
+- **LLM-assisted drafting** — `OntologyEngine.from_text(...)` drafts an ontology
+  from unstructured text; the result goes through the same validation and
+  versioning
+
+Resulting dict shape:
+
+```python
+{
+  "uri": "https://alphahub.ltmb.io/ontology/v1.0/",
+  "name": "...", "version": "1.0",
+  "classes": [{"name", "uri", "label", "comment", "properties", "entity_count"}],
+  "properties": [...],
+  "validation": {"valid": true, "consistent": true, "satisfiable": true, ...}
+}
+```
+
+## 8 · Connecting source systems
+
+The `ingest` layer feeds the ontology and knowledge graph from external systems.
+Every connector is lazy-loaded and degrades to a helpful install hint when its
+SDK is missing. Two patterns:
+
+- a **Connector** (auth + transport) paired with an **Ingestor** (extraction +
+  `export_as_documents`)
+- a unified dispatcher: `ingest(sources, source_type="db", method=...)`
+
+| Source system | Ingestor | Connection | Primary call |
+|---|---|---|---|
+| **Databricks** | `DatabricksIngestor` | `host`, `token`, `http_path`, `catalog`, `schema` (env: `DATABRICKS_HOST`, `DATABRICKS_TOKEN`, `DATABRICKS_HTTP_PATH`) | `ingest_table("silver_customers")`, `ingest_query("SELECT ...")` |
+| **SAP** (OData) | `SAPIngestor` | `base_url`, `auth="oauth2"|"basic"`, `token_url`, `client_id`, `client_secret` | `discover_service()`, `ingest_entity_set(service, entity_set, top=1000)` |
+| **Snowflake** | `SnowflakeIngestor` | `account`, `user`, `password`/`private_key`, `warehouse`, `database`, `schema`, `role` | `ingest_table(...)`, `ingest_query(...)` |
+| **Salesforce** | `SalesforceIngestor` | `username`, `password`, `security_token` (env: `SALESFORCE_USERNAME`, …) | `ingest_sobject("Account", fields=[...])`, `ingest_query(soql)` |
+| **ServiceNow** | `ServiceNowIngestor` | `instance_url`, `auth`, `username`/`client_id` + secret | `ingest_table(table="incident", query=..., limit=...)` |
+| **Power BI** | `PowerBIIngestor` | `tenant_id`, `client_id`, `client_secret`, `workspace_id` | `ingest_workspace_metadata()` |
+| **Tableau** | `TableauIngestor` | `server_url`, `site_name`, `token_name`, `token_value` | `ingest_workbooks()`, `ingest_datasources()` |
+| **Looker** | `LookerIngestor` | `base_url`, `client_id`, `client_secret` (or `looker.ini`) | `ingest_looks()`, `ingest_dashboards()`, `ingest_lookml_models()` |
+| **BigQuery** | `BigQueryIngestor` | `project`, `dataset`, `credentials_file` | `ingest_table(...)`, `ingest_query(...)` |
+| **Web / intranet** | `WebIngestor` | `user_agent`, `respect_robots=True` | `ingest_url(url)`, `crawl_sitemap(...)`, `crawl_domain(...)` |
+| **Files & cloud storage** | `FileIngestor`, `CloudStorageIngestor` | provider config for S3 / GCS / Azure Blob | `ingest_file`, `ingest_directory`, `ingest_cloud` |
+| **Git repositories** | `RepoIngestor` | repo URL | `ingest_repository(url)` + commit/code analysis |
+| **REST APIs** | `RESTIngestor` | `config={"headers": {...}}` | `ingest_endpoint(...)`, `paginated_fetch(...)` |
+| **Other MCP servers** | `MCPIngestor` | `connect(server_name, url=...)` | `ingest_resources(...)`, `ingest_tool_output(...)` |
+
+Also available: Redshift, Cassandra, MongoDB, DuckDB, Elasticsearch, Kafka-style
+streams, email, RSS/Atom feeds, Airflow, Parquet/Arrow.
+
+### Running ingestion (Python workflow)
+
+Source ingestion is a **Python-library workflow** — the Studio UI imports graph
+files (JSON/CSV), it does not call source systems directly. The bridge is
+`examples/ingest_to_studio.py`:
 
 ```bash
-pip install alphasemantichub
-alphasemantichub-mcp          # stdio MCP server
+# any text/document file — no credentials needed:
+uv run python examples/ingest_to_studio.py ./doc.txt --kind file --import
+
+# source systems (credentials via env vars, e.g. DATABRICKS_HOST/TOKEN/HTTP_PATH):
+uv run python examples/ingest_to_studio.py silver_customers --kind databricks --import
+uv run python examples/ingest_to_studio.py SalesOrder --kind sap --import
+uv run python examples/ingest_to_studio.py CUSTOMERS --kind snowflake --import
+uv run python examples/ingest_to_studio.py Account --kind salesforce --import
+uv run python examples/ingest_to_studio.py https://api.example.com/items --kind rest --import
+```
+
+What it does: ingest → `export_as_documents` → extract entities + triplets
+(NER/pattern) → build a `ContextGraph` → save JSON → `--import` POSTs it to the
+running Studio (multipart `/api/import`), where it appears in the Graph view.
+
+### A note on SharePoint
+
+There is no dedicated SharePoint connector in this repository. The recommended
+path today is **Microsoft Graph via `RESTIngestor`**:
+
+```python
+from alphasemantichub.ingest import RESTIngestor
+
+ing = RESTIngestor(config={"headers": {"Authorization": f"Bearer {token}"}})
+for page in ing.paginated_fetch("https://graph.microsoft.com/v1.0/sites/{site-id}/drive/root/children"):
+    ...
 ```
 
-Tools: graph queries, semantic extraction, reasoning, decisions, provenance,
-export/import, ontology management.
+…or `WebIngestor.ingest_url(...)` for pages, or `CloudStorageIngestor` /
+`FileIngestor` for documents synced out of SharePoint document libraries. A
+dedicated connector can be added by registering a new method in
+`ingest/methods.py`.
 
-## 7 · Development
+## 9 · Development
 
 ```bash
 git clone https://github.com/DeejayAI/alpha-semantic-hub.git
@@ -230,7 +394,7 @@ All environment variables use the **`ASHUB_`** prefix:
 `ASHUB_DISABLE_PROGRESS`, `ASHUB_LOG_LEVEL`, and more — grep the source for
 `ASHUB_` to see the full list.
 
-## 8 · Identity summary
+## 10 · Identity summary
 
 | Token | Value |
 |---|---|

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ltm-blueverse/alpha-semantic-hub",
-  "version": "0.7.1",
+  "version": "0.7.3",
   "description": "Alpha Semantic Hub \u2014 Graph-Native Infrastructure for Context and Accountable AI Systems",
   "keywords": [
     "alpha-semantic-hub",