orga 0.1.0__tar.gz

orga-0.1.0/.gitignore

@@ -0,0 +1,161 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducible builds.
+#   However, if you need to support different python versions or platforms, you might want to ignore it.
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and others
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# Editor directories and files
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Project specific
+GEMINI.md
+设计书.org

orga-0.1.0/Dockerfile

@@ -0,0 +1,28 @@
+# Use an official Python runtime as a parent image
+FROM python:3.12-slim
+
+# Set the working directory in the container
+WORKDIR /app
+
+# Install system dependencies if any (none for now)
+# RUN apt-get update && apt-get install -y ...
+
+# Copy the current directory contents into the container at /app
+COPY . /app
+
+# Install build tools and dependencies
+# We use pip to install the package in editable mode or just requirements
+# Since we have pyproject.toml, we can install directly
+RUN pip install --no-cache-dir --upgrade pip && \
+    pip install --no-cache-dir . && \
+    pip install --no-cache-dir "fastapi" "uvicorn"
+
+# Expose ports (documentary only)
+EXPOSE 8000
+EXPOSE 8001
+
+# Define environment variables
+ENV PYTHONUNBUFFERED=1
+
+# Default entrypoint (can be overridden)
+CMD ["uvicorn", "examples.extractor_service.main:app", "--host", "0.0.0.0", "--port", "8000"]

orga-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 KEJIA <DETONG.KEJI@GMAIL.COM>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

orga-0.1.0/PKG-INFO

@@ -0,0 +1,187 @@
+Metadata-Version: 2.4
+Name: orga
+Version: 0.1.0
+Summary: A modular, strategy-driven organization profile extractor.
+Author-email: Xiang Dao <xiangdao@example.com>
+License: MIT License
+        
+        Copyright (c) 2026 KEJIA <DETONG.KEJI@GMAIL.COM>
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Requires-Dist: aiolimiter
+Requires-Dist: email-validator
+Requires-Dist: extruct
+Requires-Dist: hishel
+Requires-Dist: httpx
+Requires-Dist: opentelemetry-api
+Requires-Dist: phonenumbers
+Requires-Dist: pydantic>=2.0
+Requires-Dist: pyyaml
+Requires-Dist: rapidfuzz
+Requires-Dist: selectolax
+Requires-Dist: structlog
+Requires-Dist: tenacity
+Requires-Dist: typer
+Provides-Extra: browser
+Requires-Dist: playwright; extra == 'browser'
+Provides-Extra: dev
+Requires-Dist: black; extra == 'dev'
+Requires-Dist: mypy; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: pytest-asyncio; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Provides-Extra: llm
+Requires-Dist: langextract; extra == 'llm'
+Provides-Extra: server
+Requires-Dist: fastapi; extra == 'server'
+Requires-Dist: uvicorn; extra == 'server'
+Description-Content-Type: text/markdown
+
+# ORGA: Deterministic Organization Profiler
+
+**A fast, explainable, non-LLM extraction engine for profiling institutional websites.**
+
+---
+
+## 🎯 What is ORGA?
+
+ORGA is a Python-based profiling engine and microservice suite designed to autonomously navigate an organization's website and extract a highly structured JSON profile. It discovers global locations, extracts clean contact data (phones, emails, social footprints), and determines the organization's primary industry category.
+
+**Crucially, ORGA is not an LLM.** It is built entirely on deterministic rules, semantic heuristics, JSON-LD parsing, and lightweight statistical Bayesian models.
+
+## ⚡ Output Snapshot
+
+*A minimal example of the structured JSON output generated for a hospital website:*
+
+```json
+{
+  "name": "CHEO",
+  "org_type": "Hospital",
+  "categories": ["Hospital", "NonProfit"],
+  "locations": [
+    {
+      "address": {
+        "raw": "401 Smyth Road, Ottawa ON K1H 8L1",
+        "postal_code": "K1H 8L1",
+        "city": "Ottawa"
+      },
+      "confidence": 0.9
+    }
+  ],
+  "phones": [
+    { "value": "+16137377600", "kind": "phone" }
+  ],
+  "social_links": [
+    { "value": "https://facebook.com/cheokids", "kind": "social" }
+  ]
+}
+```
+
+## 🧠 Why No LLMs?
+
+In an era of generative AI, why build a deterministic extractor?
+
+1. **Extreme Speed & Cost Efficiency:** ORGA processes a full organization (navigating up to 5 sub-pages like `/about` or `/contact`) in **under 0.7 seconds** per site. It requires negligible CPU/Memory overhead, allowing you to process 10,000 organizations for pennies rather than dollars.
+2. **100% Explainability:** Every extracted phone number, every inferred category (e.g., `Hospital` vs. `University`), and every discarded link is fully traceable. The JSON payload includes a `debug_info` block detailing the exact CSS selector, regex match, or weighted rule path that produced the result.
+3. **No Hallucinations:** When ORGA fails, it fails predictably (e.g., returning an empty field). It will never invent a phone number or confidently hallucinate an office address.
+
+## ✨ Core Features
+
+* **Intelligent Discovery:** Automatically finds high-value pages (`/contact`, `/locations`, `/about`) from a root URL.
+* **Aggressive Noise Filtering:** Employs suppression matrices and page-weighting to strip out UI navigation noise and generic boilerplate text.
+* **Layered Classification:** Identifies primary institutional types (e.g., `Government`, `Hospital`, `NonProfit`, `InternationalOrg`) using a two-tier weighted keyword and Bayesian frequency model.
+* **Concurrent Microservices:** Includes two Dockerized FastAPI services for real-time single-URL extraction and asynchronous batch processing.
+
+## 🛑 Known Boundaries & Limitations
+
+ORGA operates at the absolute ceiling of what rule-based extraction can achieve. You should understand its limits:
+
+* **Good Fit:** Generating a massive directory of structured contacts and primary categories for standard institutional sites (Hospitals, Universities, NGOs, Government Agencies).
+* **Poor Fit:** Open-world semantic reading tasks, analyzing deep PDF reports, or distinguishing nuanced corporate hierarchies (e.g., distinguishing a holding company from its subsidiary if both use identical website templates).
+* **Address Parsing:** While highly resilient, extracting perfect Street/City/Region splits from unstructured, conversational footers without NLP remains challenging and will occasionally result in `partially_parsed` raw strings.
+
+---
+
+## 🚀 Quickstart
+
+### 1. Run the Microservices via Docker
+
+Ensure you have Docker and Docker Compose installed.
+
+```bash
+git clone https://github.com/discretewater/orga.git
+cd orga
+
+# Start the Extractor (8000) and Job Manager (8001) services
+docker compose up --build -d
+```
+
+### 2. Demo: Single Extraction
+
+Extract a profile for the World Health Organization:
+
+```bash
+curl -X POST "http://127.0.0.1:8000/extract" \
+     -H "Content-Type: application/json" \
+     -d '{"url": "https://www.who.int"}' | jq .
+```
+
+*You will receive a rich JSON profile containing the WHO's global contact points, social links, and a primary classification of `InternationalOrg`.*
+
+### 3. Demo: Batch Processing
+
+Submit multiple URLs to the async Job Manager:
+
+```bash
+# 1. Submit the Job
+curl -s -X POST "http://127.0.0.1:8001/jobs" \
+     -H "Content-Type: application/json" \
+     -d '{"urls": ["https://www.harvard.edu", "https://www.cheo.on.ca"]}'
+
+# Expected output: {"job_id": "uuid-...", "status": "pending"}
+
+# 2. Poll for Results (Replace UUID with the one from the previous step)
+curl -s "http://127.0.0.1:8001/jobs/{job_id}" | jq .
+```
+
+---
+
+## 🏗️ Architecture Summary
+
+1. **Fetcher:** Utilizes `httpx` and `aiolimiter` to aggressively fetch HTML while respecting concurrency limits.
+2. **Discoverer:** Heuristically scores anchor links to branch out from the root domain into contact/about pages.
+3. **Parsers:** `selectolax`-powered extraction targeting DOM zones, JSON-LD schemas, and normalized regex patterns.
+4. **Classifier:** A tiered engine scoring terms across zones (`<title>`, `<h1>`, `<body>`) against a weighted taxonomy.
+5. **Aggregator:** An institution-level decider that weights page evidence, applies suppression rules (e.g., a strong "Hospital" signal suppresses weak "Association" noise), and yields the final profile.
+
+## 🔮 Roadmap
+
+ORGA M7.1 is currently in a frozen baseline state.
+
+Future enhancements will explore adding a **Lightweight Supervised Post-Calibration Model** (e.g., XGBoost over debug scores) to refine category boundaries without sacrificing the speed and determinism of the core extraction layer. We will not be migrating to an LLM-first architecture.

orga-0.1.0/README.md

@@ -0,0 +1,122 @@
+# ORGA: Deterministic Organization Profiler
+
+**A fast, explainable, non-LLM extraction engine for profiling institutional websites.**
+
+---
+
+## 🎯 What is ORGA?
+
+ORGA is a Python-based profiling engine and microservice suite designed to autonomously navigate an organization's website and extract a highly structured JSON profile. It discovers global locations, extracts clean contact data (phones, emails, social footprints), and determines the organization's primary industry category.
+
+**Crucially, ORGA is not an LLM.** It is built entirely on deterministic rules, semantic heuristics, JSON-LD parsing, and lightweight statistical Bayesian models.
+
+## ⚡ Output Snapshot
+
+*A minimal example of the structured JSON output generated for a hospital website:*
+
+```json
+{
+  "name": "CHEO",
+  "org_type": "Hospital",
+  "categories": ["Hospital", "NonProfit"],
+  "locations": [
+    {
+      "address": {
+        "raw": "401 Smyth Road, Ottawa ON K1H 8L1",
+        "postal_code": "K1H 8L1",
+        "city": "Ottawa"
+      },
+      "confidence": 0.9
+    }
+  ],
+  "phones": [
+    { "value": "+16137377600", "kind": "phone" }
+  ],
+  "social_links": [
+    { "value": "https://facebook.com/cheokids", "kind": "social" }
+  ]
+}
+```
+
+## 🧠 Why No LLMs?
+
+In an era of generative AI, why build a deterministic extractor?
+
+1. **Extreme Speed & Cost Efficiency:** ORGA processes a full organization (navigating up to 5 sub-pages like `/about` or `/contact`) in **under 0.7 seconds** per site. It requires negligible CPU/Memory overhead, allowing you to process 10,000 organizations for pennies rather than dollars.
+2. **100% Explainability:** Every extracted phone number, every inferred category (e.g., `Hospital` vs. `University`), and every discarded link is fully traceable. The JSON payload includes a `debug_info` block detailing the exact CSS selector, regex match, or weighted rule path that produced the result.
+3. **No Hallucinations:** When ORGA fails, it fails predictably (e.g., returning an empty field). It will never invent a phone number or confidently hallucinate an office address.
+
+## ✨ Core Features
+
+* **Intelligent Discovery:** Automatically finds high-value pages (`/contact`, `/locations`, `/about`) from a root URL.
+* **Aggressive Noise Filtering:** Employs suppression matrices and page-weighting to strip out UI navigation noise and generic boilerplate text.
+* **Layered Classification:** Identifies primary institutional types (e.g., `Government`, `Hospital`, `NonProfit`, `InternationalOrg`) using a two-tier weighted keyword and Bayesian frequency model.
+* **Concurrent Microservices:** Includes two Dockerized FastAPI services for real-time single-URL extraction and asynchronous batch processing.
+
+## 🛑 Known Boundaries & Limitations
+
+ORGA operates at the absolute ceiling of what rule-based extraction can achieve. You should understand its limits:
+
+* **Good Fit:** Generating a massive directory of structured contacts and primary categories for standard institutional sites (Hospitals, Universities, NGOs, Government Agencies).
+* **Poor Fit:** Open-world semantic reading tasks, analyzing deep PDF reports, or distinguishing nuanced corporate hierarchies (e.g., distinguishing a holding company from its subsidiary if both use identical website templates).
+* **Address Parsing:** While highly resilient, extracting perfect Street/City/Region splits from unstructured, conversational footers without NLP remains challenging and will occasionally result in `partially_parsed` raw strings.
+
+---
+
+## 🚀 Quickstart
+
+### 1. Run the Microservices via Docker
+
+Ensure you have Docker and Docker Compose installed.
+
+```bash
+git clone https://github.com/discretewater/orga.git
+cd orga
+
+# Start the Extractor (8000) and Job Manager (8001) services
+docker compose up --build -d
+```
+
+### 2. Demo: Single Extraction
+
+Extract a profile for the World Health Organization:
+
+```bash
+curl -X POST "http://127.0.0.1:8000/extract" \
+     -H "Content-Type: application/json" \
+     -d '{"url": "https://www.who.int"}' | jq .
+```
+
+*You will receive a rich JSON profile containing the WHO's global contact points, social links, and a primary classification of `InternationalOrg`.*
+
+### 3. Demo: Batch Processing
+
+Submit multiple URLs to the async Job Manager:
+
+```bash
+# 1. Submit the Job
+curl -s -X POST "http://127.0.0.1:8001/jobs" \
+     -H "Content-Type: application/json" \
+     -d '{"urls": ["https://www.harvard.edu", "https://www.cheo.on.ca"]}'
+
+# Expected output: {"job_id": "uuid-...", "status": "pending"}
+
+# 2. Poll for Results (Replace UUID with the one from the previous step)
+curl -s "http://127.0.0.1:8001/jobs/{job_id}" | jq .
+```
+
+---
+
+## 🏗️ Architecture Summary
+
+1. **Fetcher:** Utilizes `httpx` and `aiolimiter` to aggressively fetch HTML while respecting concurrency limits.
+2. **Discoverer:** Heuristically scores anchor links to branch out from the root domain into contact/about pages.
+3. **Parsers:** `selectolax`-powered extraction targeting DOM zones, JSON-LD schemas, and normalized regex patterns.
+4. **Classifier:** A tiered engine scoring terms across zones (`<title>`, `<h1>`, `<body>`) against a weighted taxonomy.
+5. **Aggregator:** An institution-level decider that weights page evidence, applies suppression rules (e.g., a strong "Hospital" signal suppresses weak "Association" noise), and yields the final profile.
+
+## 🔮 Roadmap
+
+ORGA M7.1 is currently in a frozen baseline state.
+
+Future enhancements will explore adding a **Lightweight Supervised Post-Calibration Model** (e.g., XGBoost over debug scores) to refine category boundaries without sacrificing the speed and determinism of the core extraction layer. We will not be migrating to an LLM-first architecture.

orga-0.1.0/design/address-quality-notes.org

@@ -0,0 +1,30 @@
+* Address Quality Analysis and Limited Polish Strategy
+
+** 1. Current State Assessment
+
+The ORGA address parser (M7.1) is resilient but imprecise. It successfully identifies blocks of text containing address-like signals (Postal Codes, Street suffixes) but struggles to delineate the exact boundaries of that block within noisy HTML footers.
+
+*** Common Issues:
+- **Bleeding:** The address string often includes preceding navigation headers ("Contact Us", "Locations") or succeeding contact info ("Phone: 555-0199").
+- **Fragmentation:** Multi-line addresses in `<div>` soup are sometimes split into separate entities.
+- **False Positives:** ISO numbers or copyright dates resembling postal codes.
+
+** 2. The "Limited Polish" Strategy
+
+Per M8.1 directives, we are NOT rewriting the parser. We are applying **generic termination signals**.
+
+*** Improved Termination List:
+We define a set of "Hard Stops". If the parser encounters these tokens, it assumes the address block has ended.
+
+- *Contact Labels:* "Tel:", "Phone:", "Fax:", "Email:", "Call:"
+- *Social Media:* "Follow us", "Twitter", "Facebook", "Connect"
+- *Legal:* "Copyright", "Rights Reserved", "Privacy Policy"
+- *Navigation:* "Menu", "Home", "About", "Services"
+
+** 3. Remaining (Acceptable) Artifacts
+
+Even with these stops, some noise is inevitable.
+- *Inline Descriptions:* "Main Entrance at 123 Main St." -> "Main Entrance at" is hard to separate without NLP.
+- *PO Box Variations:* Highly variable formats for P.O. Boxes may be partially captured.
+
+These are considered **Acceptable Variances** for a heuristic system.

orga-0.1.0/design/architecture-overview.org

@@ -0,0 +1,44 @@
+* ORGA Architecture Overview
+
+** 1. System Context
+
+ORGA is a standalone extraction engine. It takes a URL (or list of URLs) as input and outputs a structured JSON profile. It can run as a CLI tool or as a set of Dockerized microservices.
+
+** 2. Core Components
+
+*** 1. The Fetcher (Network Layer)
+- **Tech:** `httpx`, `aiolimiter`
+- **Role:** Handles HTTP requests, retries, user-agent rotation, and concurrency limiting.
+- **Key Feature:** "Per-host" throttling to prevent banning.
+
+*** 2. The Discoverer (Navigation Layer)
+- **Tech:** `selectolax`
+- **Role:** Analyzes the landing page to find "high-value" sub-pages (`/contact`, `/about`, `/locations`).
+- **Logic:** Uses weighted keyword heuristics on anchor tags to prioritize navigation.
+
+*** 3. The Parser (Extraction Layer)
+- **Tech:** `selectolax`, `phonenumbers`, `email-validator`
+- **Role:** Extracts raw entities from the DOM.
+- **Strategies:**
+  - *JSON-LD:* Extracts structured schema.org data (High confidence).
+  - *Regex:* Scans text for phones/emails (Medium confidence).
+  - *Heuristic DOM:* Scans footers/contact blocks for addresses.
+
+*** 4. The Classifier (Intelligence Layer)
+- **Tech:** Custom Weighted Rule Engine + Bayesian Fallback
+- **Role:** Determines the organization type (e.g., Hospital, University).
+- **Process:**
+  1. *Page Scoring:* Scores each page against a taxonomy (Tier 1 Rules).
+  2. *Aggregation:* `ClassificationAggregator` combines scores, applying weights (About > News).
+  3. *Suppression:* High-confidence tags (Hospital) suppress low-confidence noise (Association).
+
+*** 5. The Merger (Governance Layer)
+- **Role:** The final gatekeeper.
+- **Tasks:** Deduplicates contacts, normalizes phone numbers (E.164), and filters out junk (e.g., social links to "twitter.com/share").
+
+** 3. Microservice Design
+
+The system is split into two containers for scalability:
+
+- **Extractor Service:** Stateless. Handles single requests. Good for scaling horizontally.
+- **Job Service:** Stateful (in-memory or Redis). Manages long-running batches and tracks progress.

orga-0.1.0/design/demo-walkthrough.org

@@ -0,0 +1,75 @@
+* ORGA Demo Walkthrough
+
+This guide provides a step-by-step walkthrough for demonstrating ORGA to stakeholders or running it as a proof of concept.
+
+** 1. Setup (5 Minutes)
+
+**Prerequisites:** Docker and Docker Compose.
+
+1. **Clone and Enter:**
+   #+BEGIN_SRC bash
+   git clone https://github.com/discretewater/orga.git
+   cd orga
+   #+END_SRC
+
+2. **Launch Services:**
+   #+BEGIN_SRC bash
+   docker compose up --build -d
+   #+END_SRC
+
+3. **Verify Health:**
+   #+BEGIN_SRC bash
+   curl localhost:8000/health
+   # Returns: {"status":"ok"}
+   #+END_SRC
+
+** 2. Scenario A: The "Real-Time" Lookup
+
+*Goal:* Show how fast ORGA profiles a single site.
+
+**Action:**
+Run this command to profile the WHO:
+#+BEGIN_SRC bash
+curl -X POST "http://127.0.0.1:8000/extract" \
+     -H "Content-Type: application/json" \
+     -d '{"url": "https://www.who.int"}' | jq .
+#+END_SRC
+
+**Talking Points:**
+- Point out the `org_type`: "InternationalOrg".
+- Show the `social_links`: Clean, validated URLs.
+- Highlight speed: "This took < 1 second."
+
+** 3. Scenario B: The "Batch Ingestion"
+
+*Goal:* Demonstrate robustness and async processing.
+
+**Action:**
+1. Submit a job with a mix of site types:
+   #+BEGIN_SRC bash
+   curl -X POST "http://127.0.0.1:8001/jobs" \
+     -H "Content-Type: application/json" \
+     -d '{ "urls": [
+           "https://www.harvard.edu",
+           "https://www.cheo.on.ca",
+           "https://www.greenpeace.org"
+         ]}'
+   #+END_SRC
+   *Copy the `job_id` returned.*
+
+2. Poll for results:
+   #+BEGIN_SRC bash
+   curl "http://127.0.0.1:8001/jobs/<YOUR_JOB_ID>" | jq .
+   #+END_SRC
+
+**Talking Points:**
+- ORGA handles concurrency automatically.
+- It respects rate limits per host.
+- It survives blocked pages without crashing the batch.
+
+** 4. Interpreting the "Black Box" (Debug Info)
+
+In the JSON output, scroll down to `debug_info`.
+- Show `classification_debug`.
+- Explain how ORGA weighed the `/about` page higher than the `/news` page to determine the category.
+- This proves the system is **deterministic**, not a "black box" AI.