agent-os-kernel 1.1.0__py3-none-any.whl → 1.3.0__py3-none-any.whl

modules/caas/paper/CHECKLIST.md

@@ -0,0 +1,103 @@
+# Paper Submission Checklist
+
+Based on NeurIPS/ICML/ACL reproducibility requirements.
+
+## 📄 Paper Content
+
+### Abstract & Introduction
+- [ ] Clear problem statement
+- [ ] Concrete contributions (numbered list)
+- [ ] Scope and limitations stated upfront
+
+### Related Work
+- [ ] RAG systems comparison (LlamaIndex, LangChain, etc.)
+- [ ] Context management literature
+- [ ] Enterprise AI deployment challenges
+- [ ] Temporal/decay-based retrieval
+
+### Method
+- [ ] Architecture diagram
+- [ ] Algorithm pseudocode for key components
+- [ ] Complexity analysis
+- [ ] Design decisions justified
+
+### Experiments
+- [ ] Datasets described (ours + public)
+- [ ] Baselines clearly defined
+- [ ] Metrics explained
+- [ ] Statistical significance tests
+- [ ] Ablation studies for each component
+
+### Results
+- [ ] Tables with confidence intervals
+- [ ] Qualitative examples
+- [ ] Failure case analysis
+- [ ] Computational cost comparison
+
+## 🔬 Reproducibility
+
+### Code
+- [x] Code publicly available (GitHub)
+- [x] MIT License
+- [x] Installation instructions (pip install)
+- [x] Docker support
+- [ ] Exact dependency versions pinned
+- [ ] Random seeds documented
+
+### Data
+- [x] Dataset publicly available (Hugging Face)
+- [x] Dataset card with documentation
+- [ ] Data preprocessing scripts
+- [ ] Train/test splits specified
+
+### Experiments
+- [ ] Hyperparameters documented
+- [ ] Hardware requirements specified
+- [ ] Runtime estimates provided
+- [ ] Scripts to reproduce all tables/figures
+
+## 📊 Figures & Tables
+
+### Required Figures
+- [ ] Fig 1: System architecture overview
+- [ ] Fig 2: Context Triad visualization
+- [ ] Fig 3: Structure-aware indexing example
+- [ ] Fig 4: Benchmark results comparison
+
+### Required Tables
+- [ ] Table 1: Comparison with baselines
+- [ ] Table 2: Ablation study results
+- [ ] Table 3: Computational efficiency
+- [ ] Table 4: Dataset statistics
+
+## 📝 Writing Quality
+
+- [ ] Proofread for grammar/typos
+- [ ] Consistent terminology throughout
+- [ ] All acronyms defined on first use
+- [ ] References properly formatted
+- [ ] Page limit respected
+- [ ] Anonymous for review (if required)
+
+## 🤖 AI Disclosure
+
+- [x] LLM_DISCLOSURE.md prepared
+- [ ] Disclosure statement in paper (if required by venue)
+- [ ] AI-assisted sections documented
+
+## 📦 Supplementary Materials
+
+- [ ] Extended proofs (if any)
+- [ ] Additional experiments
+- [ ] Full hyperparameter tables
+- [ ] More qualitative examples
+- [ ] Video demo (optional)
+
+## 🚀 Final Submission
+
+- [ ] PDF compiles without errors
+- [ ] All figures render correctly
+- [ ] Supplementary ZIP prepared
+- [ ] Code repository cleaned
+- [ ] Dataset finalized
+- [ ] Co-author approvals obtained

modules/caas/paper/GITHUB_RELEASE_NOTES.md

@@ -0,0 +1,105 @@
+# GitHub Release v0.1.0 — Release Notes
+
+Copy and paste this content when creating the GitHub release.
+
+---
+
+## Release Title
+```
+v0.1.0 — First Public Release 🎉
+```
+
+## Release Body (Markdown)
+
+```markdown
+# Context-as-a-Service v0.1.0
+
+**🎉 First public release!** A principled framework for enterprise RAG systems that addresses seven critical fallacies in production deployments.
+
+## 🚀 Highlights
+
+- **28.1% improvement** in Precision@5 over flat-chunk baselines
+- **0.003ms routing latency** — 150,000× faster than LLM-based routing
+- **Five novel components**: Structure-Aware Indexing, Context Triad, Pragmatic Truth, Heuristic Router, Trust Gateway
+- **Enterprise-ready**: Docker support, comprehensive tests, security features
+
+## 📦 Installation
+
+```bash
+pip install context-as-a-service
+```
+
+Or with Docker:
+```bash
+docker pull ghcr.io/imran-siddique/context-as-a-service:latest
+```
+
+## 📊 Benchmark Results
+
+| Metric | Baseline | CaaS | Improvement |
+|--------|----------|------|-------------|
+| Precision@5 | 0.640 ± 0.057 | 0.820 ± 0.045 | **+28.1%** |
+| NDCG@10 | 0.610 ± 0.048 | 0.780 ± 0.042 | **+27.9%** |
+| Routing Latency | N/A | 0.003ms | ⚡ |
+
+*Statistical significance: p < 0.001, Cohen's d = 3.36 (large effect)*
+
+## 📚 Resources
+
+- **PyPI**: https://pypi.org/project/context-as-a-service/
+- **Hugging Face Dataset**: https://huggingface.co/datasets/imran-siddique/context-as-a-service
+- **Documentation**: See `/docs` folder
+- **Paper Draft**: See `/paper` folder
+
+## 🔧 Key Features
+
+### Structure-Aware Indexing
+Three-tier value hierarchy (High/Medium/Low) based on document structure.
+
+### Context Triad
+Hot/Warm/Cold prioritization with configurable token budgets:
+- Hot: 2,000 tokens (current conversation)
+- Warm: 1,000 tokens (user context)
+- Cold: 5,000 tokens (retrieved knowledge)
+
+### Pragmatic Truth
+Surfaces practical knowledge from informal sources alongside official documentation.
+
+### Heuristic Router
+Deterministic, zero-latency query routing without ML inference.
+
+### Trust Gateway
+Enterprise security layer with input validation, audit logging, and rate limiting.
+
+## 📝 What's Changed
+
+See [CHANGELOG.md](CHANGELOG.md) for full details.
+
+## 🙏 Acknowledgments
+
+Built with support from the open-source community. Special thanks to all contributors!
+
+---
+
+**Full Changelog**: https://github.com/imran-siddique/context-as-a-service/commits/v0.1.0
+```
+
+---
+
+## Instructions to Publish
+
+1. Go to: https://github.com/imran-siddique/context-as-a-service/releases/new
+2. **Choose a tag**: Select existing `v0.1.0`
+3. **Release title**: `v0.1.0 — First Public Release 🎉`
+4. **Description**: Paste the markdown content above
+5. **Attachments** (optional): 
+   - `paper/caas_paper_draft.pdf` (if compiled)
+6. Click **Publish release**
+
+## Verification Checklist
+
+- [ ] Tag v0.1.0 selected
+- [ ] Title includes version and emoji
+- [ ] Links to PyPI, HF, docs all work
+- [ ] CHANGELOG.md referenced
+- [ ] Release is NOT marked as pre-release

modules/caas/paper/README.md

@@ -0,0 +1,71 @@
+# Paper: Context-as-a-Service
+
+This directory contains materials for the academic paper submission.
+
+## 📂 Structure
+
+```
+paper/
+├── README.md              # This file
+├── CHECKLIST.md           # Submission checklist (NeurIPS/ICML/ACL style)
+├── LLM_DISCLOSURE.md      # AI assistance disclosure statement
+├── abstract.md            # Working abstract draft
+├── figures/               # Diagrams and visualizations
+│   └── .gitkeep
+├── tables/                # Result tables (generated from benchmarks)
+│   └── .gitkeep
+└── latex/                 # LaTeX source (when ready)
+    └── .gitkeep
+```
+
+## 📋 Submission Targets
+
+| Venue | Deadline | Format | Status |
+|-------|----------|--------|--------|
+| NeurIPS 2026 | May 2026 | 9 pages | 🎯 Target |
+| ICML 2026 | Feb 2026 | 8 pages | Backup |
+| ACL 2026 | TBD | 8 pages | NLP focus |
+| EMNLP 2026 | TBD | 8 pages | NLP focus |
+
+## 🏷️ Working Title
+
+**"Context-as-a-Service: A Principled Architecture for Enterprise RAG Systems"**
+
+Alternative titles:
+- "Beyond Flat Chunks: Structure-Aware Context Extraction for LLMs"
+- "Seven Fallacies of Context Management in Production RAG Systems"
+- "The Trust Gateway: Enterprise-Grade Context Routing for LLMs"
+
+## 📝 Key Contributions
+
+1. **The Seven Fallacies Framework** - Taxonomy of common RAG pitfalls
+2. **Structure-Aware Indexing** - Three-tier value hierarchy (not flat chunks)
+3. **Context Triad** - Hot/Warm/Cold intimacy-based prioritization
+4. **Pragmatic Truth** - Official vs. practical knowledge tracking
+5. **Trust Gateway** - Enterprise-grade on-prem routing architecture
+6. **Heuristic Router** - Zero-latency deterministic query routing
+7. **Open Benchmark Corpus** - 16-document enterprise evaluation set
+
+## 🔗 Resources
+
+- **Code**: https://github.com/imran-siddique/context-as-a-service
+- **PyPI**: https://pypi.org/project/context-as-a-service/
+- **Dataset**: https://huggingface.co/datasets/imran-siddique/context-as-a-service
+- **Docs**: https://github.com/imran-siddique/context-as-a-service/tree/main/docs
+
+## 📊 Generating Results
+
+```bash
+# Run benchmark evaluation
+python benchmarks/run_evaluation.py --corpus-path benchmarks/data/sample_corpus/
+
+# Generate statistical comparisons  
+python benchmarks/statistical_tests.py
+
+# Output tables for paper
+python benchmarks/run_evaluation.py --output paper/tables/
+```
+
+## ✅ Pre-submission Checklist
+
+See [CHECKLIST.md](CHECKLIST.md) for the full submission checklist.

modules/caas/paper/abstract.md

@@ -0,0 +1,24 @@
+# Abstract (Working Draft)
+
+## Version 1 (250 words)
+
+Retrieval-Augmented Generation (RAG) systems have become essential for grounding LLM outputs in factual content. However, production deployments face seven critical fallacies that current frameworks fail to address: (1) the Flat Chunk Fallacy, treating all content equally regardless of structural importance; (2) Context Amnesia, losing metadata when chunks are extracted; (3) Time-Blind Retrieval, ignoring content freshness; (4) Flat Context, lacking priority tiers for different context types; (5) Official Truth Fallacy, favoring documentation over practical knowledge; (6) Brutal Squeeze, using lossy summarization instead of precision truncation; and (7) the Middleware Gap, trusting third-party routers with sensitive data.
+
+We present **Context-as-a-Service (CaaS)**, an open-source framework that systematically addresses these challenges through five novel components: (a) **Structure-Aware Indexing** with three-tier value hierarchies; (b) **Context Triad** for Hot/Warm/Cold intimacy-based prioritization; (c) **Pragmatic Truth** tracking that surfaces practical knowledge alongside official sources; (d) **Heuristic Router** for zero-latency deterministic query routing; and (e) **Trust Gateway** for enterprise-grade on-premises deployment.
+
+We evaluate CaaS on a new benchmark corpus of 16 enterprise documents spanning code, legal, HR, and engineering domains. Our experiments demonstrate **28.1% improvement in Precision@5** and **27.9% improvement in NDCG@10** over flat-chunk baselines, with sub-millisecond routing latency (0.003ms) and only 18.4% latency overhead for the full pipeline. CaaS is available as an open-source Python package with MIT license, Docker support, and a public Hugging Face dataset for reproducibility.
+
+---
+
+## Key Claims (✅ Supported by Data)
+
+1. **Seven Fallacies** - Need literature/examples backing each one
+2. **✅ 28.1% Precision@5 improvement** - Benchmarked on 16-doc corpus
+3. **✅ 0.003ms routing latency** - Heuristic router measured
+4. **Enterprise compatibility** - Document Trust Gateway security properties
+
+## Notes
+
+- Target: 250 words (NeurIPS limit)
+- Current: ~250 words
+- Need: Concrete numbers from experiments

modules/caas/paper/build_pdf.py

@@ -0,0 +1,355 @@
+#!/usr/bin/env python3
+"""
+Build paper PDF from Markdown sources.
+
+Requirements:
+    pip install markdown weasyprint
+
+Usage:
+    python paper/build_pdf.py              # Build full paper
+    python paper/build_pdf.py --section abstract  # Build single section
+"""
+
+import argparse
+import subprocess
+import sys
+from pathlib import Path
+from datetime import datetime
+
+PAPER_DIR = Path(__file__).parent
+SECTIONS = [
+    "abstract.md",
+    "intro.md",
+    "related_work.md",
+    "method.md",
+    "experiments.md",
+    # "discussion.md",  # TODO
+    # "conclusion.md",  # TODO
+]
+
+HTML_TEMPLATE = """<!DOCTYPE html>
+<html>
+<head>
+    <meta charset="utf-8">
+    <title>Context-as-a-Service: A Principled Architecture for Enterprise RAG Systems</title>
+    <style>
+        @page {{
+            size: letter;
+            margin: 1in;
+        }}
+        body {{
+            font-family: 'Times New Roman', Times, serif;
+            font-size: 11pt;
+            line-height: 1.4;
+            max-width: 6.5in;
+            margin: 0 auto;
+            padding: 20px;
+        }}
+        h1 {{
+            font-size: 16pt;
+            text-align: center;
+            margin-bottom: 0.5em;
+        }}
+        h2 {{
+            font-size: 12pt;
+            margin-top: 1.5em;
+            margin-bottom: 0.5em;
+        }}
+        h3 {{
+            font-size: 11pt;
+            font-style: italic;
+            margin-top: 1em;
+            margin-bottom: 0.3em;
+        }}
+        p {{
+            text-align: justify;
+            margin-bottom: 0.8em;
+        }}
+        code {{
+            font-family: 'Courier New', monospace;
+            font-size: 9pt;
+            background-color: #f5f5f5;
+            padding: 1px 3px;
+        }}
+        pre {{
+            font-family: 'Courier New', monospace;
+            font-size: 9pt;
+            background-color: #f5f5f5;
+            padding: 10px;
+            overflow-x: auto;
+        }}
+        blockquote {{
+            border-left: 3px solid #ccc;
+            margin-left: 0;
+            padding-left: 1em;
+            color: #555;
+        }}
+        strong {{
+            font-weight: bold;
+        }}
+        em {{
+            font-style: italic;
+        }}
+        table {{
+            border-collapse: collapse;
+            width: 100%;
+            margin: 1em 0;
+            font-size: 10pt;
+        }}
+        th, td {{
+            border: 1px solid #ccc;
+            padding: 6px 10px;
+            text-align: left;
+        }}
+        th {{
+            background-color: #f0f0f0;
+        }}
+        .title {{
+            text-align: center;
+            margin-bottom: 2em;
+        }}
+        .author {{
+            text-align: center;
+            font-size: 11pt;
+            margin-bottom: 0.5em;
+        }}
+        .affiliation {{
+            text-align: center;
+            font-size: 10pt;
+            color: #666;
+            margin-bottom: 2em;
+        }}
+        .abstract {{
+            margin: 2em 0;
+            padding: 1em;
+            background-color: #fafafa;
+            border: 1px solid #eee;
+        }}
+        .abstract h2 {{
+            margin-top: 0;
+        }}
+        hr {{
+            border: none;
+            border-top: 1px solid #ccc;
+            margin: 2em 0;
+        }}
+        .page-break {{
+            page-break-after: always;
+        }}
+        .footer {{
+            font-size: 9pt;
+            color: #999;
+            text-align: center;
+            margin-top: 3em;
+        }}
+    </style>
+</head>
+<body>
+    <div class="title">
+        <h1>Context-as-a-Service:<br>A Principled Architecture for Enterprise RAG Systems</h1>
+    </div>
+    <div class="author">
+        Anonymous Author(s)
+    </div>
+    <div class="affiliation">
+        Anonymous Institution
+    </div>
+    
+    {content}
+    
+    <div class="footer">
+        <p>Draft generated {date} | Code: <a href="https://github.com/imran-siddique/context-as-a-service">GitHub</a> | Data: <a href="https://huggingface.co/datasets/imran-siddique/context-as-a-service">Hugging Face</a></p>
+    </div>
+</body>
+</html>
+"""
+
+
+def check_dependencies():
+    """Check if required tools are available."""
+    # Try pandoc first (preferred)
+    try:
+        result = subprocess.run(
+            ["pandoc", "--version"],
+            capture_output=True,
+            text=True
+        )
+        if result.returncode == 0:
+            return "pandoc"
+    except FileNotFoundError:
+        pass
+    
+    # Try weasyprint as fallback
+    try:
+        import weasyprint
+        return "weasyprint"
+    except ImportError:
+        pass
+    
+    # Try markdown + pdfkit
+    try:
+        import markdown
+        import pdfkit
+        return "pdfkit"
+    except ImportError:
+        pass
+    
+    return None
+
+
+def read_markdown(filepath: Path) -> str:
+    """Read and preprocess markdown file."""
+    content = filepath.read_text(encoding="utf-8")
+    
+    # Remove YAML frontmatter if present
+    if content.startswith("---"):
+        end = content.find("---", 3)
+        if end != -1:
+            content = content[end + 3:].strip()
+    
+    # Remove markdown comments
+    lines = []
+    for line in content.split("\n"):
+        if not line.strip().startswith("<!--"):
+            lines.append(line)
+    
+    return "\n".join(lines)
+
+
+def build_with_pandoc(md_files: list, output: Path):
+    """Build PDF using pandoc."""
+    cmd = [
+        "pandoc",
+        "-f", "markdown",
+        "-t", "pdf",
+        "--pdf-engine=xelatex",
+        "-V", "geometry:margin=1in",
+        "-V", "fontsize=11pt",
+        "-o", str(output),
+    ] + [str(f) for f in md_files]
+    
+    print(f"Running: {' '.join(cmd)}")
+    result = subprocess.run(cmd, capture_output=True, text=True)
+    
+    if result.returncode != 0:
+        print(f"Pandoc error: {result.stderr}")
+        return False
+    return True
+
+
+def build_with_weasyprint(md_files: list, output: Path):
+    """Build PDF using weasyprint."""
+    import markdown
+    from weasyprint import HTML
+    
+    # Combine all markdown
+    combined_md = ""
+    for f in md_files:
+        combined_md += read_markdown(f) + "\n\n---\n\n"
+    
+    # Convert to HTML
+    md_converter = markdown.Markdown(extensions=['tables', 'fenced_code'])
+    html_content = md_converter.convert(combined_md)
+    
+    # Wrap in template
+    full_html = HTML_TEMPLATE.format(
+        content=html_content,
+        date=datetime.now().strftime("%Y-%m-%d")
+    )
+    
+    # Generate PDF
+    html = HTML(string=full_html)
+    html.write_pdf(output)
+    return True
+
+
+def build_html(md_files: list, output: Path):
+    """Build HTML (always works, no dependencies)."""
+    import re
+    
+    combined_md = ""
+    for f in md_files:
+        combined_md += read_markdown(f) + "\n\n<hr>\n\n"
+    
+    # Simple markdown to HTML conversion
+    html = combined_md
+    
+    # Headers
+    html = re.sub(r'^### (.+)$', r'<h3>\1</h3>', html, flags=re.MULTILINE)
+    html = re.sub(r'^## (.+)$', r'<h2>\1</h2>', html, flags=re.MULTILINE)
+    html = re.sub(r'^# (.+)$', r'<h2>\1</h2>', html, flags=re.MULTILINE)  # Demote h1 to h2
+    
+    # Bold and italic
+    html = re.sub(r'\*\*(.+?)\*\*', r'<strong>\1</strong>', html)
+    html = re.sub(r'\*(.+?)\*', r'<em>\1</em>', html)
+    
+    # Code
+    html = re.sub(r'`(.+?)`', r'<code>\1</code>', html)
+    
+    # Paragraphs (simple)
+    html = re.sub(r'\n\n', '</p><p>', html)
+    html = f'<p>{html}</p>'
+    
+    # Wrap in template
+    full_html = HTML_TEMPLATE.format(
+        content=html,
+        date=datetime.now().strftime("%Y-%m-%d")
+    )
+    
+    output.write_text(full_html, encoding="utf-8")
+    return True
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Build paper PDF from Markdown")
+    parser.add_argument("--section", help="Build single section only")
+    parser.add_argument("--output", "-o", help="Output filename")
+    parser.add_argument("--format", choices=["pdf", "html"], default="html",
+                       help="Output format (default: html)")
+    args = parser.parse_args()
+    
+    # Determine which files to build
+    if args.section:
+        md_files = [PAPER_DIR / f"{args.section}.md"]
+        if not md_files[0].exists():
+            md_files = [PAPER_DIR / args.section]
+        if not md_files[0].exists():
+            print(f"Section not found: {args.section}")
+            return 1
+    else:
+        md_files = [PAPER_DIR / s for s in SECTIONS if (PAPER_DIR / s).exists()]
+    
+    print(f"Building from: {[f.name for f in md_files]}")
+    
+    # Determine output path
+    if args.output:
+        output = Path(args.output)
+    else:
+        output = PAPER_DIR / f"caas_paper_draft.{args.format}"
+    
+    # Build
+    if args.format == "pdf":
+        tool = check_dependencies()
+        if tool == "pandoc":
+            success = build_with_pandoc(md_files, output)
+        elif tool == "weasyprint":
+            success = build_with_weasyprint(md_files, output)
+        else:
+            print("No PDF tools available. Install pandoc or: pip install weasyprint")
+            print("Falling back to HTML...")
+            output = output.with_suffix(".html")
+            success = build_html(md_files, output)
+    else:
+        success = build_html(md_files, output)
+    
+    if success:
+        print(f"✓ Built: {output}")
+        print(f"  Size: {output.stat().st_size / 1024:.1f} KB")
+        return 0
+    else:
+        print("✗ Build failed")
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())