@zerwiz/ymir 0.1.16 → 0.1.18

package/assets/reference/docs/apodex/Final Report(4).md

@@ -0,0 +1,234 @@
+The `ApodexAI/AgentHarness` repo is the **official open‑source evaluation harness for Apodex‑1.0 models in a ReAct agent setup**. It is not a TUI, not a general agent platform like Ymir, and not a replacement for pi.dev — it’s the reference pipeline you use to **serve an Apodex model and run deep‑research benchmarks end‑to‑end.**
+
+Below is what it does and how you can meaningfully use it, given your interest in pi.dev and Ymir.
+
+---
+
+## 1. What AgentHarness actually is
+
+From the README:
+
+- Purpose:  
+  > “Evaluation harness for Apodex‑1.0 on public deep‑research benchmarks. … used to reproduce the public benchmark results for Apodex‑1.0 in a standard ReAct setup.” [1]
+
+- Benchmarked Apodex‑1.0 variants (open‑source weights) [1]:
+  - `Apodex‑1.0‑mini`
+  - `Apodex‑1.0‑4B‑SFT`
+  - `Apodex‑1.0‑2B‑SFT`
+  - `Apodex‑1.0‑0.8B‑SFT`
+
+- Example performance table across four benchmarks (BrowseComp, BrowseComp‑ZH, HLE‑Text, DeepSearchQA) [1]:
+
+  | Model             | BrowseComp | BrowseComp‑ZH | HLE‑Text | DeepSearchQA |
+  |-------------------|-----------:|--------------:|---------:|-------------:|
+  | Apodex‑1.0‑mini   | 71.5       | 80.6          | 46.8     | 82.2         |
+  | Apodex‑1.0‑4B‑SFT | 48.8       | 63.5          | 32.9     | 69.9         |
+  | Apodex‑1.0‑2B‑SFT | 27.9       | 35.0          | 18.2     | 49.9         |
+  | Apodex‑1.0‑0.8B‑SFT | 13.9     | 10.7          | 11.2     | 25.8         |
+
+- Supported benchmarks include (among others) [1]:
+  - BrowseComp, BrowseComp‑ZH
+  - xbench‑DeepResearch
+  - Humanity’s Last Exam (text‑only)
+  - SuperChem
+  - FrontierScience‑Research, FrontierScience‑Olympiad
+  - DeepSearchQA
+  - WideSearch
+
+- License: **Apache 2.0** (permissive; you can copy, modify, and integrate it in commercial or internal systems) [1].
+
+So: AgentHarness is a **reproducible evaluation runner** for a ReAct‑style Apodex agent, plus glue to datasets, search, fetch, and a code sandbox.
+
+---
+
+## 2. What it actually runs (high level)
+
+The README outlines a standard workflow [1]:
+
+1. **Install dependencies**
+
+   Uses `uv` with Python 3.12:
+
+   ```bash
+   uv sync --python 3.12
+   ```
+
+2. **Serve the model (SGLang)**
+
+   Example command for the 35B Apodex‑1.0 model:
+
+   ```bash
+   python3 -m sglang.launch_server \
+     --model-path apodex/Apodex-1.0-35B-A3B \
+     --tp 8 \
+     --host 0.0.0.0 \
+     --port 1234 \
+     --context-length 262144 \
+     --tool-call-parser qwen3_coder \
+     --reasoning-parser qwen3
+   ```
+
+   This gives you a local, OpenAI‑compatible endpoint for the Apodex model.
+
+3. **Configure environment**
+
+   ```bash
+   cp .env.example .env
+   ```
+
+   You then set (per README) [1]:
+
+   - `OPENAI_BASE_URL`, `OPENAI_API_KEY`, `OPENAI_MODEL` → point at your served Apodex or any OpenAI‑compatible agent model.
+   - `SERPER_API_KEY` → web search.
+   - `JINA_API_KEY` → web fetch.
+   - `E2B_API_KEY` → code sandbox.
+
+4. **Download benchmark datasets** [1]:
+
+   ```bash
+   wget https://huggingface.co/datasets/apodex/Deep-Research-Benchmarks/resolve/main/deep_research_benchmarks_260607.zip
+   unzip -P 'apodex*()_2026' deep_research_benchmarks_260607.zip
+   rm deep_research_benchmarks_260607.zip
+   ```
+
+   Note: HLE answers are *not* redistributed; to run HLE‑Text you must accept the license for `cais/hle` and place the standardized JSONL at:
+   `benchmarks/datasets/HLE-text/standardized_data.jsonl` [1].
+
+5. **Run a smoke test** [1]:
+
+   ```bash
+   uv run python -m benchmarks.runner.run_subprocess \
+     --benchmark browsecomp \
+     --pipeline react_base \
+     --profile default \
+     --limit 1 \
+     --concurrency 1 \
+     --out ./tmp/smoke
+   ```
+
+6. **Run a full benchmark** [1]:
+
+   ```bash
+   uv run python -m benchmarks.runner.run_subprocess \
+     --benchmark browsecomp \
+     --pipeline react_base \
+     --profile default \
+     --runs 5 \
+     --concurrency 30 \
+     --out ./bc-runs
+   ```
+
+7. **Check progress and aggregate accuracy** [1]:
+
+   ```bash
+   uv run python -m benchmarks.runner.check_progress ./bc-runs
+   ```
+
+Each question runs in its own subprocess for isolation and debuggability (per README discussion) [1].
+
+---
+
+## 3. How this helps you, concretely
+
+### 3.1 With pi.dev
+
+Your situation:
+
+- You already use **pi.dev**, which is a terminal‑first coding harness that can talk to many providers (including local/OpenAI‑compatible endpoints).
+
+How AgentHarness fits:
+
+1. **Validating a local Apodex endpoint before wiring into pi.dev**
+
+   - Use AgentHarness to **serve** Apodex‑1.0 locally via SGLang and run at least:
+     - a smoke test on `browsecomp` and
+     - a small subset of one other benchmark.
+   - If results are in the expected range from the README’s performance table, you know:
+     - your model weights,
+     - serving stack, and
+     - tool integrations (search, fetch, sandbox)
+     are configured correctly.
+
+   Once it’s validated, you can point pi.dev’s “OpenAI‑compatible” provider configuration at the same `OPENAI_BASE_URL`/`OPENAI_MODEL` you used in AgentHarness.
+
+2. **Understanding what “Apodex as a model” actually does**
+
+   - AgentHarness makes it explicit that the Apodex model is being used in a **ReAct agent loop** (tool calls + reasoning) with:
+     - web search,
+     - web fetch,
+     - code execution.
+   - That’s very close to how you’d want pi.dev to treat Apodex: not as a dumb completion model, but as a tool‑calling agent. The harness gives you a working example of how that’s wired.
+
+### 3.2 With Ymir
+
+Your earlier questions were about running Apodex (or even its TUI) *inside* Ymir, e.g. as Brokk or Kaia.
+
+AgentHarness’s role relative to Ymir:
+
+1. **Reference “sub‑agent” behavior**
+
+   - The harness shows you a clean, **ReAct‑style agent** around Apodex:
+     - structured tool calls to search/fetch/code,
+     - bounded concurrent jobs,
+     - per‑question subprocesses.
+   - In Ymir language, this is close to how an **Eindri** worker in an Utgard sandbox would behave: take a job, call tools, produce a verdict.
+
+2. **Input to designing a Ymir‑compliant worker**
+
+   If you later:
+   - keep Ymir as the outer orchestrator (Kaia + Brokk + Mimirsbrunn), and
+   - want Apodex as an inner researcher,
+
+   then AgentHarness is essentially a **template** for that inner researcher loop. You would:
+
+   - Preserve Ymir’s laws (Glitnir review, Skuld, append‑only ledger).
+   - Adapt the AgentHarness runner’s idea of:
+     - “one question = one isolated execution,”
+     - reliability under failures/hangs.
+
+   The harness itself does **not** implement Ymir’s ledger, Houses, A2A, or review gates; it’s just a clean, reproducible research runner you can embed under those rules.
+
+---
+
+## 4. What AgentHarness is *not*
+
+To avoid confusion with your previous questions:
+
+- It is **not**:
+  - a TUI or chat client.
+  - an agent platform with governance (like Ymir).
+  - a replacement for pi.dev.
+
+- It **is**:
+  - a tested, Apache‑2.0‑licensed **benchmark/evaluation harness** for deep‑research agents built on Apodex‑1.0.
+  - a practical way to:
+    - serve Apodex locally,
+    - confirm it works as intended,
+    - and then reuse that same endpoint from tools like pi.dev or Ymir.
+
+---
+
+## 5. Recommended way to use it in your stack
+
+Given everything we’ve discussed:
+
+1. **If your goal is: _“Use Apodex as a strong research agent behind my existing tools”_**
+
+   - Use AgentHarness to:
+     - spin up a local Apodex‑1.0 model with SGLang,
+     - run the provided smoke test on `browsecomp`,
+     - confirm it behaves well on a few benchmark questions.
+   - Then:
+     - point **pi.dev** at that endpoint for day‑to‑day terminal use,
+     - later, if/when you have Ymir access and want to plug Apodex in, reuse the same endpoint and ideas from the harness runner for your Eindri‑style workers.
+
+2. **If your goal is strictly benchmarking**
+
+   - Just run the harness as‑is with your chosen Apodex variant and compare your numbers against the README’s performance table. That tells you whether your hardware/serving stack is behaving as expected.
+
+If you tell me your hardware constraints (e.g., “single 24 GB GPU” vs “multi‑GPU server”) and whether you care more about **reproducible benchmarks** or **integrating as a provider**, I can advise which Apodex‑1.0 variant in the harness is realistic for you and what minimum pieces (search/fetch/sandbox) you should actually turn on.  
+
+---
+
+References  
+[1] ApodexAI/AgentHarness: Evaluation harness for Apodex‑1.0 on public deep‑research benchmarks. https://github.com/ApodexAI/AgentHarness

package/assets/reference/docs/apodex/Final Report(5).md

@@ -0,0 +1,13 @@
+**Short answer: yes — but with two clarifications and a reframing.** The product is spelled **Apodex** (not "Appdex"), and the URL you gave (`ymir.zerwiz.org`) is actually the **Ymir** platform — an alpha/invite-only self-hosted agent framework, not an Appdex page. So your question blends two things; this report answers both: using Apodex in your own workflow, and plugging it into Ymir. I've written the full assessment to `/outputs/Appdex_as_Orchestrator_Planner_Reviewer_Assessment.md`.
+
+**Verdict by role (all three attached reports + corroborated external checks):**
+
+| Role | Fit | Why |
+|---|---|---|
+| **Planner** | ✅ Strong | Heavy-duty mode decomposes complex goals across sub-agent teams with draft/revise loops and an evidence graph that maps every claim to a source — great for traceable plans. |
+| **Orchestrator** | ⚠️ Medium/conditional | It orchestrates its *own* research sub-agent teams internally (via the `/v1/responses` API + background polling), but isn't documented as a general meta-orchestrator for arbitrary external fleets. Best wired in as one node or as isolated Eindri-style workers. |
+| **Reviewer** | ⚠️ Low–medium | It has a global verifier/evidence-auditor component, useful as automated evidence checking — but no public governance gate. Human sign-off (Glitnir in Ymir, human-supervised heavy-duty mode) must stay final. |
+
+**Recommended approach:** Treat Apodex as a **strong reasoning + planning backend**, not a ready-made suite. Either (A) route planning-heavy multi-step work through its hosted API or a locally served Apache-2.0 weight (validate first via the `AgentHarness` smoke test), adding your own review gate on top; or (B) plug it into Ymir as Brokk/Kaia's brain and/or run its orchestration burst as Eindri workers — keeping Ymir's ledger, review, and access rules intact. Note Ymir is alpha/invite-only and some A2A/MCP specs are partly private, so path B needs an invite or your own fork.
+
+Limitations worth flagging: the 150+ sub-agent / 15k-step scale figure and platform billing details weren't independently verifiable here, and the verifier output should be treated as evidence for a human gate rather than a verdict.

package/assets/reference/docs/apodex/Final Report.md

@@ -0,0 +1,82 @@
+Apodex would not be a straight “upgrade” over Pi.dev; they solve different problems. For most people in your position, the right move is:
+
+- **Keep using Pi.dev for what it’s good at.**
+- **Use Apodex alongside it** when you need things Pi isn’t designed for.
+
+Here’s the actionable breakdown.
+
+---
+
+## What Pi.dev is giving you now
+
+From the information gathered, Pi.dev (“Pi”) is:
+
+- A **minimal, open-source terminal coding agent/harness** (MIT-licensed).
+- Runs **entirely on your machine**, no SaaS backend or cloud workspace.
+- Talks to **15+ model providers** and can route to **self-hosted endpoints** (e.g., Ollama, your own inference stack).
+- Highly **extensible in TypeScript** via extensions, skills, prompt templates, themes.
+- Stores **tree-structured session history locally**, with commands to navigate, branch, and export.
+
+In practice, Pi.dev is excellent when:
+
+- You live in the **terminal** and want an AI that edits code, runs tools, and works over your repo.
+- You care about **data locality** and **self-hosted models**.
+- You like a **minimal harness you can tinker with**, not a big, opinionated product.
+- You’re comfortable managing providers, keys, and extensions yourself.
+
+---
+
+## What Apodex does differently
+
+Apodex (me) is:
+
+- A **general-purpose assistant/solver** aimed at **multi-step, mission-critical tasks**, not a terminal harness.
+- Able to:
+  - Perform **web research**.
+  - **Fetch, download, and read** external documents (PDFs, docs, spreadsheets, etc.).
+  - **Run code and scripts in a sandbox** and work with files as part of the conversation.
+  - **Decompose and track tasks** explicitly so long, complex jobs don’t get lost.
+- Oriented around producing **finished artifacts** (reports, charts, code files, structured plans), not around being your terminal UI.
+
+In other words, Apodex is closer to a **research + tooling copilot**, whereas Pi.dev is a **local coding harness**.
+
+---
+
+## So, would Apodex be “better” for you?
+
+It depends what you’re actually doing day to day.
+
+### Stick with Pi.dev as your main tool if:
+
+- Your primary need is **coding inside your repos** via the terminal.
+- **Local-first and data control** are non-negotiable (self-hosted models, no extra SaaS).
+- You’re satisfied with:
+  - Pi’s TUI,
+  - its extensibility model (TypeScript extensions, skills),
+  - and manually wiring it into your workflows.
+
+In that case, there is no clear “better” — Pi is already very well-suited to that job.
+
+### Add Apodex (don’t replace Pi) if:
+
+You often need to do things like:
+
+- **Research**: Compare libraries, read specs, dig through multiple web sources and synthesize them.
+- **Document work**: Pull down a PDF or doc, extract key points, cross-reference with other materials.
+- **Multi-step pipelines**: 
+  - “Download this dataset, transform it, run this analysis, then write up a short report.”
+  - “Generate a design doc, then refine it based on constraints, then output a final structured version.”
+- **Project-sized tasks**: Where keeping track of sub-tasks, assumptions, and decisions in one place is valuable.
+
+Here, Apodex will feel “better” because it is built to string these steps together and keep state across them, not just respond in a single CLI session.
+
+---
+
+## Practical recommendation
+
+- **Don’t abandon Pi.dev** if you’re happy with it as a coding harness.
+- **Use Apodex side by side with Pi.dev**:
+  - Pi.dev for **local coding sessions** in your terminal and tight integration with your repos/models.
+  - Apodex for **research, planning, document-heavy tasks, and multi-step workflows** that go beyond “edit these files.”
+
+If you tell me one or two concrete workflows you currently run in Pi.dev (e.g., “I mainly use it to refactor code in a monorepo” or “I use it to scaffold new services”), I can give a yes/no for each: “Pi-only is fine here” vs. “Apodex would actually handle this better.”