maqam 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ajnas
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,176 @@
+# Maqam
+
+![Maqam governed agent framework hero](app/assets/maqam-readme-hero.png)
+
+Maqam is an MIT-licensed Ajnas agent framework for governed workflows. It combines a local agent runtime, policy engine, evidence ledger, skill registry, tool gateway, human-review-ready approval errors, and a crawler-backed research workflow.
+
+The crawler is no longer the product center; it is the first governed connector. Maqam is meant for enterprise agent workflows that need inspectable runs, source-backed outputs, compliance-friendly defaults, and no required hosted service.
+
+![Maqam system map](app/assets/maqam-system-map.svg)
+
+## What Ships
+
+- `AgentRuntime`: sequential workflow execution with retries, trace events, task outputs, and policy preflight.
+- `PolicyEngine`: deterministic goal and tool-call decisions for allowed tools, origins, limits, and approval gates.
+- `EvidenceLedger`: provenance records, claim links, source hashes, confidence, and unsupported-claim checks.
+- `ToolGateway`: one governed path for external tool execution.
+- `SkillRegistry`: lightweight skill metadata registration and selection.
+- `createResearchWorkflow`: crawler-backed source collection, synthesis, and quality checks.
+- `maqam`: local web console for running governed research workflows.
+- `maqam-crawl`: respectful crawler CLI that obeys `robots.txt` by default.
+
+## Why It Matters
+
+Agent systems fail in production when tools run outside policy, outputs cannot be traced to sources, and risky actions happen without approval. Maqam makes those control points explicit:
+
+- Every workflow starts with policy preflight.
+- Every tool call goes through `ToolGateway`.
+- Every source-backed claim can be recorded in `EvidenceLedger`.
+- Every run returns trace data for inspection and replay.
+- Approval-required actions fail closed with `ApprovalRequiredError`.
+- The crawler supports research and ingestion while preserving compliance defaults.
+
+## Install
+
+```bash
+npm install -g maqam
+```
+
+Run the local console:
+
+```bash
+maqam
+```
+
+Then open `http://127.0.0.1:8787`.
+
+Run without global install:
+
+```bash
+npx maqam --port 8787
+```
+
+## Crawler CLI
+
+```bash
+maqam-crawl https://example.com --max-pages 50 --jsonl --output crawl.jsonl
+```
+
+Legacy aliases `ajnas-crawl` and `ajnas-agent-crawler` are kept for compatibility.
+
+Options:
+
+- `--max-pages <n>`: maximum pages to return. Default: `50`
+- `--concurrency <n>`: concurrent workers. Default: `4`
+- `--delay <ms>`: minimum delay per origin. Default: `250`
+- `--timeout <ms>`: request timeout. Default: `15000`
+- `--sitemaps`: discover URLs from robots.txt sitemaps and `/sitemap.xml`
+- `--all-origins`: allow crawling across discovered origins
+- `--jsonl`: output JSON Lines instead of a JSON array
+- `--output <file>`: write output to a file
+- `--user-agent <ua>`: custom user agent
+
+## Framework SDK
+
+```js
+import {
+  AgentRuntime,
+  EvidenceLedger,
+  PolicyEngine,
+  ToolGateway,
+  createCrawlerTool,
+  createResearchWorkflow
+} from "maqam";
+
+const evidenceLedger = new EvidenceLedger();
+const policyEngine = new PolicyEngine({
+  allowedTools: ["crawler"],
+  allowedOrigins: ["https://github.com", "https://www.npmjs.com"]
+});
+
+const gateway = new ToolGateway({ policyEngine, evidenceLedger });
+gateway.registerTool("crawler", createCrawlerTool());
+
+const runtime = new AgentRuntime({ policyEngine, evidenceLedger, toolGateway: gateway });
+const result = await runtime.runWorkflow(
+  createResearchWorkflow({
+    seeds: ["https://github.com/apify/crawlee"],
+    maxPages: 5
+  }),
+  {
+    objective: "Research permissive OSS agent framework projects",
+    allowedTools: ["crawler"],
+    allowedOrigins: ["https://github.com"]
+  }
+);
+
+console.log(result.outputs.synthesize_report.candidates);
+```
+
+## Crawler API
+
+```js
+import { crawl } from "maqam";
+
+const pages = await crawl({
+  seeds: ["https://example.com"],
+  maxPages: 25,
+  concurrency: 4,
+  includeSitemaps: true,
+  onPage(page) {
+    console.log(page.url, page.title);
+  }
+});
+
+console.log(pages[0].markdown);
+```
+
+## Maqam Console
+
+```bash
+npm run maqam
+```
+
+The console runs a governed research workflow through:
+
+- `PolicyEngine`: allows or denies goals and tool calls.
+- `ToolGateway`: routes all external work through policy checks.
+- `EvidenceLedger`: records source-backed evidence and claim support.
+- `AgentRuntime`: executes workflow tasks with traces and retries.
+- `createResearchWorkflow`: composes crawler collection, synthesis, and quality checks.
+
+Brand assets live in `app/assets/`, including `maqam-logo.svg` and `maqam-brand-board.png`.
+
+## Principles
+
+- Respect `robots.txt` by default.
+- Use a clear user agent.
+- Rate-limit per origin.
+- Avoid bypassing access controls, paywalls, anti-bot systems, or private content.
+- No required AI provider dependency.
+- No required external hosted service.
+- Produce JSON/JSONL output that agents can consume directly.
+
+## What This Is Not
+
+Maqam is not a stealth scraper and does not include bypass tooling. It will not help evade login walls, paywalls, anti-bot protections, CAPTCHA, robots.txt, or authorization boundaries.
+
+## Development
+
+```bash
+npm install
+npm test
+npm pack --dry-run
+```
+
+## Publish
+
+```bash
+npm publish --access public
+```
+
+Publishing requires an authenticated npm session with permission to publish the `maqam` package.
+
+## License
+
+MIT

package/app/app.js

@@ -0,0 +1,113 @@
+const form = document.querySelector("#research-form");
+const statusEl = document.querySelector("#run-status");
+const candidatesEl = document.querySelector("#candidates");
+const evidenceEl = document.querySelector("#evidence-list");
+const traceEl = document.querySelector("#trace-list");
+const metricRuntime = document.querySelector("#metric-runtime");
+const metricEvidence = document.querySelector("#metric-evidence");
+const metricClaims = document.querySelector("#metric-claims");
+const metricTools = document.querySelector("#metric-tools");
+
+function setStatus(text, kind = "ready") {
+  statusEl.textContent = text;
+  statusEl.className = kind === "error" ? "error" : "";
+}
+
+function truncate(text, length = 180) {
+  if (!text) return "";
+  return text.length > length ? `${text.slice(0, length - 1)}...` : text;
+}
+
+function candidateTemplate(candidate) {
+  return `
+    <article>
+      <h3>${candidate.name || "Untitled candidate"}</h3>
+      <p>${truncate(candidate.whatItDoes || "No summary returned.")}</p>
+      <div class="tag-row">
+        <a class="tag" href="${candidate.url}" target="_blank" rel="noreferrer">Source</a>
+        <span class="tag">${candidate.recommendation || "review"}</span>
+      </div>
+    </article>
+  `;
+}
+
+function evidenceTemplate(record) {
+  return `
+    <article>
+      <h3>${record.sourceType}: ${record.source}</h3>
+      <p>${truncate(record.excerpt || "No excerpt captured.", 220)}</p>
+      <div class="tag-row">
+        <span class="tag">${record.tool || "tool"}</span>
+        <span class="tag">${Math.round((record.confidence || 0) * 100)}% confidence</span>
+      </div>
+    </article>
+  `;
+}
+
+function traceTemplate(item) {
+  return `<li>${item.taskId || item.toolName}: ${item.status || item.decision?.status || "completed"}</li>`;
+}
+
+function renderRun(payload) {
+  const run = payload.run;
+  const candidates = run.outputs?.synthesize_report?.candidates || [];
+  const evidence = run.evidence?.evidence || [];
+  const claims = run.evidence?.claims || [];
+  const toolTrace = payload.toolTrace || [];
+
+  candidatesEl.className = candidates.length ? "stack" : "stack empty";
+  candidatesEl.innerHTML = candidates.length
+    ? candidates.map(candidateTemplate).join("")
+    : "No candidates returned.";
+
+  evidenceEl.className = evidence.length ? "stack" : "stack empty";
+  evidenceEl.innerHTML = evidence.length
+    ? evidence.map(evidenceTemplate).join("")
+    : "No evidence records returned.";
+
+  traceEl.innerHTML = [
+    ...(run.trace || []).map(traceTemplate),
+    ...toolTrace.map(traceTemplate)
+  ].join("");
+
+  metricRuntime.textContent = run.status;
+  metricEvidence.textContent = `${evidence.length} records`;
+  metricClaims.textContent = `${claims.length} checked`;
+  metricTools.textContent = `${toolTrace.length} calls`;
+  setStatus(run.status === "completed" ? "Completed" : run.status, run.status === "failed" ? "error" : "ready");
+}
+
+form.addEventListener("submit", async (event) => {
+  event.preventDefault();
+  const button = form.querySelector("button");
+  const data = new FormData(form);
+  const seed = data.get("seed");
+  const maxPages = Number(data.get("maxPages"));
+  const sameOrigin = data.get("sameOrigin") === "true";
+
+  button.disabled = true;
+  setStatus("Running");
+  metricRuntime.textContent = "Running";
+  candidatesEl.className = "stack empty";
+  candidatesEl.textContent = "Collecting sources through policy gateway...";
+  evidenceEl.className = "stack empty";
+  evidenceEl.textContent = "Waiting for evidence records...";
+
+  try {
+    const response = await fetch("/api/runs/research", {
+      method: "POST",
+      headers: { "content-type": "application/json" },
+      body: JSON.stringify({ seeds: [seed], maxPages, sameOrigin })
+    });
+    const payload = await response.json();
+    if (!response.ok) throw new Error(payload.error || "Run failed");
+    renderRun(payload);
+  } catch (error) {
+    setStatus("Error", "error");
+    candidatesEl.className = "stack empty error";
+    candidatesEl.textContent = error.message || "Unable to run workflow.";
+    metricRuntime.textContent = "Error";
+  } finally {
+    button.disabled = false;
+  }
+});

package/app/assets/maqam-logo.svg

@@ -0,0 +1,17 @@
+<svg width="128" height="128" viewBox="0 0 128 128" fill="none" xmlns="http://www.w3.org/2000/svg" role="img" aria-labelledby="title desc">
+  <title id="title">Maqam logo</title>
+  <desc id="desc">A geometric M mark built from interlocking frames and a protected evidence core.</desc>
+  <rect width="128" height="128" rx="30" fill="#0C1110"/>
+  <path d="M27 91V37h17l20 28 20-28h17v54H84V62L66 88h-4L44 62v29H27Z" fill="#F4F7F3"/>
+  <path d="M44 37h20l-8 12H44V37Z" fill="#48C7FF"/>
+  <path d="M84 37h17v17H90l-6-17Z" fill="#84E07F"/>
+  <path d="M64 65 84 37h17L70 80h-12L27 37h17l20 28Z" fill="url(#g)" fill-opacity=".62"/>
+  <circle cx="64" cy="72" r="9" fill="#0C1110" stroke="#84E07F" stroke-width="4"/>
+  <path d="M35 100h58" stroke="#48C7FF" stroke-width="5" stroke-linecap="round"/>
+  <defs>
+    <linearGradient id="g" x1="27" y1="37" x2="101" y2="80" gradientUnits="userSpaceOnUse">
+      <stop stop-color="#48C7FF"/>
+      <stop offset="1" stop-color="#84E07F"/>
+    </linearGradient>
+  </defs>
+</svg>

package/app/assets/maqam-system-map.svg

@@ -0,0 +1,114 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="630" viewBox="0 0 1200 630" role="img" aria-labelledby="title desc">
+  <title id="title">Maqam system map</title>
+  <desc id="desc">A governed agent workflow from goal intake through policy, runtime, tools, evidence, approvals, and verified output.</desc>
+  <defs>
+    <linearGradient id="bg" x1="0" x2="1" y1="0" y2="1">
+      <stop offset="0" stop-color="#070A0D"/>
+      <stop offset="0.52" stop-color="#0B0D10"/>
+      <stop offset="1" stop-color="#111816"/>
+    </linearGradient>
+    <linearGradient id="blue" x1="0" x2="1">
+      <stop offset="0" stop-color="#1D4ED8"/>
+      <stop offset="1" stop-color="#3B82F6"/>
+    </linearGradient>
+    <linearGradient id="green" x1="0" x2="1">
+      <stop offset="0" stop-color="#16A34A"/>
+      <stop offset="1" stop-color="#22C55E"/>
+    </linearGradient>
+    <filter id="glow" x="-30%" y="-30%" width="160%" height="160%">
+      <feGaussianBlur stdDeviation="4" result="blur"/>
+      <feMerge>
+        <feMergeNode in="blur"/>
+        <feMergeNode in="SourceGraphic"/>
+      </feMerge>
+    </filter>
+    <style>
+      .label{font:700 20px Segoe UI,Arial,sans-serif;fill:#F5F6F8}
+      .small{font:500 14px Segoe UI,Arial,sans-serif;fill:#A9B7B0}
+      .tiny{font:600 12px Segoe UI,Arial,sans-serif;fill:#A9B7B0;letter-spacing:.08em}
+      .panel{fill:#111816;stroke:#2D3935;stroke-width:1.5}
+      .line{fill:none;stroke:#334155;stroke-width:2}
+      .blueLine{fill:none;stroke:#2563EB;stroke-width:3}
+      .greenLine{fill:none;stroke:#22C55E;stroke-width:3}
+    </style>
+  </defs>
+  <rect width="1200" height="630" rx="28" fill="url(#bg)"/>
+  <path d="M0 126H1200M0 252H1200M0 378H1200M0 504H1200M200 0V630M400 0V630M600 0V630M800 0V630M1000 0V630" stroke="#1F2937" stroke-width="1" opacity=".32"/>
+
+  <g transform="translate(54 46)">
+    <path d="M0 70V10L100 68V168H20V92L100 138" fill="none" stroke="url(#blue)" stroke-width="12" stroke-linejoin="round" filter="url(#glow)"/>
+    <path d="M204 70V10L104 68V168H184V92L104 138" fill="none" stroke="url(#green)" stroke-width="12" stroke-linejoin="round" filter="url(#glow)"/>
+    <rect x="92" y="102" width="20" height="20" fill="#F5F6F8" rx="2"/>
+  </g>
+
+  <text x="54" y="288" class="tiny" fill="#3B82F6">MAQAM</text>
+  <text x="54" y="328" class="label" font-size="44">Compose governed agents</text>
+  <text x="54" y="364" class="small">Small, trusted skills. One controlled workflow.</text>
+
+  <g transform="translate(360 70)">
+    <rect class="panel" width="232" height="86" rx="14"/>
+    <text x="24" y="36" class="label">PolicyEngine</text>
+    <text x="24" y="62" class="small">allow, deny, limit, approve</text>
+  </g>
+  <g transform="translate(360 210)">
+    <rect class="panel" width="232" height="86" rx="14"/>
+    <text x="24" y="36" class="label">AgentRuntime</text>
+    <text x="24" y="62" class="small">tasks, retries, traces</text>
+  </g>
+  <g transform="translate(360 350)">
+    <rect class="panel" width="232" height="86" rx="14"/>
+    <text x="24" y="36" class="label">SkillRegistry</text>
+    <text x="24" y="62" class="small">skills, triggers, scores</text>
+  </g>
+
+  <g transform="translate(704 70)">
+    <rect class="panel" width="232" height="86" rx="14"/>
+    <text x="24" y="36" class="label">ToolGateway</text>
+    <text x="24" y="62" class="small">crawler and connectors</text>
+  </g>
+  <g transform="translate(704 210)">
+    <rect class="panel" width="232" height="86" rx="14"/>
+    <text x="24" y="36" class="label">EvidenceLedger</text>
+    <text x="24" y="62" class="small">sources, claims, hashes</text>
+  </g>
+  <g transform="translate(704 350)">
+    <rect class="panel" width="232" height="86" rx="14"/>
+    <text x="24" y="36" class="label">Human Review</text>
+    <text x="24" y="62" class="small">approval-ready gates</text>
+  </g>
+
+  <g transform="translate(508 495)">
+    <rect width="320" height="72" rx="16" fill="#0B1220" stroke="#2563EB" stroke-width="1.5"/>
+    <text x="24" y="31" class="label">Verified Output</text>
+    <text x="24" y="54" class="small">auditable reports, JSON, and run traces</text>
+  </g>
+
+  <path class="blueLine" d="M592 113H704"/>
+  <path class="blueLine" d="M592 253H704"/>
+  <path class="greenLine" d="M592 393H704"/>
+  <path class="line" d="M476 156V210"/>
+  <path class="line" d="M476 296V350"/>
+  <path class="line" d="M820 156V210"/>
+  <path class="line" d="M820 296V350"/>
+  <path class="greenLine" d="M820 436V495"/>
+  <path class="blueLine" d="M476 436V531H508"/>
+
+  <circle cx="592" cy="113" r="7" fill="#2563EB"/>
+  <circle cx="704" cy="113" r="7" fill="#22C55E"/>
+  <circle cx="592" cy="253" r="7" fill="#2563EB"/>
+  <circle cx="704" cy="253" r="7" fill="#22C55E"/>
+  <circle cx="592" cy="393" r="7" fill="#2563EB"/>
+  <circle cx="704" cy="393" r="7" fill="#22C55E"/>
+  <circle cx="820" cy="495" r="7" fill="#22C55E"/>
+
+  <g transform="translate(990 112)">
+    <rect class="panel" width="142" height="320" rx="16"/>
+    <text x="22" y="42" class="tiny">POLICY CHIPS</text>
+    <text x="22" y="82" class="small">Access Control</text>
+    <text x="22" y="122" class="small">PII Protection</text>
+    <text x="22" y="162" class="small">Tool Allowlist</text>
+    <text x="22" y="202" class="small">Rate Limits</text>
+    <text x="22" y="262" class="tiny">STATE</text>
+    <text x="22" y="300" class="small" fill="#22C55E">Governed</text>
+  </g>
+</svg>

package/app/index.html

@@ -0,0 +1,113 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <meta name="description" content="Maqam is an enterprise agent framework console for governed research workflows.">
+    <title>Maqam | Compose governed agents</title>
+    <link rel="icon" href="/assets/maqam-logo.svg" type="image/svg+xml">
+    <link rel="stylesheet" href="/styles.css">
+  </head>
+  <body>
+    <div class="shell">
+      <header class="topbar">
+        <a class="brand" href="/" aria-label="Maqam home">
+          <img src="/assets/maqam-logo.svg" alt="" width="44" height="44">
+          <span>
+            <strong>Maqam</strong>
+            <small>Compose governed agents</small>
+          </span>
+        </a>
+        <nav class="nav" aria-label="Primary">
+          <a href="#run">Run</a>
+          <a href="#evidence">Evidence</a>
+          <a href="#policy">Policy</a>
+        </nav>
+      </header>
+
+      <main>
+        <section class="hero" aria-labelledby="hero-title">
+          <div class="hero-copy">
+            <p class="kicker">Enterprise agent framework</p>
+            <h1 id="hero-title">Build agent workflows that can be inspected, governed, and trusted.</h1>
+            <p class="lede">Maqam composes skills, tools, policy, evidence, and runtime traces into one controlled workflow surface.</p>
+          </div>
+          <div class="run-panel" id="run">
+            <div class="panel-header">
+              <span>Governed research run</span>
+              <strong id="run-status">Ready</strong>
+            </div>
+            <form id="research-form">
+              <label for="seed-url">Seed URL</label>
+              <input id="seed-url" name="seed" type="url" value="https://github.com/apify/crawlee" autocomplete="url" required>
+              <div class="form-grid">
+                <label>
+                  Max pages
+                  <input name="maxPages" type="number" min="1" max="25" value="2" autocomplete="off">
+                </label>
+                <label>
+                  Same origin
+                  <select name="sameOrigin">
+                    <option value="true" selected>On</option>
+                    <option value="false">Off</option>
+                  </select>
+                </label>
+              </div>
+              <button type="submit">Run governed workflow</button>
+              <p class="hint">The server enforces policy before tool execution and records evidence for the synthesized result.</p>
+            </form>
+          </div>
+        </section>
+
+        <section class="metrics" aria-label="Run metrics">
+          <article>
+            <span>Runtime</span>
+            <strong id="metric-runtime">Idle</strong>
+          </article>
+          <article>
+            <span>Evidence</span>
+            <strong id="metric-evidence">0 records</strong>
+          </article>
+          <article>
+            <span>Claims</span>
+            <strong id="metric-claims">0 checked</strong>
+          </article>
+          <article>
+            <span>Gateway</span>
+            <strong id="metric-tools">0 calls</strong>
+          </article>
+        </section>
+
+        <section class="workspace">
+          <div class="column">
+            <div class="section-heading">
+              <h2>Candidates</h2>
+              <p>What the workflow produced from governed source collection.</p>
+            </div>
+            <div id="candidates" class="stack empty">Run a workflow to inspect candidates.</div>
+          </div>
+          <div class="column">
+            <div class="section-heading" id="evidence">
+              <h2>Evidence ledger</h2>
+              <p>Source-backed records linked to claims.</p>
+            </div>
+            <div id="evidence-list" class="stack empty">Evidence records will appear here.</div>
+          </div>
+        </section>
+
+        <section class="policy-band" id="policy">
+          <div>
+            <h2>Policy path</h2>
+            <p>Goal preflight, tool authorization, evidence capture, synthesis, and quality checks are first-class run steps.</p>
+          </div>
+          <ol id="trace-list">
+            <li>Preflight waiting</li>
+            <li>Tool gateway waiting</li>
+            <li>Evidence ledger waiting</li>
+          </ol>
+        </section>
+      </main>
+    </div>
+    <script type="module" src="/app.js"></script>
+  </body>
+</html>