@hackerrank/astra-cli 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 HackerRank
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,205 @@
+# astra
+
+A minimal, **zero-dependency** AI coding agent for the **HackerRank AI Gateway**,
+written in plain Node.js (ESM, Node ≥ 20).
+
+astra is a small, self-contained agent that talks to the `hackerrank-ai-gateway` and
+works with **every** model on it. It runs in two modes from one engine:
+
+- an **interactive assistant** — a chat REPL that can read files and run commands
+  (with your approval), like a lightweight cline / claude-code;
+- an **autonomous task runner** — give it a task and it works to completion on its own.
+
+Every run is a resumable session, with exact token accounting and USD cost tracking.
+
+## Why it's minimal
+
+- **No dependencies.** Uses Node's built-in `fetch`, `spawn`, and `fs`. No install step.
+- **Text-based action protocol.** The model returns one fenced ` ```bash ` block per
+  turn, so it works with *every* model on the gateway — no provider tool-calling required.
+- **Linear history.** Every step just appends to the message list. The session *is* the
+  conversation; trivial to debug and inspect.
+- **Stateless actions.** Each command runs in a fresh `bash -c` subshell, so runs are
+  easy to reason about and sandbox.
+
+## Layout
+
+| File | Role |
+|---|---|
+| `src/model.js` | Gateway client (OpenAI-compatible `/chat/completions`). Handles retries, the `max_tokens` vs `max_completion_tokens` difference, and cost accounting. |
+| `src/environment.js` | Runs one command per action in a fresh subshell, with a per-command timeout that kills the whole process group and output truncation. |
+| `src/prompts.js` | System / instance / format-error / observation templates + a tiny `{{var}}` renderer, split into interactive vs autonomous rules. |
+| `src/agent.js` | The turn-based engine: query → parse one command → execute → observe → repeat. Handles submission, step/time limits, and format errors. |
+| `src/repl.js` | The interactive assistant loop: prompts, command approval, slash commands. |
+| `src/session.js` | Resumable sessions stored under `~/.astra/sessions/`. |
+| `src/config.js` | Dedicated `~/.astra/config.json`, credential resolution, interactive key prompt. |
+| `src/prices.js` | Public list prices used to estimate cost for models that don't report it. |
+| `src/cli.js` | Argument parsing, mode dispatch, and streaming output. |
+
+## Setup
+
+Node ≥ 20. Install from npm:
+
+```bash
+npm install -g @hackerrank/astra
+
+# or without a global install
+npx @hackerrank/astra -m claude-sonnet-5
+```
+
+From a git checkout of this repo you can also run the CLI directly, or link it:
+
+```bash
+node src/cli.js -m claude-sonnet-5
+npm link    # then `astra` uses this checkout
+```
+
+You also need an API key for the gateway. The CLI resolves it in this order
+(first hit wins):
+
+1. `--api-key` flag
+2. `ASTRA_GATEWAY_API_KEY` environment variable
+3. `~/.astra/config.json` — astra's own config (written with `0600` perms)
+4. **interactive prompt** when run in a terminal — offers to save the key to
+   `~/.astra/config.json` so future runs pick it up automatically
+
+The first interactive run asks for the key and can remember it. In CI, pass the
+key via flag or env var.
+
+## Usage
+
+astra has **two modes**, sharing one engine:
+
+### Interactive assistant (no task)
+
+A personal coding-assistant REPL. Commands need your approval by default; the
+whole chat is saved as a resumable session.
+
+```bash
+astra -m claude-sonnet-5            # start chatting
+astra -m claude-sonnet-5 -y         # auto-run commands (no prompts)
+```
+
+In-REPL commands: `/help /exit /clear /history /tokens /yolo`.
+
+### Autonomous task run
+
+When you pass a task with `-t`/`-f`, astra appends the autonomous rules and runs
+to completion, submitting via the sentinel when done.
+
+```bash
+# from an inline task
+astra -m claude-sonnet-5 -t "Fix the failing test in ./app and submit."
+
+# from a task file, running inside a specific repo
+astra \
+  -m gpt-5.6-sol \
+  -f ./task/instruction.md \
+  -C ./task/repo \
+  -s 60 \
+  -o ./runs/run-01.traj.json
+```
+
+### Sessions
+
+Every run (either mode) is saved under `~/.astra/sessions/<id>.json`.
+
+```bash
+astra --sessions            # list saved sessions
+astra --resume <id>         # resume (continue an interactive chat,
+                            # or inspect/continue an autonomous run)
+```
+
+Verified working models on the gateway include `claude-sonnet-5`, `claude-opus-5`,
+`gpt-5.6-sol`, `gpt-5.6-terra`, `gemini-3.7-flash`, `grok-4.6`, `kimi-k3`, and others.
+Some routes (e.g. `glm-5.2`) may return HTTP 402 when the underlying provider is out of
+credits — that's a quota issue, not a bug.
+
+### Options
+
+```
+-m, --model <id>        Model id on the gateway (required unless --sessions)
+-t, --task <text>       Task text -> autonomous mode
+-f, --task-file <path>  Read task text from a file -> autonomous mode
+-C, --cwd <path>        Working directory for commands (default: cwd)
+-o, --output <path>     Also write trajectory JSON here (autonomous mode)
+-s, --steps <n>         Step limit (default: 40)
+-w, --wall <seconds>    Wall-clock limit (default: 0 = none)
+    --timeout <seconds> Per-command timeout (default: 60)
+    --max-output <n>    Max chars of command output kept (default: 16000)
+    --base-url <url>    Override gateway base URL
+    --api-key <key>     Gateway API key (else env/config/prompt)
+    --resume <id>       Resume a saved session
+    --sessions          List saved sessions and exit
+-y, --yolo              Auto-run commands (always on in autonomous mode)
+-q, --quiet             Do not stream steps (autonomous mode)
+-h, --help              Show help
+```
+
+## How the loop works
+
+```
+system + instance prompt
+        │
+        ▼
+   ┌─► query model ──► parse ONE ```bash block ──► run in fresh subshell
+   │        ▲                    │ (bad format)            │
+   │        └── format-error ◄───┘                         ▼
+   │                                              observe <returncode>/<output>
+   └──────────────────────────────────────────────────────┘
+                     │ command echoes the submit sentinel
+                     ▼
+                  Submitted
+```
+
+In interactive mode, a reply with **no** bash block is treated as a chat turn that hands
+control back to you; in autonomous mode the loop keeps going until the task is submitted
+or a limit is hit.
+
+Termination reasons written to the session `info.exit_status`:
+
+- `Submitted` — a command printed `COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT` with exit code 0.
+- `LimitsExceeded` — hit the step limit.
+- `TimeExceeded` — hit the wall-clock limit.
+- `RepeatedFormatError` — too many unparseable replies in a row.
+- `ContextWindowExceeded` — the request exceeded the model's context window.
+
+## Session / trajectory output
+
+Each run is stored as JSON (`astra-1` format):
+
+```json
+{
+  "trajectory_format": "astra-1",
+  "info": {
+    "mode": "autonomous",
+    "exit_status": "Submitted",
+    "submission": "",
+    "n_steps": 2,
+    "model": "claude-sonnet-5",
+    "n_calls": 2,
+    "elapsed_seconds": 6,
+    "tokens": { "prompt": 3414, "completion": 208, "total": 3622, "last_context": 1361 },
+    "cost": { "usd": 0.0139, "source": "estimated", "reported_usd": 0, "estimated_usd": 0.0139 }
+  },
+  "messages": [ /* full linear history: system, user, assistant, ... , exit */ ]
+}
+```
+
+## Cost & token tracking
+
+Tokens are exact (from each response's `usage`). Cost is **hybrid**, tracking-only
+(no limits):
+
+- **Reported** — some routes (the OpenRouter-proxied models: `deepseek-v4-pro`,
+  `glm-5.2`, `grok-4.6`, `kimi-k3`, `qwen-3.8`) return an exact `usage.cost` in USD.
+  We use that number directly (`source: "reported"`).
+- **Estimated** — native routes (Claude, GPT, Gemini) return no cost, so we estimate
+  from `src/prices.js`, a table of **public list prices** ($/1M tokens). These are
+  clearly marked `source: "estimated"` and shown with a `~` in the per-step line.
+  Edit `src/prices.js` in one place if you have the real gateway pricing.
+
+A run that mixes both is tagged `source: "mixed"` with the split preserved in
+`reported_usd` / `estimated_usd`.
+
+Remote / Jenkins benchmarking lives in the sibling repo `astra-bench`, not here.

package/package.json

@@ -0,0 +1,30 @@
+{
+  "name": "@hackerrank/astra-cli",
+  "version": "0.1.0",
+  "description": "Minimal zero-dependency AI coding agent for the HackerRank AI Gateway.",
+  "type": "module",
+  "bin": {
+    "astra": "src/cli.js"
+  },
+  "files": [
+    "src",
+    "README.md",
+    "LICENSE"
+  ],
+  "engines": {
+    "node": ">=20"
+  },
+  "scripts": {
+    "start": "node src/cli.js",
+    "astra": "node src/cli.js"
+  },
+  "publishConfig": {
+    "registry": "https://registry.npmjs.org/",
+    "access": "public"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+ssh://git@github.com/interviewstreet/hrbench.git"
+  },
+  "license": "MIT"
+}

package/src/agent.js

@@ -0,0 +1,319 @@
+/**
+ * The agent loop — the whole brain of astra in one small file.
+ *
+ * One engine, two modes:
+ *   - autonomous:  run to completion; a missing/extra command is a format error;
+ *                  the SUBMIT sentinel ends the run. Used for --task / benchmark.
+ *   - interactive: a reply with a bash block runs a command and loops; a reply
+ *                  with NO bash block is a chat turn that hands control back to
+ *                  the user. Used for the REPL.
+ *
+ * Turn model: `runTurn()` performs one model call and (if a command was issued)
+ * one execution. It returns a small status object so a driver can either keep
+ * looping (autonomous) or pause for user input (interactive).
+ *
+ * Termination (autonomous):
+ *   - Submitted:   a command outputs COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT (rc 0)
+ *   - LimitsExceeded: step limit reached
+ *   - TimeExceeded:   wall-clock limit reached
+ *   - RepeatedFormatError: too many unparseable replies in a row
+ *   - ContextWindowExceeded: prompt no longer fits the model's context
+ */
+
+import fs from "node:fs";
+import path from "node:path";
+import {
+  buildSystemPrompt,
+  INSTANCE_TEMPLATE,
+  FORMAT_ERROR,
+  OBSERVATION_TEMPLATE,
+  render,
+} from "./prompts.js";
+import { ContextWindowError } from "./model.js";
+
+const SUBMIT_SENTINEL = "COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT";
+
+export class Agent {
+  constructor(model, env, opts = {}) {
+    this.model = model;
+    this.env = env;
+    this.mode = opts.mode === "interactive" ? "interactive" : "autonomous";
+    this.stepLimit = opts.stepLimit ?? 40;
+    this.wallTimeLimitSeconds = opts.wallTimeLimitSeconds ?? 0;
+    this.maxConsecutiveFormatErrors = opts.maxConsecutiveFormatErrors ?? 3;
+    this.outputPath = opts.outputPath ?? null;
+    this.onEvent = opts.onEvent ?? (() => {});
+    this.onStep = opts.onStep ?? (() => {});
+    // Approval gate: return true to allow a command, false to skip it. Default
+    // allows everything (autonomous / --yolo). Interactive mode wires a prompt.
+    this.confirm = opts.confirm ?? (() => true);
+    this.messages = [];
+    this.nSteps = 0;
+    this.formatErrorStreak = 0;
+    this.startTime = Date.now();
+    // Exact context size (prompt tokens) reported by the most recent call.
+    this.lastContextTokens = 0;
+    this.exitStatus = null; // set once the run terminates
+    this.task = opts.task ?? "";
+    // Optional session metadata; when set, save() also writes the session file.
+    this.sessionId = opts.sessionId ?? null;
+    this.sessionCreated = opts.sessionCreated ?? new Date().toISOString();
+    this.title = opts.title ?? "";
+    this._saveSession = opts.saveSession ?? null; // (id, doc) => void
+  }
+
+  templateVars(extra = {}) {
+    return {
+      ...this.env.templateVars(),
+      task: this.task ?? "",
+      ...extra,
+    };
+  }
+
+  add(role, content, extra = {}) {
+    const msg = { role, content, extra };
+    this.messages.push(msg);
+    this.onEvent(msg);
+    this.save();
+    return msg;
+  }
+
+  /**
+   * Initialize a fresh conversation. If `task` is provided, seed the first user
+   * message with the task instructions (autonomous submission workflow).
+   */
+  start(task) {
+    if (task != null) this.task = task;
+    this.messages = [];
+    const vars = this.templateVars();
+    this.add("system", buildSystemPrompt(this.mode, vars));
+    if (this.task) {
+      this.add("user", render(INSTANCE_TEMPLATE, vars));
+    }
+  }
+
+  /** Add a user chat message (interactive follow-ups). */
+  addUserMessage(text) {
+    this.add("user", text);
+  }
+
+  apiMessages() {
+    return this.messages
+      .filter((m) => m.role === "system" || m.role === "user" || m.role === "assistant")
+      .map((m) => ({ role: m.role, content: m.content }));
+  }
+
+  /**
+   * Perform one turn: query the model, then optionally run one command.
+   * Returns one of:
+   *   { kind: "chat" }                          model replied with no command
+   *   { kind: "command", returncode }           command ran, loop can continue
+   *   { kind: "format_error" }                  unparseable (autonomous only)
+   *   { kind: "exit", exit_status, submission } run terminated
+   */
+  async runTurn() {
+    this.nSteps++;
+
+    // --- 1. Query model ---
+    let content, usage;
+    try {
+      ({ content, usage } = await this.model.query(this.apiMessages()));
+    } catch (err) {
+      if (err instanceof ContextWindowError) {
+        return this.exit("ContextWindowExceeded", "");
+      }
+      throw err;
+    }
+    this.lastContextTokens = usage.prompt_tokens;
+    this.add("assistant", content, { usage });
+
+    this.onStep({
+      step: this.nSteps,
+      stepLimit: this.stepLimit,
+      nCalls: this.model.nCalls,
+      usage,
+      contextTokens: usage.prompt_tokens,
+      totalPromptTokens: this.model.totalPromptTokens,
+      totalCompletionTokens: this.model.totalCompletionTokens,
+      costUsd: usage.cost_usd,
+      costKind: usage.cost_kind,
+      totalCostUsd: this.model.totalCostUsd,
+      elapsedSeconds: (Date.now() - this.startTime) / 1000,
+    });
+
+    // --- 2. Parse a command (may be absent) ---
+    let command;
+    try {
+      command = parseCommand(content);
+      this.formatErrorStreak = 0;
+    } catch (err) {
+      // In interactive mode, "no command" is a normal chat reply that yields
+      // the turn back to the user. Only autonomous mode treats it as an error.
+      if (this.mode === "interactive" && err.code === "NO_COMMAND") {
+        return { kind: "chat", content };
+      }
+      this.formatErrorStreak++;
+      if (
+        this.maxConsecutiveFormatErrors > 0 &&
+        this.formatErrorStreak >= this.maxConsecutiveFormatErrors
+      ) {
+        return this.exit("RepeatedFormatError", "");
+      }
+      this.add("user", render(FORMAT_ERROR, { error: err.message }));
+      return { kind: "format_error", error: err.message };
+    }
+
+    // --- 3. Approval gate ---
+    const allowed = await this.confirm(command);
+    if (!allowed) {
+      this.add("user", "The user declined to run that command. Suggest an alternative or ask what to do instead.", {
+        command,
+        declined: true,
+      });
+      return { kind: "declined", command };
+    }
+
+    // --- 4. Execute ---
+    const output = await this.env.execute(command);
+
+    // --- 5. Submission sentinel ---
+    const submitted = checkSubmitted(output);
+    if (submitted != null) {
+      return this.exit("Submitted", submitted);
+    }
+
+    // --- 6. Observation ---
+    const obs = render(OBSERVATION_TEMPLATE, {
+      exception: output.exception_info ? `<exception>${output.exception_info}</exception>\n` : "",
+      returncode: output.returncode,
+      output: output.output,
+    });
+    this.add("user", obs, { command, returncode: output.returncode });
+    return { kind: "command", command, returncode: output.returncode, output: output.output };
+  }
+
+  /**
+   * Autonomous driver: keep taking turns until the run terminates.
+   * Returns { exit_status, submission }.
+   */
+  async run(task) {
+    this.start(task);
+    let result;
+    while (true) {
+      if (this.stepLimit > 0 && this.nSteps >= this.stepLimit) {
+        result = this.exit("LimitsExceeded", "");
+        break;
+      }
+      const elapsed = (Date.now() - this.startTime) / 1000;
+      if (this.wallTimeLimitSeconds > 0 && elapsed >= this.wallTimeLimitSeconds) {
+        result = this.exit("TimeExceeded", "");
+        break;
+      }
+      const turn = await this.runTurn();
+      if (turn.kind === "exit") {
+        result = { exit_status: turn.exit_status, submission: turn.submission };
+        break;
+      }
+    }
+    this.save();
+    return result;
+  }
+
+  exit(status, submission) {
+    this.exitStatus = status;
+    this.add("exit", submission, { exit_status: status, submission });
+    return { kind: "exit", exit_status: status, submission };
+  }
+
+  serialize() {
+    return {
+      trajectory_format: "astra-1",
+      id: this.sessionId ?? undefined,
+      created: this.sessionCreated,
+      title: this.title || undefined,
+      info: {
+        mode: this.mode,
+        task: this.task || undefined,
+        exit_status: this.exitStatus ?? "",
+        submission: this.messages.findLast((m) => m.role === "exit")?.extra?.submission ?? "",
+        n_steps: this.nSteps,
+        model: this.model.model,
+        n_calls: this.model.nCalls,
+        elapsed_seconds: Math.round((Date.now() - this.startTime) / 1000),
+        tokens: {
+          prompt: this.model.totalPromptTokens,
+          completion: this.model.totalCompletionTokens,
+          total: this.model.totalPromptTokens + this.model.totalCompletionTokens,
+          last_context: this.lastContextTokens,
+        },
+        cost: {
+          usd: this.model.totalCostUsd,
+          source: this.model.costSource, // reported | estimated | mixed | null
+          reported_usd: this.model.reportedCostUsd,
+          estimated_usd: this.model.estimatedCostUsd,
+        },
+      },
+      messages: this.messages,
+    };
+  }
+
+  save() {
+    const doc = this.serialize();
+    if (this.sessionId && this._saveSession) {
+      this._saveSession(this.sessionId, doc);
+    }
+    if (!this.outputPath) return;
+    fs.mkdirSync(path.dirname(this.outputPath), { recursive: true });
+    fs.writeFileSync(this.outputPath, JSON.stringify(doc, null, 2));
+  }
+
+  /** Restore messages/counters from a previously serialized session. */
+  restore(data) {
+    this.messages = Array.isArray(data?.messages) ? data.messages : [];
+    this.task = data?.info?.task ?? this.task;
+    this.mode = data?.info?.mode === "interactive" ? "interactive" : this.mode;
+    this.nSteps = data?.info?.n_steps ?? this.messages.filter((m) => m.role === "assistant").length;
+    this.lastContextTokens = data?.info?.tokens?.last_context ?? 0;
+    this.exitStatus = data?.info?.exit_status || null;
+    if (data?.id) this.sessionId = data.id;
+    if (data?.created) this.sessionCreated = data.created;
+    if (data?.title) this.title = data.title;
+  }
+}
+
+/** Extract exactly one command from a fenced bash block. */
+export function parseCommand(text) {
+  // Accept ```bash, ```sh, or a bare ``` block.
+  const re = /```(?:bash|sh)?\s*\n([\s\S]*?)```/g;
+  const blocks = [];
+  let m;
+  while ((m = re.exec(text)) !== null) blocks.push(m[1].trim());
+
+  if (blocks.length === 0) {
+    const err = new Error("No bash code block found. Provide exactly one ```bash ... ``` block.");
+    err.code = "NO_COMMAND";
+    throw err;
+  }
+  if (blocks.length > 1) {
+    const err = new Error(`Found ${blocks.length} code blocks; provide exactly one.`);
+    err.code = "MULTI_COMMAND";
+    throw err;
+  }
+  const cmd = blocks[0].trim();
+  if (!cmd) {
+    const err = new Error("The bash code block was empty.");
+    err.code = "EMPTY_COMMAND";
+    throw err;
+  }
+  return cmd;
+}
+
+/** Returns the submission string if the sentinel was echoed, else null. */
+export function checkSubmitted(output) {
+  if (output.returncode !== 0) return null;
+  const lines = output.output.replace(/^\s+/, "").split(/\r?\n/);
+  if (lines[0]?.trim() === SUBMIT_SENTINEL) {
+    return lines.slice(1).join("\n");
+  }
+  return null;
+}