website-api 1.0.1 → 1.0.3

package/README.md

@@ -1,241 +1,3 @@
 # website-api
 
-A TypeScript CLI and library published as `website-api` that uses [chrome-tools](https://github.com/user/chrome-tools) to extract decrypted Chrome cookies on macOS, then queries private/internal website APIs — no manual cookie copy-pasting needed.
-
----
-
-## Features
-
-- 🔑 **Keychain Decryption** — Leverages `chrome-tools` to decrypt Chrome cookies via the macOS Keychain.
-- 📂 **Auto-Loading Adapters** — Drop a folder into `src/website/`, and the adapter is auto-discovered at runtime.
-- 🧩 **Zero-Boilerplate Adapters** — The `defineAdapter()` factory handles cookies, user-agent, fetch, and error handling. Simple adapters are ~10 lines.
-- 👥 **Multi-Profile Support** — Target specific Chrome profiles (e.g., `Default`, `Profile 1`) or custom user data paths.
-- 🚀 **Dual Interface** — Works as both an importable ES module library and a terminal CLI.
-
----
-
-## Supported Website APIs
-
-| ID | Name | Domain | Description |
-|----|------|--------|-------------|
-| `codex-usage` | ChatGPT / Codex Usage | chatgpt.com | Fetches ChatGPT rate limit usage and quota details from the private wham/usage API. |
-| `cursor-usage` | Cursor Usage | cursor.com | Fetches the active Cursor usage summary from the private usage-summary API. |
-| `ollama-usage` | Ollama Usage | ollama.com | Fetches Ollama plan and usage details from the authenticated settings page. |
-
-Each adapter has a `request.md` file documenting the exact HTTP requests and expected responses.
-
----
-
-## Installation & Setup
-
-### Prerequisites
-
-- macOS with Google Chrome installed
-- Node.js ≥ 18
-- [pnpm](https://pnpm.io/) package manager
-
-### Install
-
-```bash
-pnpm install
-pnpm run build
-```
-
-### Link CLI globally (optional)
-
-```bash
-pnpm link --global
-website-api --help
-```
-
----
-
-## CLI Usage
-
-```bash
-# Show help
-node dist/bin/cli.js --help
-
-# List all supported adapters
-node dist/bin/cli.js list
-
-# Query a website API
-node dist/bin/cli.js codex-usage
-node dist/bin/cli.js cursor-usage
-node dist/bin/cli.js ollama-usage
-
-# Domain aliases still work too
-node dist/bin/cli.js chatgpt.com
-node dist/bin/cli.js cursor.com
-node dist/bin/cli.js ollama.com
-```
-
-### Options
-
-| Option | Description |
-|--------|-------------|
-| `--profile <name>` | Target a specific Chrome profile (e.g., `Default`, `Profile 1`) |
-| `--current-profile` | Show the currently resolved Chrome profile directory and name |
-| `-u, --user-agent <string>` | Custom User-Agent header for HTTP requests |
-
-### Universal Fallback
-
-Any URL or domain not matching a registered adapter will be fetched directly with Chrome cookies:
-
-```bash
-node dist/bin/cli.js https://example.com/api/data
-```
-
----
-
-## Library Usage (Programmatic API)
-
-```typescript
-import { queryWebsite } from "website-api";
-
-const data = await queryWebsite("chatgpt.com", {
-  profile: "Default",
-});
-
-// Command IDs also work:
-// await queryWebsite("codex-usage", { profile: "Default" });
-
-console.log(JSON.stringify(data, null, 2));
-```
-
----
-
-## How to Add a New Website Adapter
-
-### Step 1: Create the adapter folder
-
-```
-src/website/github.com/
-├── github-adapter.ts   ← adapter code
-└── request.md          ← document the endpoint (required)
-```
-
-### Step 2: Write the adapter
-
-For **simple single-endpoint** APIs — just declare the endpoint:
-
-```typescript
-// src/website/github.com/github-adapter.ts
-import { defineAdapter } from "../../base-adapter.js";
-
-export default defineAdapter({
-  id: "github.com",
-  name: "GitHub Notifications",
-  domain: "github.com",
-  description: "Fetches unread notifications from the GitHub API.",
-  endpoints: [{ url: "https://api.github.com/notifications" }],
-});
-```
-
-For **multi-step flows** (e.g., session token → API call) — override `fetchData`:
-
-```typescript
-// src/website/example.com/example-adapter.ts
-import { defineAdapter } from "../../base-adapter.js";
-
-export default defineAdapter({
-  id: "example.com",
-  name: "Example Service",
-  domain: "example.com",
-  description: "Fetches data with a two-step auth flow.",
-
-  async fetchData(cookies, options) {
-    const cookieStr = this.buildCookieString(cookies);  // shared helper
-    const ua = this.resolveUserAgent(options);           // shared helper
-
-    // Step 1: get token
-    const session = await this.fetchJson("https://example.com/auth", {
-      headers: { Cookie: cookieStr, "User-Agent": ua },
-    });
-
-    // Step 2: use token
-    return this.fetchJson("https://example.com/api/data", {
-      headers: { Authorization: `Bearer ${session.token}` },
-    });
-  },
-});
-```
-
-### Step 3: Document the endpoint
-
-Create `request.md` in the adapter folder documenting the HTTP requests, headers, and example responses. See [chatgpt.com/request.md](src/website/chatgpt.com/request.md) for reference.
-
-### Step 4: Build and test
-
-```bash
-pnpm run build
-node dist/bin/cli.js list          # verify it appears
-node dist/bin/cli.js github.com    # test it
-```
-
-That's it — **no registration needed**. The adapter is auto-discovered from the `website/` directory at runtime.
-
----
-
-## Shared Adapter Helpers
-
-Every adapter created with `defineAdapter()` automatically gets these methods via `this`:
-
-| Method | Description |
-|--------|-------------|
-| `this.buildCookieString(cookies)` | Converts cookies array to `"name=value; ..."` header string |
-| `this.resolveUserAgent(options)` | Resolves User-Agent from options → `.env` → default |
-| `this.fetchJson(url, init?)` | Fetch with error handling + automatic JSON parsing |
-
----
-
-## Configuration
-
-Create a `.env` file in the project root:
-
-```env
-# Custom User-Agent (defaults to Chrome 148 on macOS)
-userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36"
-
-# Custom Chrome profile path (defaults to standard macOS location)
-PROFILE_PATH="/Users/yourname/chrome_profile/chrome"
-PROFILE_NAME="Default"
-```
-
----
-
-## Project Structure
-
-```
-├── src/
-│   ├── website-api.ts          # Main entry — auto-loads adapters, queryWebsite()
-│   ├── base-adapter.ts         # defineAdapter() factory + shared helpers
-│   ├── types.ts                # TypeScript interfaces
-│   ├── env.ts                  # .env file loader
-│   ├── universal-adapter.ts    # Fallback for unregistered domains
-│   └── website/
-│       ├── chatgpt.com/
-│       │   ├── chatgpt-adapter.ts
-│       │   └── request.md
-│       ├── cursor.com/
-│           ├── cursor-adapter.ts
-│           └── request.md
-│       └── ollama.com/
-│           ├── ollama-adapter.ts
-│           └── request.md
-├── bin/
-│   └── cli.ts                  # CLI entry point
-├── tsconfig.json
-├── package.json
-└── .env                        # Local config (gitignored)
-```
-
----
-
-## Security Warning
-
-This utility is designed strictly for local development and authorized administrative use. macOS will request explicit Keychain authorization before allowing `chrome-tools` to access the Chrome Safe Storage Password. Only run this tool in trusted environments.
-
-## License
-
-MIT License
+CLI and library to fetch website API data

package/dist/src/base-adapter.js

@@ -1 +1 @@
-const e={buildCookieString:e=>e.map(e=>`${e.name}=${e.value}`).join("; "),resolveUserAgent:e=>e.userAgent||process.env.userAgent||process.env.USER_AGENT||"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36",async fetchJson(e,t){const n=await fetch(e,t);if(!n.ok)throw new Error(`HTTP ${n.status}: ${n.statusText}`);const s=await n.text();try{return JSON.parse(s)}catch{return{response:s}}}};async function t(e,t){if(!this.endpoints||0===this.endpoints.length)throw new Error(`Adapter "${this.id}" has no endpoints defined and no fetchData override`);const n=this.endpoints[0],s=this.buildCookieString(e),o=this.resolveUserAgent(t);return this.fetchJson(n.url,{method:n.method||"GET",headers:{Cookie:s,"User-Agent":o,Accept:"application/json, text/plain, */*",...n.headers}})}export function defineAdapter(n){const s={...n,...e,fetchData:n.fetchData??t};return s.fetchData=s.fetchData.bind(s),s}
+const t={buildCookieString:t=>t.map(t=>`${t.name}=${t.value}`).join("; "),resolveUserAgent:t=>t.userAgent||process.env.userAgent||process.env.USER_AGENT||"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36",async fetchJson(t,e){const n=await fetch(t,e);if(!n.ok)throw new Error(`HTTP ${n.status}: ${n.statusText}`);const r=await n.text();try{return JSON.parse(r)}catch{return{response:r}}},async fetchText(t,e){const n=await fetch(t,e);if(!n.ok)throw new Error(`HTTP ${n.status}: ${n.statusText}`);return n.text()},async fetchHtml(t,e){return this.fetchText(t,e)}};async function e(t,e){if(!this.endpoints||0===this.endpoints.length)throw new Error(`Adapter "${this.id}" has no endpoints defined and no fetchData override`);const n=this.endpoints[0],r=this.buildCookieString(t),s=this.resolveUserAgent(e),o=await async function(t,e,n){const r=await fetch(t.url,{method:t.method||"GET",headers:{Cookie:e,"User-Agent":n,Accept:"html"===t.responseType?"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8":"text"===t.responseType?"text/plain,*/*;q=0.8":"application/json, text/plain, */*",...t.headers}});if(!r.ok)throw new Error(`HTTP ${r.status}: ${r.statusText}`);const s=await r.text(),o=r.headers.get("content-type")?.toLowerCase()??"";if("json"===t.responseType||"text"!==t.responseType&&"html"!==t.responseType&&(o.includes("application/json")||o.includes("+json")))try{return JSON.parse(s)}catch(e){throw new Error(`Expected JSON from ${t.url}, but received invalid JSON: ${e instanceof Error?e.message:String(e)}`)}return s}(n,r,s);return n.transform?n.transform.call(this,o,t,e):o}export function defineAdapter(n){const r={...n,...t,fetchData:n.fetchData??e};return r.fetchData=r.fetchData.bind(r),r}

package/dist/src/types.d.ts

@@ -9,6 +9,13 @@ export interface Endpoint {
     method?: string;
     /** Additional headers to include in the request. */
     headers?: Record<string, string>;
+    /**
+     * Expected response body type. Defaults to auto-detect from content-type.
+     * Use "html" for text/html responses that need parsing.
+     */
+    responseType?: "auto" | "json" | "text" | "html";
+    /** Optional post-processing step for the parsed response body. */
+    transform?: (this: WebsiteAdapter, body: unknown, cookies: CookieEntry[], options: QueryOptions) => Promise<unknown> | unknown;
 }
 /**
  * Configuration for defining a website adapter.
@@ -39,6 +46,10 @@ export interface WebsiteAdapter extends AdapterConfig {
     resolveUserAgent(options: QueryOptions): string;
     /** Fetches a URL and returns parsed JSON, with error handling. */
     fetchJson(url: string, init?: RequestInit): Promise<any>;
+    /** Fetches a URL and returns the raw response text, with error handling. */
+    fetchText(url: string, init?: RequestInit): Promise<string>;
+    /** Fetches an HTML document as text, with error handling. */
+    fetchHtml(url: string, init?: RequestInit): Promise<string>;
 }
 /**
  * Options passed when querying a website API.

package/dist/src/website/ollama.com/ollama-adapter.js

@@ -1 +1 @@
-import{defineAdapter as e}from"../../base-adapter.js";export default e({id:"ollama-usage",name:"Ollama Usage",domain:"ollama.com",description:"Fetches Ollama plan and usage details from the authenticated settings page.",async fetchData(e,a){const n=this.buildCookieString(e),i=this.resolveUserAgent(a),o=await fetch("https://ollama.com/settings",{method:"GET",headers:{Cookie:n,"User-Agent":i,Accept:"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"}});if(!o.ok)throw new Error(`HTTP ${o.status}: ${o.statusText}`);const r=await o.text(),l=s(r,"Session usage"),m=s(r,"Weekly usage");return{time:(new Date).toISOString(),Plan:t(r),"Session Usage":l.usage,"Session Reset":l.reset,"Weekly Usage":m.usage,"Weekly Reset":m.reset}}});function t(e){const t=e.match(/Cloud Usage[\s\S]*?<\/span>[\s\S]*?<span[^>]*>([\s\S]*?)<\/span/i);return t?.[1]?.trim()??"unknown"}function s(e,t){const s=t.replace(/[.*+?^${}()|[\]\\]/g,"\\$&");const a=new RegExp(`<div[\\s\\S]*?<span[^>]*>\\s*${s}\\s*<\\/span>[\\s\\S]*?aria-label="${s}\\s+([^"]+)"[\\s\\S]*?data-time="([^"]+)"`,"i"),n=e.match(a);return{usage:n?.[1]?.replace(/\s+used$/i,"").trim()??"unknown",reset:n?.[2]?.trim()??"unknown"}}
+import{defineAdapter as e}from"../../base-adapter.js";export default e({id:"ollama-usage",name:"Ollama Usage",domain:"ollama.com",description:"Fetches Ollama plan and usage details from the authenticated settings page.",endpoints:[{url:"https://ollama.com/settings",responseType:"html",headers:{Accept:"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},transform(e){const t="string"==typeof e?e:String(e),n=a(t,"Session usage"),i=a(t,"Weekly usage");return{time:(new Date).toISOString(),Plan:s(t),"Session Usage":n.usage,"Session Reset":n.reset,"Weekly Usage":i.usage,"Weekly Reset":i.reset}}}]});function s(e){const s=e.match(/Cloud Usage[\s\S]*?<\/span>[\s\S]*?<span[^>]*>([\s\S]*?)<\/span/i);return s?.[1]?.trim()??"unknown"}function a(e,s){const a=s.replace(/[.*+?^${}()|[\]\\]/g,"\\$&");const t=new RegExp(`<div[\\s\\S]*?<span[^>]*>\\s*${a}\\s*<\\/span>[\\s\\S]*?aria-label="${a}\\s+([^"]+)"[\\s\\S]*?data-time="([^"]+)"`,"i"),n=e.match(t);return{usage:n?.[1]?.replace(/\s+used$/i,"").trim()??"unknown",reset:n?.[2]?.trim()??"unknown"}}

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "website-api",
-  "version": "1.0.1",
-  "description": "CLI and library to fetch website API data using chrome-tools decrypted cookies on macOS",
+  "version": "1.0.3",
+  "description": "CLI and library to fetch website API data",
   "main": "./dist/src/website-api.js",
   "type": "module",
   "bin": {