grab-url 0.9.132

package/readme.md

@@ -0,0 +1,156 @@
+
+<p align="center">
+    <img width="400px" src="https://i.imgur.com/qrQWkeb.png" />
+</p>
+<p align="center">
+    <br />
+    <a href="https://npmjs.org/package/grab-api.js">
+        <img src="https://i.imgur.com/ifE8SbX.png"
+            alt="NPM badge" />
+    </a>
+</p>
+<p align="center">
+  <a href="https://discord.gg/SJdBqBz3tV"><img src="https://img.shields.io/discord/1110227955554209923.svg?label=Chat&logo=Discord&colorB=7289da&style=flat" alt="Join Discord" /></a>
+     <a href="https://github.com/vtempest/grab-api/discussions">
+     <img alt="GitHub Stars" src="https://img.shields.io/github/stars/vtempest/grab-api" /></a>
+   <a href="https://npmjs.org/package/grab-api.js"><img alt="NPM Version" src="https://img.shields.io/npm/v/grab-api.js" /></a>
+    <a href="https://bundlephobia.com/package/grab-api.js"><img src="https://img.shields.io/bundlephobia/minzip/grab-api.js" /></a>
+    <a href="https://github.com/vtempest/grab-API/discussions"><img alt="GitHub Discussions"
+        src="https://img.shields.io/github/discussions/vtempest/grab-API" /></a>
+    <a href="https://github.blog/developer-skills/github/beginners-guide-to-github-creating-a-pull-request/"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome"/></a>
+    <a href="https://codespaces.new/vtempest/grab-API"><img src="https://github.com/codespaces/badge.svg" width="150" height="20"/></a>
+</p>
+<h3 align="center">
+  <a href="https://grab.js.org"> 📑 Docs (grab.js.org)</a>
+  <a href="https://grab.js.org/guide/Examples"> 🎯 Example Strategies </a>
+</h3>
+
+```
+npm i grab-api.js
+```
+
+### GRAB: Generate Request to API from Browser
+
+1.  **GRAB is the FBEST Request Manager: Functionally Brilliant, Elegantly Simple Tool**: One Function, no dependencies,   minimalist syntax, [more features than alternatives](https://grab.js.org/guide/Comparisons)
+2. **Auto-JSON Convert**: Pass parameters and get response or error in JSON, handling other data types as is.
+3. **isLoading Status**: Sets `.isLoading=true` on the pre-initialized response object so you can show a "Loading..." in any framework
+4. **Debug Logging**: Adds global `log()` and prints colored JSON structure, response, timing for requests in test.
+5. **Mock Server Support**: Configure `window.grab.mock` for development and testing environments
+6. **Cancel Duplicates**: Prevent this request if one is ongoing to same path & params, or cancel the ongoing request.
+7. **Timeout & Retry**: Customizable request timeout, default 30s, and auto-retry on error
+8. **DevTools**: `Ctrl+I` overlays webpage with devtools showing all requests and responses, timing, and JSON structure.
+9. **Request History**: Stores all request and response data in global `grab.log` object
+10. **Pagination Infinite Scroll**: Built-in pagination for infinite scroll to auto-load and merge next result page, with scroll position recovery.
+11. **Base URL Based on Environment**: Configure `grab.defaults.baseURL` once at the top, overide with `SERVER_API_URL` in `.env`.
+12. **Frontend Cache**: Set cache headers and retrieve from frontend memory for repeat requests to static data.
+13. **Regrab On Error**: Regrab on timeout error, or on window refocus, or on network change, or on stale data.
+14. **Framework Agnostic**: Alternatives like TanStack work only in component initialization and depend on React & others.
+15. **Globals**: Adds to window in browser or global in Node.js so you only import once: `grab()`, `log()`, `grab.log`, `grab.mock`, `grab.defaults`
+16. **TypeScript Tooltips**: Developers can hover over option names and autocomplete TypeScript.
+17. **Request Stategies**: [🎯 Examples](https://grab.js.org/guide/Examples) show common stategies like debounce, repeat, proxy, unit tests, interceptors, file upload, etc
+18. **Rate Limiting**: Built-in rate limiting to prevent multi-click cascading responses, require to wait seconds between requests.
+19. **Repeat**: Repeat request this many times, or repeat every X seconds to poll for updates.
+20. **Loading Icons**: Import from `grab-api.js/icons` to get enhanced animated loading icons.
+
+### Examples
+
+```ts
+import grab from 'grab-api.js';
+
+let res = $state({}) as {
+  results: Array<{title:string}>,
+  isLoading: boolean,
+  error: string,
+};
+
+await grab('search', {
+  response: res,
+  query: "search words",
+  post: true
+})
+ 
+grab('user').then(log)
+
+//in svelte component
+{#if res.results}
+    {res.results}
+{:else if res.isLoading}
+    ...
+{:else if res.error}
+    {res.error}
+{/if}
+
+ //Setup Mock testing server, response is object or function
+ window.grab.mock["search"] = {
+   response: (params) => {
+     return { results: [{title:`Result about ${params.query}`}] };
+   },
+   method: "POST"
+ };
+
+ //set defaults for all requests
+ grab("", { 
+   setDefaults: true,
+   baseURL: "http://localhost:8080",
+   timeout: 30,
+   debug: true,
+   rateLimit: 1,
+   cache: true,
+   cancelOngoingIfNew: true,
+ });
+
+ grab.defaults.baseURL = "http://localhost:8080/api/";
+```
+
+### Screenshots
+
+**Animated SVG Loading Icons with Customizable Colors**
+
+![icons](https://i.imgur.com/OqpWya1.gif)
+
+**Set Types for Tooltips on Request & Response**
+
+![types](https://i.imgur.com/IfR4OmC.png)
+
+**Debug Colorized log(JSON)**
+
+![Debug log](https://i.imgur.com/R8Qp6Vg.png)
+
+**Autocomplete option names**
+
+![Autocomplete](https://i.imgur.com/XlxILJ0.png)
+
+**Hover over options for info**
+
+![Info Tooltip](https://i.imgur.com/vV5jbZo.png)
+
+
+### Comparison of HTTP Request Libraries
+
+| Feature | [GRAB](https://github.com/vtempest/grab-api) | [Axios](https://github.com/axios/axios) | [TanStack Query](https://github.com/TanStack/query) | [SWR](https://github.com/vercel/swr) | [Alova](https://github.com/alovajs/alova) | [SuperAgent](https://github.com/ladjs/superagent) | [Apisauce](https://github.com/infinitered/apisauce) | [Ky](https://github.com/sindresorhus/ky) |
+| :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | 
+| Size | ✅ 3KB | ❌ 13KB | ❌ 39KB | ❌ 4.2KB | ⚠️ 4KB | ❌ 19KB | ❌ 15KB (with axios) | ⚠️ 4KB |
+| Zero Dependencies | ✅ Yes | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No | ❌ Needs Axios | ✅ Yes |
+| isLoading State Handling | ✅ Auto-managed | ❌ Manual | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Manual | ❌ Manual | ❌ Manual |
+| Auto JSON Handling | ✅ Automatic | ✅ Configurable | ❌ Manual | ❌ Manual | ✅ Automatic | ✅ Automatic | ✅ Automatic | ✅ Automatic |
+| Request Deduplication | ✅ Built-in | ❌ No | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ❌ No |
+| Caching | ✅ Multi-level | ❌ No | ✅ Advanced | ✅ Advanced | ✅ Multi-level | ❌ No | ❌ No | ❌ No |
+| Mock Testing | ✅ Easy setup | ❌ Needs MSW/etc | ❌ Needs MSW/etc | ❌ Needs MSW/etc | ⚠️ Basic | ❌ Needs separate lib | ❌ Needs separate lib | ❌ Needs MSW/etc |
+| Rate Limiting | ✅ Built-in | ❌ Manual | ❌ Manual | ❌ Manual | ⚠️ Basic | ❌ Manual | ❌ Manual | ❌ Manual |
+| Automatic Retry | ✅ Configurable | ⚠️ Via interceptors | ✅ Built-in | ✅ Built-in | ✅ Built-in | ✅ Built-in | ❌ Manual | ✅ Built-in |
+| Request Cancellation | ✅ Auto + manual | ✅ Manual | ✅ Automatic | ✅ Automatic | ✅ Manual | ✅ Manual | ✅ Manual | ✅ Manual |
+| Pagination Support | ✅ Infinite scroll | ❌ Manual | ✅ Advanced | ⚠️ Basic | ✅ Built-in | ❌ Manual | ❌ Manual | ❌ Manual |
+| Interceptors | ✅ Advanced | ✅ Advanced | ⚠️ Limited | ⚠️ Limited | ✅ Advanced | ✅ Plugins | ✅ Transforms | ✅ Hooks system |
+| Debug Logging | ✅ Colored output | ⚠️ Basic | ✅ DevTools | ✅ DevTools | ⚠️ Basic | ⚠️ Basic | ⚠️ Basic | ⚠️ Basic |
+| Request History | ✅ Built-in | ❌ Manual | ✅ DevTools | ✅ DevTools | ❌ Manual | ❌ Manual | ❌ Manual | ❌ Manual |
+| Easy Syntax | ✅ Minimal | ⚠️ Medium | ❌ High | ❌ High | ⚠️ Medium | ⚠️ Medium | ✅ Low | ✅ Minimal |
+
+
+**Stop trying to make fetch happen!** [*](https://knowyourmeme.com/memes/stop-trying-to-make-fetch-happen)
+
+**Why fetch things when you can just GRAB?**
+
+**Debugging requests is a bitch. [Make the switch!](https://grab.js.org/guide/Comparisons)**
+
+
+🌟 Star this repo so it will grow and get updates! 

package/src/grab-cli.js

@@ -0,0 +1,316 @@
+#!/usr/bin/env node
+import { grab, log } from './grab-api';
+import fs from 'fs';
+import path from 'path';
+
+// Lightweight argument parser polyfill
+class ArgParser {
+  constructor() {
+    this.commands = {};
+    this.options = {};
+    this.examples = [];
+    this.helpText = '';
+    this.versionText = '1.0.0';
+  }
+
+  usage(text) {
+    this.helpText = text;
+    return this;
+  }
+
+  command(pattern, desc, handler) {
+    const match = pattern.match(/\$0 <(\w+)>/);
+    if (match) {
+      this.commands[match[1]] = { desc, handler, required: true };
+    }
+    return this;
+  }
+
+  option(name, opts = {}) {
+    this.options[name] = opts;
+    return this;
+  }
+
+  example(cmd, desc) {
+    this.examples.push({ cmd, desc });
+    return this;
+  }
+
+  help() {
+    return this;
+  }
+
+  alias(short, long) {
+    if (this.options[long]) {
+      this.options[long].alias = short;
+    }
+    return this;
+  }
+
+  version(v) {
+    if (v) this.versionText = v;
+    return this;
+  }
+
+  strict() {
+    return this;
+  }
+
+  parseSync() {
+    const args = process.argv.slice(2);
+    const result = {};
+    const positional = [];
+    
+    // Check for help
+    if (args.includes('--help') || args.includes('-h')) {
+      this.showHelp();
+      process.exit(0);
+    }
+
+    // Check for version
+    if (args.includes('--version')) {
+      console.log(this.versionText);
+      process.exit(0);
+    }
+
+    for (let i = 0; i < args.length; i++) {
+      const arg = args[i];
+      
+      if (arg.startsWith('--')) {
+        const [key, value] = arg.split('=');
+        const optName = key.slice(2);
+        
+        if (value !== undefined) {
+          result[optName] = this.coerceValue(optName, value);
+        } else if (this.options[optName]?.type === 'boolean') {
+          result[optName] = true;
+        } else {
+          const nextArg = args[i + 1];
+          if (nextArg && !nextArg.startsWith('-')) {
+            result[optName] = this.coerceValue(optName, nextArg);
+            i++; // Skip next arg
+          } else {
+            result[optName] = true;
+          }
+        }
+      } else if (arg.startsWith('-') && arg.length === 2) {
+        const shortFlag = arg[1];
+        const longName = this.findLongName(shortFlag);
+        
+        if (longName) {
+          if (this.options[longName]?.type === 'boolean') {
+            result[longName] = true;
+          } else {
+            const nextArg = args[i + 1];
+            if (nextArg && !nextArg.startsWith('-')) {
+              result[longName] = this.coerceValue(longName, nextArg);
+              i++;
+            }
+          }
+        }
+      } else {
+        positional.push(arg);
+      }
+    }
+
+    // Handle positional arguments
+    if (positional.length > 0) {
+      result.url = positional[0];
+    }
+
+    // Set defaults
+    Object.keys(this.options).forEach(key => {
+      if (result[key] === undefined && this.options[key].default !== undefined) {
+        result[key] = this.options[key].default;
+      }
+    });
+
+    // Validate required positional args
+    if (!result.url && this.commands.url?.required) {
+      console.error('Error: Missing required argument: url');
+      this.showHelp();
+      process.exit(1);
+    }
+
+    return result;
+  }
+
+  coerceValue(optName, value) {
+    const opt = this.options[optName];
+    if (!opt) return value;
+
+    if (opt.coerce) {
+      return opt.coerce(value);
+    }
+
+    switch (opt.type) {
+      case 'number':
+        return Number(value);
+      case 'boolean':
+        return value === 'true' || value === '1';
+      default:
+        return value;
+    }
+  }
+
+  findLongName(shortFlag) {
+    return Object.keys(this.options).find(key => 
+      this.options[key].alias === shortFlag
+    );
+  }
+
+  showHelp() {
+    console.log(this.helpText || 'Usage: grab <url> [options]');
+    console.log('\nPositional arguments:');
+    Object.keys(this.commands).forEach(cmd => {
+      console.log(`  ${cmd.padEnd(20)} ${this.commands[cmd].desc}`);
+    });
+    
+    console.log('\nOptions:');
+    Object.keys(this.options).forEach(key => {
+      const opt = this.options[key];
+      const flags = opt.alias ? `-${opt.alias}, --${key}` : `--${key}`;
+      console.log(`  ${flags.padEnd(20)} ${opt.describe || ''}`);
+    });
+
+    if (this.examples.length > 0) {
+      console.log('\nExamples:');
+      this.examples.forEach(ex => {
+        console.log(`  ${ex.cmd}`);
+        console.log(`    ${ex.desc}`);
+      });
+    }
+  }
+}
+
+// Create parser instance
+const createParser = () => new ArgParser();
+
+// Parse arguments with custom parser
+const argv = createParser()
+  .usage('Usage: grab <url> [options]')
+  .command('$0 <url>', 'Fetch data from API endpoint')
+  .option('x', {
+    alias: 'exec',
+    type: 'boolean',
+    default: false,
+    describe: 'Execute flag (functionality TBD)'
+  })
+  .option('no-save', {
+    type: 'boolean',
+    default: false,
+    describe: 'Don\'t save output to file, just print to console'
+  })
+  .option('output', {
+    alias: 'o',
+    type: 'string',
+    describe: 'Output filename (default: output.json)',
+    default: null
+  })
+  .option('params', {
+    alias: 'p',
+    type: 'string',
+    describe: 'JSON string of query parameters (e.g., \'{"key":"value"}\')',
+    coerce: (arg) => {
+      if (!arg) return {};
+      try {
+        return JSON.parse(arg);
+      } catch (e) {
+        throw new Error(`Invalid JSON in params: ${arg}`);
+      }
+    }
+  })
+  .example('grab https://api.example.com/data', 'Fetch data and save to output.json')
+  .example('grab https://api.example.com/data --no-save', 'Fetch data and print to console')
+  .example('grab https://api.example.com/data -o result.json', 'Save output to result.json')
+  .example('grab https://api.example.com/data -p \'{"limit":10,"page":1}\'', 'Pass query parameters')
+  .help()
+  .alias('h', 'help')
+  .version()
+  .strict()
+  .parseSync();
+
+// Extract values from parsed arguments
+const { url, x: execFlag, 'no-save': noSave, output: outputFile, params = {} } = argv;
+// params.debug = true;
+
+// Validate URL
+if (!url || (!url.startsWith('http://') && !url.startsWith('https://'))) {
+  log('Error: URL must start with http:// or https://');
+  process.exit(1);
+}
+
+(async () => {
+  const startTime = process.hrtime(); // High-res timer
+  
+  try {
+    const res = await grab(url, params);
+    
+    if (res.error) 
+      log(`\n\nStatus: ❌ ${res.error}`);
+    
+    
+    let filePath = null;
+    let outputData;
+    let isTextData = false;
+    
+    // Determine data type and prepare output
+    if (typeof res.data === 'string') {
+      outputData = res.data;
+      isTextData = true;
+    } else if (Buffer.isBuffer(res.data) || res.data instanceof Uint8Array) {
+      // Binary data (like video files)
+      outputData = res.data;
+      isTextData = false;
+    } else if (res.data instanceof Blob) {
+      // Convert Blob to Buffer for file writing
+      const arrayBuffer = await res.data.arrayBuffer();
+      outputData = Buffer.from(arrayBuffer);
+      isTextData = false;
+    } else if (res.data && typeof res.data === 'object') {
+      // JSON or other objects
+      outputData = JSON.stringify(res.data, null, 2);
+      isTextData = true;
+    } else {
+      // Fallback - try to stringify
+      outputData = String(res.data);
+      isTextData = true;
+    }
+    
+    if (!noSave) {
+      // Determine file extension based on URL or data type
+      const urlPath = new URL(url).pathname;
+      const urlExt = path.extname(urlPath);
+      const defaultExt = isTextData ? '.json' : (urlExt || '.bin');
+      
+      filePath = outputFile
+        ? path.resolve(outputFile)
+        : path.resolve(process.cwd(), `output${defaultExt}`);
+      
+      // Write file with appropriate encoding
+      if (isTextData) {
+        fs.writeFileSync(filePath, outputData, 'utf8');
+      } else {
+        fs.writeFileSync(filePath, outputData);
+      }
+      
+      // Calculate elapsed time
+      const [seconds, nanoseconds] = process.hrtime(startTime);
+      const elapsedMs = (seconds + nanoseconds / 1e9).toFixed(2);
+      const stats = fs.statSync(filePath);
+      const fileSizeMB = (stats.size / (1024 * 1024)).toFixed(1);
+      
+      log(`⏱️ ${elapsedMs}s 📦 ${fileSizeMB}MB ✅ Saved to: ${filePath}`);
+    } else {
+      // For console output, only show text data
+      if (isTextData) {
+        // log(outputData);
+      } else {
+        log(`Binary data received (${outputData.length} bytes). Use --output to save to file.`);
+      }
+    }
+  } catch (error) {
+    log(`Error: ${error.message}`, {color: 'red'});
+    process.exit(1);
+  }
+})();