nosible 0.1.5

package/README.md

@@ -0,0 +1,438 @@
+[![NPM Version](https://img.shields.io/npm/v/nosible.svg?label=NPM&logo=npm)](https://www.npmjs.com/package/nosible)
+[![GitHub Tests](https://img.shields.io/github/actions/workflow/status/NosibleAI/nosible-js/test.yml?branch=main&label=Tests)](https://github.com/NosibleAI/nosible-js/actions)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg?logo=typescript)](https://www.typescriptlang.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+# NOSIBLE Search Client
+
+A high-level TypeScript/JavaScript client for the [NOSIBLE Search API](https://www.nosible.ai/search/v2/docs/#/).
+Easily integrate the Nosible Search API into your TypeScript and JavaScript projects.
+
+## 📄 Documentation
+
+You can find the full NOSIBLE Search Client documentation and examples in the [website/docs](./website/docs) directory.
+
+## 📦 Installation
+
+```bash
+npm install nosible
+```
+
+## ⚡ Installing with other package managers
+
+```bash
+# Using yarn
+yarn add nosible
+
+# Using bun
+bun add nosible
+```
+
+**Requirements**:
+
+- Node.js 18+
+- TypeScript 5+ (for TypeScript projects)
+
+**Dependencies**:
+
+- duckdb
+- nodejs-polars
+- apache-arrow
+- openai
+- lunr
+- papaparse
+- zod
+- bottleneck
+- simple-statistics
+
+## 🔑 Authentication
+
+1. Sign in to [NOSIBLE.AI](https://www.nosible.ai/) and grab your free API key.
+2. Set it as an environment variable or pass directly:
+
+### On Windows
+
+```powershell
+$Env:NOSIBLE_API_KEY="basic|abcd1234..."
+$Env:LLM_API_KEY="sk-..."  # for query expansions (optional)
+```
+
+### On macOS/Linux
+
+```bash
+export NOSIBLE_API_KEY="basic|abcd1234..."
+export LLM_API_KEY="sk-..."  # for query expansions (optional)
+```
+
+### Or in code
+
+As an argument:
+
+```typescript
+import {NosibleClient} from "nosible";
+
+const client = new NosibleClient({
+  apiKey: "basic|abcd1234...",
+  llmApiKey: "sk-...", // optional, for LLM features
+});
+```
+
+As an environment variable:
+
+```typescript
+import {NosibleClient} from "nosible";
+
+process.env.NOSIBLE_API_KEY = "basic|abcd1234...";
+process.env.LLM_API_KEY = "sk-...";
+
+const client = new NosibleClient();
+```
+
+## 🔍 Your first search
+
+To complete your first search:
+
+```typescript
+import {NosibleClient} from "nosible";
+
+const client = new NosibleClient({apiKey: "YOUR API KEY"});
+
+const results = await client.fastSearch({
+  question: "What is Artificial General Intelligence?",
+  nResults: 20,
+});
+
+console.log(results);
+```
+
+## 🤖 AI-Powered Search
+
+An AI agent with access to advanced search capabilities in NOSIBLE. It can understand complex queries and provide contextually relevant results.
+
+```typescript
+import {NosibleClient} from "nosible";
+
+const client = new NosibleClient({apiKey: "YOUR API KEY"});
+
+const results = await client.aiSearch({
+  // aiSearch() gives you access to AI-powered search
+  prompt: "Find me interesting technical blogs about Monte Carlo Tree Search.",
+  agent: "research",
+});
+
+console.log(results);
+```
+
+## ✨ Features
+
+- 🚀 **Fast Search** - Quick searches optimized for speed (10-100 results)
+- 🤖 **AI Search** - Advanced AI-powered searches with custom instructions
+- 📦 **Bulk Search** - Large-scale data retrieval (1000-10000 results)
+- 🔄 **Parallel Searches** - Execute multiple searches concurrently
+- 🌐 **URL Scraping** - Extract structured content from web pages
+- 📈 **Topic Trends** - Analyze topic popularity over time
+- 🎯 **Advanced Filtering** - SQL filters, date ranges, geographic filters, and more
+- ✅ **Type Safe** - Full TypeScript support with IntelliSense
+- 🔒 **Secure** - API key authentication
+
+## 📖 Usage Examples
+
+### Fast Search
+
+Quick searches with configurable result limits (10-100 results):
+
+```typescript
+import {NosibleClient} from "nosible";
+
+const client = new NosibleClient();
+
+// Basic fast search
+const results = await client.fastSearch({
+  question: "latest AI technology trends",
+  nResults: 20,
+});
+
+console.log(`Found ${results.length} results`);
+for (const result of results) {
+  console.log(`Title: ${result.title}`);
+  console.log(`URL: ${result.url}`);
+  console.log(`Similarity: ${result.similarity}`);
+  console.log(`Published: ${result.published}`);
+}
+```
+
+### Advanced Fast Search with Filters
+
+```typescript
+// Search with advanced filtering options
+const filteredResults = await client.fastSearch({
+  question: "machine learning startups",
+  nResults: 50,
+  algorithm: "hybrid-2",
+  minSimilarity: 0.7,
+  mustInclude: ["artificial intelligence", "funding"],
+  mustExclude: ["cryptocurrency"],
+  continent: "North America",
+  brandSafety: "high",
+});
+```
+
+### AI-Powered Search
+
+Advanced searches with custom instructions:
+
+```typescript
+// AI search with custom instructions
+const aiResults = await client.aiSearch({
+  prompt:
+    "What are the environmental impacts of renewable energy? Focus on recent studies and quantitative data from the last 2 years.",
+  agent: "research",
+});
+
+console.log(aiResults);
+```
+
+### Parallel Searches
+
+Execute multiple searches concurrently:
+
+```typescript
+// Search multiple topics in parallel
+const results = await client.fastSearches({
+  questions: ["AI in healthcare", "AI in finance", "AI in education"],
+  nResults: 20,
+  algorithm: "hybrid-2",
+});
+
+// Each result is a separate ResultSet
+results.forEach((resultSet, index) => {
+  console.log(`Query ${index + 1}: ${resultSet.length} results`);
+});
+```
+
+### Using the Search Class
+
+Create reusable search configurations:
+
+```typescript
+import {NosibleClient, Search} from "nosible";
+
+const client = new NosibleClient();
+
+// Create a search with specific parameters
+const search = new Search({
+  question: "quantum computing breakthroughs",
+  nResults: 30,
+  algorithm: "hybrid-3",
+  continent: "Europe",
+  publishStart: new Date("2024-01-01"),
+  publishEnd: new Date("2025-01-01"),
+});
+
+// Execute the search
+const results = await client.fastSearch({search});
+```
+
+### Bulk Search
+
+Large-scale data collection (1000-10000 results):
+
+```typescript
+// Bulk search for comprehensive data gathering
+const bulkResults = await client.bulkSearch({
+  question: "all articles about artificial intelligence",
+  nResults: 5000,
+  algorithm: "hybrid-2",
+  minSimilarity: 0.6,
+});
+
+console.log(`Retrieved ${bulkResults.length} results`);
+
+// Export to JSON for analysis
+const data = bulkResults.toJSON();
+```
+
+### URL Scraping
+
+Extract structured content from web pages:
+
+```typescript
+// Scrape a specific URL
+const webpage = await client.scrapeUrl("https://example.com/article");
+
+console.log(`Title: ${webpage.title}`);
+console.log(`Author: ${webpage.author}`);
+console.log(`Published: ${webpage.published}`);
+console.log(`Content: ${webpage.content}`);
+
+// Scrape search results
+const searchResults = await client.fastSearch({
+  question: "latest tech news",
+  nResults: 5,
+});
+
+for (const result of searchResults) {
+  const fullArticle = await client.scrapeUrl(result.url);
+  console.log(`Full article: ${fullArticle.content}`);
+}
+```
+
+### Topic Trends Analysis
+
+Analyze topic popularity over time:
+
+```typescript
+// Analyze trends for a topic
+const trends = await client.topicTrend("artificial intelligence");
+
+console.log(`Topic: ${trends.query}`);
+console.log(`Data points: ${trends.data.length}`);
+
+trends.data.forEach((point) => {
+  console.log(`Date: ${point.date}, Popularity: ${point.popularity}`);
+});
+
+// Trend analysis with filtering
+const filteredTrends = await client.topicTrend(
+  "machine learning",
+  'continent = "North America" AND published >= "2023-01-01"'
+);
+```
+
+### Advanced Filtering with SQL Filters
+
+```typescript
+// Search with date range filters
+const recentResults = await client.fastSearch({
+  question: "climate change solutions",
+  nResults: 50,
+  publishStart: new Date("2024-01-01"),
+  publishEnd: new Date("2025-01-01"),
+  certain: true, // Only verified/certain results
+  includeNetlocs: ["nature.com", "sciencedaily.com"],
+  excludeNetlocs: ["example.com"],
+});
+
+// Search specific companies
+const companyResults = await client.fastSearch({
+  question: "product launches",
+  nResults: 30,
+  companies: ["Apple", "Google", "Microsoft"],
+  industry: "Technology",
+});
+```
+
+### Client Configuration with Defaults
+
+```typescript
+// Initialize client with default search parameters
+const client = new NosibleClient({
+  apiKey: "your-api-key",
+  searchDefaults: {
+    nResults: 50,
+    algorithm: "hybrid-2",
+    continent: "North America",
+    brandSafety: "high",
+  },
+});
+
+// All searches will use these defaults unless overridden
+const results = await client.fastSearch({
+  question: "tech startups",
+  // Will use nResults: 50, algorithm: 'hybrid-2', etc.
+});
+```
+
+### Working with ResultSet
+
+```typescript
+// ResultSet provides enhanced functionality
+const results = await client.fastSearch({
+  question: "renewable energy",
+  nResults: 100,
+});
+
+// Access results as array
+console.log(`Total: ${results.length}`);
+
+// Iterate through results
+for (const result of results) {
+  console.log(result.title);
+}
+
+// Convert to JSON for export
+const jsonData = results.toJSON();
+
+// Filter and sort
+const highQuality = results.filter((r) => r.similarity > 0.8);
+```
+
+## 🛠️ Development
+
+Contributing to this package:
+
+```bash
+# Install dependencies
+bun install
+
+# Run tests
+bun test
+
+# Build the package
+bun run build
+
+# Generate documentation
+bun run docs:generate
+
+# Start documentation dev server
+bun run docs:dev
+```
+
+## 🏗️ Project Structure
+
+```text
+nosible-js/
+├── src/
+│   ├── api/            # API client and schemas
+│   ├── client.ts       # Main NosibleClient class
+│   ├── search/         # Search functionality and classes
+│   ├── scrape/         # URL scraping functionality
+│   ├── topicTrend/     # Topic trend analysis
+│   ├── utils/          # Utility functions
+│   └── index.ts        # Package exports
+├── website/            # Docusaurus documentation site
+│   ├── docs/           # Documentation pages
+│   └── src/            # Custom pages and components
+├── dist/               # Built package (generated)
+└── package.json        # Package configuration
+```
+
+## 📚 Additional Examples
+
+For more detailed examples and guides:
+
+- [Getting Started Guide](./website/docs/getting-started.md)
+- [Examples](./website/docs/examples.md)
+- [API Reference](./website/docs/api/) (auto-generated with TypeDoc)
+
+Run the documentation site locally:
+
+```bash
+bun run docs:dev
+```
+
+## 🤝 Contributing
+
+Contributions are welcome! Please ensure tests pass and documentation is updated.
+
+## 📡 API Reference
+
+View the complete [NOSIBLE Search API Swagger Documentation](https://www.nosible.ai/search/v2/docs/#/) for detailed endpoint information.
+
+---
+
+© 2025 Nosible Inc. | [Privacy Policy](https://www.nosible.ai/privacy) | [Terms](https://www.nosible.ai/terms)
+
+## 📄 License
+
+MIT