@inference-gateway/sdk 0.1.4

package/CHANGELOG.md

@@ -0,0 +1,56 @@
+## [0.1.4](https://github.com/inference-gateway/typescript-sdk/compare/v0.1.3...v0.1.4) (2025-01-23)
+
+### 🔧 Miscellaneous
+
+* **release:** bump version to 0.1.2 in package.json ([ae64176](https://github.com/inference-gateway/typescript-sdk/commit/ae641767f3ba44edef0e9073d42421c2df05f36b))
+
+## [0.1.3](https://github.com/inference-gateway/typescript-sdk/compare/v0.1.2...v0.1.3) (2025-01-23)
+
+### 🐛 Bug Fixes
+
+* Update release workflow environment variable and package.json repository URL format ([8ea1290](https://github.com/inference-gateway/typescript-sdk/commit/8ea1290ed6e2c122cbce7c311478e9814d09e36d))
+
+## [0.1.2](https://github.com/inference-gateway/typescript-sdk/compare/v0.1.1...v0.1.2) (2025-01-23)
+
+### 👷 CI
+
+* Update permissions in release workflow for issues and pull requests ([ae1a835](https://github.com/inference-gateway/typescript-sdk/commit/ae1a83586b211a7b468fa2fc1b07f30eb02effb2))
+
+## [0.1.1](https://github.com/inference-gateway/typescript-sdk/compare/v0.1.0...v0.1.1) (2025-01-23)
+
+### ♻️ Improvements
+
+* Refactor imports and update TypeScript configuration for improved module resolution and testing ([f74b6b1](https://github.com/inference-gateway/typescript-sdk/commit/f74b6b1dbc7371da01991ba832120c92b36d9c91))
+
+### 🐛 Bug Fixes
+
+* Update tag format in .releaserc.yaml and add npm plugin ([6e55661](https://github.com/inference-gateway/typescript-sdk/commit/6e5566147c05e5ace4306197cc5250cca0e5a948))
+
+### 👷 CI
+
+* Update Node.js version in CI workflow from 20.x to 22.x ([1ecf62a](https://github.com/inference-gateway/typescript-sdk/commit/1ecf62ab2af9787cbf9ca02fb84377d5c1a08255))
+* Update Node.js version to 22.x and add global dependencies for semantic release ([0888fae](https://github.com/inference-gateway/typescript-sdk/commit/0888fae0c4a98a879808dc367e83e15d236dabab))
+
+### 🔧 Miscellaneous
+
+* Add @semantic-release/npm to Dockerfile and release workflow ([8b94e8c](https://github.com/inference-gateway/typescript-sdk/commit/8b94e8c59f705d3c7e79e29275854dbd1ad21010))
+* Add job names to CI and release workflows ([e053535](https://github.com/inference-gateway/typescript-sdk/commit/e05353554c1eb62b7f0fd6b20ac8f8c75ec0685b))
+* Bump OS version to the latest with updated version of NodeJS and NPM ([8739216](https://github.com/inference-gateway/typescript-sdk/commit/8739216acfbf26eba724fabf68103ed59cf73439))
+* **release:** 0.1.1 [skip ci] ([1c340b4](https://github.com/inference-gateway/typescript-sdk/commit/1c340b47fedd8f78220dc49b08acb72ba7f760fe))
+* **release:** bump version to 0.1.1 in package.json ([bd9fbb2](https://github.com/inference-gateway/typescript-sdk/commit/bd9fbb2346adcb89e6377a1212c3f9c257d25c0a))
+* Standardize quotes in .releaserc.yaml configuration ([b4c4f5b](https://github.com/inference-gateway/typescript-sdk/commit/b4c4f5bb31721dac3355b70bf3e04398c0f8491b))
+
+## [0.1.1](https://github.com/inference-gateway/typescript-sdk/compare/v0.1.0...0.1.1) (2025-01-23)
+
+### ♻️ Improvements
+
+* Refactor imports and update TypeScript configuration for improved module resolution and testing ([f74b6b1](https://github.com/inference-gateway/typescript-sdk/commit/f74b6b1dbc7371da01991ba832120c92b36d9c91))
+
+### 👷 CI
+
+* Update Node.js version in CI workflow from 20.x to 22.x ([1ecf62a](https://github.com/inference-gateway/typescript-sdk/commit/1ecf62ab2af9787cbf9ca02fb84377d5c1a08255))
+* Update Node.js version to 22.x and add global dependencies for semantic release ([0888fae](https://github.com/inference-gateway/typescript-sdk/commit/0888fae0c4a98a879808dc367e83e15d236dabab))
+
+### 🔧 Miscellaneous
+
+* Bump OS version to the latest with updated version of NodeJS and NPM ([8739216](https://github.com/inference-gateway/typescript-sdk/commit/8739216acfbf26eba724fabf68103ed59cf73439))

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Inference Gateway
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,127 @@
+# Inference Gateway Typescript SDK
+
+An SDK written in Typescript for the [Inference Gateway](https://github.com/edenreich/inference-gateway).
+
+- [Inference Gateway Typescript SDK](#inference-gateway-typescript-sdk)
+  - [Installation](#installation)
+  - [Usage](#usage)
+    - [Creating a Client](#creating-a-client)
+    - [Listing Models](#listing-models)
+    - [Generating Content](#generating-content)
+    - [Health Check](#health-check)
+  - [License](#license)
+
+## Installation
+
+Run `npm i @inference-gateway/sdk`.
+
+## Usage
+
+### Creating a Client
+
+```typescript
+import {
+  InferenceGatewayClient,
+  Message,
+  Provider,
+} from '@inference-gateway/sdk';
+
+async function main() {
+  const client = new InferenceGatewayClient('http://localhost:8080');
+
+  try {
+    // List available models
+    const models = await client.listModels();
+    models.forEach((providerModels) => {
+      console.log(`Provider: ${providerModels.provider}`);
+      providerModels.models.forEach((model) => {
+        console.log(`Model: ${model.id}`);
+      });
+    });
+
+    const response = await client.generateContent({
+      provider: Provider.Ollama,
+      model: 'llama2',
+      messages: [
+        {
+          role: 'system',
+          content: 'You are a helpful llama',
+        },
+        {
+          role: 'user',
+          content: 'Tell me a joke',
+        },
+      ],
+    });
+
+    console.log('Response:', response);
+  } catch (error) {
+    console.error('Error:', error);
+  }
+}
+
+main();
+```
+
+### Listing Models
+
+To list available models, use the `listModels` method:
+
+```typescript
+try {
+  const models = await client.listModels();
+  models.forEach((providerModels) => {
+    console.log(`Provider: ${providerModels.provider}`);
+    providerModels.models.forEach((model) => {
+      console.log(`Model: ${model.id}`);
+    });
+  });
+} catch (error) {
+  console.error('Error:', error);
+}
+```
+
+### Generating Content
+
+To generate content using a model, use the `generateContent` method:
+
+```typescript
+try {
+  const response = await client.generateContent({
+    provider: Provider.Ollama,
+    model: 'llama2',
+    messages: [
+      {
+        role: 'system',
+        content: 'You are a helpful llama',
+      },
+      {
+        role: 'user',
+        content: 'Tell me a joke',
+      },
+    ],
+  });
+
+  console.log('Provider:', response.provider);
+  console.log('Response:', response.response);
+} catch (error) {
+  console.error('Error:', error);
+}
+```
+
+### Health Check
+
+To check if the Inference Gateway is running, use the `healthCheck` method:
+
+```typescript
+try {
+  const isHealthy = await client.healthCheck();
+  console.log('API is healthy:', isHealthy);
+} catch (error) {
+  console.error('Error:', error);
+}
+```
+
+## License
+
+This SDK is distributed under the MIT License, see [LICENSE](LICENSE) for more information.

package/dist/src/client.d.ts

@@ -0,0 +1,10 @@
+import { GenerateContentRequest, GenerateContentResponse, ProviderModels } from './types';
+export declare class InferenceGatewayClient {
+    private baseUrl;
+    private authToken?;
+    constructor(baseUrl: string, authToken?: string);
+    private request;
+    listModels(): Promise<ProviderModels[]>;
+    generateContent(params: GenerateContentRequest): Promise<GenerateContentResponse>;
+    healthCheck(): Promise<boolean>;
+}

package/dist/src/client.js

@@ -0,0 +1,51 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.InferenceGatewayClient = void 0;
+class InferenceGatewayClient {
+    baseUrl;
+    authToken;
+    constructor(baseUrl, authToken) {
+        this.baseUrl = baseUrl.replace(/\/$/, '');
+        this.authToken = authToken;
+    }
+    async request(path, options = {}) {
+        const headers = new Headers({
+            'Content-Type': 'application/json',
+            ...options.headers,
+        });
+        if (this.authToken) {
+            headers.set('Authorization', `Bearer ${this.authToken}`);
+        }
+        const response = await fetch(`${this.baseUrl}${path}`, {
+            ...options,
+            headers,
+        });
+        if (!response.ok) {
+            const error = await response.json();
+            throw new Error(error.error || `HTTP error! status: ${response.status}`);
+        }
+        return response.json();
+    }
+    async listModels() {
+        return this.request('/llms');
+    }
+    async generateContent(params) {
+        return this.request(`/llms/${params.provider}/generate`, {
+            method: 'POST',
+            body: JSON.stringify({
+                model: params.model,
+                messages: params.messages,
+            }),
+        });
+    }
+    async healthCheck() {
+        try {
+            await this.request('/health');
+            return true;
+        }
+        catch {
+            return false;
+        }
+    }
+}
+exports.InferenceGatewayClient = InferenceGatewayClient;

package/dist/src/index.d.ts

@@ -0,0 +1,2 @@
+export * from './client';
+export * from './types';

package/dist/src/index.js

@@ -0,0 +1,18 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+__exportStar(require("./client"), exports);
+__exportStar(require("./types"), exports);

package/dist/src/types/index.d.ts

@@ -0,0 +1,36 @@
+export declare enum Provider {
+    Ollama = "ollama",
+    Groq = "groq",
+    OpenAI = "openai",
+    Google = "google",
+    Cloudflare = "cloudflare",
+    Cohere = "cohere",
+    Anthropic = "anthropic"
+}
+export interface Message {
+    role: 'system' | 'user' | 'assistant';
+    content: string;
+}
+export interface Model {
+    id: string;
+    object: string;
+    owned_by: string;
+    created: number;
+}
+export interface ProviderModels {
+    provider: Provider;
+    models: Model[];
+}
+export interface GenerateContentRequest {
+    provider: Provider;
+    model: string;
+    messages: Message[];
+}
+export interface GenerateContentResponse {
+    provider: string;
+    response: {
+        role: 'assistant';
+        model: string;
+        content: string;
+    };
+}

package/dist/src/types/index.js

@@ -0,0 +1,13 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.Provider = void 0;
+var Provider;
+(function (Provider) {
+    Provider["Ollama"] = "ollama";
+    Provider["Groq"] = "groq";
+    Provider["OpenAI"] = "openai";
+    Provider["Google"] = "google";
+    Provider["Cloudflare"] = "cloudflare";
+    Provider["Cohere"] = "cohere";
+    Provider["Anthropic"] = "anthropic";
+})(Provider || (exports.Provider = Provider = {}));

package/dist/tests/client.test.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/tests/client.test.js

@@ -0,0 +1,98 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+const client_1 = require("@/client");
+const types_1 = require("@/types");
+describe('InferenceGatewayClient', () => {
+    let client;
+    const mockBaseUrl = 'http://localhost:8080';
+    beforeEach(() => {
+        client = new client_1.InferenceGatewayClient(mockBaseUrl);
+        global.fetch = jest.fn();
+    });
+    describe('listModels', () => {
+        it('should fetch available models', async () => {
+            const mockResponse = [
+                {
+                    provider: types_1.Provider.Ollama,
+                    models: [
+                        {
+                            id: 'llama2',
+                            object: 'model',
+                            owned_by: 'ollama',
+                            created: 1234567890,
+                        },
+                    ],
+                },
+            ];
+            global.fetch.mockResolvedValueOnce({
+                ok: true,
+                json: () => Promise.resolve(mockResponse),
+            });
+            const result = await client.listModels();
+            expect(result).toEqual(mockResponse);
+            expect(global.fetch).toHaveBeenCalledWith(`${mockBaseUrl}/llms`, expect.objectContaining({
+                headers: expect.any(Headers),
+            }));
+        });
+    });
+    describe('generateContent', () => {
+        it('should generate content with the specified provider', async () => {
+            const mockRequest = {
+                provider: types_1.Provider.Ollama,
+                model: 'llama2',
+                messages: [
+                    { role: 'system', content: 'You are a helpful assistant' },
+                    { role: 'user', content: 'Hello' },
+                ],
+            };
+            const mockResponse = {
+                provider: types_1.Provider.Ollama,
+                response: {
+                    role: 'assistant',
+                    model: 'llama2',
+                    content: 'Hi there!',
+                },
+            };
+            global.fetch.mockResolvedValueOnce({
+                ok: true,
+                json: () => Promise.resolve(mockResponse),
+            });
+            const result = await client.generateContent(mockRequest);
+            expect(result).toEqual(mockResponse);
+            expect(global.fetch).toHaveBeenCalledWith(`${mockBaseUrl}/llms/${mockRequest.provider}/generate`, expect.objectContaining({
+                method: 'POST',
+                body: JSON.stringify({
+                    model: mockRequest.model,
+                    messages: mockRequest.messages,
+                }),
+            }));
+        });
+    });
+    describe('healthCheck', () => {
+        it('should return true when API is healthy', async () => {
+            global.fetch.mockResolvedValueOnce({
+                ok: true,
+                json: () => Promise.resolve({}),
+            });
+            const result = await client.healthCheck();
+            expect(result).toBe(true);
+            expect(global.fetch).toHaveBeenCalledWith(`${mockBaseUrl}/health`, expect.any(Object));
+        });
+        it('should return false when API is unhealthy', async () => {
+            global.fetch.mockRejectedValueOnce(new Error('API error'));
+            const result = await client.healthCheck();
+            expect(result).toBe(false);
+        });
+    });
+    describe('error handling', () => {
+        it('should throw error when API request fails', async () => {
+            const errorMessage = 'Bad Request';
+            global.fetch.mockResolvedValueOnce({
+                ok: false,
+                status: 400,
+                json: () => Promise.resolve({ error: errorMessage }),
+            });
+            await expect(client.listModels()).rejects.toThrow(errorMessage);
+        });
+    });
+});

package/package.json

@@ -0,0 +1,74 @@
+{
+  "name": "@inference-gateway/sdk",
+  "version": "0.1.4",
+  "description": "An SDK written in Typescript for the [Inference Gateway](https://github.com/inference-gateway/inference-gateway).",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "type": "commonjs",
+  "private": false,
+  "keywords": [
+    "inference",
+    "inference-gateway",
+    "gateway",
+    "sdk",
+    "llm",
+    "ai",
+    "openai",
+    "anthropic",
+    "ollama",
+    "cloudflare",
+    "cohere",
+    "typescript"
+  ],
+  "author": "Eden Reich <eden.reich@gmail.com>",
+  "license": "MIT",
+  "files": [
+    "dist",
+    "README.md",
+    "LICENSE",
+    "CHANGELOG.md"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/inference-gateway/typescript-sdk.git"
+  },
+  "bugs": {
+    "url": "https://github.com/inference-gateway/typescript-sdk/issues"
+  },
+  "homepage": "https://github.com/inference-gateway/typescript-sdk#README.md",
+  "peerDependencies": {
+    "node-fetch": "^2.7.0"
+  },
+  "peerDependenciesMeta": {
+    "node-fetch": {
+      "optional": true
+    }
+  },
+  "scripts": {
+    "build": "tsc",
+    "test": "jest",
+    "lint": "eslint src/**/*.ts",
+    "prepare": "npm run build"
+  },
+  "devDependencies": {
+    "@eslint/js": "^9.18.0",
+    "@types/jest": "^29.5.14",
+    "@types/node": "^22.10.9",
+    "@typescript-eslint/eslint-plugin": "^8.21.0",
+    "@typescript-eslint/parser": "^8.21.0",
+    "eslint": "^9.18.0",
+    "eslint-plugin-prettier": "^5.2.3",
+    "jest": "^29.7.0",
+    "ts-jest": "^29.2.5",
+    "typescript": "^5.7.3"
+  },
+  "engines": {
+    "node": ">=22.12.0",
+    "npm": ">=10.9.0"
+  },
+  "publishConfig": {
+    "registry": "https://registry.npmjs.org/",
+    "tag": "latest",
+    "access": "public"
+  }
+}