inibase 1.0.0-rc.0 → 1.0.0-rc.100

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2023 Inibase
+Copyright (c) 2024 Inibase
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -1,26 +1,32 @@
-![Inibase banner](./.github/assets/banner.jpg)
+# Inibase :pencil:
 
-# Inibase
+> A file-based & memory-efficient, serverless, ACID compliant, relational database management system :fire:
 
-[![npmjs](https://img.shields.io/npm/dm/inibase.svg?style=flat)](https://www.npmjs.org/package/inibase) [![Node.js Version](https://img.shields.io/badge/node-18.11.0-blue)](https://nodejs.org/) [![License](https://img.shields.io/github/license/inicontent/inibase.svg?style=flat&colorA=18181B&colorB=28CF8D)](./LICENSE) [![Activity](https://img.shields.io/github/commit-activity/m/inicontent/inibase)](https://github.com/inicontent/inibase/pulse) [![GitHub stars](https://img.shields.io/github/stars/inicontent/inibase?style=social)](https://github.com/inicontent/inibase)
+[![Inibase banner](./.github/assets/banner.jpg)](https://github.com/inicontent/inibase)
 
-> File-based relational database, simple to use and can handle large data :fire:
+[![npmjs](https://img.shields.io/npm/dm/inibase.svg?style=flat)](https://www.npmjs.org/package/inibase) [![License](https://img.shields.io/github/license/inicontent/inibase.svg?style=flat&colorA=18181B&colorB=28CF8D)](./LICENSE) [![Activity](https://img.shields.io/github/commit-activity/m/inicontent/inibase)](https://github.com/inicontent/inibase/pulse) [![GitHub stars](https://img.shields.io/github/stars/inicontent/inibase?style=social)](https://github.com/inicontent/inibase)
 
 ## Features
 
-- **Lightweight** 🪶 (~50kb)
-- **Minimalist** :white_circle:
-- **TypeScript** :large_blue_diamond:
-- **Super-Fast** :turtle:
-- **Suitable for large data** :page_with_curl:
-- **Safe** :lock:
+- **Lightweight** 🪶
+- **Minimalist** :white_circle: (but powerful)
+- **100% TypeScript** :large_blue_diamond:
+- **Super-Fast** :zap: (built-in caching system)
+- **ATOMIC** :lock: File lock for writing
+- **Built-in** form validation (+unique values :new: ) :sunglasses:
+- **Suitable for large data** :page_with_curl: (tested with 4M records)
+- **Support Compression** :eight_spoked_asterisk: (using built-in nodejs zlib)
+- **Support Table Joins** :link:
+- **Low memory-usage** :chart_with_downwards_trend: (3-5mb)
+- **Safe** :lock: (no sql or javascript injections)
+- **Easy to use** :bread:
 - **...** and much more :rocket:
 
 ## Usage
 
 ```js
 import Inibase from "inibase";
-const db = new Inibase("/database_name");
+const db = new Inibase("databaseName");
 
 // Get all items from "user" table
 const users = await db.get("user");
@@ -29,45 +35,117 @@ const users = await db.get("user");
 const users = await db.get("user", undefined, { page: 2, per_page: 15 });
 
 // Get only required columns to improve speed
-const users = await InibaseDB.get("user", undefined, {
-  columns: ["username", "address.street", "hobbies.*.name"],
+const users = await db.get("user", undefined, {
+  columns: ["username", "address.street", "hobbies.name"],
 });
 
-// Get items from "user" table where "favoriteFoods" does not includes "Pizza"
-const users = await InibaseDB.get("user", { favoriteFoods: "![]Pizza" });
+// Get items from "user" table where "favoriteFoods" does not includes "Pizza" or "Burger"
+const users = await db.get("user", { favoriteFoods: "![]Pizza,Burger" });
 ```
 
-If you like Inibase, please sponsor: [GitHub Sponsors](https://github.com/sponsors/inicontent) || [Paypal](https://paypal.me/KarimAmahtil).
+> [!NOTE]
+> Enjoy using Inibase? Consider sponsoring us via [PayPal](https://paypal.me/KarimAmahtil) <br>
+> Your support helps us maintain and improve our services. <br>
+> Thank you! 🫰
 
-## Sponsors
+## Install
 
-<br>
-<br>
+```js
+<npm|pnpm|yarn|bun> install inibase
+```
 
-Become a sponsor and have your company logo here 👉 [GitHub Sponsors](https://github.com/sponsors/inicontent) || [paypal](https://paypal.me/KarimAmahtil).
+## How it works?
 
-## Install
+`Inibase` organizes data into databases, tables, and columns, each stored in separate files.
 
-```js
-// npm
-npm install inibase
+- **POST**: New data is appended to column files efficiently.
+- **GET**: Data retrieval is optimized by reading files line-by-line.
+- **PUT**: Updates are streamlined, with only the relevant file being modified.
+- **DELETE**: Removes lines from column files for swift deletion.
+
+This structure ensures efficient storage, retrieval, and updates, making our system scalable and high-performing for diverse datasets and applications.
 
-// pnpm
-pnpm add inibase
+## Inibase CLI
 
-// yarn
-yarn add inibase
+```shell
+npx inibase -p <databaseFolderPath>
+# by default it will diplay a list of available commands (or type 'help')
 ```
 
 ## Examples
 
-#### POST
+<details>
+<summary>Tables</summary>
+<blockquote>
+
+<details>
+<summary>Config</summary>
+<blockquote>
+
+```ts
+interface {
+  compression: boolean;
+  cache: boolean;
+  prepend: boolean;
+}
+```
+
+</blockquote>
+</details>
+
+<details>
+<summary>Schema</summary>
+<blockquote>
+
+```ts
+interface {
+  id: number; // stored as a Number but displayed as a hashed ID
+  key: string;
+  required?: boolean;
+  unique?: boolean;
+  type: "string" | "number" | "boolean" | "date" | "email" | "url" | "password" | "html" | "ip" | "json" | "id";
+}
+interface Table {
+  id: number;
+  key: string;
+  required?: boolean;
+  type: "table";
+  table: string;
+}
+interface Array {
+  id: number;
+  key: string;
+  required?: boolean;
+  type: "array";
+  children: string|string[];
+}
+interface ObjectOrArrayOfObjects {
+  id: number;
+  key: string;
+  required?: boolean;
+  type: "object" | "array";
+  children: Schema;
+}
+```
+
+</blockquote>
+</details>
+
+<details>
+<summary>Create Table</summary>
+<blockquote>
 
 ```js
 import Inibase from "inibase";
-const db = new Inibase("/database_name");
+const db = new Inibase("/databaseName");
 
-const user_schema = [
+const userTableConfig = {
+  compression: true,
+  cache: true,
+  prepend: false
+}
+
+const userTableSchema = [
   {
     key: "username",
     type: "string",
@@ -133,7 +211,156 @@ const user_schema = [
   },
 ];
 
-const user_data = [
+await db.createTable("user", userTableSchema, userTableConfig);
+```
+
+</blockquote>
+</details>
+
+<details>
+<summary>Update Table</summary>
+<blockquote>
+  
+<details>
+<summary>Change Name</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
+
+// this will change table name also in joined tables
+await db.updateTable("user", undefined, {name: "userV2"});
+```
+
+</blockquote>
+</details>
+
+<details>
+<summary>Update field</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+import { setField } from "inibase/utils";
+
+const db = new Inibase("/databaseName");
+
+const userTableSchema = (await db.getTable("user")).schema;
+setField("username", userTableSchema, {key: "fullName"});
+await db.updateTable("user", newUserTableSchema);
+```
+
+</blockquote>
+</details>
+
+<details>
+<summary>Remove field</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+import { unsetField } from "inibase/utils";
+
+const db = new Inibase("/databaseName");
+
+const userTableSchema = (await db.getTable("user")).schema;
+unsetField("fullName", userTableSchema);
+await db.updateTable("user", newUserTableSchema);
+```
+
+</blockquote>
+</details>
+
+</blockquote>
+</details>
+
+<details>
+<summary>Join Tables</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
+
+const productTableSchema = [
+  {
+    key: "title",
+    type: "string",
+    required: true,
+  },
+  {
+    key: "price",
+    type: "number",
+  },
+  {
+    key: "createdBy",
+    type: "table",
+    table: "user",
+    required: true,
+  },
+];
+
+await db.createTable("product", productTableSchema);
+
+const productTableData = [
+  {
+    title: "Product 1",
+    price: 16,
+    createdBy: "1d88385d4b1581f8fb059334dec30f4c",
+  },
+  {
+    title: "Product 2",
+    price: 10,
+    createdBy: "5011c230aa44481bf7e8dcfe0710474f",
+  },
+];
+
+const product = await db.post("product", productTableData);
+// [
+//   {
+//     "id": "1d88385d4b1581f8fb059334dec30f4c",
+//     "title": "Product 1",
+//     "price": 16,
+//     "createdBy": {
+//       "id": "1d88385d4b1581f8fb059334dec30f4c",
+//       "username": "user1",
+//       "email": "user1@example.com",
+//       ...
+//     }
+//   },
+//   {
+//     "id": "5011c230aa44481bf7e8dcfe0710474f",
+//     "title": "Product 2",
+//     "price": 10,
+//     "createdBy": {
+//       "id": "5011c230aa44481bf7e8dcfe0710474f",
+//       "username": "user2",
+//       ...
+//     }
+//   }
+// ]
+```
+
+</blockquote>
+</details>
+
+</blockquote>
+</details>
+
+<details>
+<summary>Methods</summary>
+<blockquote>
+
+<details>
+<summary>POST</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
+
+const userTableData = [
   {
     username: "user1",
     email: "user1@example.com",
@@ -168,7 +395,7 @@ const user_data = [
   },
 ];
 
-const users = await db.post("user", user_data);
+const users = await db.post("user", userTableData);
 // [
 //   {
 //     "id": "1d88385d4b1581f8fb059334dec30f4c",
@@ -206,75 +433,21 @@ const users = await db.post("user", user_data);
 // ]
 ```
 
-Link two tables: "product" with "user"
+</blockquote>
+</details>
 
-```js
-import Inibase from "inibase";
-const db = new Inibase("/database_name");
-
-const product_schema = [
-  {
-    key: "title",
-    type: "string",
-    required: true,
-  },
-  {
-    key: "price",
-    type: "number",
-  },
-  {
-    key: "user",
-    type: "table",
-    required: true,
-  },
-];
+<details>
+<summary>GET</summary>
+<blockquote>
 
-const product_data = [
-  {
-    title: "Product 1",
-    price: 16,
-    user: "1d88385d4b1581f8fb059334dec30f4c",
-  },
-  {
-    title: "Product 2",
-    price: 10,
-    user: "5011c230aa44481bf7e8dcfe0710474f",
-  },
-];
-
-const product = await db.post("product", product_data);
-// [
-//   {
-//     "id": "1d88385d4b1581f8fb059334dec30f4c",
-//     "title": "Product 1",
-//     "price": 16,
-//     "user": {
-//       "id": "1d88385d4b1581f8fb059334dec30f4c",
-//       "username": "user1",
-//       "email": "user1@example.com",
-//       ...
-//     }
-//   },
-//   {
-//     "id": "5011c230aa44481bf7e8dcfe0710474f",
-//     "title": "Product 2",
-//     "price": 10,
-//     "user": {
-//       "id": "5011c230aa44481bf7e8dcfe0710474f",
-//       "username": "user2",
-//       ...
-//     }
-//   }
-// ]
-```
-
-#### GET
+<details>
+<summary>GET by ID</summary>
+<blockquote>
 
 ```js
 import Inibase from "inibase";
-const db = new Inibase("/database_name");
+const db = new Inibase("/databaseName");
 
-// Get "user" by id
 const user = await db.get("user", "1d88385d4b1581f8fb059334dec30f4c");
 // {
 //     "id": "1d88385d4b1581f8fb059334dec30f4c",
@@ -303,8 +476,19 @@ const user = await db.get("user", "1d88385d4b1581f8fb059334dec30f4c");
 //         "country": "Sampleland"
 //     }
 // }
+```
+
+</blockquote>
+</details>
+
+<details>
+<summary>GET by criteria</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
 
-// Get "user" by Criteria: where "favoriteFoods" includes "Pizza"
 const users = await db.get("user", { favoriteFoods: "[]Pizza" });
 // [
 //   {
@@ -338,11 +522,36 @@ const users = await db.get("user", { favoriteFoods: "[]Pizza" });
 // ]
 ```
 
-#### PUT
+</blockquote>
+</details>
+
+<details>
+<summary>GET with columns</summary>
+<blockquote>
 
 ```js
 import Inibase from "inibase";
-const db = new Inibase("/database_name");
+const db = new Inibase("/databaseName");
+
+// Get all "user" columns except "username" & "address.street"
+const users = await db.get("user", undefined, {
+  columns: ["!username", "!address.street"],
+});
+```
+
+</blockquote>
+</details>
+
+</blockquote>
+</details>
+
+<details>
+<summary>PUT</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
 
 // set "isActive" to "false" for all items in table "user"
 await db.put("user", { isActive: false });
@@ -354,11 +563,16 @@ await db.put("user", { isActive: false }, "1d88385d4b1581f8fb059334dec30f4c");
 await db.put("user", { isActive: false }, { isActive: true });
 ```
 
-#### DELETE
+</blockquote>
+</details>
+
+<details>
+<summary>DELETE</summary>
+<blockquote>
 
 ```js
 import Inibase from "inibase";
-const db = new Inibase("/database_name");
+const db = new Inibase("/databaseName");
 
 // delete all items in "user" table
 await db.delete("user");
@@ -370,46 +584,108 @@ await db.put("user", "1d88385d4b1581f8fb059334dec30f4c");
 await db.put("user", { isActive: false });
 ```
 
-## Typescript
+</blockquote>
+</details>
+
+<details>
+<summary>SUM</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
+
+// get the sum of column "age" in "user" table
+await db.sum("user", "age");
+
+// get the sum of column "age" by criteria (where "isActive" is equal to "false") in "user" table
+await db.sum("user", ["age", ...], { isActive: false });
+```
+
+</blockquote>
+</details>
+
+<details>
+<summary>MAX</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
+
+// get the biggest number of column "age" in "user" table
+await db.max("user", "age");
+
+// get the biggest number of column "age" by criteria (where "isActive" is equal to "false") in "user" table
+await db.max("user", ["age", ...], { isActive: false });
+```
+
+</blockquote>
+</details>
 
-#### Schema
+<details>
+<summary>MIN</summary>
+<blockquote>
 
 ```js
-type FieldType =
-  | "string"
-  | "number"
-  | "boolean"
-  | "date"
-  | "email"
-  | "url"
-  | "table"
-  | "object"
-  | "array";
-type Field =
-  | {
-      id?: string | number | null | undefined,
-      key: string,
-      type: Exclude<FieldType, "array" | "object">,
-      required?: boolean,
-    }
-  | {
-      id?: string | number | null | undefined,
-      key: string,
-      type: "array",
-      required?: boolean,
-      children: FieldType | FieldType[] | Schema,
-    }
-  | {
-      id?: string | number | null | undefined,
-      key: string,
-      type: "object",
-      required?: boolean,
-      children: Schema,
-    };
-
-type Schema = Field[];
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
+
+// get the smallest number of column "age" in "user" table
+await db.min("user", "age");
+
+// get the smallest number of column "age" by criteria (where "isActive" is equal to "false") in "user" table
+await db.min("user", ["age", ...], { isActive: false });
 ```
 
+</blockquote>
+</details>
+
+<details>
+<summary>SORT</summary>
+<blockquote>
+
+```js
+import Inibase from "inibase";
+const db = new Inibase("/databaseName");
+
+// order users by the age column
+await db.get("user", undefined, { sort: "age" });
+
+// order users by the age and username columns
+await db.get("user", undefined, { sort: ["age", "username"] });
+await db.get("user", undefined, { sort: {age: -1, username: "asc"} });
+```
+
+</blockquote>
+</details>
+
+</blockquote>
+</details>
+
+## Benchmark
+
+### Bulk
+
+|        | 10              | 100             | 1000            |
+|--------|-----------------|-----------------|-----------------|
+| POST   | 11 ms (0.65 mb) | 19 ms (1.00 mb) | 85 ms (4.58 mb) |
+| GET    | 14 ms (2.77 mb) | 12 ms (3.16 mb) | 34 ms (1.38 mb) |
+| PUT    | 6 ms (1.11 mb)  | 5 ms (1.37 mb)  | 10 ms (1.12 mb) |
+| DELETE | 17 ms (1.68 mb) | 14 ms (5.45 mb) | 25 ms (5.94 mb) |
+
+### Single
+
+|        | 10                | 100                | 1000               |
+|--------|-------------------|--------------------|--------------------|
+| POST   | 43 ms (4.70 mb)   | 387 ms (6.36 mb)   | 5341 ms (24.73 mb) |
+| GET    | 99 ms (12.51 mb)  | 846 ms (30.68 mb)  | 7103 ms (30.86 mb) |
+| PUT    | 33 ms (10.29 mb)  | 312 ms (11.06 mb)  | 3539 ms (14.87 mb) |
+| DELETE | 134 ms (13.50 mb) | 1224 ms (16.57 mb) | 7339 ms (11.46 mb) |
+
+> Default testing uses a table with username, email, and password fields, ensuring password encryption is included in the process<br>
+> To run benchmarks, install _typescript_ & _[tsx](https://github.com/privatenumber/tsx)_ globally and run `benchmark` by default bulk, for single use `benchmark --single|-s`
+
 ## Roadmap
 
 - [x] Actions:
@@ -417,11 +693,14 @@ type Schema = Field[];
     - [x] Pagination
     - [x] Criteria
     - [x] Columns
-    - [ ] Order
+    - [x] Sort
   - [x] POST
   - [x] PUT
   - [x] DELETE
-- [ ] Schema supported types:
+  - [x] SUM
+  - [x] MAX
+  - [x] MIN
+- [x] Schema supported types:
   - [x] String
   - [x] Number
   - [x] Boolean
@@ -431,9 +710,23 @@ type Schema = Field[];
   - [x] Table
   - [x] Object
   - [x] Array
-  - [ ] Password
-  - [ ] IP
-- [ ] Suggest [new feature +](https://github.com/inicontent/inibase/discussions/new?category=ideas)
+  - [x] Password
+  - [x] IP
+  - [x] HTML
+  - [x] Id
+  - [x] JSON
+- [ ] TO-DO:
+  - [ ] Ability to search in JSON fields
+  - [ ] Re-check used exec functions
+  - [ ] Use smart caching (based on N° of queries)
+  - [ ] Commenting the code
+  - [ ] Add Backup feature (generate a tar.gz)
+  - [ ] Add Custom field validation property to schema (using RegEx?)
+- [ ] Features:
+  - [ ] Encryption
+  - [x] Data Compression
+  - [x] Caching System
+  - [ ] Suggest [new feature +](https://github.com/inicontent/inibase/discussions/new?category=ideas)
 
 ## License