@oino-ts/db-bunsqlite 0.0.15 → 0.0.17

package/README.md

@@ -10,15 +10,15 @@
  # GETTING STARTED
  
  ### Setup
- Install the `@oino-ts/core` npm package and necessary database packages and import them in your code.
+ Install the `@oino-ts/db` npm package and necessary database packages and import them in your code.
  ```
- bun install @oino-ts/core
- bun install @oino-ts/bunsqlite
+ bun install @oino-ts/db
+ bun install @oino-ts/db-bunsqlite
  ```
  
  ```
- import { OINODb, OINOApi, OINOFactory } from "@oino-ts/core";
- import { OINODbBunSqlite } from "@oino-ts/bunsqlite"
+ import { OINODb, OINOApi, OINOFactory } from "@oino-ts/db";
+ import { OINODbBunSqlite } from "@oino-ts/db-bunsqlite"
  ```
  
  ### Register database and logger
@@ -51,7 +51,7 @@
  ### Write results back to HTTP Response
  The results for a GET request will contain [`OINOModelSet`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbModelSet.html) data that can be written out as JSON or CSV as needed. For other requests result is just success or error with messages.
  ```
- return new Response(result.modelset.writeString(OINOContentType.json))
+ return new Response(result.data.writeString(OINOContentType.json))
  ```
  
  
@@ -59,20 +59,8 @@
  
  ## RESTfull
  OINO maps HTTP methods GET/POST/PUT/DELETE to SQL operations SELECT/INSERT/UPDATE/DELETE. The GET/POST requests can be made without URL ID to get all rows or insert new ones and others target a single row using URL ID. 
- 
- ### HTTP GET
- ```
- Request and response:
- > curl.exe -X GET http://localhost:3001/orderdetails/11077:77
- [
-   {"_OINOID_":"11077:77","OrderID":11077,"ProductID":77,"UnitPrice":13,"Quantity":2,"Discount":0}
- ]
-
- SQL:
- SELECT "OrderID","ProductID","UnitPrice","Quantity","Discount" FROM [OrderDetails] WHERE ("OrderID"=11077 AND "ProductID"=77);
- ```
 
- ### HTTP POST 
+ For example HTTP POST 
  ```
  Request and response:
  > curl.exe -X POST http://localhost:3001/orderdetails -H "Content-Type: application/json" --data '[{\"OrderID\":11077,\"ProductID\":99,\"UnitPrice\":19,\"Quantity\":1,\"Discount\":0}]'
@@ -82,31 +70,12 @@
  INSERT INTO [OrderDetails] ("OrderID","ProductID","UnitPrice","Quantity","Discount") VALUES (11077,99,19,1,0);
  ```
 
- ### HTTP PUT
- ```
- Request and response:
- > curl.exe -X PUT http://localhost:3001/orderdetails/11077:99 -H "Content-Type: application/json" --data '[{\"UnitPrice\":20}]'
- {"success":true,"statusCode":200,"statusMessage":"OK","messages":[]}
 
- SQL:
- UPDATE [OrderDetails] SET "UnitPrice"=20 WHERE ("OrderID"=11077 AND "ProductID"=99);
- ```
- 
- ### HTTP DELETE
- ```
- Request and response:
- > curl.exe -X DELETE http://localhost:3001/orderdetails/11077:99
- {"success":true,"statusCode":200,"statusMessage":"OK","messages":[]}
-
- SQL:
- DELETE FROM [OrderDetails] WHERE ("OrderID"=11077 AND "ProductID"=99);
- ```
-  
  ## Universal Serialization
  OINO handles serialization of data to JSON/CSV/etc. and back based on the data model. It knows what columns exist, what is their data type and how to convert each to JSON/CSV and back. This allows also partial data to be sent, i.e. you can send only columns that need updating or even send extra columns and have them ignored.
 
  ### Features
- - Files can be sent to BLOB fields using BASE64 encoding.
+ - Files can be sent to BLOB fields using BASE64 or MIME multipart encoding.
  - Datetimes are (optionally) normalized to ISO 8601 format.
  - Extended JSON-encoding
    - Unquoted literal `undefined` can be used to represent non-existent values (leaving property out works too but preserving structure might be easier e.g. when translating data).
@@ -129,12 +98,13 @@
  - Bun Sqlite through Bun native implementation
  - Postgresql through [pg](https://www.npmjs.com/package/pg)-package
  - Mariadb / Mysql-support through [mariadb](https://www.npmjs.com/package/mariadb)-package
+ - Sql Server through [mssql](https://www.npmjs.com/package/mssql)-package
 
  ## Complex Keys
  To support tables with multipart primary keys OINO generates a composite key `_OINOID_` that is included in the result and can be used as the REST ID. For example in the example above table `OrderDetails` has two primary keys `OrderID` and `ProductID` making the `_OINOID_` of form `11077:99`. 
 
  ## Power Of SQL
- Since OINO controls the SQL, WHERE-conditions can be defined with [`OINOSqlFilter`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlFilter.html) and order with [`OINOSqlOrder`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlOrder.html) that are passed as HTTP request parameters. No more API development where you make unique API endpoints for each filter that fetch all data with original API and filter in backend code. Every API can be filtered when and as needed without unnessecary data tranfer and utilizing SQL indexing when available.
+ Since OINO is just generating SQL, WHERE-conditions can be defined with [`OINOSqlFilter`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlFilter.html), order with [`OINOSqlOrder`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlOrder.html) and limits/paging with [`OINOSqlOrder`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlLimit.html) that are passed as HTTP request parameters. No more API development where you make unique API endpoints for each filter that fetch all data with original API and filter in backend code. Every API can be filtered when and as needed without unnessecary data tranfer and utilizing SQL indexing when available.
 
  ## Swagger Support
  Swagger is great as long as the definitions are updated and with OINO you can automatically get a Swagger definition including a data model schema.
@@ -158,7 +128,7 @@
  # STATUS
  OINO is currently a hobby project which should and should considered in alpha status. That also means compatibility breaking changes can be made without prior notice when architectual issues are discovered.
 
- ## Beta
+ ## Beta 
  For a beta status following milestones are planned:
 
  ### Realistic app
@@ -178,8 +148,8 @@
  ### Batch updates
  Supporting batch updates similar to batch inserts is slightly bending the RESTfull principles but would still be a useful optional feature.
 
- ### Aggregation and limits
- Similar to filtering and ordering, aggregation and limits can be implemented as HTTP request parameters telling what column is aggregated or used for ordering or how many results to return.
+ ### Aggregation
+ Similar to filtering, ordering and limits, aggregation could be implemented as HTTP request parameters telling what column is aggregated or used for ordering or how many results to return.
 
  ### Streaming
  One core idea is to be efficient in not making unnecessary copies of the data and minimizing garbage collection debt. This can be taken further by implementing streaming, allowing large dataset to be written to HTTP response as SQL result rows are received.
@@ -205,18 +175,19 @@
  # LINKS
  - [Github repository](https://github.com/pragmatta/oino-ts)
  - [NPM repository](https://www.npmjs.com/org/oino-ts) 
-
+ 
  
  # ACKNOWLEDGEMENTS
  
  ## Libraries
  OINO uses the following open source libraries and npm packages and I would like to thank everyone for their contributions:
- - Postgresql support by [node-postgres package](https://www.npmjs.com/package/pg)
- - Mariadb / Mysql-support by [mariadb package](https://www.npmjs.com/package/mariadb)
+ - Postgresql [node-postgres package](https://www.npmjs.com/package/pg)
+ - Mariadb / Mysql [mariadb package](https://www.npmjs.com/package/mariadb)
+ - Sql Server [mssql package](https://www.npmjs.com/package/mssql)
+ - Custom base encoding [base-x package](https://www.npmjs.com/package/base-x)
 
  ## Bun
  OINO has been developed using the Bun runtime, not because of the speed improvements but for the first class Typescript support and integrated developper experience. Kudos on the bun team for making Typescript work more exiting again.
  
  ## SQL Scripts
  The SQL scripts for creating the sample Northwind database are based on [Google Code archive](https://code.google.com/archive/p/northwindextended/downloads) and have been further customized to ensure they would have identical data (in the scope of the automated testing).
-

package/dist/cjs/OINODbBunSqlite.js

@@ -24,6 +24,7 @@ class OINOBunSqliteDataset extends db_1.OINODbMemoryDataSet {
 class OINODbBunSqlite extends db_1.OINODb {
     static _tableDescriptionRegex = /^CREATE TABLE\s*[\"\[]?\w+[\"\]]?\s*\(\s*(.*)\s*\)\s*(WITHOUT ROWID)?$/msi;
     static _tablePrimarykeyRegex = /PRIMARY KEY \(([^\)]+)\)/i;
+    static _tableForeignkeyRegex = /FOREIGN KEY \(\[([^\)]+)\]\)/i;
     static _tableFieldTypeRegex = /[\"\[\s]?(\w+)[\"\]\s]\s?(INTEGER|REAL|DOUBLE|NUMERIC|DECIMAL|TEXT|BLOB|VARCHAR|DATETIME|DATE|BOOLEAN)(\s?\((\d+)\s?\,?\s?(\d*)?\))?/i;
     _db;
     /**
@@ -36,7 +37,7 @@ class OINODbBunSqlite extends db_1.OINODb {
         if (!this._params.url.startsWith("file://")) {
             throw new Error(db_1.OINO_ERROR_PREFIX + ": OINODbBunSqlite url must be a file://-url!");
         }
-        db_1.OINOLog.debug("OINODbBunSqlite.constructor", { params: params });
+        // OINOLog.debug("OINODbBunSqlite.constructor", {params:params})
         if (this._params.type !== "OINODbBunSqlite") {
             throw new Error(db_1.OINO_ERROR_PREFIX + ": Not OINODbBunSqlite-type: " + this._params.type);
         }
@@ -44,6 +45,7 @@ class OINODbBunSqlite extends db_1.OINODb {
     _parseDbFieldParams(fieldStr) {
         const result = {
             isPrimaryKey: fieldStr.indexOf("PRIMARY KEY") >= 0,
+            isForeignKey: false,
             isAutoInc: fieldStr.indexOf("AUTOINCREMENT") >= 0,
             isNotNull: fieldStr.indexOf("NOT NULL") >= 0
         };
@@ -134,7 +136,7 @@ class OINODbBunSqlite extends db_1.OINODb {
     connect() {
         const filepath = this._params.url.substring(7);
         try {
-            db_1.OINOLog.debug("OINODbBunSqlite.connect", { params: this._params });
+            // OINOLog.debug("OINODbBunSqlite.connect", {params:this._params})
             this._db = bun_sqlite_1.Database.open(filepath, { create: true, readonly: false, readwrite: true });
             // OINOLog.debug("OINODbBunSqlite.connect done")
             return Promise.resolve(true);
@@ -191,6 +193,7 @@ class OINODbBunSqlite extends db_1.OINODb {
     async initializeApiDatamodel(api) {
         const res = await this.sqlSelect("select sql from sqlite_schema WHERE name='" + api.params.tableName + "'");
         const sql_desc = (res?.getRow()[0]);
+        const foreign_keys = [];
         // OINOLog.debug("OINODbBunSqlite.initDatamodel.sql_desc=" + sql_desc)
         let table_matches = OINODbBunSqlite._tableDescriptionRegex.exec(sql_desc);
         // OINOLog.debug("OINODbBunSqlite.initDatamodel", {table_matches:table_matches})
@@ -208,7 +211,8 @@ class OINODbBunSqlite extends db_1.OINODb {
                 // OINOLog.debug("initDatamodel next field", {field_str:field_str, field_match:field_match, field_params:field_params})
                 if ((!field_match) || (field_match.length < 3)) {
                     let primarykey_match = OINODbBunSqlite._tablePrimarykeyRegex.exec(field_str);
-                    // OINOLog.debug("initDatamodel non-field definition", {primarykey_match:primarykey_match})
+                    let foreignkey_match = OINODbBunSqlite._tableForeignkeyRegex.exec(field_str);
+                    db_1.OINOLog.debug("initDatamodel non-field definition", { primarykey_match: primarykey_match, foreignkey_match: foreignkey_match });
                     if (primarykey_match && primarykey_match.length >= 2) {
                         const primary_keys = primarykey_match[1].replaceAll("\"", "").split(','); // not sure if will have space or not so split by comma and trim later
                         for (let i = 0; i < primary_keys.length; i++) {
@@ -220,6 +224,14 @@ class OINODbBunSqlite extends db_1.OINODb {
                             }
                         }
                     }
+                    else if (foreignkey_match && foreignkey_match.length >= 2) {
+                        const fk = foreignkey_match[1].trim();
+                        for (let j = 0; j < api.datamodel.fields.length; j++) {
+                            if (api.datamodel.fields[j].name == fk) {
+                                api.datamodel.fields[j].fieldParams.isForeignKey = true;
+                            }
+                        }
+                    }
                     else {
                         db_1.OINOLog.info("OINODbBunSqlite.initializeApiDatamodel: Unsupported field definition skipped.", { field: field_str });
                     }

package/dist/esm/OINODbBunSqlite.js

@@ -21,6 +21,7 @@ class OINOBunSqliteDataset extends OINODbMemoryDataSet {
 export class OINODbBunSqlite extends OINODb {
     static _tableDescriptionRegex = /^CREATE TABLE\s*[\"\[]?\w+[\"\]]?\s*\(\s*(.*)\s*\)\s*(WITHOUT ROWID)?$/msi;
     static _tablePrimarykeyRegex = /PRIMARY KEY \(([^\)]+)\)/i;
+    static _tableForeignkeyRegex = /FOREIGN KEY \(\[([^\)]+)\]\)/i;
     static _tableFieldTypeRegex = /[\"\[\s]?(\w+)[\"\]\s]\s?(INTEGER|REAL|DOUBLE|NUMERIC|DECIMAL|TEXT|BLOB|VARCHAR|DATETIME|DATE|BOOLEAN)(\s?\((\d+)\s?\,?\s?(\d*)?\))?/i;
     _db;
     /**
@@ -33,7 +34,7 @@ export class OINODbBunSqlite extends OINODb {
         if (!this._params.url.startsWith("file://")) {
             throw new Error(OINO_ERROR_PREFIX + ": OINODbBunSqlite url must be a file://-url!");
         }
-        OINOLog.debug("OINODbBunSqlite.constructor", { params: params });
+        // OINOLog.debug("OINODbBunSqlite.constructor", {params:params})
         if (this._params.type !== "OINODbBunSqlite") {
             throw new Error(OINO_ERROR_PREFIX + ": Not OINODbBunSqlite-type: " + this._params.type);
         }
@@ -41,6 +42,7 @@ export class OINODbBunSqlite extends OINODb {
     _parseDbFieldParams(fieldStr) {
         const result = {
             isPrimaryKey: fieldStr.indexOf("PRIMARY KEY") >= 0,
+            isForeignKey: false,
             isAutoInc: fieldStr.indexOf("AUTOINCREMENT") >= 0,
             isNotNull: fieldStr.indexOf("NOT NULL") >= 0
         };
@@ -131,7 +133,7 @@ export class OINODbBunSqlite extends OINODb {
     connect() {
         const filepath = this._params.url.substring(7);
         try {
-            OINOLog.debug("OINODbBunSqlite.connect", { params: this._params });
+            // OINOLog.debug("OINODbBunSqlite.connect", {params:this._params})
             this._db = BunSqliteDb.open(filepath, { create: true, readonly: false, readwrite: true });
             // OINOLog.debug("OINODbBunSqlite.connect done")
             return Promise.resolve(true);
@@ -188,6 +190,7 @@ export class OINODbBunSqlite extends OINODb {
     async initializeApiDatamodel(api) {
         const res = await this.sqlSelect("select sql from sqlite_schema WHERE name='" + api.params.tableName + "'");
         const sql_desc = (res?.getRow()[0]);
+        const foreign_keys = [];
         // OINOLog.debug("OINODbBunSqlite.initDatamodel.sql_desc=" + sql_desc)
         let table_matches = OINODbBunSqlite._tableDescriptionRegex.exec(sql_desc);
         // OINOLog.debug("OINODbBunSqlite.initDatamodel", {table_matches:table_matches})
@@ -205,7 +208,8 @@ export class OINODbBunSqlite extends OINODb {
                 // OINOLog.debug("initDatamodel next field", {field_str:field_str, field_match:field_match, field_params:field_params})
                 if ((!field_match) || (field_match.length < 3)) {
                     let primarykey_match = OINODbBunSqlite._tablePrimarykeyRegex.exec(field_str);
-                    // OINOLog.debug("initDatamodel non-field definition", {primarykey_match:primarykey_match})
+                    let foreignkey_match = OINODbBunSqlite._tableForeignkeyRegex.exec(field_str);
+                    OINOLog.debug("initDatamodel non-field definition", { primarykey_match: primarykey_match, foreignkey_match: foreignkey_match });
                     if (primarykey_match && primarykey_match.length >= 2) {
                         const primary_keys = primarykey_match[1].replaceAll("\"", "").split(','); // not sure if will have space or not so split by comma and trim later
                         for (let i = 0; i < primary_keys.length; i++) {
@@ -217,6 +221,14 @@ export class OINODbBunSqlite extends OINODb {
                             }
                         }
                     }
+                    else if (foreignkey_match && foreignkey_match.length >= 2) {
+                        const fk = foreignkey_match[1].trim();
+                        for (let j = 0; j < api.datamodel.fields.length; j++) {
+                            if (api.datamodel.fields[j].name == fk) {
+                                api.datamodel.fields[j].fieldParams.isForeignKey = true;
+                            }
+                        }
+                    }
                     else {
                         OINOLog.info("OINODbBunSqlite.initializeApiDatamodel: Unsupported field definition skipped.", { field: field_str });
                     }

package/dist/types/OINODbBunSqlite.d.ts

@@ -6,6 +6,7 @@ import { OINODb, OINODbParams, OINODbDataSet, OINODbApi, OINODataCell } from "@o
 export declare class OINODbBunSqlite extends OINODb {
     private static _tableDescriptionRegex;
     private static _tablePrimarykeyRegex;
+    private static _tableForeignkeyRegex;
     private static _tableFieldTypeRegex;
     private _db;
     /**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@oino-ts/db-bunsqlite",
-  "version": "0.0.15",
+  "version": "0.0.17",
   "description": "OINO TS package for using Bun Sqlite databases.",
   "author": "Matias Kiviniemi (pragmatta)",
   "license": "MPL-2.0",
@@ -20,7 +20,7 @@
   "module": "./dist/esm/index.js",
   "types": "./dist/types/index.d.ts",
   "dependencies": {
-    "@oino-ts/db": "^0.0.15"
+    "@oino-ts/db": "^0.0.17"
   },
   "devDependencies": {
     "@types/node": "^20.12.7",

package/src/OINODbBunSqlite.ts

@@ -25,6 +25,7 @@ class OINOBunSqliteDataset extends OINODbMemoryDataSet {
 export class OINODbBunSqlite extends OINODb {
     private static _tableDescriptionRegex = /^CREATE TABLE\s*[\"\[]?\w+[\"\]]?\s*\(\s*(.*)\s*\)\s*(WITHOUT ROWID)?$/msi
     private static _tablePrimarykeyRegex = /PRIMARY KEY \(([^\)]+)\)/i
+    private static _tableForeignkeyRegex = /FOREIGN KEY \(\[([^\)]+)\]\)/i
     private static _tableFieldTypeRegex = /[\"\[\s]?(\w+)[\"\]\s]\s?(INTEGER|REAL|DOUBLE|NUMERIC|DECIMAL|TEXT|BLOB|VARCHAR|DATETIME|DATE|BOOLEAN)(\s?\((\d+)\s?\,?\s?(\d*)?\))?/i
 
     private _db:BunSqliteDb|null
@@ -39,7 +40,7 @@ export class OINODbBunSqlite extends OINODb {
         if (!this._params.url.startsWith("file://")) {
             throw new Error(OINO_ERROR_PREFIX + ": OINODbBunSqlite url must be a file://-url!")
         }
-        OINOLog.debug("OINODbBunSqlite.constructor", {params:params})
+        // OINOLog.debug("OINODbBunSqlite.constructor", {params:params})
         
         if (this._params.type !== "OINODbBunSqlite") {
             throw new Error(OINO_ERROR_PREFIX + ": Not OINODbBunSqlite-type: " + this._params.type)
@@ -49,6 +50,7 @@ export class OINODbBunSqlite extends OINODb {
     private _parseDbFieldParams(fieldStr:string): OINODbDataFieldParams {
         const result:OINODbDataFieldParams = {
             isPrimaryKey: fieldStr.indexOf("PRIMARY KEY") >= 0,
+            isForeignKey: false, 
             isAutoInc: fieldStr.indexOf("AUTOINCREMENT") >= 0,
             isNotNull: fieldStr.indexOf("NOT NULL") >= 0
         }
@@ -144,7 +146,7 @@ export class OINODbBunSqlite extends OINODb {
     connect(): Promise<boolean> {
         const filepath:string = this._params.url.substring(7)
         try {
-            OINOLog.debug("OINODbBunSqlite.connect", {params:this._params})
+            // OINOLog.debug("OINODbBunSqlite.connect", {params:this._params})
             this._db = BunSqliteDb.open(filepath, { create: true, readonly: false, readwrite: true })        
             // OINOLog.debug("OINODbBunSqlite.connect done")
             return Promise.resolve(true)
@@ -203,6 +205,7 @@ export class OINODbBunSqlite extends OINODb {
     async initializeApiDatamodel(api:OINODbApi): Promise<void> {
         const res:OINODbDataSet|null = await this.sqlSelect("select sql from sqlite_schema WHERE name='" + api.params.tableName + "'")
         const sql_desc:string = (res?.getRow()[0]) as string
+        const foreign_keys:string[] = []
         // OINOLog.debug("OINODbBunSqlite.initDatamodel.sql_desc=" + sql_desc)
         let table_matches = OINODbBunSqlite._tableDescriptionRegex.exec(sql_desc)
         // OINOLog.debug("OINODbBunSqlite.initDatamodel", {table_matches:table_matches})
@@ -220,7 +223,8 @@ export class OINODbBunSqlite extends OINODb {
                 // OINOLog.debug("initDatamodel next field", {field_str:field_str, field_match:field_match, field_params:field_params})
                 if ((!field_match) || (field_match.length < 3))  {
                     let primarykey_match = OINODbBunSqlite._tablePrimarykeyRegex.exec(field_str)
-                    // OINOLog.debug("initDatamodel non-field definition", {primarykey_match:primarykey_match})
+                    let foreignkey_match = OINODbBunSqlite._tableForeignkeyRegex.exec(field_str)
+                    OINOLog.debug("initDatamodel non-field definition", {primarykey_match:primarykey_match, foreignkey_match:foreignkey_match})
                     if (primarykey_match && primarykey_match.length >= 2) {
                         const primary_keys:string[] = primarykey_match[1].replaceAll("\"", "").split(',') // not sure if will have space or not so split by comma and trim later
                         for (let i:number=0; i<primary_keys.length; i++) {
@@ -232,6 +236,14 @@ export class OINODbBunSqlite extends OINODb {
                             }
                         }
 
+                    } else if (foreignkey_match && foreignkey_match.length >= 2) {
+                        const fk:string = foreignkey_match[1].trim() 
+                        for (let j:number=0; j<api.datamodel.fields.length; j++) {
+                            if (api.datamodel.fields[j].name == fk) {
+                                api.datamodel.fields[j].fieldParams.isForeignKey = true
+                            }
+                        }
+
                     } else {
                         OINOLog.info("OINODbBunSqlite.initializeApiDatamodel: Unsupported field definition skipped.", { field: field_str })
                     }