@oino-ts/db-mariadb 0.0.15 → 0.0.17

package/README.md

@@ -10,15 +10,15 @@
  # GETTING STARTED
  
  ### Setup
- Install the `@oino-ts/core` npm package and necessary database packages and import them in your code.
+ Install the `@oino-ts/db` npm package and necessary database packages and import them in your code.
  ```
- bun install @oino-ts/core
- bun install @oino-ts/bunsqlite
+ bun install @oino-ts/db
+ bun install @oino-ts/db-bunsqlite
  ```
  
  ```
- import { OINODb, OINOApi, OINOFactory } from "@oino-ts/core";
- import { OINODbBunSqlite } from "@oino-ts/bunsqlite"
+ import { OINODb, OINOApi, OINOFactory } from "@oino-ts/db";
+ import { OINODbBunSqlite } from "@oino-ts/db-bunsqlite"
  ```
  
  ### Register database and logger
@@ -51,7 +51,7 @@
  ### Write results back to HTTP Response
  The results for a GET request will contain [`OINOModelSet`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbModelSet.html) data that can be written out as JSON or CSV as needed. For other requests result is just success or error with messages.
  ```
- return new Response(result.modelset.writeString(OINOContentType.json))
+ return new Response(result.data.writeString(OINOContentType.json))
  ```
  
  
@@ -59,20 +59,8 @@
  
  ## RESTfull
  OINO maps HTTP methods GET/POST/PUT/DELETE to SQL operations SELECT/INSERT/UPDATE/DELETE. The GET/POST requests can be made without URL ID to get all rows or insert new ones and others target a single row using URL ID. 
- 
- ### HTTP GET
- ```
- Request and response:
- > curl.exe -X GET http://localhost:3001/orderdetails/11077:77
- [
-   {"_OINOID_":"11077:77","OrderID":11077,"ProductID":77,"UnitPrice":13,"Quantity":2,"Discount":0}
- ]
-
- SQL:
- SELECT "OrderID","ProductID","UnitPrice","Quantity","Discount" FROM [OrderDetails] WHERE ("OrderID"=11077 AND "ProductID"=77);
- ```
 
- ### HTTP POST 
+ For example HTTP POST 
  ```
  Request and response:
  > curl.exe -X POST http://localhost:3001/orderdetails -H "Content-Type: application/json" --data '[{\"OrderID\":11077,\"ProductID\":99,\"UnitPrice\":19,\"Quantity\":1,\"Discount\":0}]'
@@ -82,31 +70,12 @@
  INSERT INTO [OrderDetails] ("OrderID","ProductID","UnitPrice","Quantity","Discount") VALUES (11077,99,19,1,0);
  ```
 
- ### HTTP PUT
- ```
- Request and response:
- > curl.exe -X PUT http://localhost:3001/orderdetails/11077:99 -H "Content-Type: application/json" --data '[{\"UnitPrice\":20}]'
- {"success":true,"statusCode":200,"statusMessage":"OK","messages":[]}
 
- SQL:
- UPDATE [OrderDetails] SET "UnitPrice"=20 WHERE ("OrderID"=11077 AND "ProductID"=99);
- ```
- 
- ### HTTP DELETE
- ```
- Request and response:
- > curl.exe -X DELETE http://localhost:3001/orderdetails/11077:99
- {"success":true,"statusCode":200,"statusMessage":"OK","messages":[]}
-
- SQL:
- DELETE FROM [OrderDetails] WHERE ("OrderID"=11077 AND "ProductID"=99);
- ```
-  
  ## Universal Serialization
  OINO handles serialization of data to JSON/CSV/etc. and back based on the data model. It knows what columns exist, what is their data type and how to convert each to JSON/CSV and back. This allows also partial data to be sent, i.e. you can send only columns that need updating or even send extra columns and have them ignored.
 
  ### Features
- - Files can be sent to BLOB fields using BASE64 encoding.
+ - Files can be sent to BLOB fields using BASE64 or MIME multipart encoding.
  - Datetimes are (optionally) normalized to ISO 8601 format.
  - Extended JSON-encoding
    - Unquoted literal `undefined` can be used to represent non-existent values (leaving property out works too but preserving structure might be easier e.g. when translating data).
@@ -129,12 +98,13 @@
  - Bun Sqlite through Bun native implementation
  - Postgresql through [pg](https://www.npmjs.com/package/pg)-package
  - Mariadb / Mysql-support through [mariadb](https://www.npmjs.com/package/mariadb)-package
+ - Sql Server through [mssql](https://www.npmjs.com/package/mssql)-package
 
  ## Complex Keys
  To support tables with multipart primary keys OINO generates a composite key `_OINOID_` that is included in the result and can be used as the REST ID. For example in the example above table `OrderDetails` has two primary keys `OrderID` and `ProductID` making the `_OINOID_` of form `11077:99`. 
 
  ## Power Of SQL
- Since OINO controls the SQL, WHERE-conditions can be defined with [`OINOSqlFilter`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlFilter.html) and order with [`OINOSqlOrder`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlOrder.html) that are passed as HTTP request parameters. No more API development where you make unique API endpoints for each filter that fetch all data with original API and filter in backend code. Every API can be filtered when and as needed without unnessecary data tranfer and utilizing SQL indexing when available.
+ Since OINO is just generating SQL, WHERE-conditions can be defined with [`OINOSqlFilter`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlFilter.html), order with [`OINOSqlOrder`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlOrder.html) and limits/paging with [`OINOSqlOrder`](https://pragmatta.github.io/oino-ts/classes/db_src.OINODbSqlLimit.html) that are passed as HTTP request parameters. No more API development where you make unique API endpoints for each filter that fetch all data with original API and filter in backend code. Every API can be filtered when and as needed without unnessecary data tranfer and utilizing SQL indexing when available.
 
  ## Swagger Support
  Swagger is great as long as the definitions are updated and with OINO you can automatically get a Swagger definition including a data model schema.
@@ -158,7 +128,7 @@
  # STATUS
  OINO is currently a hobby project which should and should considered in alpha status. That also means compatibility breaking changes can be made without prior notice when architectual issues are discovered.
 
- ## Beta
+ ## Beta 
  For a beta status following milestones are planned:
 
  ### Realistic app
@@ -178,8 +148,8 @@
  ### Batch updates
  Supporting batch updates similar to batch inserts is slightly bending the RESTfull principles but would still be a useful optional feature.
 
- ### Aggregation and limits
- Similar to filtering and ordering, aggregation and limits can be implemented as HTTP request parameters telling what column is aggregated or used for ordering or how many results to return.
+ ### Aggregation
+ Similar to filtering, ordering and limits, aggregation could be implemented as HTTP request parameters telling what column is aggregated or used for ordering or how many results to return.
 
  ### Streaming
  One core idea is to be efficient in not making unnecessary copies of the data and minimizing garbage collection debt. This can be taken further by implementing streaming, allowing large dataset to be written to HTTP response as SQL result rows are received.
@@ -205,18 +175,19 @@
  # LINKS
  - [Github repository](https://github.com/pragmatta/oino-ts)
  - [NPM repository](https://www.npmjs.com/org/oino-ts) 
-
+ 
  
  # ACKNOWLEDGEMENTS
  
  ## Libraries
  OINO uses the following open source libraries and npm packages and I would like to thank everyone for their contributions:
- - Postgresql support by [node-postgres package](https://www.npmjs.com/package/pg)
- - Mariadb / Mysql-support by [mariadb package](https://www.npmjs.com/package/mariadb)
+ - Postgresql [node-postgres package](https://www.npmjs.com/package/pg)
+ - Mariadb / Mysql [mariadb package](https://www.npmjs.com/package/mariadb)
+ - Sql Server [mssql package](https://www.npmjs.com/package/mssql)
+ - Custom base encoding [base-x package](https://www.npmjs.com/package/base-x)
 
  ## Bun
  OINO has been developed using the Bun runtime, not because of the speed improvements but for the first class Typescript support and integrated developper experience. Kudos on the bun team for making Typescript work more exiting again.
  
  ## SQL Scripts
  The SQL scripts for creating the sample Northwind database are based on [Google Code archive](https://code.google.com/archive/p/northwindextended/downloads) and have been further customized to ensure they would have identical data (in the scope of the automated testing).
-

package/dist/cjs/OINODbMariadb.js

@@ -38,14 +38,25 @@ class OINOMariadbData extends db_1.OINODbDataSet {
     }
     _currentRow;
     _eof;
+    /**
+     * Is data set empty.
+     *
+     */
     isEmpty() {
         return (this._rows.length == 0);
     }
-    // EOF means "there is no more content", i.e. either dataset is empty or we have moved beyond last line
+    /**
+     * Is there no more content, i.e. either dataset is empty or we have moved beyond last line
+     *
+     */
     isEof() {
         return (this._eof);
     }
-    next() {
+    /**
+     * Attempts to moves dataset to the next row, possibly waiting for more data to become available. Returns !isEof().
+     *
+     */
+    async next() {
         // OINOLog.debug("OINODbDataSet.next", {currentRow:this._currentRow, length:this.sqlResult.data.length})
         if (this._currentRow < this._rows.length - 1) {
             this._currentRow = this._currentRow + 1;
@@ -53,8 +64,12 @@ class OINOMariadbData extends db_1.OINODbDataSet {
         else {
             this._eof = true;
         }
-        return !this._eof;
+        return Promise.resolve(!this._eof);
     }
+    /**
+     * Gets current row of data.
+     *
+     */
     getRow() {
         if ((this._currentRow >= 0) && (this._currentRow < this._rows.length)) {
             return this._rows[this._currentRow];
@@ -71,7 +86,6 @@ class OINOMariadbData extends db_1.OINODbDataSet {
 class OINODbMariadb extends db_1.OINODb {
     static _fieldLengthRegex = /([^\(\)]+)(\s?\((\d+)\s?\,?\s?(\d*)?\))?/i;
     static _exceptionMessageRegex = /\(([^\)]*)\) (.*)\nsql\:(.*)?/i;
-    static _tableSchemaSql = `SHOW COLUMNS from `;
     _pool;
     /**
      * Constructor of `OINODbMariadb`
@@ -256,7 +270,7 @@ class OINODbMariadb extends db_1.OINODb {
         }
         catch (err) {
             // ... error checks
-            throw new Error(db_1.OINO_ERROR_PREFIX + ": Error connecting to Postgresql server: " + err);
+            throw new Error(db_1.OINO_ERROR_PREFIX + ": Error connecting to OINODbMariadb server: " + err);
         }
     }
     /**
@@ -299,6 +313,21 @@ class OINODbMariadb extends db_1.OINODb {
         db_1.OINOBenchmark.end("OINODb", "sqlExec");
         return result;
     }
+    _getSchemaSql(dbName, tableName) {
+        const sql = `SELECT
+    c.COLUMN_NAME,
+    c.COLUMN_TYPE,
+    c.IS_NULLABLE,
+    c.COLUMN_KEY,
+    c.COLUMN_DEFAULT,
+    c.EXTRA,
+    KCU.CONSTRAINT_NAME AS ForeignKeyName 
+FROM information_schema.COLUMNS C
+	LEFT JOIN information_schema.KEY_COLUMN_USAGE KCU ON KCU.TABLE_SCHEMA = C.TABLE_SCHEMA AND KCU.TABLE_NAME = C.TABLE_NAME AND C.COLUMN_NAME = KCU.COLUMN_NAME and KCU.REFERENCED_TABLE_NAME IS NOT NULL
+WHERE C.TABLE_SCHEMA = '${dbName}' AND C.TABLE_NAME = '${tableName}'
+ORDER BY C.ORDINAL_POSITION;`;
+        return sql;
+    }
     /**
      * Initialize a data model by getting the SQL schema and populating OINODbDataFields of
      * the model.
@@ -307,7 +336,7 @@ class OINODbMariadb extends db_1.OINODb {
      *
      */
     async initializeApiDatamodel(api) {
-        const res = await this.sqlSelect(OINODbMariadb._tableSchemaSql + this._params.database + "." + api.params.tableName + ";");
+        const res = await this.sqlSelect(this._getSchemaSql(this._params.database, api.params.tableName));
         while (!res.isEof()) {
             const row = res.getRow();
             // OINOLog.debug("OINODbMariadb.initializeApiDatamodel", { description:row })
@@ -320,6 +349,7 @@ class OINODbMariadb extends db_1.OINODb {
             const extra = row[5]?.toString() || "";
             const field_params = {
                 isPrimaryKey: row[3] == "PRI",
+                isForeignKey: row[6] != null,
                 isAutoInc: extra.indexOf('auto_increment') >= 0,
                 isNotNull: row[2] == "NO"
             };
@@ -361,7 +391,7 @@ class OINODbMariadb extends db_1.OINODb {
                     api.datamodel.addField(new db_1.OINOStringDataField(this, field_name, sql_type, field_params, 0));
                 }
             }
-            res.next();
+            await res.next();
         }
         db_1.OINOLog.debug("OINODbMariadb.initializeDatasetModel:\n" + api.datamodel.printDebug("\n"));
         return Promise.resolve();

package/dist/esm/OINODbMariadb.js

@@ -35,14 +35,25 @@ class OINOMariadbData extends OINODbDataSet {
     }
     _currentRow;
     _eof;
+    /**
+     * Is data set empty.
+     *
+     */
     isEmpty() {
         return (this._rows.length == 0);
     }
-    // EOF means "there is no more content", i.e. either dataset is empty or we have moved beyond last line
+    /**
+     * Is there no more content, i.e. either dataset is empty or we have moved beyond last line
+     *
+     */
     isEof() {
         return (this._eof);
     }
-    next() {
+    /**
+     * Attempts to moves dataset to the next row, possibly waiting for more data to become available. Returns !isEof().
+     *
+     */
+    async next() {
         // OINOLog.debug("OINODbDataSet.next", {currentRow:this._currentRow, length:this.sqlResult.data.length})
         if (this._currentRow < this._rows.length - 1) {
             this._currentRow = this._currentRow + 1;
@@ -50,8 +61,12 @@ class OINOMariadbData extends OINODbDataSet {
         else {
             this._eof = true;
         }
-        return !this._eof;
+        return Promise.resolve(!this._eof);
     }
+    /**
+     * Gets current row of data.
+     *
+     */
     getRow() {
         if ((this._currentRow >= 0) && (this._currentRow < this._rows.length)) {
             return this._rows[this._currentRow];
@@ -68,7 +83,6 @@ class OINOMariadbData extends OINODbDataSet {
 export class OINODbMariadb extends OINODb {
     static _fieldLengthRegex = /([^\(\)]+)(\s?\((\d+)\s?\,?\s?(\d*)?\))?/i;
     static _exceptionMessageRegex = /\(([^\)]*)\) (.*)\nsql\:(.*)?/i;
-    static _tableSchemaSql = `SHOW COLUMNS from `;
     _pool;
     /**
      * Constructor of `OINODbMariadb`
@@ -253,7 +267,7 @@ export class OINODbMariadb extends OINODb {
         }
         catch (err) {
             // ... error checks
-            throw new Error(OINO_ERROR_PREFIX + ": Error connecting to Postgresql server: " + err);
+            throw new Error(OINO_ERROR_PREFIX + ": Error connecting to OINODbMariadb server: " + err);
         }
     }
     /**
@@ -296,6 +310,21 @@ export class OINODbMariadb extends OINODb {
         OINOBenchmark.end("OINODb", "sqlExec");
         return result;
     }
+    _getSchemaSql(dbName, tableName) {
+        const sql = `SELECT
+    c.COLUMN_NAME,
+    c.COLUMN_TYPE,
+    c.IS_NULLABLE,
+    c.COLUMN_KEY,
+    c.COLUMN_DEFAULT,
+    c.EXTRA,
+    KCU.CONSTRAINT_NAME AS ForeignKeyName 
+FROM information_schema.COLUMNS C
+	LEFT JOIN information_schema.KEY_COLUMN_USAGE KCU ON KCU.TABLE_SCHEMA = C.TABLE_SCHEMA AND KCU.TABLE_NAME = C.TABLE_NAME AND C.COLUMN_NAME = KCU.COLUMN_NAME and KCU.REFERENCED_TABLE_NAME IS NOT NULL
+WHERE C.TABLE_SCHEMA = '${dbName}' AND C.TABLE_NAME = '${tableName}'
+ORDER BY C.ORDINAL_POSITION;`;
+        return sql;
+    }
     /**
      * Initialize a data model by getting the SQL schema and populating OINODbDataFields of
      * the model.
@@ -304,7 +333,7 @@ export class OINODbMariadb extends OINODb {
      *
      */
     async initializeApiDatamodel(api) {
-        const res = await this.sqlSelect(OINODbMariadb._tableSchemaSql + this._params.database + "." + api.params.tableName + ";");
+        const res = await this.sqlSelect(this._getSchemaSql(this._params.database, api.params.tableName));
         while (!res.isEof()) {
             const row = res.getRow();
             // OINOLog.debug("OINODbMariadb.initializeApiDatamodel", { description:row })
@@ -317,6 +346,7 @@ export class OINODbMariadb extends OINODb {
             const extra = row[5]?.toString() || "";
             const field_params = {
                 isPrimaryKey: row[3] == "PRI",
+                isForeignKey: row[6] != null,
                 isAutoInc: extra.indexOf('auto_increment') >= 0,
                 isNotNull: row[2] == "NO"
             };
@@ -358,7 +388,7 @@ export class OINODbMariadb extends OINODb {
                     api.datamodel.addField(new OINOStringDataField(this, field_name, sql_type, field_params, 0));
                 }
             }
-            res.next();
+            await res.next();
         }
         OINOLog.debug("OINODbMariadb.initializeDatasetModel:\n" + api.datamodel.printDebug("\n"));
         return Promise.resolve();

package/dist/types/OINODbMariadb.d.ts

@@ -6,7 +6,6 @@ import { OINODb, OINODbParams, OINODbDataSet, OINODbApi, OINODataCell } from "@o
 export declare class OINODbMariadb extends OINODb {
     private static _fieldLengthRegex;
     private static _exceptionMessageRegex;
-    private static _tableSchemaSql;
     private _pool;
     /**
      * Constructor of `OINODbMariadb`
@@ -67,6 +66,7 @@ export declare class OINODbMariadb extends OINODb {
      *
      */
     sqlExec(sql: string): Promise<OINODbDataSet>;
+    private _getSchemaSql;
     /**
      * Initialize a data model by getting the SQL schema and populating OINODbDataFields of
      * the model.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@oino-ts/db-mariadb",
-  "version": "0.0.15",
+  "version": "0.0.17",
   "description": "OINO TS package for using Mariadb databases.",
   "author": "Matias Kiviniemi (pragmatta)",
   "license": "MPL-2.0",
@@ -21,7 +21,7 @@
   "module": "./dist/esm/index.js",
   "types": "./dist/types/index.d.ts",
   "dependencies": {
-    "@oino-ts/db": "^0.0.15",
+    "@oino-ts/db": "^0.0.17",
     "mariadb": "3.2.3"
   },
   "devDependencies": {

package/src/OINODbMariadb.ts

@@ -41,25 +41,40 @@ class OINOMariadbData extends OINODbDataSet {
     private _currentRow: number
     private _eof: boolean
     
+    /**
+     * Is data set empty.
+     *
+     */
     isEmpty():boolean {
         return (this._rows.length == 0)
     }
 
-    // EOF means "there is no more content", i.e. either dataset is empty or we have moved beyond last line
+    /**
+     * Is there no more content, i.e. either dataset is empty or we have moved beyond last line
+     *
+     */
     isEof():boolean {
         return (this._eof)
     }
 
-    next():boolean {
+    /**
+     * Attempts to moves dataset to the next row, possibly waiting for more data to become available. Returns !isEof().
+     *
+     */
+    async next():Promise<boolean> {
         // OINOLog.debug("OINODbDataSet.next", {currentRow:this._currentRow, length:this.sqlResult.data.length})
         if (this._currentRow < this._rows.length-1) {
             this._currentRow = this._currentRow + 1
         } else {
             this._eof = true
         }
-        return !this._eof
+        return Promise.resolve(!this._eof)
     }
 
+    /**
+     * Gets current row of data.
+     *
+     */
     getRow(): OINODataRow {
         if ((this._currentRow >=0) && (this._currentRow < this._rows.length)) {
             return this._rows[this._currentRow]
@@ -77,7 +92,6 @@ export class OINODbMariadb extends OINODb {
     
     private static _fieldLengthRegex = /([^\(\)]+)(\s?\((\d+)\s?\,?\s?(\d*)?\))?/i
     private static _exceptionMessageRegex = /\(([^\)]*)\) (.*)\nsql\:(.*)?/i
-    private static _tableSchemaSql:string = `SHOW COLUMNS from ` 
     
     private _pool:mariadb.Pool
 
@@ -269,7 +283,7 @@ export class OINODbMariadb extends OINODb {
             return Promise.resolve(true)
         } catch (err) {
             // ... error checks
-            throw new Error(OINO_ERROR_PREFIX + ": Error connecting to Postgresql server: " + err)
+            throw new Error(OINO_ERROR_PREFIX + ": Error connecting to OINODbMariadb server: " + err)
         }        
     }
 
@@ -315,6 +329,23 @@ export class OINODbMariadb extends OINODb {
         return result
     }
 
+    private _getSchemaSql(dbName:string, tableName:string):string {
+        const sql = 
+`SELECT
+    c.COLUMN_NAME,
+    c.COLUMN_TYPE,
+    c.IS_NULLABLE,
+    c.COLUMN_KEY,
+    c.COLUMN_DEFAULT,
+    c.EXTRA,
+    KCU.CONSTRAINT_NAME AS ForeignKeyName 
+FROM information_schema.COLUMNS C
+	LEFT JOIN information_schema.KEY_COLUMN_USAGE KCU ON KCU.TABLE_SCHEMA = C.TABLE_SCHEMA AND KCU.TABLE_NAME = C.TABLE_NAME AND C.COLUMN_NAME = KCU.COLUMN_NAME and KCU.REFERENCED_TABLE_NAME IS NOT NULL
+WHERE C.TABLE_SCHEMA = '${dbName}' AND C.TABLE_NAME = '${tableName}'
+ORDER BY C.ORDINAL_POSITION;`
+        return sql
+    }
+
     /**
      * Initialize a data model by getting the SQL schema and populating OINODbDataFields of 
      * the model.
@@ -324,7 +355,7 @@ export class OINODbMariadb extends OINODb {
      */
     async initializeApiDatamodel(api:OINODbApi): Promise<void> {
         
-        const res:OINODbDataSet = await this.sqlSelect(OINODbMariadb._tableSchemaSql + this._params.database + "." + api.params.tableName + ";")
+        const res:OINODbDataSet = await this.sqlSelect(this._getSchemaSql(this._params.database, api.params.tableName))
         while (!res.isEof()) {
             const row:OINODataRow = res.getRow()
             // OINOLog.debug("OINODbMariadb.initializeApiDatamodel", { description:row })
@@ -337,6 +368,7 @@ export class OINODbMariadb extends OINODb {
             const extra:string = row[5]?.toString() || ""
             const field_params:OINODbDataFieldParams = {
                 isPrimaryKey: row[3] == "PRI",
+                isForeignKey: row[6] != null,
                 isAutoInc: extra.indexOf('auto_increment') >= 0,
                 isNotNull: row[2] == "NO"
             }            
@@ -375,7 +407,7 @@ export class OINODbMariadb extends OINODb {
                     api.datamodel.addField(new OINOStringDataField(this, field_name, sql_type, field_params, 0))
                 }   
             }
-            res.next()
+            await res.next()
         }
         OINOLog.debug("OINODbMariadb.initializeDatasetModel:\n" + api.datamodel.printDebug("\n"))
         return Promise.resolve()