dbmodel 0.1.2 → 5.2.2-alpha.12

package/LICENSE

@@ -1,21 +1,21 @@
-MIT License
-
-Copyright (c) 2019 dbshell
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+MIT License
+
+Copyright (c) 2019 dbshell
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,116 +1,102 @@
-# dbmodel
-Deploy, load or build script from model of SQL database. Can be used as command-line tool or as javascript functions
-
-Model is stored as a collection of files:
-* tables - stored as YAML files
-  * columns
-  * indexes
-  * primary keys
-  * foreign keys
-* views - stored as SQL file with extension **.view.sql**
-* stored procedures - stored as SQL file with extension **.proc.sql**
-* functions - stored as SQL file with extension **.func.sql**
-
-Currently only supported client is **mssql** .
-
-## Installation - as global tool
-
-    npm install --global dbmodel
-
-## Installation - as regular package
-
-    npm install --save dbmodel
-
-## Available commands
-* **load** - loads structure of database, saves it to local directory (called *project*). Also can download data of enlisted tables (use --load-data-condition options)
-* **deploy** - deploys structure from local directory (*project*) to database. *Deploy does not perform any actions leading to data loss, these changes must be made manually.*
-  * creates not existing tables
-  * creates not existing columns of existing tables
-  * checks column NULL/NOT NULL flag, alters colums
-  * checks tables, which are in database, but not in project, list of these tables are reported
-  * checks columns, which are in database, but not in project, list of these columns are reported
-  * checks indexes and its definitions, indexes are created or recreated, if neccessary (*but not deleted*)
-  * checks and creates foreign keys
-  * checks, creates new or changes existing views, stored procedures and functions
-  * updates and creates static data (included in table yaml files)
-* **build** - builds script from project folder. This operation is complete offline, no database connection is needed. Built script makes subset of deploy command. It can be executed on empty database, but also it can convert existing database to current structure (but only using operations below).
-  * creates not existing tables
-  * creates not existing columns of existing tables
-  * creates not existing indexes (checked only by name)
-  * creates not existing foreign keys
-  * creates new or changes existing views, stored procedures and functions
-  * updates and creates static data (included in table yaml files)
-
-## Command line interface
-
-```sh
-# load from existing database
-dbmodel load -s localhost -u USERNAME -p PASSWORD -d DATABASE -c mssql OUTPUT_FOLDER
-
-# deploy project to database
-dbmodel deploy -s localhost -u USERNAME -p PASSWORD -d DATABASE -c mssql PROJECT_FOLDER
-
-# build SQL script from project
-dbmodel build -c mssql PROJECT_FOLDER OUTPUT_FILE.sql
-```
-
-## JavaScript interface
-
-```javascript
-const dbmodel = require('dbmodel');
-
-await dbmodel.deploy({
-  client: 'mssql',
-  connection: {
-    server: '127.0.0.1',
-    user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
-    database: 'Chinook_Model',
-  },
-  hooks: [dbmodel.hooks.autoIndexForeignKeys], // this hook adds indexes to all foreign keys
-  projectDir: 'model',
-})
-```
-
-list of dbmodel exported functions:
-* build - builds SQL script
-* deploy - deploys model to database 
-* dump - dumps loaded model into directory
-* load - loads model from database
-* read - reads model from directory
-* connect - creates database connection defined in options
-
-## Table yaml file documentation
-
-```yaml
-name: Album # table name
-columns:
-  - name: AlbumId # column name
-    type: int # data type. is used directly in target SQL engine 
-    autoIncrement: true # column is autoincrement
-    notNull: true # column is not nullable (default: is nullable)
-  - name: Title
-    type: nvarchar
-    length: 160 # maximum character length
-    notNull: true
-  - name: ArtistId
-    type: int
-    references: Artist # name of table. Is used for creating foreign key
-  - name: isDeleted
-    type: bit
-    notNull: true
-    default: 0 # default value
-primaryKey:
-  - AlbumId # list of primary key column names
-indexes:
-  - name: UQ_AlbumTitleArtistId # index name
-    unique: true # whether index is unique. default=false
-    columns: # list of index columns
-      - Title
-      - ArtistId
-    filter: isDeleted=0 # if defined, filtered index (with WHERE condition) is created
-    continueOnError: true # if true and there was error in creating this index, continue (suitable for lately added unique indexes)
-data: # static data (only for list tables)
-  - AlbumId: -1 # values for all columns, which should be filled
-    Title: Predefined static album
-```
+# dbmodel
+Deploy, load or build script from model of SQL database. Can be used as command-line tool. Uses [DbGate](https://dbgate.org) tooling and plugins for connecting many different databases.
+
+If you want to use this tool from JavaScript interface, please use [dbgate-api](https://www.npmjs.com/package/dbgate-api) package.
+
+Model is stored as a collection of files:
+* tables - stored as YAML files
+  * columns
+  * indexes
+  * primary keys
+  * foreign keys
+* views - stored as SQL file with extension **.view.sql**
+* stored procedures - stored as SQL file with extension **.proc.sql**
+* functions - stored as SQL file with extension **.func.sql**
+
+## Installation - as global tool
+
+    npm install --global dbmodel
+
+## Installation - as regular package
+
+    npm install --save dbmodel
+
+## Available commands
+* **load** - loads structure of database, saves it to local directory (called *project*). Also can download data of enlisted tables (use --load-data-condition options)
+* **deploy** - deploys structure from local directory (*project*) to database. *Deploy does not perform any actions leading to data loss, these changes must be made manually.*
+  * creates not existing tables
+  * creates not existing columns of existing tables
+  * checks column NULL/NOT NULL flag, alters colums
+  * checks tables, which are in database, but not in project, list of these tables are reported
+  * checks columns, which are in database, but not in project, list of these columns are reported
+  * checks indexes and its definitions, indexes are created or recreated, if neccessary (*but not deleted*)
+  * checks and creates foreign keys
+  * checks, creates new or changes existing views, stored procedures and functions
+  * updates and creates static data (included in table yaml files)
+* **build** - builds script from project folder. This operation is complete offline, no database connection is needed. Built script makes subset of deploy command. It can be executed on empty database, but also it can convert existing database to current structure (but only using operations below).
+  * creates not existing tables
+  * creates not existing columns of existing tables
+  * creates not existing indexes (checked only by name)
+  * creates not existing foreign keys
+  * creates new or changes existing views, stored procedures and functions
+  * updates and creates static data (included in table yaml files)
+
+## Command line interface
+
+```sh
+# load from existing database
+dbmodel load -s localhost -u USERNAME -p PASSWORD -d DATABASE -e mssql@dbgate-plugin-mssql OUTPUT_FOLDER
+
+# deploy project to database
+dbmodel deploy -s localhost -u USERNAME -p PASSWORD -d DATABASE -e mssql@dbgate-plugin-mssql PROJECT_FOLDER
+
+# build SQL script from project
+dbmodel build -e mssql@dbgate-plugin-mssql PROJECT_FOLDER OUTPUT_FILE.sql
+```
+
+Parameter -e (or --engine) specifies database dialect and connection driver to be used
+Supported databases:
+- MySQL - `-e mysql@dbgate-plugin-mysql`
+- MS SQL Server - `-e mssql@dbgate-plugin-mssql`
+- PostgreSQL - `-e postgres@dbgate-plugin-postgres`
+- SQLite - `-e sqlite@dbgate-plugin-sqlite`
+- Oracle - `-e oracle@dbgate-plugin-oracle`
+- MariaDB - `-e mariadb@dbgate-plugin-mysql`
+- CockroachDB - `-e cockroach@dbgate-plugin-postgres`
+- Amazon Redshift - `-e redshift@dbgate-plugin-postgres`
+
+
+## Table yaml file documentation
+
+```yaml
+name: Album # table name
+columns:
+  - name: AlbumId # column name
+    type: int # data type. is used directly in target SQL engine 
+    autoIncrement: true # column is autoincrement
+    notNull: true # column is not nullable (default: is nullable)
+  - name: Title
+    type: nvarchar
+    length: 160 # maximum character length
+    notNull: true
+  - name: ArtistId
+    type: int
+    references: Artist # name of table. Is used for creating foreign key
+  - name: isDeleted
+    type: bit
+    notNull: true
+    default: 0 # default value
+primaryKey:
+  - AlbumId # list of primary key column names
+indexes:
+  - name: UQ_AlbumTitleArtistId # index name
+    unique: true # whether index is unique. default=false
+    columns: # list of index columns
+      - Title
+      - ArtistId
+    filter: isDeleted=0 # if defined, filtered index (with WHERE condition) is created
+    continueOnError: true # if true and there was error in creating this index, continue (suitable for lately added unique indexes)
+data: # static data (only for list tables)
+  - AlbumId: -1 # values for all columns, which should be filled
+    Title: Predefined static album
+```

package/bin/dbmodel.js

@@ -1,81 +1,118 @@
-#!/usr/bin/env node
-
-const program = require('commander');
-const dbmodel = require('../lib');
-
-program
-  .option('-s, --server <server>', 'server host')
-  .option('-u, --user <user>', 'user name')
-  .option('-p, --password <password>', 'password')
-  .option('-d, --database <database>', 'database name')
-  .option('--auto-index-foreign-keys', 'automatically adds indexes to all foreign keys')
-  .option(
-    '--load-data-condition <condition>',
-    'regex, which table data will be loaded and stored in model (in load command)'
-  )
-  .requiredOption('-c, --client <client>', 'client name, must be mssql');
-
-program
-  .command('deploy <projectDir>')
-  .description('Deploys model to database')
-  .action(projectDir => {
-    const { client, server, user, password, database } = program;
-    const hooks = [];
-    if (program.autoIndexForeignKeys) hooks.push(dbmodel.hooks.autoIndexForeignKeys);
-    dbmodel.runAndExit(
-      dbmodel.deploy({
-        client,
-        connection: {
-          server,
-          user,
-          password,
-          database,
-        },
-        hooks,
-        projectDir,
-      })
-    );
-  });
-
-program
-  .command('load <outputDir>')
-  .description('Loads model from database')
-  .action(outputDir => {
-    const { client, server, user, password, database } = program;
-    const loadDataCondition = program.loadDataCondition
-      ? table => table.name.match(new RegExp(program.loadDataCondition, 'i'))
-      : null;
-    const hooks = [];
-    dbmodel.runAndExit(
-      dbmodel.dump({
-        client,
-        connection: {
-          server,
-          user,
-          password,
-          database,
-        },
-        hooks,
-        outputDir,
-        loadDataCondition,
-      })
-    );
-  });
-
-program
-  .command('build <projectDir> <outputFile>')
-  .description('Builds single SQL script from project')
-  .action((projectDir, outputFile) => {
-    const { client } = program;
-    const hooks = [];
-    dbmodel.runAndExit(
-      dbmodel.build({
-        client,
-        hooks,
-        projectDir,
-        outputFile,
-      })
-    );
-  });
-
-program.parse(process.argv);
+#!/usr/bin/env node
+
+const path = require('path');
+require('dotenv').config();
+
+global.API_PACKAGE = path.dirname(path.dirname(require.resolve('dbgate-api')));
+global.PLUGINS_DIR = process.env.DEVMODE
+  ? path.join(path.dirname(path.dirname(global.API_PACKAGE)), 'plugins')
+  : path.dirname(global.API_PACKAGE);
+global.IS_NPM_DIST = true;
+
+const program = require('commander');
+const dbgateApi = require('dbgate-api');
+const { createLogger } = require('pinomin');
+
+const logger = createLogger('dbmodel');
+
+async function runAndExit(promise) {
+  try {
+    await promise;
+    logger.info('Success');
+    process.exit();
+  } catch (err) {
+    logger.error({ err }, 'Processing failed');
+    process.exit(1);
+  }
+}
+
+program
+  .option('-s, --server <server>', 'server host')
+  .option('-u, --user <user>', 'user name')
+  .option('-p, --password <password>', 'password')
+  .option('-d, --database <database>', 'database name')
+  .option('--auto-index-foreign-keys', 'automatically adds indexes to all foreign keys')
+  .option(
+    '--load-data-condition <condition>',
+    'regex, which table data will be loaded and stored in model (in load command)'
+  )
+  .requiredOption('-e, --engine <engine>', 'engine name, eg. mysql@dbgate-plugin-mysql');
+
+program
+  .command('deploy <modelFolder>')
+  .description('Deploys model to database')
+  .action(modelFolder => {
+    const { engine, server, user, password, database } = program.opts();
+    // const hooks = [];
+    // if (program.autoIndexForeignKeys) hooks.push(dbmodel.hooks.autoIndexForeignKeys);
+
+    runAndExit(
+      dbgateApi.deployDb({
+        connection: {
+          engine,
+          server,
+          user,
+          password,
+          database,
+        },
+        modelFolder,
+      })
+    );
+  });
+
+//     runAndExit(
+//       dbmodel.deploy({
+//         connection: {
+//           engine,
+//           server,
+//           user,
+//           password,
+//           database,
+//         },
+//         hooks,
+//         projectDir,
+//       })
+//     );
+
+program
+  .command('load <outputDir>')
+  .description('Loads model from database')
+  .action(outputDir => {
+    const { engine, server, user, password, database } = program.opts();
+    // const loadDataCondition = program.loadDataCondition
+    //   ? table => table.name.match(new RegExp(program.loadDataCondition, 'i'))
+    //   : null;
+    // const hooks = [];
+
+    runAndExit(
+      dbgateApi.loadDatabase({
+        connection: {
+          engine,
+          server,
+          user,
+          password,
+          database,
+        },
+        outputDir,
+      })
+    );
+  });
+
+program
+  .command('build <modelFolder> <outputFile>')
+  .description('Builds single SQL script from project')
+  .action((modelFolder, outputFile) => {
+    const { engine } = program.opts();
+    // const hooks = [];
+    runAndExit(
+      dbgateApi.generateModelSql({
+        // client,
+        // hooks,
+        modelFolder,
+        outputFile,
+        engine,
+      })
+    );
+  });
+
+program.parse(process.argv);

package/package.json

@@ -1,47 +1,45 @@
-{
-  "name": "dbmodel",
-  "version": "0.1.2",
-  "description": "Deploy, load or build script from model of SQL database",
-  "main": "lib/index.js",
-  "scripts": {
-    "test": "echo \"Error: no test specified\" && exit 1",
-    "lint": "eslint"
-  },
-  "repository": {
-    "type": "git",
-    "url": "git+https://github.com/dbshell/dbmodel.git"
-  },
-  "bin": {
-    "dbmodel": "./bin/dbmodel.js"
-  },
-  "keywords": [
-    "sql",
-    "model",
-    "deploy",
-    "mssql"
-  ],
-  "author": "Jan Prochazka",
-  "license": "MIT",
-  "bugs": {
-    "url": "https://github.com/dbshell/dbmodel/issues"
-  },
-  "homepage": "https://github.com/dbshell/dbmodel#readme",
-  "devDependencies": {
-    "eslint": "^6.7.0"
-  },
-  "files": [
-    "LICENSE",
-    "README.md",
-    "bin",
-    "lib"
-  ],
-  "dependencies": {
-    "commander": "^4.0.1",
-    "fs-readdir-recursive": "^1.1.0",
-    "js-yaml": "^3.13.1",
-    "json-normalize": "^1.1.2",
-    "mssql": "^6.0.1",
-    "source-map-support": "^0.5.16",
-    "toposort": "^2.0.2"
-  }
-}
+{
+  "name": "dbmodel",
+  "version": "5.2.2-alpha.12",
+  "homepage": "https://dbgate.org/",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/dbgate/dbgate.git"
+  },
+  "description": "Deploy, load or build script from model of SQL database",
+  "author": "Jan Prochazka",
+  "license": "MIT",
+  "bin": {
+    "dbmodel": "./bin/dbmodel.js"
+  },
+  "keywords": [
+    "sql",
+    "dbgate",
+    "web"
+  ],
+  "scripts": {
+    "dbmodel": "node ./bin/dbmodel.js",
+    "dbmodel:load": "cross-env DEVMODE=1 node ./bin/dbmodel.js load testdata/db -e postgres@dbgate-plugin-postgres -s localhost -u postgres -p Pwd2020Db -d zradlo",
+    "dbmodel:deploy": "cross-env DEVMODE=1 node ./bin/dbmodel.js deploy testdata/db -e postgres@dbgate-plugin-postgres -s localhost -u postgres -p Pwd2020Db -d deployed",
+    "dbmodel:build": "cross-env DEVMODE=1 node ./bin/dbmodel.js build testdata/db testdata/db.sql -e postgres@dbgate-plugin-postgres "
+  },
+  "files": [
+    "LICENSE",
+    "README.md",
+    "bin"
+  ],
+  "dependencies": {
+    "commander": "^10.0.0",
+    "dbgate-api": "^5.2.2-alpha.12",
+    "dbgate-plugin-csv": "^5.2.2-alpha.12",
+    "dbgate-plugin-excel": "^5.2.2-alpha.12",
+    "dbgate-plugin-mongo": "^5.2.2-alpha.12",
+    "dbgate-plugin-mssql": "^5.2.2-alpha.12",
+    "dbgate-plugin-mysql": "^5.2.2-alpha.12",
+    "dbgate-plugin-postgres": "^5.2.2-alpha.12",
+    "dbgate-plugin-xml": "^5.2.2-alpha.12",
+    "dbgate-web": "^5.2.2-alpha.12",
+    "dotenv": "^16.0.0",
+    "pinomin": "^1.0.1"
+  }
+}

package/lib/build.js

@@ -1,24 +0,0 @@
-var fs = require('fs');
-const read = require('./read');
-const runHooks = require('./hooks/runHooks');
-
-async function build(options) {
-  const { client } = options;
-  await read(options);
-
-  const { outputFile } = options;
-  options.outputDescriptor = fs.openSync(outputFile, 'w');
-
-  runHooks(options, 'before-build');
-
-  const buildFunc = require(`./clients/${client}/build`);
-  await buildFunc(options);
-
-  runHooks(options, 'after-build');
-
-  fs.close(options.outputDescriptor);
-
-  return options;
-}
-
-module.exports = build;