dt-toolbox 7.0.0 → 7.1.1

package/Changelog.md

@@ -1,5 +1,20 @@
 ## Release History
 
+### 7.1.1 (2023-09-05 )
+- [x] Fix: Multiple queries in same programming scope;
+
+
+
+### 7.1.0 (2023-08-28)
+- [x] Method `extractList` was added;
+- [x] Fix: Method `model` recognizes '**files**' as a model;
+- [x] Fix: Tuple model if data is just an array is wrong;
+- [x] Fix: Convertor to files shows 'root' in the beginning of the path;
+- [x] Fix: Convertor to breadcrumbs shows 'root' in the beginning of the path;
+- [ ] Bug: Multiple queries in same programming scope;
+
+
+
 ### 7.0.0 (2023-07-15)
 - [x] Method `look` has new arguments: functions `next` and `finish`;
 - [x] To stop iteration on current dt-line `return next()`;
@@ -7,7 +22,12 @@
 - [x] Method `insert` was renamed to `insertSegment`;
 - [x] Method `listSegments` was added to show list of all segments in dt-object;
 - [x] Method `insertSegment` expect the incoming data as dt-object, but if is not - will assume it as a standard javascript object and will convert it to dt-object automatically;
-- [] Method `insertSegment` can recognize a dt model object.
+- [ ] Method `insertSegment` can recognize a dt model object.
+- [ ] Bug: Method `model` doesn't recognize '**files**' as a model;
+- [ ] Bug: Tuple model if data is just an array is wrong;
+- [ ] Bug: Convertor to files shows 'root' in the beginning of the path;
+- [ ] Bug: Convertor to breadcrumbs shows 'root' in the beginning of the path;
+- [ ] Bug: Multiple queries in same programming scope;
 
 
 

package/Migration.guide.md

@@ -48,7 +48,24 @@ Method `insert` was renamed to `insertSegment` to be clear that data is not mixe
 
 Method `listSegments` was added to show list of all segments in dt-object.
 
+After version 7.1.x method `extractList` was added. Method helps to extract from dt-object multiple segments and properties, defined in a list. Options are coming as a second argument. Use them to define a model of extracted data.
 
+```js
+const 
+    first = { name: 'first', data: 'first data' }
+  , second = { name: 'second', data: 'second data' }
+  , third = { name: 'third', data: 'third data' }
+  ;
+const storage = dtbox.init ( first );   // first will become a root segment
+storage.insertSegment ( 'second', second );
+storage.insertSegment ( 'third', third );
+
+const [ a, b c, firstData ] = storage.extractList ( ['first', 'second', 'third', 'data' ], { type: 'std' } ));
+// a -> { name: 'first', data: 'first data' }
+// b -> { name: 'second', data: 'second data' }
+// c -> { name: 'third', data: 'third data' }
+// firstData -> 'first data' // Field 'data' from main dt-line of 'root' segment.
+```
 
 ## From v.4.x.x  - v.6.x.x
 

package/README.md

@@ -3,23 +3,14 @@
 ![version](https://img.shields.io/github/package-json/v/peterNaydenov/dt-toolbox)
 ![license](https://img.shields.io/github/license/peterNaydenov/dt-toolbox)
 
- - [Documentatation for old v.6.x.x is here](htts://github.com/PeterNaydenov/dt-toolbox/blob/master/README_v.6.x.x.md)
 
 
+## Last Updates
 
-## Breaking changes 
-* In version 7 look functions have 2 new arguments: Function `finish` and `next`;
-* Instead of returning a string 'next'(in version 6) to stop iteration on current dt-line, now you have to `return next()`;
-* Function `finish` is a new option. Call `return finish()` to stop iteration on all dt-lines;
-* Method `insert` was renamed to `insertSegment` to be clear that data is not mixed. Segments are separated peaces of data;
+After version 7.1.x `dt-object` api has a new method `extractList` that helps to extract a multiple segments or properties, defined as a list. Use '**options**'(the second argument) to define a model of extracted data if needed. Reed about `extractList` bellow.
 
 
 
-## Other changes
-* Method `listSegments` was added to show list of all segments in dt-object;
-* Method `insertSegment` expect the incoming data as dt-object, but if is not - will assume it as a standard javascript object and will convert it to dt-object automatically;
-
-
 ## Description
 
 DT-Toolbox is created to simplify the work with deep nested javascript objects. The library was created as an immutable data-storage(dt-object), internally based on data-model called `DT-model`.
@@ -132,6 +123,7 @@ Take a look on the library APIs and see the '**Examples**' section bellow.
   , setupFilter : 'Evaluate data according "filter" function and create a shorter scan list that can be used by "dt-storage" during execution of query and model functions'
   , index    : 'Provides a copy of specified dt-line by breadcrumbs'
   , listSegments : 'Returns a list of all segments in dt-object'
+  , extractList : 'Extracts a list of segments and/or properties from dt-object'
 ```
 
 ### dt-storage API Fast Reference
@@ -463,9 +455,10 @@ function modelFn ( store, a,v,g ) { // optional arguments will come directly aft
                 as : 'std' // property 'as' will execute final conversion. Model name should be from supported list of the library.
                 }
     }
+// Supported model-names are:
+// 'standard', 'std', 'midFlat', 'tuple', 'tuples', 'breadcrumb', 'breadcrumbs'
 
 const result = dt.model ( modelFn, a,v,g ) // modelFn is the only required argument. All other arguments are optional.
-
 ```
 
 ### dt.setupFilter ()
@@ -523,6 +516,30 @@ dt.listSegments ()
 // [ 'root', 'extra' ]
 ```
 
+### dt.extractList ()
+
+After version 7.1.x method `extractList` was added. Method can extract a list of segments and properties as a single instruction. Options are coming as a second argument. Use optioins(the second argument) to define a model of extracted data if needed. Modeling is applied only on objects. 
+
+If requested segment or property is not available, response will be '**null**'.
+Segments have priority over properties. If there is a segment with the same name as a property, segment will be extracted.
+
+```js
+const 
+    first = { name: 'first', data: 'first data' }
+  , second = { name: 'second', data: 'second data' }
+  , third = { name: 'third', data: 'third data' }
+  ;
+const storage = dtbox.init ( first );   // first will become a root segment
+storage.insertSegment ( 'second', second );
+storage.insertSegment ( 'third', third );
+
+const [ a, b c, firstData, otherData ] = storage.extractList ( ['first', 'second', 'third', 'data', 'secondData' ], { type: 'std' } ));
+// a -> { name: 'first', data: 'first data' }
+// b -> { name: 'second', data: 'second data' }
+// c -> { name: 'third', data: 'third data' }
+// firstData -> 'first data' // Field 'data' from main dt-line of 'root' segment.
+// otherData -> null // There is no 'secondData' in 'root' segment. No segment 'secondData' as well.
+```
 
 
 
@@ -1157,8 +1174,8 @@ console.log ( result )
 
 
 ## External Links
-- [Migration guide](https://github.com/PeterNaydenov/dt-toolbox/blob/master/Migration.guide.md)
 - [History of changes](https://github.com/PeterNaydenov/dt-toolbox/blob/master/Changelog.md)
+- [Migration guide](https://github.com/PeterNaydenov/dt-toolbox/blob/master/Migration.guide.md)
 - [DT-Queries. Some useful query functions for DT-Toolbox](https://github.com/PeterNaydenov/dt-queries)
 - [Documentation v.4.x.x](https://github.com/PeterNaydenov/dt-toolbox/blob/master/README_v.4.x.x.md)
 - [Documentation v.2.x.x](https://github.com/PeterNaydenov/dt-toolbox/blob/master/README_v.2.x.x.md)

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dt-toolbox",
   "description": "Data manipulation tool",
-  "version": "7.0.0",
+  "version": "7.1.1",
   "license": "MIT",
   "author": "Peter Naydenov",
   "keywords": [
@@ -32,7 +32,7 @@
   },
   "devDependencies": {
     "c8": "^8.0.1",
-    "chai": "^4.3.7",
+    "chai": "^4.3.8",
     "mocha": "^10.2.0"
   },
   "c8": {

package/src/convertors/index.js

@@ -65,7 +65,7 @@ function to ( type, dependencies, inData ) {   // convert from flat to 'data-typ
                             return tuples.toType ( inData )
             case 'files':
                            tups = tuples.toType ( inData );
-                           return tups.map ( ([k,v]) => `${k}/${v}`)
+                           return tups.map ( ([k,v]) =>  (( k === 'root' ) ? v : `${k}/${v}`)   )
             case 'breadcrumb' :
             case 'breadcrumbs':
                            let current;
@@ -77,7 +77,7 @@ function to ( type, dependencies, inData ) {   // convert from flat to 'data-typ
                                                                         current = k
                                                                         count = 0
                                                             }
-                                                        let key = `${k}/${count}`;
+                                                        let key = ( k === 'root' ) ? count : `${k}/${count}` 
                                                         breadcrumbs[key] = v
                                                         count++
                                                     }

package/src/convertors/tuples.js

@@ -140,18 +140,18 @@ function toFlat ( dependencies, d ) {   // Converts data to 'dt' model
 
 function toType ( dt ) {   // Converts data to tuples
         let result = [];
+        
         dt.forEach ( line => {
                     const 
                           [ name, flatData, breadcrumbs ] = line
                         , isArray = flatData instanceof Array ? true : false
-                        // , result = []
                         ;
                     let nakedBread = '';
                     if ( name !== 'root' )    nakedBread = breadcrumbs.replace ( `root/`, '' )
 
                     if ( isArray ) {
-                                if ( nakedBread.length == 0 )   result.push (['root', el])
-                                else   flatData.forEach ( el => result.push ([nakedBread, el]))
+                                if ( nakedBread.length == 0 )   flatData.forEach ( (el,i) => result.push ([`root`, el]))
+                                else                            flatData.forEach (  el    => result.push ([nakedBread, el]))
                                 return
                         }
                     

package/src/draft/connect.js

@@ -0,0 +1,50 @@
+'use strict'
+
+
+function connect ( selectionIx ) {
+return function connect ( list ) {
+            list.forEach ( item => {
+                        let arr = item.split ( '/' )
+                        const 
+                               child = arr.pop ()
+                             , parent = arr.pop ()
+                            , line = selectionIx[parent]
+                            , childLine = selectionIx[child]
+                            ;
+                            
+                        if ( !childLine     )   return this
+                        if ( !line          )   return this
+                        if ( line[1][child] )   return this
+
+                        let breadcrumbs = `${line[2]}/${child}`;
+
+                        updateBreadcrumbs ( selectionIx , child, breadcrumbs )
+                        if ( line[3].includes(breadcrumbs) )   return this
+                        line[3].push ( breadcrumbs )
+                        return this
+                })
+}} // connect func.
+
+
+
+function updateBreadcrumbs ( selectionIx, child, bread ) {
+        const 
+              childLine = selectionIx[child]
+            , oldBread = childLine [2]
+            ;
+
+        childLine[2] = bread
+        selectionIx[oldBread] = childLine
+
+        childLine[3].forEach ( (br,i) => {
+                        const search = new RegExp (`^${oldBread}\/`);
+                        const brUpdate = br.replace ( search, `${bread}/` );
+                        updateBreadcrumbs ( selectionIx[br], br, brUpdate )
+                })
+} // updateBreadcrumbs func.
+
+
+
+export default connect
+
+

package/src/draft/index.js

@@ -0,0 +1,13 @@
+import push from './push.js'
+import set from './set.js'
+import connect from './connect.js'
+import save from './save.js'
+
+
+
+export default {
+                      push
+                    , set
+                    , connect
+                    , save
+            }

package/src/flatData/get.js

@@ -8,7 +8,7 @@
 
 
 function get ( flatIO ) {
-return function find ( name ) {
+return function get ( name ) {
         const list = [];
         flatIO.getScanList().every ( line => {
                         const loc = line[2];

package/src/flatData/index.js

@@ -4,11 +4,6 @@ import insert      from './insert.js'
 import ex          from './export.js'
 import setupFilter from './setupFilter.js'
 
-import set from './set.js'
-import connect from './connect.js'
-import save from './save.js'
-import push from './push.js'
-
 import fr   from './from.js'
 import use  from './use.js'
 import find  from './find.js'
@@ -16,13 +11,14 @@ import like from './like.js'
 import look from './look.js'
 import get  from './get.js'
 
-
 // Default filters
 import list       from './filters/list.js'
 import listObject from './filters/listObject.js'
 import object     from './filters/object.js'
 import root       from './filters/root.js'
 
+
+
 function flatData ( dependencies, d ) {
           const 
               [ copy, indexes, flatData ] = d
@@ -31,11 +27,7 @@ function flatData ( dependencies, d ) {
             , filterFn = { list, listObject, object, root } // { filter functions}
             ;
 
-        let 
-              selection = []          // Place for building a query/model result;
-            , selectionIx = {}
-            , scanList  = flatData     // Where 'look' function will be executed;
-            ;
+        let scanList  = flatData;   // Where 'look' function will be executed;
             
         const  
                 getFilterNames = () => Object.keys ( filterFn )
@@ -60,35 +52,23 @@ function flatData ( dependencies, d ) {
                           insert  : insert ( flatStorage, filtering, getFilterNames )                     // Insert a new rows to existing flatData;
                         , export  : ex ( flatStorage )                                                   // Exports a copy of flatData;
                         , getCopy : name => ( copy[name] ) ? copy[name] : null 
-                        , getSelection  : () => selection
-                        , getSelectionIx : () => selectionIx
                         , getIndexes    : () => indexes
                         , getLine       : k => indexes[k] ? indexes[k] : null
                         , getFilters    : () => filters
                         , getScanList   : () => scanList
                         , setupScanList : l =>  scanList = l
                         , setupFilter : setupFilter ( flatStorage, filterFn, filtering, getFilterNames )   // Add and apply a new filter function to flatData;
-                        , reset         : () => {  selection = [], scanList = flatData    }              // Removes 'selection' and reset the scanList to use all flatData;
-                        , resetScan     : () => { scanList = flatData }
+                        , resetScan     : () => { scanList = flatData }                                  // Reset the scanList to use all flatData;
                     }
-            , flatStoreSelection_API = {
-                          set     : set ( selection, selectionIx )      // Selection: Creates a new row;
-                        , connect : connect ( selectionIx )             // Selection: Creates an edges among rows;
-                        , save    : save ( selectionIx )                // Selection: Sets property+value in specified selection row;
-                        , push    : push ( selectionIx )                // Selection: Adds a new value to selection row if data is an array structure;
-                  }
+            
             , flatStore_API = {
-                          set     : flatStoreSelection_API.set  
-                        , connect : flatStoreSelection_API.connect
-                        , save    : flatStoreSelection_API.save
-                        , push    : flatStoreSelection_API.push
-                        
-                        , from : fr ( flatIO_API )                               // Scan: Scan deeper from object;
-                        , use  : use ( flatIO_API )                              // Scan: Use filter to select objects to scan;
-                        , get  : get ( flatIO_API )                              // Scan: Take a single object with specific breadcrumbs;
-                        , find  : find ( flatIO_API  )                             // Scan: String search for exact object name;
-                        , like : like ( flatIO_API )                             // Scan: String search in object name;
-                        , look : look ( flatIO_API )     // Scan: Executes on each object property in the selection list;
+                          // Query and Model functions will add methods related to new structure: set, connect, save, push;
+                          from : fr ( flatIO_API )     // Scan: Scan deeper from object;
+                        , use  : use ( flatIO_API )    // Scan: Use filter to select objects to scan;
+                        , get  : get ( flatIO_API )    // Scan: Take a single object with specific breadcrumbs;
+                        , find  : find ( flatIO_API  )   // Scan: String search for exact object name;
+                        , like : like ( flatIO_API )   // Scan: String search in object name;
+                        , look : look ( flatIO_API )   // Scan: Executes on each object property in the selection list;
                     }
             ;
         return [ flatIO_API , flatStore_API ]

package/src/flatObject/extractList.js

@@ -0,0 +1,46 @@
+'use strict'
+
+
+
+function extractList ( dependencies, flatIO, indexFn ) {
+return function exportList ( list, options={} ) {
+            const 
+                      root     = indexFn ( 'root' )
+                    , { main:{load} } = dependencies ()
+                    , possibleAs = ['standard', 'std', 'midFlat', 'tuple', 'tuples', 'breadcrumb', 'breadcrumbs', 'files', 'file' ]
+                    ;
+            let 
+                  error = false
+                , errorMsg = ''
+                ;
+
+            if ( options ) {
+                    if ( !options.as ) {
+                            error = true
+                            errorMsg = `Options should be an object and property "as" is required`
+                        }
+                    if ( !error && !possibleAs.includes(options.as) ) {  
+                            error = true
+                            errorMsg = `Invalid option "as" value: ${options.as}. Choose one of: std, standard, ''`
+                        }
+                    if ( error )  throw new Error ( `Invalid option "as" value: ${options.as}` )
+                }
+
+            return list
+                        .map ( name => {
+                                            let lines = flatIO.export ( name );
+                                            if ( lines.length === 0 )  return root[1][name] ? root[1][name] : null
+                                            else                       return lines
+                                        })
+                        .map ( item => {
+                                            if ( item == null            )   return null
+                                            if ( !(item instanceof Array))   return item
+                                            return options ? load(item).model(() => options ) : load(item)
+                                        })
+}} // extractList func.
+
+
+
+export default extractList
+
+

package/src/flatObject/index.js

@@ -8,6 +8,9 @@ import copy        from './copy.js'
 import model       from './model.js'
 import query       from './query.js'
 import setupFilter from './setupFilter.js'
+import extractList from './extractList.js'
+
+
 
 
 
@@ -31,20 +34,24 @@ function flatObject ( dependencies, d ) {
                                                               return res
                                                         }, [] )
                                 }
-                    , index  : n => {
-                                        if ( n == null )   return null
-                                        let r  = flatIO.getLine(n)
-                                        if ( r ) {
-                                              let [ name, d, breadcrumbs, links ] = r;
-                                              let copy;
-                                              if ( d instanceof Array )   copy = [...d ]
-                                              else                        copy = {...d }
-                                              return [ name, copy, breadcrumbs, [...links] ]
-                                          }
-                                        else return null
-                                  } // index
+                    , index
                 }
             ;
+
+        function index (n) {
+                        if ( n == null )   return null
+                        let r  = flatIO.getLine(n)
+                        if ( r ) {
+                              let [ name, d, breadcrumbs, links ] = r;
+                              let copy;
+                              if ( d instanceof Array )   copy = [...d ]
+                              else                        copy = {...d }
+                              return [ name, copy, breadcrumbs, [...links] ]
+                          }
+                        else return null
+            } // index func.
+            
+        objectAPI [ 'extractList' ] = extractList ( dependencies, flatIO, index )
         return objectAPI
     } // flatObject func.
 

package/src/flatObject/insert.js

@@ -8,7 +8,6 @@ return function insert ( name, inData ) {
             if ( isDTO(inData) ) {
                         d    = inData.export ()
                         copy = inData.copy ()
-                        
                }
             else if ( isDTM(inData) ) {
                         d = walk ({ data : inData })

package/src/flatObject/model.js

@@ -4,13 +4,22 @@
 
 function model ( dependencies, flatIO ,flatStore ) {
 return function model () {
+    let 
+        selection = []          // Place for building a query/model result;
+      , selectionIx = {}
+      ;
     const 
-          [ fn, ...args ] = arguments
-        , { as } = fn ( flatStore, ...args ) || {}
-        , { convert } = dependencies ()
-        , selection = flatIO.getSelection ()
+          { convert, draft } = dependencies ()
+        , draft_API = {
+                          set     : draft.set ( selection, selectionIx )      // Selection: Creates a new row;
+                        , connect : draft.connect ( selectionIx )             // Selection: Creates an edges among rows;
+                        , save    : draft.save ( selectionIx )                // Selection: Sets property+value in specified selection row;
+                        , push    : draft.push ( selectionIx )                // Selection: Adds a new value to selection row if data is an array structure;
+                    }
+        , [ fn, ...args ] = arguments
+        , { as } = fn ( {...flatStore,...draft_API}, ...args ) || {}
         , data = ( selection.length === 0 ) ? flatIO.export() : selection
-        , CONVERTOR_NAMES = [ 'standard', 'std', 'midFlat', 'tuple', 'tuples', 'breadcrumb', 'breadcrumbs' ]
+        , CONVERTOR_NAMES = [ 'standard', 'std', 'midFlat', 'tuple', 'tuples', 'breadcrumb', 'breadcrumbs', 'files', 'file' ]
         , hasConvertor = as ? CONVERTOR_NAMES.includes ( as ) : false
         ;
 
@@ -19,9 +28,10 @@ return function model () {
                 console.error ( `Model '${as}' is unknown data-model.` )
                 return null
         }
+        
     if ( as )   result = convert.to ( as, dependencies, data )
     else        result = data
-    flatIO.reset ()
+    flatIO.resetScan ()
     return result
 }} // model func.
 

package/src/flatObject/query.js

@@ -2,19 +2,28 @@
 
 
 
-function query ( dependencies, flatIO ,flatStore ) {
+function query ( dependencies, flatIO, flatStore ) {
 return function query () {
     const 
-          { main } = dependencies ()
-        , { load } = main ()
+          { main:{load}, draft } = dependencies ()
         , [ fn, ...args ] = arguments
         ;
+    let 
+        selection = []          // Place for building a query/model result;
+      , selectionIx = {}
+      ;
+
+    const draft_API = {
+                  set     : draft.set ( selection, selectionIx )      // Selection: Creates a new row;
+                , connect : draft.connect ( selectionIx )             // Selection: Creates an edges among rows;
+                , save    : draft.save ( selectionIx )                // Selection: Sets property+value in specified selection row;
+                , push    : draft.push ( selectionIx )                // Selection: Adds a new value to selection row if data is an array structure;
+        }
     
-    fn ( flatStore, ...args )
-    const result = flatIO.getSelection ();
-    flatIO.reset ()
-    if ( result.length === 0 )   return this
-    return load ( result )
+    fn ( {...flatStore, ...draft_API}, ...args )
+    flatIO.resetScan ()
+    if ( selection.length === 0 )   return this
+    return load ( selection )
 }} // query func.
 
 

package/src/main.js

@@ -4,7 +4,7 @@
     DT object & DT Toolbox
     =======================
     
-    Version 6.0.0
+    Version 7.1.0
 
     History notes:
      - Idea was born on March 17th, 2016.
@@ -30,6 +30,10 @@
          * Data inserts;
          * Filters for faster data scan;
          * Model and query functions to shape results;
+    
+     - Version 7.1.0 published on August 29th, 2023
+            * Some fixes on version v.7.0.0;
+            * New method `extractList` for multiple data extraction from dt-model;
 */
 
 

package/src/mainLib.js

@@ -4,9 +4,10 @@
 
 import walk from '@peter.naydenov/walk'
 
-import flatObject from './flatObject/index.js'        // Flat data api.
-import flatData   from './flatData/index.js'            // Query data api. Used in model functions 
-import convert   from './convertors/index.js'        // Convertors
+import flatObject from './flatObject/index.js'     // Flat data api.
+import flatData   from './flatData/index.js'       // Query data api. Used in model functions 
+import convert   from './convertors/index.js'    // Convertors
+import draft     from './draft/index.js'         // Build a dt-model api
 
 
 
@@ -31,7 +32,7 @@ const mainLib = {
                             , flatObject
                             , convert
                             , INIT_DATA_TYPES
-                            , main : () => ({ load : mainLib.load })
+                            , main : { load : mainLib.load }
                             , isDTO : d  => typeof d.insertSegment  === 'function'
                             , isDTM : d => {
                                                 if ( !(d    instanceof Array))   return false
@@ -40,7 +41,7 @@ const mainLib = {
                                                 if ( d[0][0] !== 'root'      )   return false
                                                 return true
                                         }
-                            , 
+                            , draft
                         }
             } // dependencies func.
 
@@ -56,11 +57,8 @@ const mainLib = {
                             console.error ( `Can't understand your data-model: ${model}. Please, find what is possible on https://github.com/PeterNaydenov/dt-toolbox` )
                             return null
                         }
-                    const 
-                          d  = convert.from(model).toFlat ( dependencies, inData )
-                        , flat =  flatObject ( dependencies, d )
-                        ;
-                    return flat
+                    const d  = convert.from(model).toFlat ( dependencies, inData );
+                    return flatObject ( dependencies, d )
             } // init func.