@speckle/objectloader 2.3.0 → 2.3.17-alpha.2819

package/.eslintrc.cjs

@@ -0,0 +1,16 @@
+/**
+ * Extends repo root config, only put changes here that are scoped to this specific package
+ * (if you already are - evaluate whether you really need package scoped linting rules)
+ */
+
+/** @type {import("eslint").Linter.Config} */
+const config = {
+  env: {
+    browser: true
+  },
+  parserOptions: {
+    sourceType: 'module'
+  }
+}
+
+module.exports = config

package/LICENSE

@@ -0,0 +1,208 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2020 AEC Systems
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+   
+   NOTICE: Unless otherwise described, the code in this repository is 
+   licensed under the license above. Some modules, extensions or code herein 
+   might be otherwise licensed. This is indicated either in the root of the 
+   containing folder under a different license file, or in the respective 
+   file's header. If you have any questions, don't hesitate to get in touch 
+   with us via [email](mailto:hello@speckle.systems).

package/examples/browser/index.html

@@ -1,19 +1,21 @@
 <!DOCTYPE html>
 <html lang="en" class="no-js">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width" />
 
-<head>
-  <meta charset="UTF-8">
-  <meta name="viewport" content="width=device-width">
+    <title>Object Loader Test</title>
+  </head>
 
-  <title>Object Loader Test</title>
-</head>
+  <body>
+    <h1>This is a test.</h1>
+    <p>All the magic is in the console.</p>
+    <p>
+      To ensure this example runs correctly, please serve this file from a local http
+      server - if you manually open the file in a browser, it might not work.
+    </p>
+    <button onclick="loadData()">PRESS ME</button>
+  </body>
 
-<body>
-  <h1>This is a test.</h1>
-  <p>All the magic is in the console.</p>
-  <p>To ensure this example runs correctly, please serve this file from a local http server - if you manually open the file in a browser, it might not work.</p>
-  <button onclick="loadData()">PRESS ME</button>
-</body>
-  
   <script src="script.js" type="module"></script>
-</html>
+</html>

package/examples/browser/script.js

@@ -9,25 +9,23 @@ window.ObjectLoader = ObjectLoader
 
 // https://latest.speckle.dev/streams/92b620fb17/objects/7cd9d41b5b5f3c8908536aec2a05f1a1
 // let loader = new ObjectLoader({
-//   serverUrl:"https://latest.speckle.dev", 
-//   streamId:"92b620fb17", 
+//   serverUrl:"https://latest.speckle.dev",
+//   streamId:"92b620fb17",
 //   objectId:"878c426bb213ddb4d580da74922a2b16"
 // })
 
 // https://latest.speckle.dev/streams/3ed8357f29/objects/0408ab9caaa2ebefb2dd7f1f671e7555
-let loader = new ObjectLoader({
-  serverUrl:"https://latest.speckle.dev", 
-  streamId:"3ed8357f29", 
-  objectId:"0408ab9caaa2ebefb2dd7f1f671e7555"
+const loader = new ObjectLoader({
+  serverUrl: 'https://latest.speckle.dev',
+  streamId: '3ed8357f29',
+  objectId: '0408ab9caaa2ebefb2dd7f1f671e7555'
 })
 
-
 window.loadData = async function loadData() {
-  
-  let obj = await loader.getAndConstructObject((e) =>{
+  const obj = await loader.getAndConstructObject((e) => {
     console.log(e) // log progress!
   })
 
   console.log('Done!')
-  console.log( obj )
-}
+  console.log(obj)
+}

package/examples/node/script.js

@@ -1 +1,22 @@
-// NOTE: This lib is not working in node, because node-fetch returns node-native readable streams - we need a workaround first. 
+// Since Node v<18 does not provide fetch, we need to pass it in the options object. Note that fetch must return a WHATWG compliant stream, so cross-fetch won't work, but node/undici's implementation will.
+
+import { fetch } from 'undici'
+import ObjectLoader from '../../index.js'
+
+const loader = new ObjectLoader({
+  serverUrl: 'https://latest.speckle.dev',
+  streamId: '3ed8357f29',
+  objectId: '0408ab9caaa2ebefb2dd7f1f671e7555',
+  options: { enableCaching: false, excludeProps: [], fetch }
+})
+
+const loadData = async function loadData() {
+  const obj = await loader.getAndConstructObject((e) => {
+    console.log(e) // log progress!
+  })
+
+  console.log('Done!')
+  console.log(obj)
+}
+
+loadData()

package/index.js

@@ -3,14 +3,23 @@
  * TODO: Object construction progress reporting is weird.
  */
 
-
 export default class ObjectLoader {
-
   /**
    * Creates a new object loader instance.
    * @param {*} param0
    */
-  constructor( { serverUrl, streamId, token, objectId, options = { enableCaching: true, fullyTraverseArrays: false, excludeProps: [ ] } } ) {
+  constructor({
+    serverUrl,
+    streamId,
+    token,
+    objectId,
+    options = {
+      enableCaching: true,
+      fullyTraverseArrays: false,
+      excludeProps: [],
+      fetch: null
+    }
+  }) {
     this.INTERVAL_MS = 20
     this.TIMEOUT_MS = 180000 // three mins
 
@@ -19,16 +28,16 @@ export default class ObjectLoader {
     this.objectId = objectId
     console.log('Object loader constructor called!')
     try {
-      this.token = token || localStorage.getItem( 'AuthToken' )
+      this.token = token || localStorage.getItem('AuthToken')
     } catch (error) {
-        // Accessing localStorage may throw when executing on sandboxed document, ignore.
+      // Accessing localStorage may throw when executing on sandboxed document, ignore.
     }
 
     this.headers = {
-      'Accept': 'text/plain'
+      Accept: 'text/plain'
     }
 
-    if( this.token ) {
+    if (this.token) {
       this.headers['Authorization'] = `Bearer ${this.token}`
     }
 
@@ -48,6 +57,13 @@ export default class ObjectLoader {
     this.lastAsyncPause = Date.now()
     this.existingAsyncPause = null
 
+    // we can't simply bind fetch to this.fetch, so instead we have to do some acrobatics:
+    // https://stackoverflow.com/questions/69337187/uncaught-in-promise-typeerror-failed-to-execute-fetch-on-workerglobalscope#comment124731316_69337187
+    this.preferredFetch = options.fetch
+    this.fetch = function (...args) {
+      const currentFetch = this.preferredFetch || fetch
+      return currentFetch(...args)
+    }
   }
 
   async asyncPause() {
@@ -55,19 +71,19 @@ export default class ObjectLoader {
     // while ( this.existingAsyncPause ) {
     //   await this.existingAsyncPause
     // }
-    if ( Date.now() - this.lastAsyncPause >= 100 ) {
+    if (Date.now() - this.lastAsyncPause >= 100) {
       this.lastAsyncPause = Date.now()
-      this.existingAsyncPause = new Promise( resolve => setTimeout( resolve, 0 ) )
+      this.existingAsyncPause = new Promise((resolve) => setTimeout(resolve, 0))
       await this.existingAsyncPause
       this.existingAsyncPause = null
-      if (Date.now() - this.lastAsyncPause > 500) console.log("Loader Event loop lag: ", Date.now() - this.lastAsyncPause)
+      if (Date.now() - this.lastAsyncPause > 500)
+        console.log('Loader Event loop lag: ', Date.now() - this.lastAsyncPause)
     }
-
   }
 
   dispose() {
     this.buffer = []
-    this.intervals.forEach( i => clearInterval( i.interval ) )
+    this.intervals.forEach((i) => clearInterval(i.interval))
   }
 
   /**
@@ -75,30 +91,34 @@ export default class ObjectLoader {
    * @param {*} onProgress
    * @returns
    */
-  async getAndConstructObject( onProgress ) {
-
-    ;( await this.downloadObjectsInBuffer( onProgress ) ) // Fire and forget; PS: semicolon of doom
+  async getAndConstructObject(onProgress) {
+    await this.downloadObjectsInBuffer(onProgress) // Fire and forget; PS: semicolon of doom
 
-    let rootObject = await this.getObject( this.objectId )
-    return this.traverseAndConstruct( rootObject, onProgress )
+    const rootObject = await this.getObject(this.objectId)
+    return this.traverseAndConstruct(rootObject, onProgress)
   }
 
   /**
    * Internal function used to download all the objects in a local buffer.
    * @param {*} onProgress
    */
-  async downloadObjectsInBuffer( onProgress ) {
+  async downloadObjectsInBuffer(onProgress) {
     let first = true
     let downloadNum = 0
 
-    for await ( let obj of this.getObjectIterator() ) {
-      if( first ) {
+    for await (const obj of this.getObjectIterator()) {
+      if (first) {
         this.totalChildrenCount = obj.totalChildrenCount
         first = false
         this.isLoading = true
       }
       downloadNum++
-      if( onProgress ) onProgress( { stage: 'download', current: downloadNum, total: this.totalChildrenCount } )
+      if (onProgress)
+        onProgress({
+          stage: 'download',
+          current: downloadNum,
+          total: this.totalChildrenCount
+        })
     }
     this.isLoading = false
   }
@@ -109,51 +129,69 @@ export default class ObjectLoader {
    * @param {*} onProgress
    * @returns
    */
-  async traverseAndConstruct( obj, onProgress ) {
-    if( !obj ) return
-    if ( typeof obj !== 'object' ) return obj
+  async traverseAndConstruct(obj, onProgress) {
+    if (!obj) return
+    if (typeof obj !== 'object') return obj
 
     // Handle arrays
-    if ( Array.isArray( obj ) &&  obj.length !== 0 ) {
-      let arr = []
-      for ( let element of obj ) {
-        if ( typeof element !== 'object' && ! this.options.fullyTraverseArrays ) return obj
+    if (Array.isArray(obj) && obj.length !== 0) {
+      const arr = []
+      for (const element of obj) {
+        if (typeof element !== 'object' && !this.options.fullyTraverseArrays) return obj
 
         // Dereference element if needed
-        let deRef = element.referencedId ? await this.getObject( element.referencedId ) : element
-        if( element.referencedId && onProgress ) onProgress( { stage: 'construction', current: ++this.traversedReferencesCount > this.totalChildrenCount ? this.totalChildrenCount : this.traversedReferencesCount, total: this.totalChildrenCount } )
+        const deRef = element.referencedId
+          ? await this.getObject(element.referencedId)
+          : element
+        if (element.referencedId && onProgress)
+          onProgress({
+            stage: 'construction',
+            current:
+              ++this.traversedReferencesCount > this.totalChildrenCount
+                ? this.totalChildrenCount
+                : this.traversedReferencesCount,
+            total: this.totalChildrenCount
+          })
 
         // Push the traversed object in the array
-        arr.push( await this.traverseAndConstruct( deRef, onProgress ) )
+        arr.push(await this.traverseAndConstruct(deRef, onProgress))
       }
 
       // De-chunk
-      if( arr[0]?.speckle_type?.toLowerCase().includes('datachunk') ) {
-        return arr.reduce( ( prev, curr ) => prev.concat( curr.data ), [] )
+      if (arr[0]?.speckle_type?.toLowerCase().includes('datachunk')) {
+        return arr.reduce((prev, curr) => prev.concat(curr.data), [])
       }
 
       return arr
-     }
+    }
 
     // Handle objects
     // 1) Purge ignored props
-    for( let ignoredProp of this.options.excludeProps ) {
-      delete obj[ ignoredProp ]
+    for (const ignoredProp of this.options.excludeProps) {
+      delete obj[ignoredProp]
     }
 
     // 2) Iterate through obj
-    for( let prop in obj ) {
-      if( typeof obj[prop] !== 'object' || obj[prop] === null ) continue // leave alone primitive props
-
-      if( obj[prop].referencedId ) {
-        obj[prop] = await this.getObject( obj[prop].referencedId )
-        if( onProgress ) onProgress( { stage: 'construction', current: ++this.traversedReferencesCount > this.totalChildrenCount ? this.totalChildrenCount : this.traversedReferencesCount, total: this.totalChildrenCount } )
+    for (const prop in obj) {
+      if (typeof obj[prop] !== 'object' || obj[prop] === null) continue // leave alone primitive props
+
+      if (obj[prop].referencedId) {
+        obj[prop] = await this.getObject(obj[prop].referencedId)
+        if (onProgress)
+          onProgress({
+            stage: 'construction',
+            current:
+              ++this.traversedReferencesCount > this.totalChildrenCount
+                ? this.totalChildrenCount
+                : this.traversedReferencesCount,
+            total: this.totalChildrenCount
+          })
       }
 
-      obj[prop] = await this.traverseAndConstruct( obj[prop], onProgress )
+      obj[prop] = await this.traverseAndConstruct(obj[prop], onProgress)
     }
 
-     return obj
+    return obj
   }
 
   /**
@@ -161,68 +199,71 @@ export default class ObjectLoader {
    * @param {*} id
    * @returns
    */
-  async getObject( id ){
-    if ( this.buffer[id] ) return this.buffer[id]
+  async getObject(id) {
+    if (this.buffer[id]) return this.buffer[id]
 
-    let promise = new Promise( ( resolve, reject ) => {
-      this.promises.push( { id, resolve, reject } )
+    const promise = new Promise((resolve, reject) => {
+      this.promises.push({ id, resolve, reject })
       // Only create a new interval checker if none is already present!
-      if ( this.intervals[id] ) {
+      if (this.intervals[id]) {
         this.intervals[id].elapsed = 0 // reset elapsed
       } else {
-        let intervalId = setInterval( this.tryResolvePromise.bind( this ), this.INTERVAL_MS, id )
+        const intervalId = setInterval(
+          this.tryResolvePromise.bind(this),
+          this.INTERVAL_MS,
+          id
+        )
         this.intervals[id] = { interval: intervalId, elapsed: 0 }
       }
-    } )
+    })
     return promise
   }
 
-  tryResolvePromise( id ) {
+  tryResolvePromise(id) {
     this.intervals[id].elapsed += this.INTERVAL_MS
-    if ( this.buffer[id] ) {
-      for ( let p of this.promises.filter( p => p.id === id ) ) {
-        p.resolve( this.buffer[id] )
+    if (this.buffer[id]) {
+      for (const p of this.promises.filter((p) => p.id === id)) {
+        p.resolve(this.buffer[id])
       }
 
-      clearInterval( this.intervals[id].interval )
+      clearInterval(this.intervals[id].interval)
       delete this.intervals[id]
       // this.promises = this.promises.filter( p => p.id !== p.id ) // clearing out promises too early seems to nuke loading
       return
     }
 
-    if ( this.intervals[id].elapsed > this.TIMEOUT_MS ) {
-      console.warn( `Timeout resolving ${id}. HIC SVNT DRACONES.` )
-      clearInterval( this.intervals[id].interval )
-      this.promises.filter( p => p.id === id ).forEach( p => p.reject() )
-      this.promises = this.promises.filter( p => p.id !== p.id ) // clear out
+    if (this.intervals[id].elapsed > this.TIMEOUT_MS) {
+      console.warn(`Timeout resolving ${id}. HIC SVNT DRACONES.`)
+      clearInterval(this.intervals[id].interval)
+      this.promises.filter((p) => p.id === id).forEach((p) => p.reject())
+      this.promises = this.promises.filter((p) => p.id !== p.id) // clear out
     }
   }
 
-  async * getObjectIterator(  ) {
-    let t0 = Date.now()
+  async *getObjectIterator() {
+    const t0 = Date.now()
     let count = 0
-    for await ( let line of this.getRawObjectIterator() ) {
-      let { id, obj } = this.processLine( line )
-      this.buffer[ id ] = obj
+    for await (const line of this.getRawObjectIterator()) {
+      const { id, obj } = this.processLine(line)
+      this.buffer[id] = obj
       count += 1
       yield obj
     }
     console.log(`Loaded ${count} objects in: ${(Date.now() - t0) / 1000}`)
   }
 
-  processLine( chunk ) {
-    var pieces = chunk.split( '\t' )
-    return { id: pieces[0], obj: JSON.parse( pieces[1] ) }
+  processLine(chunk) {
+    const pieces = chunk.split('\t')
+    return { id: pieces[0], obj: JSON.parse(pieces[1]) }
   }
 
-  async * getRawObjectIterator() {
-    let tSTART = Date.now()
-
-    if ( this.options.enableCaching && window.indexedDB && this.cacheDB === null) {
+  async *getRawObjectIterator() {
+    if (this.options.enableCaching && window.indexedDB && this.cacheDB === null) {
       await safariFix()
-      let idbOpenRequest = indexedDB.open('speckle-object-cache', 1)
-      idbOpenRequest.onupgradeneeded = () => idbOpenRequest.result.createObjectStore('objects');
-      this.cacheDB = await this.promisifyIdbRequest( idbOpenRequest )
+      const idbOpenRequest = indexedDB.open('speckle-object-cache', 1)
+      idbOpenRequest.onupgradeneeded = () =>
+        idbOpenRequest.result.createObjectStore('objects')
+      this.cacheDB = await this.promisifyIdbRequest(idbOpenRequest)
     }
 
     const rootObjJson = await this.getRawRootObject()
@@ -231,162 +272,171 @@ export default class ObjectLoader {
     yield `${this.objectId}\t${rootObjJson}`
 
     const rootObj = JSON.parse(rootObjJson)
-    if ( !rootObj.__closure ) return
+    if (!rootObj.__closure) return
 
-    let childrenIds = Object.keys(rootObj.__closure).sort( (a, b) => rootObj.__closure[a] - rootObj.__closure[b] )
-    if ( childrenIds.length === 0 ) return
+    let childrenIds = Object.keys(rootObj.__closure).sort(
+      (a, b) => rootObj.__closure[a] - rootObj.__closure[b]
+    )
+    if (childrenIds.length === 0) return
 
     let splitHttpRequests = []
 
-    if ( childrenIds.length > 50 ) {
+    if (childrenIds.length > 50) {
       // split into 5%, 15%, 40%, 40% (5% for the high priority children: the ones with lower minDepth)
-      let splitBeforeCacheCheck = [ [], [], [], [] ]
+      const splitBeforeCacheCheck = [[], [], [], []]
       let crtChildIndex = 0
 
-      for ( ; crtChildIndex < 0.05 * childrenIds.length; crtChildIndex++ ) {
-        splitBeforeCacheCheck[0].push( childrenIds[ crtChildIndex ] )
+      for (; crtChildIndex < 0.05 * childrenIds.length; crtChildIndex++) {
+        splitBeforeCacheCheck[0].push(childrenIds[crtChildIndex])
       }
-      for ( ; crtChildIndex < 0.2 * childrenIds.length; crtChildIndex++ ) {
-        splitBeforeCacheCheck[1].push( childrenIds[ crtChildIndex ] )
+      for (; crtChildIndex < 0.2 * childrenIds.length; crtChildIndex++) {
+        splitBeforeCacheCheck[1].push(childrenIds[crtChildIndex])
       }
-      for ( ; crtChildIndex < 0.6 * childrenIds.length; crtChildIndex++ ) {
-        splitBeforeCacheCheck[2].push( childrenIds[ crtChildIndex ] )
+      for (; crtChildIndex < 0.6 * childrenIds.length; crtChildIndex++) {
+        splitBeforeCacheCheck[2].push(childrenIds[crtChildIndex])
       }
-      for ( ; crtChildIndex < childrenIds.length; crtChildIndex++ ) {
-        splitBeforeCacheCheck[3].push( childrenIds[ crtChildIndex ] )
+      for (; crtChildIndex < childrenIds.length; crtChildIndex++) {
+        splitBeforeCacheCheck[3].push(childrenIds[crtChildIndex])
       }
 
+      console.log('Cache check for: ', splitBeforeCacheCheck)
 
-      console.log("Cache check for: ", splitBeforeCacheCheck)
+      const newChildren = []
+      let nextCachePromise = this.cacheGetObjects(splitBeforeCacheCheck[0])
 
-      let newChildren = []
-      let nextCachePromise = this.cacheGetObjects( splitBeforeCacheCheck[ 0 ] )
+      for (let i = 0; i < 4; i++) {
+        const cachedObjects = await nextCachePromise
+        if (i < 3) nextCachePromise = this.cacheGetObjects(splitBeforeCacheCheck[i + 1])
 
-      for ( let i = 0; i < 4; i++ ) {
-        let cachedObjects = await nextCachePromise
-        if ( i < 3 ) nextCachePromise = this.cacheGetObjects( splitBeforeCacheCheck[ i + 1 ] )
-
-        let sortedCachedKeys = Object.keys(cachedObjects).sort( (a, b) => rootObj.__closure[a] - rootObj.__closure[b] )
-        for ( let id of sortedCachedKeys ) {
-          yield `${id}\t${cachedObjects[ id ]}`
+        const sortedCachedKeys = Object.keys(cachedObjects).sort(
+          (a, b) => rootObj.__closure[a] - rootObj.__closure[b]
+        )
+        for (const id of sortedCachedKeys) {
+          yield `${id}\t${cachedObjects[id]}`
         }
-        let newChildrenForBatch = splitBeforeCacheCheck[i].filter( id => !( id in cachedObjects ) )
-        newChildren.push( ...newChildrenForBatch )
+        const newChildrenForBatch = splitBeforeCacheCheck[i].filter(
+          (id) => !(id in cachedObjects)
+        )
+        newChildren.push(...newChildrenForBatch)
       }
 
-      if ( newChildren.length === 0 ) return
+      if (newChildren.length === 0) return
 
-      if ( newChildren.length <= 50 ) {
+      if (newChildren.length <= 50) {
         // we have almost all of children in the cache. do only 1 requests for the remaining new children
-        splitHttpRequests.push( newChildren )
+        splitHttpRequests.push(newChildren)
       } else {
         // we now set up the batches for 4 http requests, starting from `newChildren` (already sorted by priority)
-        splitHttpRequests = [ [], [], [], [] ]
+        splitHttpRequests = [[], [], [], []]
         crtChildIndex = 0
 
-        for ( ; crtChildIndex < 0.05 * newChildren.length; crtChildIndex++ ) {
-          splitHttpRequests[0].push( newChildren[ crtChildIndex ] )
+        for (; crtChildIndex < 0.05 * newChildren.length; crtChildIndex++) {
+          splitHttpRequests[0].push(newChildren[crtChildIndex])
         }
-        for ( ; crtChildIndex < 0.2 * newChildren.length; crtChildIndex++ ) {
-          splitHttpRequests[1].push( newChildren[ crtChildIndex ] )
+        for (; crtChildIndex < 0.2 * newChildren.length; crtChildIndex++) {
+          splitHttpRequests[1].push(newChildren[crtChildIndex])
         }
-        for ( ; crtChildIndex < 0.6 * newChildren.length; crtChildIndex++ ) {
-          splitHttpRequests[2].push( newChildren[ crtChildIndex ] )
+        for (; crtChildIndex < 0.6 * newChildren.length; crtChildIndex++) {
+          splitHttpRequests[2].push(newChildren[crtChildIndex])
         }
-        for ( ; crtChildIndex < newChildren.length; crtChildIndex++ ) {
-          splitHttpRequests[3].push( newChildren[ crtChildIndex ] )
+        for (; crtChildIndex < newChildren.length; crtChildIndex++) {
+          splitHttpRequests[3].push(newChildren[crtChildIndex])
         }
       }
-
     } else {
       // small object with <= 50 children. check cache and make only 1 request
-      const cachedObjects = await this.cacheGetObjects( childrenIds )
-      let sortedCachedKeys = Object.keys(cachedObjects).sort( (a, b) => rootObj.__closure[a] - rootObj.__closure[b] )
-      for ( let id of sortedCachedKeys ) {
-        yield `${id}\t${cachedObjects[ id ]}`
+      const cachedObjects = await this.cacheGetObjects(childrenIds)
+      const sortedCachedKeys = Object.keys(cachedObjects).sort(
+        (a, b) => rootObj.__closure[a] - rootObj.__closure[b]
+      )
+      for (const id of sortedCachedKeys) {
+        yield `${id}\t${cachedObjects[id]}`
       }
-      childrenIds = childrenIds.filter(id => !( id in cachedObjects ) )
-      if ( childrenIds.length === 0 ) return
+      childrenIds = childrenIds.filter((id) => !(id in cachedObjects))
+      if (childrenIds.length === 0) return
 
       // only 1 http request with the remaining children ( <= 50 )
-      splitHttpRequests.push( childrenIds )
+      splitHttpRequests.push(childrenIds)
     }
 
     // Starting http requests for batches in `splitHttpRequests`
 
     const decoders = []
     const readers = []
-    const readPromisses = []
+    const readPromises = []
     const startIndexes = []
     const readBuffers = []
     const finishedRequests = []
 
     for (let i = 0; i < splitHttpRequests.length; i++) {
       decoders.push(new TextDecoder())
-      readers.push( null )
-      readPromisses.push( null )
-      startIndexes.push( 0 )
-      readBuffers.push( '' )
-      finishedRequests.push( false )
-
-      fetch(
-        this.requestUrlChildren,
-        {
-          method: 'POST',
-          headers: { ...this.headers, 'Content-Type': 'application/json' },
-          body: JSON.stringify( { objects: JSON.stringify( splitHttpRequests[i] ) } )
-        }
-      ).then( crtResponse => {
-        let crtReader = crtResponse.body.getReader()
+      readers.push(null)
+      readPromises.push(null)
+      startIndexes.push(0)
+      readBuffers.push('')
+      finishedRequests.push(false)
+
+      this.fetch(this.requestUrlChildren, {
+        method: 'POST',
+        headers: { ...this.headers, 'Content-Type': 'application/json' },
+        body: JSON.stringify({ objects: JSON.stringify(splitHttpRequests[i]) })
+      }).then((crtResponse) => {
+        const crtReader = crtResponse.body.getReader()
         readers[i] = crtReader
-        let crtReadPromise = crtReader.read().then(x => { x.reqId = i; return x })
-        readPromisses[i] = crtReadPromise
+        const crtReadPromise = crtReader.read().then((x) => {
+          x.reqId = i
+          return x
+        })
+        readPromises[i] = crtReadPromise
       })
     }
 
-    while ( true ) {
-      let validReadPromises = readPromisses.filter(x => x != null)
-      if ( validReadPromises.length === 0 ) {
+    while (true) {
+      const validReadPromises = readPromises.filter((x) => !!x)
+      if (validReadPromises.length === 0) {
         // Check if all requests finished
-        if ( finishedRequests.every(x => x) ) {
+        if (finishedRequests.every((x) => x)) {
           break
         }
         // Sleep 10 ms
-        await new Promise( ( resolve ) => {
-          setTimeout( resolve, 10 )
-        } )
+        await new Promise((resolve) => {
+          setTimeout(resolve, 10)
+        })
         continue
       }
 
       // Wait for data on any running request
-      let data = await Promise.any( validReadPromises )
+      const data = await Promise.any(validReadPromises)
+      // eslint-disable-next-line prefer-const
       let { value: crtDataChunk, done: readerDone, reqId } = data
-      finishedRequests[ reqId ] = readerDone
+      finishedRequests[reqId] = readerDone
 
       // Replace read promise on this request with a new `read` call
-      if ( !readerDone ) {
-         let crtReadPromise = readers[ reqId ].read().then(x => { x.reqId = reqId; return x })
-         readPromisses[ reqId ] = crtReadPromise
+      if (!readerDone) {
+        const crtReadPromise = readers[reqId].read().then((x) => {
+          x.reqId = reqId
+          return x
+        })
+        readPromises[reqId] = crtReadPromise
       } else {
         // This request finished. "Flush any non-newline-terminated text"
-        if ( readBuffers[ reqId ].length > 0 ) {
-          yield readBuffers[ reqId ]
-          readBuffers[ reqId ] = ''
+        if (readBuffers[reqId].length > 0) {
+          yield readBuffers[reqId]
+          readBuffers[reqId] = ''
         }
         // no other read calls for this request
-        readPromisses[ reqId ] = null
+        readPromises[reqId] = null
       }
 
-      if ( !crtDataChunk )
-        continue
+      if (!crtDataChunk) continue
 
-      crtDataChunk = decoders[ reqId ].decode( crtDataChunk )
-      let unprocessedText = readBuffers[ reqId ] + crtDataChunk
-      let unprocessedLines = unprocessedText.split(/\r\n|\n|\r/)
-      let remainderText = unprocessedLines.pop()
-      readBuffers[ reqId ] = remainderText
+      crtDataChunk = decoders[reqId].decode(crtDataChunk)
+      const unprocessedText = readBuffers[reqId] + crtDataChunk
+      const unprocessedLines = unprocessedText.split(/\r\n|\n|\r/)
+      const remainderText = unprocessedLines.pop()
+      readBuffers[reqId] = remainderText
 
-      for ( let line of unprocessedLines ) {
+      for (const line of unprocessedLines) {
         yield line
       }
       this.cacheStoreObjects(unprocessedLines)
@@ -394,43 +444,46 @@ export default class ObjectLoader {
   }
 
   async getRawRootObject() {
-    const cachedRootObject = await this.cacheGetObjects( [ this.objectId ] )
-    if ( cachedRootObject[ this.objectId ] )
-      return cachedRootObject[ this.objectId ]
-    const response = await fetch( this.requestUrlRootObj, { headers: this.headers } )
+    const cachedRootObject = await this.cacheGetObjects([this.objectId])
+    if (cachedRootObject[this.objectId]) return cachedRootObject[this.objectId]
+    const response = await this.fetch(this.requestUrlRootObj, { headers: this.headers })
     const responseText = await response.text()
-    this.cacheStoreObjects( [ `${this.objectId}\t${responseText}` ] )
+    this.cacheStoreObjects([`${this.objectId}\t${responseText}`])
     return responseText
   }
 
   promisifyIdbRequest(request) {
     return new Promise((resolve, reject) => {
-      request.oncomplete = request.onsuccess = () => resolve(request.result);
-      request.onabort = request.onerror = () => reject(request.error);
+      request.oncomplete = request.onsuccess = () => resolve(request.result)
+      request.onabort = request.onerror = () => reject(request.error)
     })
   }
 
   async cacheGetObjects(ids) {
-    if ( !this.options.enableCaching || !window.indexedDB ) {
+    if (!this.options.enableCaching || !window.indexedDB) {
       return {}
     }
 
-    let ret = {}
+    const ret = {}
 
     for (let i = 0; i < ids.length; i += 500) {
-      let idsChunk = ids.slice(i, i + 500)
-      let t0 = Date.now()
+      const idsChunk = ids.slice(i, i + 500)
 
-      let store = this.cacheDB.transaction('objects', 'readonly').objectStore('objects')
-      let idbChildrenPromises = idsChunk.map( id => this.promisifyIdbRequest( store.get( id ) ).then( data => ( { id, data } ) ) )
-      let cachedData = await Promise.all(idbChildrenPromises)
+      const store = this.cacheDB
+        .transaction('objects', 'readonly')
+        .objectStore('objects')
+      const idbChildrenPromises = idsChunk.map((id) =>
+        this.promisifyIdbRequest(store.get(id)).then((data) => ({ id, data }))
+      )
+      const cachedData = await Promise.all(idbChildrenPromises)
 
       // console.log("Cache check for : ", idsChunk.length, Date.now() - t0)
 
-      for ( let cachedObj of cachedData ) {
-        if ( !cachedObj.data ) // non-existent objects are retrieved with `undefined` data
+      for (const cachedObj of cachedData) {
+        if (!cachedObj.data)
+          // non-existent objects are retrieved with `undefined` data
           continue
-        ret[ cachedObj.id ] = cachedObj.data
+        ret[cachedObj.id] = cachedObj.data
       }
     }
 
@@ -438,21 +491,22 @@ export default class ObjectLoader {
   }
 
   cacheStoreObjects(objects) {
-    if ( !this.options.enableCaching || !window.indexedDB ) {
+    if (!this.options.enableCaching || !window.indexedDB) {
       return {}
     }
 
-    let store = this.cacheDB.transaction('objects', 'readwrite').objectStore('objects')
-    for ( let obj of objects ) {
-      let idAndData = obj.split( '\t' )
+    const store = this.cacheDB
+      .transaction('objects', 'readwrite')
+      .objectStore('objects')
+    for (const obj of objects) {
+      const idAndData = obj.split('\t')
       store.put(idAndData[1], idAndData[0])
     }
 
-    return this.promisifyIdbRequest( store.transaction )
+    return this.promisifyIdbRequest(store.transaction)
   }
 }
 
-
 // Credits and more info: https://github.com/jakearchibald/safari-14-idb-fix
 function safariFix() {
   const isSafari =
@@ -465,9 +519,9 @@ function safariFix() {
 
   let intervalId
 
-  return new Promise( ( resolve ) => {
+  return new Promise((resolve) => {
     const tryIdb = () => indexedDB.databases().finally(resolve)
     intervalId = setInterval(tryIdb, 100)
     tryIdb()
-  }).finally( () => clearInterval(intervalId) )
+  }).finally(() => clearInterval(intervalId))
 }

package/jsconfig.json

@@ -0,0 +1,7 @@
+{
+  "extends": "../../jsconfig.base.json",
+  "compilerOptions": {
+    "module": "es2020"
+  },
+  "include": ["index.js", "examples"]
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@speckle/objectloader",
-  "version": "2.3.0",
+  "version": "2.3.17-alpha.2819",
   "description": "Simple API helper to stream in objects from the Speckle Server.",
   "main": "index.js",
   "homepage": "https://speckle.systems",
@@ -11,9 +11,21 @@
     "directory": "packages/objectloader"
   },
   "scripts": {
-    "test": "echo \"Error: no test specified\" && exit 1"
+    "test": "echo \"Error: no test specified\" && exit 1",
+    "lint": "eslint . --ext .js,.ts"
   },
-  "keywords": ["speckle", "aec", "speckle api"],
+  "keywords": [
+    "speckle",
+    "aec",
+    "speckle api"
+  ],
   "author": "AEC Systems",
-  "license": "Apache-2.0"
+  "license": "Apache-2.0",
+  "devDependencies": {
+    "eslint": "^8.11.0",
+    "eslint-config-prettier": "^8.5.0",
+    "prettier": "^2.5.1",
+    "undici": "^4.14.1"
+  },
+  "gitHead": "78dc2b89ab4864e3e97988417bab06c906ef454a"
 }

package/readme.md

@@ -12,7 +12,9 @@ Comprehensive developer and user documentation can be found in our:
 
 This is a small utility class that helps you stream an object and all its sub-components from the Speckle Server API. It is intended to be used in contexts where you want to "download" the whole object, or iteratively traverse its whole tree.
 
-Here's a sample way on how to use it, pfilfered from the [3d viewer package](../viewer):
+### In the browser
+
+Here's a sample way on how to use it, pilfered from the [3d viewer package](../viewer):
 
 ```js
 
@@ -36,23 +38,36 @@ async load( { serverUrl, token, streamId, objectId } ) {
 
 ```
 
-If you do not want to process the objects one by one as they are streamed to you, you can use the `getAndConstructObject()` method. Here's an example: 
+If you do not want to process the objects one by one as they are streamed to you, you can use the `getAndConstructObject()` method. Here's an example:
 
-```js
+````js
 
 let loader = new ObjectLoader( {
-  serverUrl: "https://latest.speckle.dev", 
-  streamId: "3ed8357f29", 
+  serverUrl: "https://latest.speckle.dev",
+  streamId: "3ed8357f29",
   objectId: "0408ab9caaa2ebefb2dd7f1f671e7555",
   options: {
     fullyTraverseArrays: false, // Default: false. By default, if an array starts with a primitive type, it will not be traversed. Set it to true if you want to capture scenarios in which lists can have intersped objects and primitives, e.g. [ 1, 2, "a", { important object } ]
-    excludeProps: [ 'displayValue', 'displayMesh', '__closure' ] // Default: []. Any prop names that you pass in here will be ignored from object construction traversal. 
+    excludeProps: [ 'displayValue', 'displayMesh', '__closure' ] // Default: []. Any prop names that you pass in here will be ignored from object construction traversal.
   }
 } )
 
 let obj = await loader.getAndConstructObject( ( e ) => console.log( 'Progress', e ) )
 
-```
+### On the server
+
+Since Node.js does not yet support the [`fetch API`](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch), you'll need to provide your own `fetch` function in the options object. Note that `fetch` must return a [Web Stream](https://nodejs.org/api/webstreams.html), so [node-fetch](https://github.com/node-fetch/node-fetch) won't work, but [node/undici's](https://undici.nodejs.org/) implementation will.
+
+```js
+import { fetch } from 'undici'
+
+let loader = new ObjectLoader({
+  serverUrl: 'https://latest.speckle.dev',
+  streamId: '3ed8357f29',
+  objectId: '0408ab9caaa2ebefb2dd7f1f671e7555',
+  options: { enableCaching: false, excludeProps: [], fetch },
+})
+````
 
 ## Community