@vizabi/reader-ddfcsv 4.5.1 → 4.5.3

package/README.md

@@ -1,112 +1,122 @@
 # Vizabi DDFcsv reader
+This package implements [DDF model](https://open-numbers.github.io/ddf.html) for datasets made of csv files and a datapackage.json. 
+Once initialised, this reader can receive DDFQL queries, perform data searching, filtering, joinery etc and output an tidy data array for data visualisation.
 
+See live example here https://observablehq.com/@vizabi/ddfcsv-reader
 ## Install
-
-```
+```bash
 npm i @vizabi/reader-ddfcsv
 ```
-
 ## Usage
-
-```
-const Vizabi = require('vizabi');
-const ddfCsvReader = require('reader-ddfcsv');
-const readerObject = ddfCsvReader.getDDFCsvReaderObject();
-Vizabi.Reader.extend('ddf-csv-reader', readerObject);
-// ...
+There are two implementations: for frontend and backend. Additionally, you can customise file reading plugin, so that for example backend implementation would fetch files online instead of reading locally.
+
+* Backend implementation is at `lib/src/index.js`, also usable via es6 import `import DDFCsvReader from "@vizabi/reader-ddfcsv"`, as aliased by "main" field in package.json
+
+* Frontend implementation is at `lib-web/src/index-web.js`, also bundled and minified to `dist/reader-ddfcsv.js`
+### On Backend
+```js
+//this will import "lib/src/index.js", aliased by "main" in package.json
+import DDFCsvReader from "@vizabi/reader-ddfcsv"; 
+
+//init
+const readerInstance = new DDFCsvReader.getDDFCsvReaderObject();
+readerInstance.init({path: "path/to/local/dataset/root"});
+
+//test reading some concepts
+const concepts = await readerInstance.read({
+	from: "concepts",
+	language: "en",
+	select: {key: ["concept"], value: ["concept_type", "name"]},
+	where: {}
+})
 ```
-
-## Build
+### On Frontend
+On a plain webpage use
+```js
+import "@vizabi/reader-ddfcsv/dist/reader-ddfcsv.js";
+
+//init
+const ddfReader = new DDFCsvReader.getDDFCsvReaderObject();
+reader.init({
+	path: "https://github.com/open-numbers/ddf--gapminder--fasttrack.git"
+})
+
+const data = await readerInstance.read({
+	from: "datapoints",
+	language: "en",
+	select: {key: ["country", "time"], value: ["gdp_pcap", "lex", "pop"]},
+	where: {}
+})
 
 ```
-git clone https://github.com/vizabi/reader-ddfcsv.git
-cd reader-ddfcsv
-npm i
-npm run build
-```
-
-### Run tests
-
+  
+In Observable notebooks use
 ```
-npm test
+DDFCsvReader = require('https://unpkg.com/@vizabi/reader-ddfcsv@4.5.2')
 ```
 
-And after this you can see `dist` folder that contains two sets:
-
-  * `bundle.js` and `bundle.js.map`
-  * `bundle.web.js` and `bundle.web.js.map`
-  
-First one is for using with electron app or tests. Second one is only for using in browser.
+which will redirect to https://unpkg.com/@vizabi/reader-ddfcsv@4.5.1/dist/reader-ddfcsv.js
 
-## File readers
+See the [Live demo](https://observablehq.com/@vizabi/ddfcsv-reader)
+## Custom file readers
+Each implementation has its own default file readers built-in:
+* FrontendFileReader is a part of `lib-web/src/index-web.js` version, also usable from `dist/reader-ddfcsv.js`
+* BackendFileReader is a part of `lib/src/index.js` version.
 
-`Vizabi DDFcsv reader` has 2 file readers:
- 
-  * FrontendFileReader is a part of `bundle.js` version.
-  * BackendFileReader is a part of `bundle.web.js` version.
-  
+You can find in the source code and import them readers separately
+```js
+import { FrontendFileReader } from "@vizabi/reader-ddfcsv/lib-web/src/index-web.js";
+```
+### FrontendFileReader
+This reader is designed for file reading via HTTP protocol. It uses `fetch`, which since v4.5.2 is no longer polyfilled
 ### BackendFileReader
+This reader is designed for file reading via OS file system. It uses `fs` and `path`
+### Frankensteining: Using FrontendFileReader in a Backend instance
+The backend version of `DDFCsvReader` defaults to `fs` for local files.
+To fetch data from GitHub (or any remote source), inject the `FrontendFileReader`:
 
-This reader is designed for file reading via OS file system.
-  
-### FrontendFileReader
+```js
 
-This reader is designed for file reading via HTTP protocol.
+import DDFCsvReader from "@vizabi/reader-ddfcsv";
+import { FrontendFileReader } from "@vizabi/reader-ddfcsv/lib-web/src/index-web.js";
 
-## DDFcsv reader usage for DDF data reading examples:
+//init
+const readerInstance = new DDFCsvReader.prepareDDFCsvReaderObject(new FrontendFileReader)();
+readerInstance.init({ path: "https://github.com/open-numbers/ddf--gapminder--ai_worldview_benchmark.git" });
 
-```
-import {BackendFileReader, Ddf} from 'reader-ddfcsv';
-
-const backendFileReader = new BackendFileReader();
-const ddf = new Ddf('your-ddf-folder', backendFileReader);
-
-ddf.getIndex(indexErr => {
-  // process indexErr here ...
-    
-  ddf.getConcepts((conceptsErr, conceptsData) => {
-    // process conceptsErr here ...
-
-    // process conceptsData here ...
-
-    ddf.getAllDataPointsContent(
-      (dataPointsFileErr, dataPointsData) => {
-        // process dataPointsFileErr here ...
-        
-        // process dataPointsData here ...
-      },
-      dataPointsAllErr => {
-        // process dataPointsAllErr here ...
-
-        // data reading finish
-      });
-  });
-});
-```
+//usage example
+const concepts = await readerInstance.read({
+	from: "concepts",
+	language: "en",
+	select: {key: ["concept"], value: ["concept_type", "name"]},
+	where: {}
+})
 
-or 
+```
 
+## Data reading examples:
+Once you can access concepts of a dataset you can run any other DDFQL query, see [DDFQL specs](https://open-numbers.github.io/ddf.html)
+
+```js
+//typical datapoint query
+data = readerInstance.read({
+	from: "datapoints",
+	language: "en",
+	select: {key: ["country", "time"], value: ["gdp_pcap", "lex", "pop"]},
+	where: {}
+})
 ```
-import {BackendFileReader, Ddf} from 'reader-ddfcsv';
-
-const backendFileReader = new BackendFileReader();
-const ddf = new Ddf('your-ddf-folder', backendFileReader);
-// for example, :
-const query = {
-  from: 'entities',
-  animatable: 'time',
-    select: {
-      key: ['geo'],
-      value: ['geo.name', '_default', 'geo.world_4region']
-    },
-    where: {'geo.is--country': true},
-    grouping: {},
-    orderBy: null
-  };
-  
-ddf.processRequest(query, (err, data) => {
-    // process err here ...
 
-    // process expected data here ...
-};
+interactive example: https://observablehq.com/@vizabi/ddfcsv-reader
+
+## Build
+```bash
+git clone https://github.com/vizabi/reader-ddfcsv.git
+cd reader-ddfcsv
+npm i
+npm run build
 ```
+## Run tests
+```bash
+npm test
+```