qvdjs 2.1.0 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -3
- package/dist/index.cjs +722 -143
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +723 -144
- package/dist/index.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -67,11 +67,11 @@ when it loads a file. QVD is also QlikView's format, but QlikView is not part of
|
|
|
67
67
|
|
|
68
68
|
→ [Checked against Qlik Sense](https://qvdjs.ptarmiganlabs.com/v2.0/overview/checked-against-qlik-sense/)
|
|
69
69
|
|
|
70
|
-
##
|
|
70
|
+
## Six ways to open a file
|
|
71
71
|
|
|
72
72
|
`fromQvd` is the general one, and often not the one you want. A QVD is an XML header, then a symbol
|
|
73
73
|
table holding every distinct value, then a bit-packed index table of one code per cell. Two things
|
|
74
|
-
separate
|
|
74
|
+
separate these calls: how far into the file a read has to go, and how much of it the read then
|
|
75
75
|
holds.
|
|
76
76
|
|
|
77
77
|
| You want | Call | How far it reads, and what it holds |
|
|
@@ -81,6 +81,7 @@ holds.
|
|
|
81
81
|
| A few columns of a large file | `QvdColumnTable.fromQvd(path)` | Everything, but stops before building rows — 141 MiB against 385 MiB |
|
|
82
82
|
| Only the schema: names, row count, types | `QvdDataFrame.readMetadata(path)` | The header alone. Constant cost, whatever the file's size |
|
|
83
83
|
| To know whether a read will fit first | `QvdDataFrame.checkRead(path, options)` | The header alone. Answers rather than reads; a read it approves is not refused for memory |
|
|
84
|
+
| Pages, by position and in any order | `QvdDataFrame.open(path, options)` | The header once, then a page per call — `rows()`, `columns()`, a free `check()`, `close()` |
|
|
84
85
|
|
|
85
86
|
```javascript
|
|
86
87
|
import {QvdDataFrame, QvdColumnTable} from 'qvdjs';
|
|
@@ -91,6 +92,11 @@ const {columns, rowCount} = await QvdDataFrame.readMetadata('sales.qvd');
|
|
|
91
92
|
// Will reading it fit? Also the header alone, and every suggestion it gives has been checked.
|
|
92
93
|
const {fits, suggestions} = await QvdDataFrame.checkRead('sales.qvd');
|
|
93
94
|
|
|
95
|
+
// Paging: the header is read once, and a page can be asked for by position, forwards or back.
|
|
96
|
+
const qvd = await QvdDataFrame.open('sales.qvd');
|
|
97
|
+
const page = await qvd.rows({offset: 5_000_000, limit: 100});
|
|
98
|
+
await qvd.close();
|
|
99
|
+
|
|
94
100
|
// Sum one column without ever building a row.
|
|
95
101
|
const table = await QvdColumnTable.fromQvd('sales.qvd');
|
|
96
102
|
let total = 0;
|
|
@@ -107,7 +113,7 @@ for await (const chunk of QvdDataFrame.iterate('sales.qvd', {chunkSize: 50_000})
|
|
|
107
113
|
const df = await QvdDataFrame.fromQvd('sales.qvd');
|
|
108
114
|
```
|
|
109
115
|
|
|
110
|
-
Reaching for `fromQvd` when you wanted one of the
|
|
116
|
+
Reaching for `fromQvd` when you wanted one of the others is the common mistake, and
|
|
111
117
|
`fromQvd(path, {maxRows: 0})` is **not** a substitute for `readMetadata`: it loads no rows but still
|
|
112
118
|
parses the whole symbol table, which grows with the data.
|
|
113
119
|
|