qvdjs 2.1.0 → 2.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -67,11 +67,11 @@ when it loads a file. QVD is also QlikView's format, but QlikView is not part of
67
67
 
68
68
  → [Checked against Qlik Sense](https://qvdjs.ptarmiganlabs.com/v2.0/overview/checked-against-qlik-sense/)
69
69
 
70
- ## Four ways to open a file
70
+ ## Six ways to open a file
71
71
 
72
72
  `fromQvd` is the general one, and often not the one you want. A QVD is an XML header, then a symbol
73
73
  table holding every distinct value, then a bit-packed index table of one code per cell. Two things
74
- separate the four calls: how far into the file a read has to go, and how much of it the read then
74
+ separate these calls: how far into the file a read has to go, and how much of it the read then
75
75
  holds.
76
76
 
77
77
  | You want | Call | How far it reads, and what it holds |
@@ -81,6 +81,7 @@ holds.
81
81
  | A few columns of a large file | `QvdColumnTable.fromQvd(path)` | Everything, but stops before building rows — 141 MiB against 385 MiB |
82
82
  | Only the schema: names, row count, types | `QvdDataFrame.readMetadata(path)` | The header alone. Constant cost, whatever the file's size |
83
83
  | To know whether a read will fit first | `QvdDataFrame.checkRead(path, options)` | The header alone. Answers rather than reads; a read it approves is not refused for memory |
84
+ | Pages, by position and in any order | `QvdDataFrame.open(path, options)` | The header once, then a page per call — `rows()`, `columns()`, a free `check()`, `close()` |
84
85
 
85
86
  ```javascript
86
87
  import {QvdDataFrame, QvdColumnTable} from 'qvdjs';
@@ -91,6 +92,11 @@ const {columns, rowCount} = await QvdDataFrame.readMetadata('sales.qvd');
91
92
  // Will reading it fit? Also the header alone, and every suggestion it gives has been checked.
92
93
  const {fits, suggestions} = await QvdDataFrame.checkRead('sales.qvd');
93
94
 
95
+ // Paging: the header is read once, and a page can be asked for by position, forwards or back.
96
+ const qvd = await QvdDataFrame.open('sales.qvd');
97
+ const page = await qvd.rows({offset: 5_000_000, limit: 100});
98
+ await qvd.close();
99
+
94
100
  // Sum one column without ever building a row.
95
101
  const table = await QvdColumnTable.fromQvd('sales.qvd');
96
102
  let total = 0;
@@ -107,7 +113,7 @@ for await (const chunk of QvdDataFrame.iterate('sales.qvd', {chunkSize: 50_000})
107
113
  const df = await QvdDataFrame.fromQvd('sales.qvd');
108
114
  ```
109
115
 
110
- Reaching for `fromQvd` when you wanted one of the other three is the common mistake, and
116
+ Reaching for `fromQvd` when you wanted one of the others is the common mistake, and
111
117
  `fromQvd(path, {maxRows: 0})` is **not** a substitute for `readMetadata`: it loads no rows but still
112
118
  parses the whole symbol table, which grows with the data.
113
119