jbrowse-plugin-zarr 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. package/README.md +249 -2
  2. package/dist/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.d.ts +50 -0
  3. package/dist/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.js +336 -0
  4. package/dist/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.js.map +1 -0
  5. package/dist/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.test.d.ts +1 -0
  6. package/dist/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.test.js +466 -0
  7. package/dist/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.test.js.map +1 -0
  8. package/dist/MultiWiggleZarrAdapter/configSchema.d.ts +43 -0
  9. package/dist/MultiWiggleZarrAdapter/configSchema.js +72 -0
  10. package/dist/MultiWiggleZarrAdapter/configSchema.js.map +1 -0
  11. package/dist/MultiWiggleZarrAdapter/index.d.ts +2 -0
  12. package/dist/MultiWiggleZarrAdapter/index.js +12 -0
  13. package/dist/MultiWiggleZarrAdapter/index.js.map +1 -0
  14. package/dist/MultiWiggleZarrAdapter/signalMatrixMetadata.d.ts +25 -0
  15. package/dist/MultiWiggleZarrAdapter/signalMatrixMetadata.js +115 -0
  16. package/dist/MultiWiggleZarrAdapter/signalMatrixMetadata.js.map +1 -0
  17. package/dist/index.d.ts +7 -0
  18. package/dist/index.js +14 -0
  19. package/dist/index.js.map +1 -0
  20. package/dist/jbrowse-plugin-zarr.umd.production.min.js +2 -0
  21. package/dist/jbrowse-plugin-zarr.umd.production.min.js.map +7 -0
  22. package/dist/zarrStore.d.ts +9 -0
  23. package/dist/zarrStore.js +56 -0
  24. package/dist/zarrStore.js.map +1 -0
  25. package/package.json +62 -3
  26. package/src/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.test.ts +569 -0
  27. package/src/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.ts +462 -0
  28. package/src/MultiWiggleZarrAdapter/configSchema.ts +88 -0
  29. package/src/MultiWiggleZarrAdapter/index.ts +19 -0
  30. package/src/MultiWiggleZarrAdapter/signalMatrixMetadata.ts +171 -0
  31. package/src/declare.d.ts +1 -0
  32. package/src/index.ts +15 -0
  33. package/src/zarrStore.ts +77 -0
package/README.md CHANGED
@@ -1,3 +1,250 @@
1
- # Temporary Holding Version
1
+ # jbrowse-plugin-zarr
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Reads multi-sample quantitative signal from a [Zarr v3](https://zarr.dev/) store
4
+ into a JBrowse 2
5
+ [`MultiQuantitativeTrack`](https://jbrowse.org/jb2/docs/user_guides/multiquantitative_track).
6
+ The store holds one samples-by-bins array per resolution level, described by the
7
+ [`jbrowse_signal_matrix`](#store-format) metadata below, so this reads stores
8
+ written for it rather than Zarr in general.
9
+
10
+ ![Copy number for 2504 individuals as a red and blue heatmap, one row per person, over a 190 kb window of human chromosome 17](img/zarr_cohort.png)
11
+
12
+ Every individual in the 1000 Genomes panel over the CCL3L1 locus, one row each,
13
+ clustered so the copy-number classes separate. White is two copies, red a gain,
14
+ blue a loss. The whole screen is three requests against one store. Built in
15
+ [Population copy number with the 1000 Genomes panel](https://jbrowse.org/jb2-staging/docs/tutorials/population_cnv/),
16
+ this adapter's worked example, which also covers writing the store.
17
+
18
+ The copy-number values are not ours. They are
19
+ [QuicK-mer2](https://github.com/KiddLab/QuicK-mer2) estimates over the 30x 1000
20
+ Genomes panel, produced by the Kidd lab at the University of Michigan and
21
+ published as the [KiddLab/kmer_1KG](https://github.com/KiddLab/kmer_1KG) track
22
+ hub. If you use them, cite Shen F and Kidd JM,
23
+ [Rapid, Paralog-Sensitive CNV Analysis of 2457 Human Genomes Using QuicK-mer2](https://doi.org/10.3390/genes11020141),
24
+ Genes 2020, 11(2):141.
25
+
26
+ ## Why
27
+
28
+ A multi-sample signal track built from one BigWig per sample is bound by round
29
+ trips, not bytes. Each file needs its header, chrom B-tree and R-tree index
30
+ before it knows where a region's values live, so a screen costs three to four
31
+ sequential requests per sample however small the values are. At 2500 samples
32
+ that is thousands of requests for a few kilobytes of data.
33
+
34
+ A samples-by-bins array chunked `[all samples, N bins]` answers the same screen
35
+ in one or two chunk reads, whatever the sample count, straight off static
36
+ hosting with no tile server.
37
+
38
+ ### Consider a multi-column bedGraph first
39
+
40
+ Most of that gain comes from putting every sample in one file, and a
41
+ Tabix-indexed bedGraph with one value column per sample does the same. JBrowse
42
+ reads it with the built-in `BedGraphTabixAdapter`, so it needs no plugin. For
43
+ the 2,504-sample panel on jbrowse.org,
44
+ [jb2bench](https://github.com/cmdcolin/jb2bench) (`scripts/cohort-signal/`)
45
+ measured:
46
+
47
+ | view | 2,504 BigWigs | this store | multi-column bedGraph |
48
+ | ---------------------- | ---------------------------- | --------------------------- | --------------------------- |
49
+ | 190 kb of chr17 | 15,048 requests, 48 MB, 14 s | 3 requests, 0.19 MB, 0.06 s | 3 requests, 0.31 MB, 0.08 s |
50
+ | all of chr17, 1,500 px | 15,048 requests, 69 MB, 15 s | 40 requests, 9.5 MB, 0.5 s | 4 requests, 5.9 MB, 1.1 s |
51
+
52
+ At a locus the two single-file layouts are the same. The store is ahead only
53
+ when zoomed out: a bedGraph has one bin size, so a whole-chromosome view
54
+ returns every row, and parsing that text is most of its 1.1 s.
55
+
56
+ ### Per-base signal
57
+
58
+ Copy number changes slowly and repeats across samples, so one bedGraph row
59
+ stands for a long stretch. Per-base signal does neither. For 100 ENCODE
60
+ ATAC-seq fold-change tracks over chr1:10–20 Mb at their native resolution, the
61
+ same jb2bench script (`SET=perbase`) measured:
62
+
63
+ | view | 100 BigWigs | this store | multi-column bedGraph |
64
+ | ------ | --------------------------- | -------------------------- | --------------------------- |
65
+ | 1 kb | 692 requests, 2.3 MB, 0.9 s | 7 requests, 0.05 MB, 0.1 s | 3 requests, 1.0 MB, 0.2 s |
66
+ | 100 kb | 674 requests, 5.8 MB, 0.9 s | 46 requests, 2.1 MB, 0.2 s | 4 requests, 6.5 MB, 1.5 s |
67
+ | 1 Mb | 600 requests, 5.2 MB, 1.0 s | 46 requests, 2.4 MB, 0.4 s | 14 requests, 51 MB, 11 s |
68
+ | 10 Mb | 600 requests, 3.2 MB, 0.9 s | 46 requests, 2.1 MB, 0.2 s | 111 requests, 502 MB, 109 s |
69
+
70
+ A hundred samples rarely share a value at adjacent bases, so the bedGraph keeps
71
+ 9.0 million rows for 10 million bases after merging equal ones, and every view
72
+ reads all of them. The store reads a coarser level instead. The BigWigs have
73
+ zoom levels too and hold near 1 s, at 600–700 requests a screen.
74
+
75
+ The store is the largest of the three on disk: 1.0 GB against 634 MB for the
76
+ BigWigs and 495 MB for the bedGraph. Its 10 bp level with min and max is 390 MB
77
+ of that, more than the 1 bp level, so start the pyramid coarser if disk
78
+ matters more than views around 10–50 kb.
79
+
80
+ Use this plugin for per-base signal across many samples, for genome-wide views
81
+ of a large cohort, or when the [summary levels](#summary-levels) matter at
82
+ coarse zoom. For binned data viewed at a locus, a multi-column bedGraph is
83
+ simpler, smaller (178 MB against this store's 662 MB for the whole-genome
84
+ copy-number panel) and needs no plugin.
85
+
86
+ ## Install
87
+
88
+ Needs JBrowse 5. Install it from the
89
+ [plugin store](https://jbrowse.org/jb2/docs/user_guides/plugin_store), or name
90
+ it in a config by its store name, which resolves to the newest release the host
91
+ supports (see
92
+ [configuring plugins](https://jbrowse.org/jb2/docs/config_guides/plugins)):
93
+
94
+ ```json
95
+ {
96
+ "plugins": [{ "storePlugin": "Zarr" }],
97
+ "tracks": [
98
+ {
99
+ "type": "MultiQuantitativeTrack",
100
+ "trackId": "cohort_signal",
101
+ "name": "Cohort signal",
102
+ "assemblyNames": ["hg38"],
103
+ "adapter": {
104
+ "type": "MultiWiggleZarrAdapter",
105
+ "uri": "https://example.com/cohort.zarr"
106
+ },
107
+ "displayDefaults": { "defaultRendering": "multirowdensity" }
108
+ }
109
+ ]
110
+ }
111
+ ```
112
+
113
+ The adapter config is the store's location and nothing else. Sample names, bin
114
+ size and resolution levels come from the store's own metadata.
115
+
116
+ ## Store format
117
+
118
+ A Zarr v3 group whose root attributes carry a `jbrowse_signal_matrix` object,
119
+ plus one `float32` array per resolution level:
120
+
121
+ ```json
122
+ {
123
+ "jbrowse_signal_matrix": {
124
+ "version": 2,
125
+ "samples": [{ "name": "HG00551", "group": "PUR" }],
126
+ "decimals": 2,
127
+ "levels": [
128
+ {
129
+ "path": "bin1000",
130
+ "binSize": 1000,
131
+ "refs": {
132
+ "chr17": { "start": 35000000, "binOffset": 0, "numBins": 2500 }
133
+ }
134
+ },
135
+ {
136
+ "path": "bin10000",
137
+ "binSize": 10000,
138
+ "minPath": "bin10000_min",
139
+ "maxPath": "bin10000_max",
140
+ "refs": {
141
+ "chr17": { "start": 35000000, "binOffset": 0, "numBins": 250 }
142
+ }
143
+ }
144
+ ]
145
+ }
146
+ }
147
+ ```
148
+
149
+ Each level array is shaped `[samples, bins]`, C order, with every refName laid
150
+ end to end on the bin axis at its own `binOffset`. `start` is the genomic
151
+ coordinate of that refName's first bin, so a store can cover one window of one
152
+ chromosome rather than a whole genome. Unmeasured bins are the array's `NaN`
153
+ fill value and are dropped rather than drawn as zero.
154
+
155
+ Chunk each level with the whole sample axis inside one chunk,
156
+ `[all samples, 256 bins]` or thereabouts. This is the opposite of the natural
157
+ per-sample layout, and it is what the format buys: split the samples across
158
+ chunks instead and a screenful costs a read per sample again.
159
+
160
+ Levels are a resolution pyramid: the adapter reads the coarsest level whose bins
161
+ are still no wider than a screen pixel.
162
+
163
+ ### Summary levels
164
+
165
+ A level may name two sibling arrays holding the min and max of the finer bins
166
+ each of its bins averages, the same thing a BigWig zoom record carries beside
167
+ its mean. They are the same shape and chunking as the level itself:
168
+
169
+ - `minPath` and `maxPath` are optional and always travel together. One without
170
+ the other is rejected, because the display would draw a whisker against the
171
+ mean on the other side and it would read as real asymmetry in the data.
172
+ - The finest level has neither: it has nothing below it to summarize.
173
+ - Their presence is the feature test, not `version`. A store written before v2
174
+ has none, and a reader that predates them opens `path` and behaves as before.
175
+
176
+ Without them a zoomed-out view can only show a mean, and anything narrower than
177
+ a coarse bin averages back into the baseline at exactly the zoom someone would
178
+ be scanning for it. With them, the multi-wiggle display's `mark.extent`
179
+ `min-max` and `encoding.y.field` `minScore` or `maxScore` do what they do for a
180
+ BigWig. A view at a summary level reads all three arrays whatever it draws, as a
181
+ BigWig ships its zoom record whole, so changing either setting redraws without
182
+ a refetch.
183
+
184
+ The min and max are over the store's own finer bins, not over whatever the input
185
+ was. If the finest level is itself an aggregate of something finer, the whiskers
186
+ describe the store, not the source.
187
+
188
+ ### Size
189
+
190
+ Two knobs, both read-side transparent, so neither needs a reader to know:
191
+
192
+ - **`chunk-bins`** trades requests against overfetch, not total bytes: a view
193
+ needs the bins it needs. 256 leaves under 20% edge overfetch on a typical
194
+ view; going much smaller multiplies requests to save very little.
195
+ - **Stored precision** is usually the bigger lever. gzip on this layout is
196
+ mostly matching repeated values rather than modelling a distribution, so the
197
+ number of distinct values dominates. Quantizing copy number to 0.01 roughly
198
+ halves a store; it does nothing for genuinely continuous data. Record what was
199
+ used in `decimals` — provenance only, since the values arrive already rounded.
200
+
201
+ The coarse levels are not the cheap ones. Averaging manufactures distinct values
202
+ and destroys the repetition gzip was feeding on, so a level with a third of the
203
+ bins is not a third of the bytes. Measure before assuming.
204
+
205
+ Write one with
206
+ [`build_signal_zarr.ts`](https://github.com/GMOD/jbrowse-components/blob/main/scripts/build_signal_zarr.ts),
207
+ which packs a list of BigWigs into this layout and takes both knobs as flags.
208
+
209
+ ## Codecs
210
+
211
+ This build ships zarrita's own codecs (`bytes`, `gzip`, `zlib`, `transpose`,
212
+ `crc32c`) and stubs the numcodecs ones (`blosc`, `lz4`, `zstd`), which are
213
+ 1.35 MB of wasm against a 52 KB plugin. A store compressed with one of those
214
+ fails with a message naming the codec rather than silently misreading. Write
215
+ stores with gzip.
216
+
217
+ That is less of a compromise than it looks. On a 2504-sample copy-number chunk,
218
+ gzip-9 gets 9.9x and zstd-9 gets 11.4x, so the wasm buys 13%. A byte-shuffle
219
+ filter ahead of either makes it _worse_, by 73%: shuffle helps floats that vary
220
+ smoothly in their mantissa, and this layout's wins come from long runs of
221
+ identical values that shuffle splits across four distant regions of the buffer.
222
+ Reducing stored precision beats every codec choice here, and costs no bytes on
223
+ the wire to support.
224
+
225
+ ## Develop
226
+
227
+ ```bash
228
+ pnpm install
229
+ pnpm test # vitest, adapter round trip against an in-memory store
230
+ pnpm start # esbuild watch, serves dist/out.js on :9000
231
+ pnpm build # dist/jbrowse-plugin-zarr.umd.production.min.js
232
+ ```
233
+
234
+ Point a JBrowse config's plugin `url` at `http://localhost:9000/dist/out.js` to
235
+ develop against the watch build.
236
+
237
+ ## Release
238
+
239
+ ```bash
240
+ pnpm version patch # lint, typecheck, test, build, then push the v* tag
241
+ ```
242
+
243
+ The tag publishes to npm from CI by trusted publishing, and the plugin store
244
+ takes it from npm at its next update.
245
+
246
+ `pnpm betabuild` still pushes an unreleased build to
247
+ `https://jbrowse.org/demos/zarr/jbrowse-plugin-zarr.umd.production.min.js` for
248
+ trying a change in a config before releasing it. It re-downloads the entry point
249
+ through CloudFront afterwards and fails if it does not match what was just
250
+ built.
@@ -0,0 +1,50 @@
1
+ import { BaseFeatureDataAdapter } from '@jbrowse/core/data_adapters/BaseAdapter';
2
+ import type { SignalMatrixMetadata } from './signalMatrixMetadata';
3
+ import type { BaseOptions } from '@jbrowse/core/data_adapters/BaseAdapter';
4
+ import type { Feature, Region } from '@jbrowse/core/util';
5
+ import type { Observable } from 'rxjs';
6
+ import type { Array as ZarrArray, DataType, FetchStore, Group as ZarrGroup } from 'zarrita';
7
+ export interface RawFeatureArrays {
8
+ starts: Int32Array;
9
+ ends: Int32Array;
10
+ scores: Float32Array;
11
+ summary?: {
12
+ minScores: Float32Array;
13
+ maxScores: Float32Array;
14
+ };
15
+ count: number;
16
+ }
17
+ interface WiggleOptions extends BaseOptions {
18
+ bpPerPx?: number;
19
+ resolution?: number;
20
+ sources?: {
21
+ name: string;
22
+ }[];
23
+ }
24
+ export default class MultiWiggleZarrAdapter extends BaseFeatureDataAdapter {
25
+ static capabilities: string[];
26
+ private setupP?;
27
+ private setupPre;
28
+ setup(opts?: BaseOptions): Promise<{
29
+ meta: SignalMatrixMetadata;
30
+ group: ZarrGroup<FetchStore>;
31
+ arrays: Map<string, Promise<ZarrArray<DataType, FetchStore>>>;
32
+ }>;
33
+ private openArray;
34
+ getRefNames(opts?: BaseOptions): Promise<string[]>;
35
+ private placeRegion;
36
+ private planFetch;
37
+ private fetchRegionSlabs;
38
+ getMultiSourceFeatureArraysMulti(regions: Region[], opts?: WiggleOptions): Promise<{
39
+ source: string;
40
+ raws: RawFeatureArrays[];
41
+ }[]>;
42
+ getSources(_regions: Region[]): Promise<{
43
+ source: string;
44
+ name: string;
45
+ group?: string;
46
+ color?: string;
47
+ }[]>;
48
+ getFeatures(region: Region, opts?: WiggleOptions): Observable<Feature>;
49
+ }
50
+ export {};
@@ -0,0 +1,336 @@
1
+ import { BaseFeatureDataAdapter } from '@jbrowse/core/data_adapters/BaseAdapter';
2
+ import { SimpleFeature, updateStatus } from '@jbrowse/core/util';
3
+ import { ObservableCreate } from '@jbrowse/core/util/rxjs';
4
+ import { parseSignalMatrixMetadata, pickLevel } from './signalMatrixMetadata';
5
+ import { openZarrGroup, zarritaP } from '../zarrStore';
6
+ const EMPTY_SLAB = {
7
+ numBins: 0,
8
+ binSize: 1,
9
+ start: 0,
10
+ stride: 0,
11
+ offset: 0,
12
+ values: new Float32Array(0),
13
+ mins: undefined,
14
+ maxs: undefined,
15
+ };
16
+ const EMPTY_RAW = {
17
+ starts: new Int32Array(0),
18
+ ends: new Int32Array(0),
19
+ scores: new Float32Array(0),
20
+ count: 0,
21
+ };
22
+ // The subtree filter in the clustering sidebar narrows the source list the
23
+ // display sends down, so this is a real user-driven filter and not just the
24
+ // full list echoed back. `undefined` means "no filter", which is not the same
25
+ // as an empty one.
26
+ function wantedNames(sources) {
27
+ return (sources === null || sources === void 0 ? void 0 : sources.length) ? new Set(sources.map(s => s.name)) : undefined;
28
+ }
29
+ // A bin the converter never wrote (no k-mers, a gap, or outside the store's
30
+ // covered window) comes back as the array's NaN fill value. Dropping those keeps
31
+ // an unmeasured region visibly empty rather than drawing it as zero coverage.
32
+ function slabToRaw(slab, sampleIndex) {
33
+ const { numBins, binSize, start, stride, offset, values, mins, maxs } = slab;
34
+ const lo = sampleIndex * stride + offset;
35
+ const hi = lo + numBins;
36
+ const row = values.subarray(lo, hi);
37
+ const rowSummary = mins && maxs
38
+ ? { minScores: mins.subarray(lo, hi), maxScores: maxs.subarray(lo, hi) }
39
+ : undefined;
40
+ let count = 0;
41
+ for (let i = 0; i < numBins; i++) {
42
+ if (!Number.isNaN(row[i])) {
43
+ count++;
44
+ }
45
+ }
46
+ // The whole row is measured, which is the ordinary case for a binned store
47
+ // and universal at the coarse levels (a 1Mb bin is empty only where the
48
+ // reference is). Then the bin grid is the region's own, identical for every
49
+ // sample, and the values are already a contiguous run of the slab: hand back
50
+ // one shared grid and three views instead of five fresh arrays per sample.
51
+ //
52
+ // This is what decides whether a cohort renders at all. The interface is one
53
+ // RawFeatureArrays per sample PER REGION, so a 24-contig view of 2504 samples
54
+ // asks for 60,096 of them; copying meant ~300,000 typed arrays for a frame
55
+ // holding 31 MB of actual values, and the tab died before it drew. Sharing is
56
+ // safe because the consumer copies into its own buffers
57
+ // (processFeaturesFromArrays) and the transferable collector dedupes the
58
+ // buffers, so 2504 views of one slab transfer once rather than 2504 times.
59
+ if (count === numBins) {
60
+ let { gridStarts, gridEnds } = slab;
61
+ if (!gridStarts || !gridEnds) {
62
+ gridStarts = new Int32Array(numBins);
63
+ gridEnds = new Int32Array(numBins);
64
+ for (let i = 0; i < numBins; i++) {
65
+ gridStarts[i] = start + i * binSize;
66
+ gridEnds[i] = start + (i + 1) * binSize;
67
+ }
68
+ slab.gridStarts = gridStarts;
69
+ slab.gridEnds = gridEnds;
70
+ }
71
+ return {
72
+ starts: gridStarts,
73
+ ends: gridEnds,
74
+ scores: row,
75
+ summary: rowSummary,
76
+ count,
77
+ };
78
+ }
79
+ const starts = new Int32Array(count);
80
+ const ends = new Int32Array(count);
81
+ const scores = new Float32Array(count);
82
+ // Allocated only on a summary level, so a base-resolution read still transfers
83
+ // three arrays rather than five.
84
+ const summary = rowSummary && {
85
+ minScores: new Float32Array(count),
86
+ maxScores: new Float32Array(count),
87
+ };
88
+ let out = 0;
89
+ for (let i = 0; i < numBins; i++) {
90
+ const score = row[i];
91
+ if (!Number.isNaN(score)) {
92
+ starts[out] = start + i * binSize;
93
+ ends[out] = start + (i + 1) * binSize;
94
+ scores[out] = score;
95
+ // The converter writes all three from the same set of finer bins, so a
96
+ // NaN here would mean a torn store; fall back to the mean rather than
97
+ // emit a NaN the renderer would carry into its scale.
98
+ if (summary) {
99
+ const min = rowSummary.minScores[i];
100
+ const max = rowSummary.maxScores[i];
101
+ summary.minScores[out] = Number.isNaN(min) ? score : min;
102
+ summary.maxScores[out] = Number.isNaN(max) ? score : max;
103
+ }
104
+ out++;
105
+ }
106
+ }
107
+ return { starts, ends, scores, summary, count };
108
+ }
109
+ // Checked rather than trusted, because the failure is silent otherwise: a level
110
+ // with fewer rows than the metadata claims makes the last samples' `subarray`
111
+ // run off the end, and `Number.isNaN(undefined)` is false, so those rows come
112
+ // back as full-length runs of NaN-scored features instead of an error.
113
+ async function openLevelArray(group, path, numSamples) {
114
+ const zarr = await zarritaP;
115
+ const array = await zarr.open.v3(group.resolve(path), { kind: 'array' });
116
+ const { shape } = array;
117
+ if (shape.length !== 2 || shape[0] !== numSamples) {
118
+ throw new Error(`signal matrix level "${path}" is shaped [${shape.join(', ')}], expected [${numSamples}, bins] to match the ${numSamples} samples in the store metadata`);
119
+ }
120
+ return array;
121
+ }
122
+ // A samples-by-bins quantitative matrix in a Zarr v3 store: one chunk holds
123
+ // every sample over a span of bins, so a screenful of a 2500-sample cohort is a
124
+ // couple of range requests instead of the 3-4 sequential index reads per file
125
+ // that a BigWig-per-sample track pays. The store's own attributes carry the
126
+ // sample list, bin size and resolution levels — see signalMatrixMetadata.ts.
127
+ class MultiWiggleZarrAdapter extends BaseFeatureDataAdapter {
128
+ async setupPre(opts) {
129
+ const { statusCallback = () => { } } = opts !== null && opts !== void 0 ? opts : {};
130
+ const group = await updateStatus('Downloading zarr metadata', statusCallback, () => openZarrGroup({
131
+ adapterName: 'MultiWiggleZarrAdapter',
132
+ location: this.getConf('zarrLocation'),
133
+ pluginManager: this.pluginManager,
134
+ }));
135
+ return {
136
+ group,
137
+ meta: parseSignalMatrixMetadata(group.attrs),
138
+ arrays: new Map(),
139
+ };
140
+ }
141
+ async setup(opts) {
142
+ var _a;
143
+ (_a = this.setupP) !== null && _a !== void 0 ? _a : (this.setupP = this.setupPre(opts).catch((e) => {
144
+ this.setupP = undefined;
145
+ throw e;
146
+ }));
147
+ return this.setupP;
148
+ }
149
+ // One open per array for the life of the adapter: opening an array is its own
150
+ // metadata read, and the level is re-picked on every navigation. Keyed by
151
+ // path rather than by level so a level's mean/min/max are three cache entries
152
+ // and the summary pair is only ever opened on a level that declares it.
153
+ async openArray(path, opts) {
154
+ const { group, arrays, meta } = await this.setup(opts);
155
+ let arrayP = arrays.get(path);
156
+ if (!arrayP) {
157
+ // The rejection is not kept. A failed open is usually a transient fetch,
158
+ // and the level is re-picked on every navigation, so caching it would
159
+ // leave that zoom level broken for the life of the adapter with no way
160
+ // back short of reloading the page.
161
+ arrayP = openLevelArray(group, path, meta.samples.length).catch((e) => {
162
+ arrays.delete(path);
163
+ throw e;
164
+ });
165
+ arrays.set(path, arrayP);
166
+ }
167
+ return arrayP;
168
+ }
169
+ async getRefNames(opts) {
170
+ const { meta } = await this.setup(opts);
171
+ return [...new Set(meta.levels.flatMap(l => Object.keys(l.refs)))];
172
+ }
173
+ // Where one region lands on a level's flat bin axis, or undefined if the
174
+ // level does not cover it.
175
+ placeRegion(region, level) {
176
+ const span = level.refs[region.refName];
177
+ if (!span) {
178
+ return undefined;
179
+ }
180
+ const { binSize } = level;
181
+ const first = Math.max(0, Math.floor((region.start - span.start) / binSize));
182
+ const last = Math.min(span.numBins, Math.ceil((region.end - span.start) / binSize));
183
+ return last <= first
184
+ ? undefined
185
+ : {
186
+ from: span.binOffset + first,
187
+ to: span.binOffset + last,
188
+ start: span.start + first * binSize,
189
+ };
190
+ }
191
+ // Merge the placed regions into as few reads as the layout allows.
192
+ //
193
+ // Every refName sits end to end on one bin axis in karyotype order, so a
194
+ // multi-region view is usually a contiguous run of it and a whole-genome view
195
+ // always is: 24 contigs at the coarsest level are ~3100 adjacent bins, which
196
+ // is a dozen chunks. Asking per region instead turns that into 24 slices, and
197
+ // 72 once a summary level triples them, for bytes the store would have handed
198
+ // over in one pass.
199
+ //
200
+ // Two runs merge when the gap between them is under a chunk, because that
201
+ // chunk is fetched either way. That is the whole rule: it never reads a chunk
202
+ // it would not have read, so two far-apart windows stay two reads and do not
203
+ // drag the space between them along.
204
+ planFetch(placed, chunkBins) {
205
+ const order = placed
206
+ .map((_, i) => i)
207
+ .sort((a, b) => placed[a].from - placed[b].from);
208
+ const runs = [];
209
+ for (const i of order) {
210
+ const { from, to } = placed[i];
211
+ const last = runs.at(-1);
212
+ if (last && from - last.to <= chunkBins) {
213
+ last.to = Math.max(last.to, to);
214
+ }
215
+ else {
216
+ runs.push({ from, to });
217
+ }
218
+ }
219
+ return runs;
220
+ }
221
+ async fetchRegionSlabs(regions, opts = {}) {
222
+ var _a;
223
+ const { bpPerPx = 0, resolution = 1 } = opts;
224
+ const { meta } = await this.setup(opts);
225
+ const resolutionMultiplier = this.getConf('resolutionMultiplier');
226
+ // One level for the whole call: bpPerPx describes the view, not a region.
227
+ const level = pickLevel(meta.levels, (bpPerPx / resolution) * resolutionMultiplier);
228
+ const placed = regions.map(region => this.placeRegion(region, level));
229
+ const present = placed.filter(p => p !== undefined);
230
+ if (!present.length) {
231
+ return regions.map(() => EMPTY_SLAB);
232
+ }
233
+ const zarr = await zarritaP;
234
+ // Read whenever the level declares them, as the host's BigWig adapter
235
+ // ships every summary its file holds: the host sends no summary mode, and
236
+ // one fetch serves the mean, the extremes and the band between them, so
237
+ // switching the y field or the extent re-encodes without a refetch.
238
+ const { minPath, maxPath } = level;
239
+ const paths = [
240
+ level.path,
241
+ ...(minPath && maxPath ? [minPath, maxPath] : []),
242
+ ];
243
+ const arrays = await Promise.all(paths.map(p => this.openArray(p, opts)));
244
+ const chunkBins = (_a = arrays[0].chunks[1]) !== null && _a !== void 0 ? _a : 1;
245
+ const runs = this.planFetch(present, chunkBins);
246
+ // Every read of every run at once: they are independent range requests and
247
+ // the point of the layout is that a screenful is a couple of them.
248
+ const blocks = await Promise.all(runs.map(async (run) => {
249
+ const selection = [null, zarr.slice(run.from, run.to)];
250
+ const datas = await Promise.all(arrays.map(async (array, i) => {
251
+ const { data } = await zarr.get(array, selection);
252
+ if (!(data instanceof Float32Array)) {
253
+ throw new Error(`signal matrix level "${paths[i]}" must be float32, got ${array.dtype}`);
254
+ }
255
+ return data;
256
+ }));
257
+ return { run, values: datas[0], mins: datas[1], maxs: datas[2] };
258
+ }));
259
+ return placed.map(p => {
260
+ if (!p) {
261
+ return EMPTY_SLAB;
262
+ }
263
+ const block = blocks.find(b => b.run.from <= p.from && p.to <= b.run.to);
264
+ return {
265
+ numBins: p.to - p.from,
266
+ binSize: level.binSize,
267
+ start: p.start,
268
+ stride: block.run.to - block.run.from,
269
+ offset: p.from - block.run.from,
270
+ values: block.values,
271
+ mins: block.mins,
272
+ maxs: block.maxs,
273
+ };
274
+ });
275
+ }
276
+ async getMultiSourceFeatureArraysMulti(regions, opts = {}) {
277
+ var _a;
278
+ const { meta } = await this.setup(opts);
279
+ const wanted = wantedNames(opts.sources);
280
+ const slabs = await updateStatus('Downloading signal matrix', (_a = opts.statusCallback) !== null && _a !== void 0 ? _a : (() => { }), () => this.fetchRegionSlabs(regions, opts));
281
+ return meta.samples
282
+ .map((sample, sampleIndex) => ({ sample, sampleIndex }))
283
+ .filter(({ sample }) => !wanted || wanted.has(sample.name))
284
+ .map(({ sample, sampleIndex }) => ({
285
+ source: sample.name,
286
+ raws: slabs.map(slab => slab.numBins ? slabToRaw(slab, sampleIndex) : EMPTY_RAW),
287
+ }));
288
+ }
289
+ async getSources(_regions) {
290
+ const { meta } = await this.setup();
291
+ return meta.samples.map(sample => ({ ...sample, source: sample.name }));
292
+ }
293
+ // The feature-object path, which the multiwiggle render never takes — it uses
294
+ // the typed arrays above. This is what clustering (getScoreMatrix) reads, and
295
+ // it costs one SimpleFeature per sample per bin, so it emits straight off the
296
+ // slab rather than building per-sample typed arrays it would immediately throw
297
+ // away.
298
+ getFeatures(region, opts = {}) {
299
+ return ObservableCreate(async (observer) => {
300
+ const { meta } = await this.setup(opts);
301
+ const wanted = wantedNames(opts.sources);
302
+ const [slab] = await this.fetchRegionSlabs([region], opts);
303
+ const { numBins, binSize, start, stride, offset, values } = slab;
304
+ const { refName } = region;
305
+ for (const [sampleIndex, sample] of meta.samples.entries()) {
306
+ if (wanted && !wanted.has(sample.name)) {
307
+ continue;
308
+ }
309
+ const lo = sampleIndex * stride + offset;
310
+ const row = values.subarray(lo, lo + numBins);
311
+ for (let i = 0; i < numBins; i++) {
312
+ const score = row[i];
313
+ if (Number.isNaN(score)) {
314
+ continue;
315
+ }
316
+ const featureStart = start + i * binSize;
317
+ observer.next(new SimpleFeature({
318
+ // refName included: getFeaturesInMultipleRegions concatenates
319
+ // regions, and two of them can hold the same bin coordinate on
320
+ // different contigs.
321
+ uniqueId: `${sample.name}-${refName}-${featureStart}`,
322
+ refName,
323
+ start: featureStart,
324
+ end: featureStart + binSize,
325
+ score,
326
+ source: sample.name,
327
+ }));
328
+ }
329
+ }
330
+ observer.complete();
331
+ }, opts.signal);
332
+ }
333
+ }
334
+ MultiWiggleZarrAdapter.capabilities = ['hasResolution'];
335
+ export default MultiWiggleZarrAdapter;
336
+ //# sourceMappingURL=MultiWiggleZarrAdapter.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"MultiWiggleZarrAdapter.js","sourceRoot":"","sources":["../../src/MultiWiggleZarrAdapter/MultiWiggleZarrAdapter.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,sBAAsB,EAAE,MAAM,yCAAyC,CAAA;AAChF,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAA;AAChE,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAA;AAE1D,OAAO,EAAE,yBAAyB,EAAE,SAAS,EAAE,MAAM,wBAAwB,CAAA;AAC7E,OAAO,EAAE,aAAa,EAAE,QAAQ,EAAE,MAAM,cAAc,CAAA;AA+DtD,MAAM,UAAU,GAAe;IAC7B,OAAO,EAAE,CAAC;IACV,OAAO,EAAE,CAAC;IACV,KAAK,EAAE,CAAC;IACR,MAAM,EAAE,CAAC;IACT,MAAM,EAAE,CAAC;IACT,MAAM,EAAE,IAAI,YAAY,CAAC,CAAC,CAAC;IAC3B,IAAI,EAAE,SAAS;IACf,IAAI,EAAE,SAAS;CAChB,CAAA;AAED,MAAM,SAAS,GAAqB;IAClC,MAAM,EAAE,IAAI,UAAU,CAAC,CAAC,CAAC;IACzB,IAAI,EAAE,IAAI,UAAU,CAAC,CAAC,CAAC;IACvB,MAAM,EAAE,IAAI,YAAY,CAAC,CAAC,CAAC;IAC3B,KAAK,EAAE,CAAC;CACT,CAAA;AAED,2EAA2E;AAC3E,4EAA4E;AAC5E,8EAA8E;AAC9E,mBAAmB;AACnB,SAAS,WAAW,CAAC,OAAuC;IAC1D,OAAO,CAAA,OAAO,aAAP,OAAO,uBAAP,OAAO,CAAE,MAAM,EAAC,CAAC,CAAC,IAAI,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAA;AACxE,CAAC;AAED,4EAA4E;AAC5E,iFAAiF;AACjF,8EAA8E;AAC9E,SAAS,SAAS,CAAC,IAAgB,EAAE,WAAmB;IACtD,MAAM,EAAE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,EAAE,GAAG,IAAI,CAAA;IAC5E,MAAM,EAAE,GAAG,WAAW,GAAG,MAAM,GAAG,MAAM,CAAA;IACxC,MAAM,EAAE,GAAG,EAAE,GAAG,OAAO,CAAA;IACvB,MAAM,GAAG,GAAG,MAAM,CAAC,QAAQ,CAAC,EAAE,EAAE,EAAE,CAAC,CAAA;IACnC,MAAM,UAAU,GACd,IAAI,IAAI,IAAI;QACV,CAAC,CAAC,EAAE,SAAS,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,SAAS,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE;QACxE,CAAC,CAAC,SAAS,CAAA;IACf,IAAI,KAAK,GAAG,CAAC,CAAA;IACb,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,EAAE,CAAC,EAAE,EAAE,CAAC;QACjC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAE,CAAC,EAAE,CAAC;YAC3B,KAAK,EAAE,CAAA;QACT,CAAC;IACH,CAAC;IAED,2EAA2E;IAC3E,wEAAwE;IACxE,4EAA4E;IAC5E,6EAA6E;IAC7E,2EAA2E;IAC3E,EAAE;IACF,6EAA6E;IAC7E,8EAA8E;IAC9E,2EAA2E;IAC3E,8EAA8E;IAC9E,wDAAwD;IACxD,yEAAyE;IACzE,2EAA2E;IAC3E,IAAI,KAAK,KAAK,OAAO,EAAE,CAAC;QACtB,IAAI,EAAE,UAAU,EAAE,QAAQ,EAAE,GAAG,IAAI,CAAA;QACnC,IAAI,CAAC,UAAU,IAAI,CAAC,QAAQ,EAAE,CAAC;YAC7B,UAAU,GAAG,IAAI,UAAU,CAAC,OAAO,CAAC,CAAA;YACpC,QAAQ,GAAG,IAAI,UAAU,CAAC,OAAO,CAAC,CAAA;YAClC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,EAAE,CAAC,EAAE,EAAE,CAAC;gBACjC,UAAU,CAAC,CAAC,CAAC,GAAG,KAAK,GAAG,CAAC,GAAG,OAAO,CAAA;gBACnC,QAAQ,CAAC,CAAC,CAAC,GAAG,KAAK,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,OAAO,CAAA;YACzC,CAAC;YACD,IAAI,CAAC,UAAU,GAAG,UAAU,CAAA;YAC5B,IAAI,CAAC,QAAQ,GAAG,QAAQ,CAAA;QAC1B,CAAC;QACD,OAAO;YACL,MAAM,EAAE,UAAU;YAClB,IAAI,EAAE,QAAQ;YACd,MAAM,EAAE,GAAG;YACX,OAAO,EAAE,UAAU;YACnB,KAAK;SACN,CAAA;IACH,CAAC;IAED,MAAM,MAAM,GAAG,IAAI,UAAU,CAAC,KAAK,CAAC,CAAA;IACpC,MAAM,IAAI,GAAG,IAAI,UAAU,CAAC,KAAK,CAAC,CAAA;IAClC,MAAM,MAAM,GAAG,IAAI,YAAY,CAAC,KAAK,CAAC,CAAA;IACtC,+EAA+E;IAC/E,iCAAiC;IACjC,MAAM,OAAO,GAAG,UAAU,IAAI;QAC5B,SAAS,EAAE,IAAI,YAAY,CAAC,KAAK,CAAC;QAClC,SAAS,EAAE,IAAI,YAAY,CAAC,KAAK,CAAC;KACnC,CAAA;IACD,IAAI,GAAG,GAAG,CAAC,CAAA;IACX,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,EAAE,CAAC,EAAE,EAAE,CAAC;QACjC,MAAM,KAAK,GAAG,GAAG,CAAC,CAAC,CAAE,CAAA;QACrB,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,CAAC;YACzB,MAAM,CAAC,GAAG,CAAC,GAAG,KAAK,GAAG,CAAC,GAAG,OAAO,CAAA;YACjC,IAAI,CAAC,GAAG,CAAC,GAAG,KAAK,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,OAAO,CAAA;YACrC,MAAM,CAAC,GAAG,CAAC,GAAG,KAAK,CAAA;YACnB,uEAAuE;YACvE,sEAAsE;YACtE,sDAAsD;YACtD,IAAI,OAAO,EAAE,CAAC;gBACZ,MAAM,GAAG,GAAG,UAAU,CAAC,SAAS,CAAC,CAAC,CAAE,CAAA;gBACpC,MAAM,GAAG,GAAG,UAAU,CAAC,SAAS,CAAC,CAAC,CAAE,CAAA;gBACpC,OAAO,CAAC,SAAS,CAAC,GAAG,CAAC,GAAG,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAA;gBACxD,OAAO,CAAC,SAAS,CAAC,GAAG,CAAC,GAAG,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAA;YAC1D,CAAC;YACD,GAAG,EAAE,CAAA;QACP,CAAC;IACH,CAAC;IACD,OAAO,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,KAAK,EAAE,CAAA;AACjD,CAAC;AAED,gFAAgF;AAChF,8EAA8E;AAC9E,8EAA8E;AAC9E,uEAAuE;AACvE,KAAK,UAAU,cAAc,CAC3B,KAA4B,EAC5B,IAAY,EACZ,UAAkB;IAElB,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAA;IAC3B,MAAM,KAAK,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAA;IACxE,MAAM,EAAE,KAAK,EAAE,GAAG,KAAK,CAAA;IACvB,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,UAAU,EAAE,CAAC;QAClD,MAAM,IAAI,KAAK,CACb,wBAAwB,IAAI,gBAAgB,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,gBAAgB,UAAU,wBAAwB,UAAU,gCAAgC,CACzJ,CAAA;IACH,CAAC;IACD,OAAO,KAAK,CAAA;AACd,CAAC;AAED,4EAA4E;AAC5E,gFAAgF;AAChF,8EAA8E;AAC9E,4EAA4E;AAC5E,6EAA6E;AAC7E,MAAqB,sBAAuB,SAAQ,sBAAsB;IAShE,KAAK,CAAC,QAAQ,CAAC,IAAkB;QACvC,MAAM,EAAE,cAAc,GAAG,GAAG,EAAE,GAAE,CAAC,EAAE,GAAG,IAAI,aAAJ,IAAI,cAAJ,IAAI,GAAI,EAAE,CAAA;QAChD,MAAM,KAAK,GAAG,MAAM,YAAY,CAC9B,2BAA2B,EAC3B,cAAc,EACd,GAAG,EAAE,CACH,aAAa,CAAC;YACZ,WAAW,EAAE,wBAAwB;YACrC,QAAQ,EAAE,IAAI,CAAC,OAAO,CAAC,cAAc,CAAC;YACtC,aAAa,EAAE,IAAI,CAAC,aAAa;SAClC,CAAC,CACL,CAAA;QACD,OAAO;YACL,KAAK;YACL,IAAI,EAAE,yBAAyB,CAAC,KAAK,CAAC,KAAK,CAAC;YAC5C,MAAM,EAAE,IAAI,GAAG,EAAoD;SACpE,CAAA;IACH,CAAC;IAED,KAAK,CAAC,KAAK,CAAC,IAAkB;;QAC5B,MAAA,IAAI,CAAC,MAAM,oCAAX,IAAI,CAAC,MAAM,GAAK,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,CAAC,CAAU,EAAE,EAAE;YACvD,IAAI,CAAC,MAAM,GAAG,SAAS,CAAA;YACvB,MAAM,CAAC,CAAA;QACT,CAAC,CAAC,EAAA;QACF,OAAO,IAAI,CAAC,MAAM,CAAA;IACpB,CAAC;IAED,8EAA8E;IAC9E,0EAA0E;IAC1E,8EAA8E;IAC9E,wEAAwE;IAChE,KAAK,CAAC,SAAS,CAAC,IAAY,EAAE,IAAkB;QACtD,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAA;QACtD,IAAI,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAA;QAC7B,IAAI,CAAC,MAAM,EAAE,CAAC;YACZ,yEAAyE;YACzE,sEAAsE;YACtE,uEAAuE;YACvE,oCAAoC;YACpC,MAAM,GAAG,cAAc,CAAC,KAAK,EAAE,IAAI,EAAE,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,KAAK,CAC7D,CAAC,CAAU,EAAE,EAAE;gBACb,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAA;gBACnB,MAAM,CAAC,CAAA;YACT,CAAC,CACF,CAAA;YACD,MAAM,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,CAAC,CAAA;QAC1B,CAAC;QACD,OAAO,MAAM,CAAA;IACf,CAAC;IAEM,KAAK,CAAC,WAAW,CAAC,IAAkB;QACzC,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAA;QACvC,OAAO,CAAC,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAA;IACpE,CAAC;IAED,yEAAyE;IACzE,2BAA2B;IACnB,WAAW,CAAC,MAAc,EAAE,KAAwB;QAC1D,MAAM,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,CAAA;QACvC,IAAI,CAAC,IAAI,EAAE,CAAC;YACV,OAAO,SAAS,CAAA;QAClB,CAAC;QACD,MAAM,EAAE,OAAO,EAAE,GAAG,KAAK,CAAA;QACzB,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,OAAO,CAAC,CAAC,CAAA;QAC5E,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CACnB,IAAI,CAAC,OAAO,EACZ,IAAI,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,GAAG,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,OAAO,CAAC,CAC/C,CAAA;QACD,OAAO,IAAI,IAAI,KAAK;YAClB,CAAC,CAAC,SAAS;YACX,CAAC,CAAC;gBACE,IAAI,EAAE,IAAI,CAAC,SAAS,GAAG,KAAK;gBAC5B,EAAE,EAAE,IAAI,CAAC,SAAS,GAAG,IAAI;gBACzB,KAAK,EAAE,IAAI,CAAC,KAAK,GAAG,KAAK,GAAG,OAAO;aACpC,CAAA;IACP,CAAC;IAED,mEAAmE;IACnE,EAAE;IACF,yEAAyE;IACzE,8EAA8E;IAC9E,6EAA6E;IAC7E,8EAA8E;IAC9E,8EAA8E;IAC9E,oBAAoB;IACpB,EAAE;IACF,0EAA0E;IAC1E,8EAA8E;IAC9E,6EAA6E;IAC7E,qCAAqC;IAC7B,SAAS,CAAC,MAAsC,EAAE,SAAiB;QACzE,MAAM,KAAK,GAAG,MAAM;aACjB,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC;aAChB,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,MAAM,CAAC,CAAC,CAAE,CAAC,IAAI,GAAG,MAAM,CAAC,CAAC,CAAE,CAAC,IAAI,CAAC,CAAA;QACpD,MAAM,IAAI,GAAmC,EAAE,CAAA;QAC/C,KAAK,MAAM,CAAC,IAAI,KAAK,EAAE,CAAC;YACtB,MAAM,EAAE,IAAI,EAAE,EAAE,EAAE,GAAG,MAAM,CAAC,CAAC,CAAE,CAAA;YAC/B,MAAM,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAA;YACxB,IAAI,IAAI,IAAI,IAAI,GAAG,IAAI,CAAC,EAAE,IAAI,SAAS,EAAE,CAAC;gBACxC,IAAI,CAAC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,CAAC,CAAA;YACjC,CAAC;iBAAM,CAAC;gBACN,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,CAAA;YACzB,CAAC;QACH,CAAC;QACD,OAAO,IAAI,CAAA;IACb,CAAC;IAEO,KAAK,CAAC,gBAAgB,CAC5B,OAAiB,EACjB,OAAsB,EAAE;;QAExB,MAAM,EAAE,OAAO,GAAG,CAAC,EAAE,UAAU,GAAG,CAAC,EAAE,GAAG,IAAI,CAAA;QAC5C,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAA;QACvC,MAAM,oBAAoB,GAAW,IAAI,CAAC,OAAO,CAAC,sBAAsB,CAAC,CAAA;QACzE,0EAA0E;QAC1E,MAAM,KAAK,GAAG,SAAS,CACrB,IAAI,CAAC,MAAM,EACX,CAAC,OAAO,GAAG,UAAU,CAAC,GAAG,oBAAoB,CAC9C,CAAA;QACD,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,IAAI,CAAC,WAAW,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC,CAAA;QACrE,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,KAAK,SAAS,CAAC,CAAA;QACnD,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC;YACpB,OAAO,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,UAAU,CAAC,CAAA;QACtC,CAAC;QAED,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAA;QAC3B,sEAAsE;QACtE,0EAA0E;QAC1E,wEAAwE;QACxE,oEAAoE;QACpE,MAAM,EAAE,OAAO,EAAE,OAAO,EAAE,GAAG,KAAK,CAAA;QAClC,MAAM,KAAK,GAAG;YACZ,KAAK,CAAC,IAAI;YACV,GAAG,CAAC,OAAO,IAAI,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;SAClD,CAAA;QACD,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC,CAAC,CAAA;QACzE,MAAM,SAAS,GAAG,MAAA,MAAM,CAAC,CAAC,CAAE,CAAC,MAAM,CAAC,CAAC,CAAC,mCAAI,CAAC,CAAA;QAC3C,MAAM,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,EAAE,SAAS,CAAC,CAAA;QAE/C,2EAA2E;QAC3E,mEAAmE;QACnE,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,GAAG,CAC9B,IAAI,CAAC,GAAG,CAAC,KAAK,EAAC,GAAG,EAAC,EAAE;YACnB,MAAM,SAAS,GAAG,CAAC,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC,CAAA;YACtD,MAAM,KAAK,GAAG,MAAM,OAAO,CAAC,GAAG,CAC7B,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAC5B,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,SAAS,CAAC,CAAA;gBACjD,IAAI,CAAC,CAAC,IAAI,YAAY,YAAY,CAAC,EAAE,CAAC;oBACpC,MAAM,IAAI,KAAK,CACb,wBAAwB,KAAK,CAAC,CAAC,CAAC,0BAA0B,KAAK,CAAC,KAAK,EAAE,CACxE,CAAA;gBACH,CAAC;gBACD,OAAO,IAAI,CAAA;YACb,CAAC,CAAC,CACH,CAAA;YACD,OAAO,EAAE,GAAG,EAAE,MAAM,EAAE,KAAK,CAAC,CAAC,CAAE,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,CAAA;QACnE,CAAC,CAAC,CACH,CAAA;QAED,OAAO,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE;YACpB,IAAI,CAAC,CAAC,EAAE,CAAC;gBACP,OAAO,UAAU,CAAA;YACnB,CAAC;YACD,MAAM,KAAK,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,IAAI,CAAC,CAAC,IAAI,IAAI,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC,GAAG,CAAC,EAAE,CAAE,CAAA;YACzE,OAAO;gBACL,OAAO,EAAE,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,IAAI;gBACtB,OAAO,EAAE,KAAK,CAAC,OAAO;gBACtB,KAAK,EAAE,CAAC,CAAC,KAAK;gBACd,MAAM,EAAE,KAAK,CAAC,GAAG,CAAC,EAAE,GAAG,KAAK,CAAC,GAAG,CAAC,IAAI;gBACrC,MAAM,EAAE,CAAC,CAAC,IAAI,GAAG,KAAK,CAAC,GAAG,CAAC,IAAI;gBAC/B,MAAM,EAAE,KAAK,CAAC,MAAM;gBACpB,IAAI,EAAE,KAAK,CAAC,IAAI;gBAChB,IAAI,EAAE,KAAK,CAAC,IAAI;aACjB,CAAA;QACH,CAAC,CAAC,CAAA;IACJ,CAAC;IAEM,KAAK,CAAC,gCAAgC,CAC3C,OAAiB,EACjB,OAAsB,EAAE;;QAExB,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAA;QACvC,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;QACxC,MAAM,KAAK,GAAG,MAAM,YAAY,CAC9B,2BAA2B,EAC3B,MAAA,IAAI,CAAC,cAAc,mCAAI,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,EACjC,GAAG,EAAE,CAAC,IAAI,CAAC,gBAAgB,CAAC,OAAO,EAAE,IAAI,CAAC,CAC3C,CAAA;QACD,OAAO,IAAI,CAAC,OAAO;aAChB,GAAG,CAAC,CAAC,MAAM,EAAE,WAAW,EAAE,EAAE,CAAC,CAAC,EAAE,MAAM,EAAE,WAAW,EAAE,CAAC,CAAC;aACvD,MAAM,CAAC,CAAC,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,CAAC,MAAM,IAAI,MAAM,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;aAC1D,GAAG,CAAC,CAAC,EAAE,MAAM,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC,CAAC;YACjC,MAAM,EAAE,MAAM,CAAC,IAAI;YACnB,IAAI,EAAE,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CACrB,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC,CAAC,SAAS,CACxD;SACF,CAAC,CAAC,CAAA;IACP,CAAC;IAEM,KAAK,CAAC,UAAU,CAAC,QAAkB;QACxC,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,EAAE,CAAA;QACnC,OAAO,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,EAAE,GAAG,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC,CAAC,CAAA;IACzE,CAAC;IAED,8EAA8E;IAC9E,8EAA8E;IAC9E,8EAA8E;IAC9E,+EAA+E;IAC/E,QAAQ;IACD,WAAW,CAChB,MAAc,EACd,OAAsB,EAAE;QAExB,OAAO,gBAAgB,CAAU,KAAK,EAAC,QAAQ,EAAC,EAAE;YAChD,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAA;YACvC,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;YACxC,MAAM,CAAC,IAAI,CAAC,GAAG,MAAM,IAAI,CAAC,gBAAgB,CAAC,CAAC,MAAM,CAAC,EAAE,IAAI,CAAC,CAAA;YAC1D,MAAM,EAAE,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,IAAK,CAAA;YACjE,MAAM,EAAE,OAAO,EAAE,GAAG,MAAM,CAAA;YAC1B,KAAK,MAAM,CAAC,WAAW,EAAE,MAAM,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC;gBAC3D,IAAI,MAAM,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;oBACvC,SAAQ;gBACV,CAAC;gBACD,MAAM,EAAE,GAAG,WAAW,GAAG,MAAM,GAAG,MAAM,CAAA;gBACxC,MAAM,GAAG,GAAG,MAAM,CAAC,QAAQ,CAAC,EAAE,EAAE,EAAE,GAAG,OAAO,CAAC,CAAA;gBAC7C,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,EAAE,CAAC,EAAE,EAAE,CAAC;oBACjC,MAAM,KAAK,GAAG,GAAG,CAAC,CAAC,CAAE,CAAA;oBACrB,IAAI,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,CAAC;wBACxB,SAAQ;oBACV,CAAC;oBACD,MAAM,YAAY,GAAG,KAAK,GAAG,CAAC,GAAG,OAAO,CAAA;oBACxC,QAAQ,CAAC,IAAI,CACX,IAAI,aAAa,CAAC;wBAChB,8DAA8D;wBAC9D,+DAA+D;wBAC/D,qBAAqB;wBACrB,QAAQ,EAAE,GAAG,MAAM,CAAC,IAAI,IAAI,OAAO,IAAI,YAAY,EAAE;wBACrD,OAAO;wBACP,KAAK,EAAE,YAAY;wBACnB,GAAG,EAAE,YAAY,GAAG,OAAO;wBAC3B,KAAK;wBACL,MAAM,EAAE,MAAM,CAAC,IAAI;qBACpB,CAAC,CACH,CAAA;gBACH,CAAC;YACH,CAAC;YACD,QAAQ,CAAC,QAAQ,EAAE,CAAA;QACrB,CAAC,EAAE,IAAI,CAAC,MAAM,CAAC,CAAA;IACjB,CAAC;;AAhQa,mCAAY,GAAG,CAAC,eAAe,CAAC,CAAA;eAD3B,sBAAsB"}