data-restructor 3.3.4 → 3.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/codeql-analysis.yml +4 -4
- package/.github/workflows/continuous-integration.yaml +84 -0
- package/CHANGELOG.md +98 -76
- package/CONTRIBUTING.md +51 -43
- package/Commands.md +15 -15
- package/README.md +188 -137
- package/SECURITY.md +2 -2
- package/coverage/coverage-summary.json +3 -3
- package/coverage/datarestructor.js.html +133 -133
- package/coverage/describedfield.js.html +45 -45
- package/coverage/index.html +1 -1
- package/coverage/templateResolver.js.html +26 -26
- package/devdist/datarestructor-ie.js +1479 -1665
- package/devdist/datarestructor.js +1439 -1608
- package/devdist/describedfield-ie.js +259 -279
- package/devdist/describedfield.js +257 -277
- package/devdist/templateResolver-ie.js +220 -252
- package/devdist/templateResolver.js +196 -227
- package/dist/datarestructor-ie.js +1 -1
- package/dist/datarestructor-ie.js.map +1 -1
- package/dist/datarestructor.js +1 -1
- package/dist/datarestructor.js.map +1 -1
- package/dist/describedfield-ie.js +1 -1
- package/dist/describedfield-ie.js.map +1 -1
- package/dist/describedfield.js +1 -1
- package/dist/describedfield.js.map +1 -1
- package/dist/templateResolver-ie.js +1 -1
- package/dist/templateResolver-ie.js.map +1 -1
- package/dist/templateResolver.js +1 -1
- package/dist/templateResolver.js.map +1 -1
- package/docs/datarestructor.js.html +1 -1
- package/docs/describedfield.js.html +1 -1
- package/docs/index.html +49 -44
- package/docs/module-datarestructor.DescribedEntryCreator.html +1 -1
- package/docs/module-datarestructor.PropertyStructureDescriptionBuilder.html +1 -1
- package/docs/module-datarestructor.Restructor.html +1 -1
- package/docs/module-datarestructor.Transform.html +1 -1
- package/docs/module-datarestructor.html +1 -1
- package/docs/module-described_field.DescribedDataFieldBuilder.html +1 -1
- package/docs/module-described_field.DescribedDataFieldGroup.html +1 -1
- package/docs/module-described_field.html +1 -1
- package/docs/module-template_resolver.Resolver.html +1 -1
- package/docs/module-template_resolver.html +1 -1
- package/docs/templateResolver.js.html +1 -1
- package/package.json +16 -14
- package/renovate.json +20 -0
- package/.github/workflows/action.yaml +0 -44
package/README.md
CHANGED
|
@@ -1,85 +1,100 @@
|
|
|
1
|
+
# data-restructor-js
|
|
2
|
+
|
|
1
3
|
[](https://opensource.org/licenses/Apache-2.0)
|
|
2
4
|

|
|
3
|
-

|
|
5
|
+

|
|
4
6
|
](https://aleen42.github.io/badges/src/npm.svg)
|
|
5
7
|
](https://aleen42.github.io/badges/src/jasmine.svg)
|
|
6
8
|
](https://aleen42.github.io/badges/src/eslint.svg)
|
|
7
9
|

|
|
8
10
|

|
|
9
|
-

|
|
12
12
|
|
|
13
13
|
When parsing JSON on client-side, the structure of it attracts most of our attention.
|
|
14
14
|
If the structure evolves over time, it leads to recurring changes in the code that depends on it.
|
|
15
15
|
|
|
16
|
-
## Features
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
16
|
+
## Features
|
|
17
|
+
|
|
18
|
+
- Adapter that takes e.g. parsed JSON and transforms it into a uniform structure
|
|
19
|
+
- Multiple transformation steps including flattening, removing duplicates, grouping, ...
|
|
20
|
+
- Takes descriptions that reflect the incoming structure and define the uniform output
|
|
21
|
+
- Reusable and flexible
|
|
22
|
+
- Supports most browser including IE 5
|
|
22
23
|
|
|
23
|
-
## Not intended to be used when
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
24
|
+
## Not intended to be used when
|
|
25
|
+
|
|
26
|
+
- a "backend for frontend" exists, that is responsible for delivering the structure and content the way the client needs it.
|
|
27
|
+
- the structure of the data is already stable, well abstracted and/or rather generic.
|
|
28
|
+
- the code, that depends on the structure of the data, can easily be changed (only a view lines, same team, ...).
|
|
27
29
|
|
|
28
30
|
## Quickstart
|
|
31
|
+
|
|
29
32
|
Use the following command to install the library using npm:
|
|
30
|
-
|
|
33
|
+
|
|
34
|
+
```shell
|
|
31
35
|
npm install data-restructor
|
|
32
36
|
```
|
|
37
|
+
|
|
33
38
|
Alternatively, the sources can be found inside the
|
|
34
39
|
[source folder](https://github.com/JohT/data-restructor-js/tree/master/src/):
|
|
35
|
-
|
|
36
|
-
- [
|
|
37
|
-
- [
|
|
40
|
+
|
|
41
|
+
- [datarestructor.js](https://github.com/JohT/data-restructor-js/blob/master/src/js/datarestructor.js)
|
|
42
|
+
- [templateResolver.js](https://github.com/JohT/data-restructor-js/blob/master/src/js/templateResolver.js)
|
|
43
|
+
- [describedfield.js](https://github.com/JohT/data-restructor-js/blob/master/src/js/describedfield.js)
|
|
38
44
|
|
|
39
45
|
The development artifacts (not minified) can be found inside the
|
|
40
46
|
[devdist folder](https://github.com/JohT/data-restructor-js/tree/master/devdist):
|
|
47
|
+
|
|
41
48
|
- [datarestructor-ie-global-all.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/datarestructor-ie-global-all.js) all sources including polyfills merged to be used without module system
|
|
42
49
|
- [datarestructor-global-all.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/datarestructor-global-all.js) all sources without polyfills merged to be used without module system
|
|
43
|
-
- [datarestructor.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/datarestructor.js)
|
|
44
|
-
- [templateResolver.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/templateResolver.js)
|
|
45
|
-
- [describedfield.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/describedfield.js)
|
|
50
|
+
- [datarestructor.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/datarestructor.js)
|
|
51
|
+
- [templateResolver.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/templateResolver.js)
|
|
52
|
+
- [describedfield.js](https://github.com/JohT/data-restructor-js/blob/master/devdist/describedfield.js)
|
|
46
53
|
|
|
47
54
|
Here are some code examples on how these modules can be imported:
|
|
55
|
+
|
|
48
56
|
```javascript
|
|
49
57
|
var template_resolver = template_resolver || require("data-restructor/devdist/templateResolver"); // supports vanilla js
|
|
50
58
|
var described_field = described_field || require("data-restructor/devdist/describedfield"); // supports vanilla js
|
|
51
59
|
var datarestructor = datarestructor || require("data-restructor/devdist/datarestructor"); // supports vanilla js
|
|
52
60
|
```
|
|
53
61
|
|
|
54
|
-
The built (minified) versions can be found inside the
|
|
62
|
+
The built (minified) versions can be found inside the
|
|
55
63
|
[distribution folder](https://github.com/JohT/data-restructor-js/tree/master/dist):
|
|
64
|
+
|
|
56
65
|
- [datarestructor-ie-global-all-min.js](https://github.com/JohT/data-restructor-js/blob/master/dist/datarestructor-ie-global-all-min.js) all sources including polyfills merged and minified to be used without module system
|
|
57
66
|
- [datarestructor-global-all-min.js](https://github.com/JohT/data-restructor-js/blob/master/dist/datarestructor-global-all-min.js) all sources without polyfills merged and minified to be used without module system
|
|
58
|
-
- [datarestructor.js](https://github.com/JohT/data-restructor-js/blob/master/dist/datarestructor.js)
|
|
67
|
+
- [datarestructor.js](https://github.com/JohT/data-restructor-js/blob/master/dist/datarestructor.js)
|
|
59
68
|
- [datarestructor-ie.js](https://github.com/JohT/data-restructor-js/blob/master/dist/datarestructor-ie.js) (full compatibility with IE)
|
|
60
|
-
- [templateResolver.js](https://github.com/JohT/data-restructor-js/blob/master/dist/templateResolver.js)
|
|
69
|
+
- [templateResolver.js](https://github.com/JohT/data-restructor-js/blob/master/dist/templateResolver.js)
|
|
61
70
|
- [templateResolver-ie.js](https://github.com/JohT/data-restructor-js/blob/master/dist/templateResolver-ie.js) (full compatibility with IE)
|
|
62
|
-
- [describedfield.js](https://github.com/JohT/data-restructor-js/blob/master/dist/describedfield.js)
|
|
71
|
+
- [describedfield.js](https://github.com/JohT/data-restructor-js/blob/master/dist/describedfield.js)
|
|
63
72
|
- [describedfield-ie.js](https://github.com/JohT/data-restructor-js/blob/master/dist/describedfield-ie.js) (full compatibility with IE)
|
|
64
73
|
|
|
65
74
|
## Code Documentation
|
|
75
|
+
|
|
66
76
|
The [code documentation](https://joht.github.io/data-restructor-js) is generated using [JSDoc](https://jsdoc.app) and is published using [GitHub Pages](https://pages.github.com) at [https://joht.github.io/data-restructor-js](https://joht.github.io/data-restructor-js).
|
|
67
77
|
|
|
68
78
|
## Build all
|
|
69
79
|
|
|
70
80
|
Use the following [commands](COMMANDS.md) to build and package the module. A list of all commands can be found in [COMMANDS.md](COMMANDS.md).
|
|
71
|
-
|
|
81
|
+
|
|
82
|
+
```shell
|
|
72
83
|
npm install merger-js -g
|
|
73
84
|
npm install
|
|
74
85
|
npm run package
|
|
75
86
|
```
|
|
87
|
+
|
|
76
88
|
**Note:** merger.js prompts to select a source file. Please select "ALL" using the arrow keys and press enter to continue.
|
|
77
89
|
|
|
78
90
|
## Example
|
|
79
|
-
|
|
80
|
-
|
|
91
|
+
|
|
92
|
+
As a starting point you may have a look at the following example.
|
|
93
|
+
A running, comprehensive example can be found here:
|
|
81
94
|
[DataRestructorUseCaseTest.js](https://github.com/JohT/data-restructor-js/blob/master/test/js/DataRestructorUseCaseTest.js)
|
|
95
|
+
|
|
82
96
|
### Input Object
|
|
97
|
+
|
|
83
98
|
```json
|
|
84
99
|
{
|
|
85
100
|
"responses": [
|
|
@@ -109,6 +124,7 @@ A running, comprehensive example can be found here:
|
|
|
109
124
|
```
|
|
110
125
|
|
|
111
126
|
### Code
|
|
127
|
+
|
|
112
128
|
```javascript
|
|
113
129
|
function restructureJson(jsonData) {
|
|
114
130
|
var allDescriptions = [];
|
|
@@ -142,7 +158,9 @@ function detailsDescription() {
|
|
|
142
158
|
```
|
|
143
159
|
|
|
144
160
|
### Output Java Object
|
|
145
|
-
|
|
161
|
+
|
|
162
|
+
An Javascript object with mainly this structure (see [DescribedEntry](#describedentry) for more details) and content is returned, when the function `restructureJson` from above is called:
|
|
163
|
+
|
|
146
164
|
```yaml
|
|
147
165
|
category: "account"
|
|
148
166
|
displayName: "Accountnumber"
|
|
@@ -187,11 +205,13 @@ details:
|
|
|
187
205
|
value: "active, online"
|
|
188
206
|
```
|
|
189
207
|
|
|
190
|
-
## Transformation Steps
|
|
208
|
+
## Transformation Steps
|
|
191
209
|
|
|
192
|
-
### 1. Flatten hierarchical data object
|
|
193
|
-
|
|
210
|
+
### 1. Flatten hierarchical data object
|
|
211
|
+
|
|
212
|
+
The input data object, e.g. parsed from JSON, is converted to an array of point separated property names and their values.
|
|
194
213
|
For example this structure...
|
|
214
|
+
|
|
195
215
|
```json
|
|
196
216
|
{
|
|
197
217
|
"responses": [
|
|
@@ -212,32 +232,39 @@ For example this structure...
|
|
|
212
232
|
]
|
|
213
233
|
}
|
|
214
234
|
```
|
|
235
|
+
|
|
215
236
|
...is flattened to...
|
|
216
|
-
|
|
237
|
+
|
|
238
|
+
```javascript
|
|
217
239
|
responses[0].hits.total.value=1
|
|
218
240
|
responses[0].hits.hits[0]._source.accountnumber=123
|
|
219
241
|
```
|
|
220
242
|
|
|
221
243
|
### 2. Add array value properties ending with "_comma_separated_values"
|
|
244
|
+
|
|
222
245
|
To make it easier to e.g. display array values like tags,
|
|
223
246
|
an additional property is added that combines the array values to a single property,
|
|
224
|
-
that contains the values in a comma separated way.
|
|
225
|
-
This newly created property gets the name of the array property followed by "_comma_separated_values"
|
|
247
|
+
that contains the values in a comma separated way.
|
|
248
|
+
This newly created property gets the name of the array property followed by "_comma_separated_values"
|
|
226
249
|
and is inserted right after the single array values.
|
|
227
250
|
|
|
228
251
|
For example these lines...
|
|
229
|
-
|
|
252
|
+
|
|
253
|
+
```javascript
|
|
230
254
|
responses[0].hits.total.value=1
|
|
231
255
|
responses[0].hits.hits[0]._source.tags[0]=active
|
|
232
256
|
responses[0].hits.hits[0]._source.tags[1]=online
|
|
233
257
|
```
|
|
234
|
-
|
|
235
|
-
|
|
258
|
+
|
|
259
|
+
...will lead to an additional property that looks like this...
|
|
260
|
+
|
|
261
|
+
```javascript
|
|
236
262
|
responses[0].hits.hits[0]._source.tags_comma_separated_values=active, online
|
|
237
263
|
```
|
|
238
264
|
|
|
239
|
-
### 3. Attach description to matching properties
|
|
240
|
-
|
|
265
|
+
### 3. Attach description to matching properties
|
|
266
|
+
|
|
267
|
+
For every given description, all properties are searched for matches.
|
|
241
268
|
If a description matches a property, the description gets attached to it.
|
|
242
269
|
This can be used to categorize and filter properties.
|
|
243
270
|
The description builder accepts these ways to configure property matching:
|
|
@@ -247,63 +274,71 @@ The property name needs to match the described pattern exactly. It is not needed
|
|
|
247
274
|
The field name will be (by default) taken from the right most (after the last separator `.`) element of the property name.
|
|
248
275
|
In the example below the field name will be "accountnumber".
|
|
249
276
|
Example:
|
|
277
|
+
|
|
250
278
|
```javascript
|
|
251
279
|
new datarestructor.PropertyStructureDescriptionBuilder()
|
|
252
280
|
.propertyPatternEqualMode()
|
|
253
281
|
.propertyPattern("responses.hits.hits._source.accountnumber")
|
|
254
282
|
...
|
|
255
283
|
```
|
|
284
|
+
|
|
256
285
|
- Pattern Mode:
|
|
257
|
-
The property name needs to start with the described pattern.
|
|
286
|
+
The property name needs to start with the described pattern.
|
|
258
287
|
The pattern may contain variables inside double curly brackets.
|
|
259
288
|
The variable `{{fieldName}}` is a special case which describes from where the field name should be taken.
|
|
260
289
|
If `{{fieldName}}` is not specified, the field name will be taken from the right most (after the last separator `.`) element of the property name, which is the same behavior as in "Equal Mode".
|
|
261
290
|
This mode needs to set using `propertyPatternTemplateMode`, since the default mode is `propertyPatternEqualMode`.
|
|
262
291
|
Example:
|
|
292
|
+
|
|
263
293
|
```javascript
|
|
264
294
|
new datarestructor.PropertyStructureDescriptionBuilder()
|
|
265
295
|
.propertyPatternTemplateMode()
|
|
266
296
|
.propertyPattern("responses.hits.hits._source.{{fieldName}}")
|
|
267
297
|
...
|
|
268
298
|
```
|
|
299
|
+
|
|
269
300
|
- Index Matching (Optional):
|
|
270
|
-
If the source data is structured in an top level array and all property names look pretty much the same
|
|
271
|
-
it may be needed to describe data based on the array index.
|
|
301
|
+
If the source data is structured in an top level array and all property names look pretty much the same
|
|
302
|
+
it may be needed to describe data based on the array index.
|
|
272
303
|
The index of an property is taken out of its array qualifiers.
|
|
273
304
|
For example, the property name `responses[0].hits.hits[1]._source.tags[2]` has the index `0.1.2`.
|
|
274
305
|
Index Matching can be combined with property name matching.
|
|
275
|
-
This example restricts the description to the first top level array:
|
|
306
|
+
This example restricts the description to the first top level array:
|
|
307
|
+
|
|
276
308
|
```javascript
|
|
277
309
|
new datarestructor.PropertyStructureDescriptionBuilder()
|
|
278
310
|
.indexStartsWith("0.")
|
|
279
311
|
...
|
|
280
312
|
```
|
|
281
313
|
|
|
282
|
-
### 4. Removing duplicates (deduplication)
|
|
314
|
+
### 4. Removing duplicates (deduplication)
|
|
315
|
+
|
|
283
316
|
To remove duplicate properties or to override properties with other ones when they exist,
|
|
284
317
|
a `deduplicationPattern` can be defined.<br/><br/>
|
|
285
|
-
Variables (listed below) are put into double curly brackets and will be replaced with the contents
|
|
286
|
-
of the description and the matching property.
|
|
318
|
+
Variables (listed below) are put into double curly brackets and will be replaced with the contents of the description and the matching property.
|
|
287
319
|
If there are two entries with the same resolved `deduplicationPattern` (=`_identifier.deduplicationId`),
|
|
288
|
-
the second one will override the first (the first one will be removed).
|
|
320
|
+
the second one will override the first (the first one will be removed).
|
|
289
321
|
Example:
|
|
322
|
+
|
|
290
323
|
```javascript
|
|
291
324
|
new datarestructor.PropertyStructureDescriptionBuilder()
|
|
292
325
|
.deduplicationPattern("{{category}}--{{type}}--{{index[0]}}--{{index[1]}}--{{fieldName}}")
|
|
293
326
|
...
|
|
294
327
|
```
|
|
295
328
|
|
|
296
|
-
### 5. Grouping
|
|
329
|
+
### 5. Grouping
|
|
330
|
+
|
|
297
331
|
Since data had been flattened in the step 1., it is structured as a list of property names and their values.
|
|
298
332
|
This non-hierarchical structure is ideal to add further properties, attach descriptions and remove duplicates.
|
|
299
333
|
After all, a fully flat structure might not be suitable to display overviews/details or to collect options. <br/><br/>
|
|
300
334
|
The `groupName` defines the name of the group attribute (defaults to "group" if not set). <br/><br/>
|
|
301
335
|
The `groupPattern` describes, which properties belong to the same group.
|
|
302
|
-
Variables (listed below) are put into double curly brackets and will be replaced with the contents
|
|
336
|
+
Variables (listed below) are put into double curly brackets and will be replaced with the contents
|
|
303
337
|
of the description and the matching property.
|
|
304
338
|
The `groupPattern` will be resolved to the `_identifier.groupId`. Every property, that leads to a
|
|
305
339
|
new groupId gets a new attribute named by the `groupName`, where this entry and all others of the
|
|
306
340
|
same group will be put into. Example:
|
|
341
|
+
|
|
307
342
|
```javascript
|
|
308
343
|
new datarestructor.PropertyStructureDescriptionBuilder()
|
|
309
344
|
.groupName("details")
|
|
@@ -311,15 +346,17 @@ same group will be put into. Example:
|
|
|
311
346
|
...
|
|
312
347
|
```
|
|
313
348
|
|
|
314
|
-
### 6. Moving groups (destination group)
|
|
315
|
-
|
|
349
|
+
### 6. Moving groups (destination group)
|
|
350
|
+
|
|
351
|
+
After grouping in step 5., every property containing a group and the remaining non-grouped properties
|
|
316
352
|
are listed one after another. To organize them further, a group can be moved beneath another (destination) group. <br/><br/>
|
|
317
353
|
The `groupDestinationPattern` contains the pattern of the group to where the own group should be moved.
|
|
318
|
-
Variables (listed below) are put into double curly brackets and will be replaced with the contents
|
|
354
|
+
Variables (listed below) are put into double curly brackets and will be replaced with the contents
|
|
319
355
|
of the description and the matching property.
|
|
320
356
|
Optionally, the `groupDestinationName` can be specified to rename the group when it is moved. Default is the value of `groupName`.
|
|
321
|
-
Example, where the details group is moved to the summary, because the group destination pattern
|
|
322
|
-
of the details resolves to the same id as the resolved group pattern of the summary:
|
|
357
|
+
Example, where the details group is moved to the summary, because the group destination pattern
|
|
358
|
+
of the details resolves to the same id as the resolved group pattern of the summary:
|
|
359
|
+
|
|
323
360
|
```javascript
|
|
324
361
|
var summaryDescription = new datarestructor.PropertyStructureDescriptionBuilder()
|
|
325
362
|
.category("account")
|
|
@@ -333,87 +370,97 @@ of the details resolves to the same id as the resolved group pattern of the summ
|
|
|
333
370
|
.groupDestinationName("details")
|
|
334
371
|
...
|
|
335
372
|
```
|
|
336
|
-
### 7. Convert data into an array of DescribedFields:
|
|
337
|
-
The result is finally converted into an array of [DescribedDataField](#DescribedDataField)s.
|
|
338
373
|
|
|
339
|
-
|
|
374
|
+
### 7. Convert data into an array of DescribedFields
|
|
375
|
+
|
|
376
|
+
The result is finally converted into an array of [DescribedDataField](#describeddatafield)s.
|
|
377
|
+
|
|
378
|
+
## Types, fields, variables
|
|
379
|
+
|
|
340
380
|
This section lists the types and their fields in detail (mostly taken from jsdoc).
|
|
341
381
|
Every field can be used as variable in double curly brackets inside pattern properties.
|
|
342
382
|
Additionally, single elements of the index can be used by specifying the index position e.g. `{{index[0]}}` (first), `{{index[1]}}` (second),...
|
|
343
383
|
|
|
344
|
-
|
|
345
384
|
### PropertyStructureDescription (input description)
|
|
346
385
|
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
386
|
+
- **type** - ""(default). Some examples: "summary" for e.g. a list overview. "detail" e.g. when a summary is selected. "filter" e.g. for field/value pair results that can be selected as search parameters.
|
|
387
|
+
- **category** - name of the category. Default = "". Could contain a symbol character or a short domain name. (e.g. "city")
|
|
388
|
+
- **abbreviation** - ""(default). One optional character, a symbol character or a short abbreviation of the category.
|
|
389
|
+
- **image** - ""(default). One optional path to an image resource.
|
|
390
|
+
- **propertyPatternTemplateMode** - boolean "false"(default): property name needs to be equal to the pattern. "true" allows variables like `{{fieldname}}` inside the pattern.
|
|
391
|
+
- **propertyPattern** - property name pattern (without array indices) to match. A pattern may contain variables in double curly brackets {{variable}}. See also: [variables](#describeddatafield-public-fields), [further details](#describeddatafield-public-functions)
|
|
392
|
+
- **indexStartsWith** - ""(default) matches all ids. String that needs to match the beginning of the id. E.g. "1." will match id="1.3.4" but not "0.1.2".
|
|
393
|
+
- **groupName** - name of the property, that contains grouped entries. Default="group".
|
|
394
|
+
- **groupPattern** - Pattern that describes how to group entries. "groupName" defines the name of this group. A pattern may contain variables in double curly brackets {{variable}}. See also: [variables](#describeddatafield-public-fields), [further details](#describeddatafield-public-functions)
|
|
395
|
+
- **groupDestinationPattern** - Pattern that describes where the group should be moved to. Default=""=Group will not be moved. A pattern may contain variables in double curly brackets {{variable}}. See also: [variables](#describeddatafield-public-fields), [further details](#describeddatafield-public-functions)
|
|
396
|
+
- **groupDestinationName** - (default=groupName) Name of the group when it had been moved to the destination.
|
|
397
|
+
- **deduplicationPattern** - Pattern to use to remove duplicate entries. A pattern may contain variables in double curly brackets {{variable}}. See also: [variables](#describeddatafield-public-fields), [further details](#describeddatafield-public-functions)
|
|
360
398
|
|
|
361
399
|
### DescribedDataField
|
|
362
|
-
|
|
363
|
-
|
|
400
|
+
|
|
401
|
+
This is the data structure of a single output element representing a field.
|
|
402
|
+
Beside the properties described below, the described data field can also contain
|
|
364
403
|
custom properties containing groups (arrays) of sub fields of type DescribedDataField.
|
|
365
404
|
|
|
366
|
-
Before version 3.0.0 this structure was named [DescribedEntry](#
|
|
367
|
-
Since 3.0.0 and above, [DescribedEntry](#
|
|
405
|
+
Before version 3.0.0 this structure was named [DescribedEntry](#describedentry) and also contained internal fields.
|
|
406
|
+
Since 3.0.0 and above, [DescribedEntry](#describedentry) is only used internally and is not public any more.
|
|
407
|
+
|
|
408
|
+
#### DescribedDataField Public Fields
|
|
368
409
|
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
* **value** - content of the field
|
|
410
|
+
- **category** - category of the result from the PropertyStructureDescription using a short name or e.g. a symbol character
|
|
411
|
+
- **type** - type of the result from PropertyStructureDescription
|
|
412
|
+
- **abbreviation** - one optional character, a symbol character or a short abbreviation of the category
|
|
413
|
+
- **image** - one optional path to an image resource
|
|
414
|
+
- **index** - contains an array of numbers representing the hierarchical index for list entries (and their sub lists ...). Example: `"responses[2].hits.hits[4]._source.name"` will have an index of [2,4].
|
|
415
|
+
- **groupNames** - contains an array of String names. Every name represents a group that had been dynamically added as property. Groups should be added using [DescribedDataFieldGroup](#describeddatafieldgroup), which will also update the group names.
|
|
416
|
+
- **displayName** - display name extracted from the point separated hierarchical property name, e.g. "Name"
|
|
417
|
+
- **fieldName** - field name extracted from the point separated hierarchical property name, e.g. "name"
|
|
418
|
+
- **value** - content of the field
|
|
379
419
|
|
|
380
|
-
#### Public
|
|
381
|
-
|
|
420
|
+
#### DescribedDataField Public Functions
|
|
421
|
+
|
|
422
|
+
Since version 3.0.0 and above, there are no functions any more.
|
|
382
423
|
|
|
383
424
|
#### Described groups
|
|
384
|
-
* **"name of described group"** as described in PropertyStructureDescription
|
|
385
|
-
* **"names of moved groups"** as described in PropertyStructureDescription of the group that had been moved
|
|
386
425
|
|
|
426
|
+
- **"name of described group"** as described in PropertyStructureDescription
|
|
427
|
+
- **"names of moved groups"** as described in PropertyStructureDescription of the group that had been moved
|
|
387
428
|
|
|
388
429
|
### DescribedDataFieldGroup
|
|
389
|
-
This helper was added with version 3.0.0. It adds groups to [DescribedDataField](#DescribedDataField)s.
|
|
390
|
-
These groups are dynamically added properties that contain an array of sub fields also of type [DescribedDataField](#DescribedDataField).
|
|
391
430
|
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
431
|
+
This helper was added with version 3.0.0. It adds groups to [DescribedDataField](#describeddatafield)s.
|
|
432
|
+
These groups are dynamically added properties that contain an array of sub fields also of type [DescribedDataField](#describeddatafield).
|
|
433
|
+
|
|
434
|
+
#### DescribedDataFieldGroup Public Functions
|
|
435
|
+
|
|
436
|
+
- **addGroupEntry(groupName, entry)** Adds an entry to the given group. If the group does not exist, it will be created and added to the "groupNames".
|
|
437
|
+
- **addGroupEntries(groupName, entries)** Adds an array of entries to the given group. If the group does not exist, it will be created and added to the "groupNames".
|
|
396
438
|
|
|
397
439
|
### DescribedEntry
|
|
440
|
+
|
|
398
441
|
Since 3.0.0 and above, DescribedEntry is only used internally and is not public any more.
|
|
399
442
|
It is documented here for sake of completeness and for maintenance purposes.
|
|
400
443
|
See JSDoc for a more comprehensive reference.
|
|
444
|
+
|
|
401
445
|
#### Properties
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
446
|
+
|
|
447
|
+
- **describedField** - contains the [DescribedDataField](#describeddatafield)
|
|
448
|
+
- **isMatchingIndex** - true, if _identifier.index matches the described "indexStartsWith"
|
|
449
|
+
- **_identifier** - internal structure for identifier. Avoid using it outside since it may change.
|
|
450
|
+
- **_identifier.index** - array indices in hierarchical order separated by points, e.g. "0.0"
|
|
451
|
+
- **_identifier.value** - the (single) value of the "flattened" property, e.g. "Smith"
|
|
452
|
+
- **_identifier.propertyNamesWithArrayIndices** - the "original" flattened property name in hierarchical order separated by points, e.g. "responses[0].hits.hits[0]._source.name"
|
|
453
|
+
- **_identifier.propertyNameWithoutArrayIndices** - same as propertyNamesWithArrayIndices but without array indices, e.g. "responses.hits.hits._source.name"
|
|
454
|
+
- **_identifier.groupId** - Contains the resolved groupPattern from the PropertyStructureDescription. Entries with the same id will be grouped into the "groupName" of the PropertyStructureDescription.
|
|
455
|
+
- **_identifier.groupDestinationId** - Contains the resolved groupDestinationPattern from the PropertyStructureDescription. Entries with this id will be moved to the given destination group.
|
|
456
|
+
- **_identifier.deduplicationId** - Contains the resolved deduplicationPattern from the PropertyStructureDescription. Entries with the same id will be considered to be a duplicate and hence removed.
|
|
457
|
+
- **_description** - PropertyStructureDescription for internal use. Avoid using it outside since it may change.
|
|
413
458
|
|
|
414
459
|
### Template Resolver
|
|
460
|
+
|
|
415
461
|
An simple template resolver is included and provided as separate module.
|
|
416
462
|
Here is an example on how to use it:
|
|
463
|
+
|
|
417
464
|
```javaScript
|
|
418
465
|
var template_resolver = require("templateResolver");
|
|
419
466
|
var sourceDataObject = {type: "MyType", category: "MyCategory"};
|
|
@@ -422,49 +469,53 @@ var template = "{{type}}-{{category}}";
|
|
|
422
469
|
var resolvedString = resolver.resolveTemplate(template);
|
|
423
470
|
//resolvedString will contain "MyType-MyCategory"
|
|
424
471
|
```
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
472
|
+
|
|
473
|
+
#### Template Resolver Public Functions
|
|
474
|
+
|
|
475
|
+
- **resolveTemplate** - resolves the given template string. The template may contain variables in double curly brackets:
|
|
476
|
+
- All [public fields](#describeddatafield-public-fields) can be used as variables, e.g. `"{{fieldName}}"`, `"{{displayName}}"`, `"{{value}}"`.
|
|
477
|
+
- Described groups that contain an array of [described entries](#describeddatafield) can also be used, e.g. `"{{summaries[0].value}}"`.
|
|
478
|
+
- Parts of the index can be inserted by using e.g. `"{{index[1]}}"`.
|
|
479
|
+
- Besides the meta data, a described field can be used directly by its "fieldName", e.g. `"{{customernumber}}"` will be replaced by `123`, if the structure contains `fieldname="customernumber", value="123"`. This also applies to sub groups, e.g. `"{{details.customernumber}}"` will be replaced by `321`, if the structure contains `details[4].fieldname="customernumber", details[4].value="321"`.
|
|
431
480
|
|
|
432
481
|
### TransformConfig
|
|
433
|
-
|
|
482
|
+
|
|
483
|
+
An comprehensive and up to date reference can be found here: [TransformConfig JSDoc](https://joht.github.io/data-restructor-js/module-datarestructor.html#.TransformConfig).
|
|
434
484
|
|
|
435
485
|
The restructured data is by nature hierarchical and may contain cyclic data references. Fields may contain groups of fields that may contain groups of fields....
|
|
436
|
-
Since JSON can't be generated out of objects with cyclic references, sub-structures are expressed by copies.
|
|
486
|
+
Since JSON can't be generated out of objects with cyclic references, sub-structures are expressed by copies.
|
|
437
487
|
That leads to recursion and duplication, that need to be limited. This can be configured here.
|
|
438
488
|
|
|
439
|
-
#### Properties
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
489
|
+
#### TransformConfig Properties
|
|
490
|
+
|
|
491
|
+
- **debugMode** boolean value, that enables/disables detailed logging
|
|
492
|
+
- **maxRecursionDepth** numeric value that defines the maximum recursion depth
|
|
493
|
+
- **removeDuplicationAboveRecursionDepth** numeric value that defines the recursion depth, above which duplications inside groups will be removed.
|
|
443
494
|
|
|
444
495
|
#### Public functions (provides by "Transform")
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
496
|
+
|
|
497
|
+
- **enableDebugMode(boolean)** boolean value, that enables/disables detailed logging
|
|
498
|
+
- **setMaxRecursionDepth(number)** numeric value that defines the maximum recursion depth
|
|
499
|
+
- **setRemoveDuplicationAboveRecursionDepth(number)** numeric value that defines the recursion depth, above which duplications inside groups will be removed.
|
|
448
500
|
|
|
449
501
|
## References
|
|
450
|
-
* [Mozilla MDN web docs - polyfill for 'Array.filter'](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter#Polyfill) for browser compatibility
|
|
451
|
-
* [Mozilla MDN web docs - polyfill for 'Array.forEach'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach#Polyfill) for browser compatibility (references [es5.github.io](http://es5.github.io/#x15.4.4.18))
|
|
452
|
-
* [Mozilla MDN web docs - polyfill for 'Array.indexOf'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Array/indexOf#Polyfill) for browser compatibility
|
|
453
|
-
* [Mozilla MDN web docs - polyfill for 'Array.isArray'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Array/isArray#Compatibility) for browser compatibility
|
|
454
|
-
* [Mozilla MDN web docs - polyfill for 'String.startsWith'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/String/startsWith#Polyfill) for browser compatibility
|
|
455
|
-
* [Token Posts - polyfill for 'Object.keys'](http://tokenposts.blogspot.com/2012/04/javascript-objectkeys-browser.html) for browser compatibility
|
|
456
502
|
|
|
503
|
+
- [Mozilla MDN web docs - polyfill for 'Array.filter'](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter#Polyfill) for browser compatibility
|
|
504
|
+
- [Mozilla MDN web docs - polyfill for 'Array.forEach'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach#Polyfill) for browser compatibility (references [es5.github.io](http://es5.github.io/#x15.4.4.18))
|
|
505
|
+
- [Mozilla MDN web docs - polyfill for 'Array.indexOf'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Array/indexOf#Polyfill) for browser compatibility
|
|
506
|
+
- [Mozilla MDN web docs - polyfill for 'Array.isArray'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Array/isArray#Compatibility) for browser compatibility
|
|
507
|
+
- [Mozilla MDN web docs - polyfill for 'String.startsWith'](https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/String/startsWith#Polyfill) for browser compatibility
|
|
508
|
+
- [Token Posts - polyfill for 'Object.keys'](http://tokenposts.blogspot.com/2012/04/javascript-objectkeys-browser.html) for browser compatibility
|
|
457
509
|
|
|
458
510
|
## Credits
|
|
459
511
|
|
|
460
512
|
Although this project doesn't use any runtime dependencies, it is created using these great tools:
|
|
461
513
|
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
514
|
+
- [ESLint](https://eslint.org) - [MIT License](https://github.com/eslint/eslint/blob/master/LICENSE)
|
|
515
|
+
- [istanbul-badges-readme](https://www.npmjs.com/package/istanbul-badges-readme) - [MIT License](https://github.com/olavoparno/istanbul-badges-readme/blob/develop/LICENSE)
|
|
516
|
+
- [Jasmine](https://jasmine.github.io) - [MIT License](https://github.com/jasmine/jasmine/blob/main/MIT.LICENSE)
|
|
517
|
+
- [JSDoc](https://jsdoc.app) - [Apache Licence 2.0](https://github.com/jsdoc/jsdoc/blob/master/LICENSE)
|
|
518
|
+
- [merger-js](https://github.com/joao-neves95/merger-js) - [GNU General Public License v3.0](https://github.com/joao-neves95/merger-js/blob/master/LICENSE.md)
|
|
519
|
+
- [NYC aka Istanbul](https://istanbul.js.org) - [ISC License](https://github.com/istanbuljs/nyc/blob/master/LICENSE.txt)
|
|
520
|
+
- [PARCEL](https://v2.parceljs.org) - [MIT License](https://github.com/parcel-bundler/parcel/blob/v2/LICENSE)
|
|
521
|
+
- [merger-js](https://github.com/joao-neves95/merger-js) - [GNU General Public License v3.0](https://github.com/joao-neves95/merger-js/blob/master/LICENSE.md)
|
package/SECURITY.md
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
|
|
10
10
|
## Reporting a Vulnerability
|
|
11
11
|
|
|
12
|
-
Create an [issue](https://github.com/JohT/search-menu-ui/issues/new/choose)
|
|
13
|
-
and add the label "security" to report a vulnerability. Since this is right now maintained in free time,
|
|
12
|
+
Create an [issue](https://github.com/JohT/search-menu-ui/issues/new/choose)
|
|
13
|
+
and add the label "security" to report a vulnerability. Since this is right now maintained in free time,
|
|
14
14
|
it might take a while. Providing a detailed description or a pull request with the fix are
|
|
15
15
|
the best ways to support and speed up the process.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{"total": {"lines":{"total":461,"covered":455,"skipped":0,"pct":98.7},"statements":{"total":470,"covered":462,"skipped":0,"pct":98.3},"functions":{"total":108,"covered":106,"skipped":0,"pct":98.15},"branches":{"total":167,"covered":158,"skipped":0,"pct":94.61}}
|
|
2
|
-
,"/
|
|
3
|
-
,"/
|
|
4
|
-
,"/
|
|
2
|
+
,"/home/runner/work/data-restructor-js/data-restructor-js/src/js/datarestructor.js": {"lines":{"total":340,"covered":335,"skipped":0,"pct":98.53},"functions":{"total":76,"covered":75,"skipped":0,"pct":98.68},"statements":{"total":349,"covered":342,"skipped":0,"pct":97.99},"branches":{"total":128,"covered":119,"skipped":0,"pct":92.97}}
|
|
3
|
+
,"/home/runner/work/data-restructor-js/data-restructor-js/src/js/describedfield.js": {"lines":{"total":70,"covered":69,"skipped":0,"pct":98.57},"functions":{"total":22,"covered":21,"skipped":0,"pct":95.45},"statements":{"total":70,"covered":69,"skipped":0,"pct":98.57},"branches":{"total":21,"covered":21,"skipped":0,"pct":100}}
|
|
4
|
+
,"/home/runner/work/data-restructor-js/data-restructor-js/src/js/templateResolver.js": {"lines":{"total":51,"covered":51,"skipped":0,"pct":100},"functions":{"total":10,"covered":10,"skipped":0,"pct":100},"statements":{"total":51,"covered":51,"skipped":0,"pct":100},"branches":{"total":18,"covered":18,"skipped":0,"pct":100}}
|
|
5
5
|
}
|