@twin.org/data-processing-extractors 0.0.3-next.1 → 0.0.3-next.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. package/README.md +1 -1
  2. package/docs/changelog.md +48 -34
  3. package/docs/examples.md +69 -1
  4. package/docs/reference/classes/JsonPathExtractorConnector.md +4 -4
  5. package/package.json +5 -5
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # TWIN Data Processing Extractors
2
2
 
3
- Data extractor connector implementations.
3
+ Connector implementations that extract data from external systems for processing.
4
4
 
5
5
  ## Installation
6
6
 
package/docs/changelog.md CHANGED
@@ -1,22 +1,36 @@
1
- # @twin.org/data-processing-extractors - Changelog
1
+ # Changelog
2
2
 
3
- ## [0.0.3-next.1](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.3-next.0...data-processing-extractors-v0.0.3-next.1) (2025-11-11)
3
+ ## [0.0.3-next.2](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.3-next.1...data-processing-extractors-v0.0.3-next.2) (2026-05-11)
4
4
 
5
5
 
6
6
  ### Features
7
7
 
8
- * add context id features ([#25](https://github.com/twinfoundation/data-processing/issues/25)) ([2a2a955](https://github.com/twinfoundation/data-processing/commit/2a2a9559d54e8c703e97e2c05996e678416ef560))
9
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
10
- * add validate-locales ([9af0874](https://github.com/twinfoundation/data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
11
- * eslint migration to flat config ([a8138b0](https://github.com/twinfoundation/data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
12
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
13
- * update framework core ([96f6e80](https://github.com/twinfoundation/data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
14
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
8
+ * typescript 6 update ([b67debb](https://github.com/iotaledger/twin-data-processing/commit/b67debb8aff83c0f3b179c6db038ed733f7308b1))
9
+
10
+
11
+ ### Dependencies
12
+
13
+ * The following workspace dependencies were updated
14
+ * dependencies
15
+ * @twin.org/data-processing-models bumped from 0.0.3-next.1 to 0.0.3-next.2
16
+
17
+ ## [0.0.3-next.1](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.3-next.0...data-processing-extractors-v0.0.3-next.1) (2025-11-11)
18
+
19
+
20
+ ### Features
21
+
22
+ * add context id features ([#25](https://github.com/iotaledger/twin-data-processing/issues/25)) ([2a2a955](https://github.com/iotaledger/twin-data-processing/commit/2a2a9559d54e8c703e97e2c05996e678416ef560))
23
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
24
+ * add validate-locales ([9af0874](https://github.com/iotaledger/twin-data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
25
+ * eslint migration to flat config ([a8138b0](https://github.com/iotaledger/twin-data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
26
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
27
+ * update framework core ([96f6e80](https://github.com/iotaledger/twin-data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
28
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
15
29
 
16
30
 
17
31
  ### Bug Fixes
18
32
 
19
- * supporting array usage ([#20](https://github.com/twinfoundation/data-processing/issues/20)) ([d2c68cd](https://github.com/twinfoundation/data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
33
+ * supporting array usage ([#20](https://github.com/iotaledger/twin-data-processing/issues/20)) ([d2c68cd](https://github.com/iotaledger/twin-data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
20
34
 
21
35
 
22
36
  ### Dependencies
@@ -25,12 +39,12 @@
25
39
  * dependencies
26
40
  * @twin.org/data-processing-models bumped from 0.0.3-next.0 to 0.0.3-next.1
27
41
 
28
- ## [0.0.2-next.4](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.3...data-processing-extractors-v0.0.2-next.4) (2025-10-22)
42
+ ## [0.0.2-next.4](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.3...data-processing-extractors-v0.0.2-next.4) (2025-10-22)
29
43
 
30
44
 
31
45
  ### Bug Fixes
32
46
 
33
- * supporting array usage ([#20](https://github.com/twinfoundation/data-processing/issues/20)) ([d2c68cd](https://github.com/twinfoundation/data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
47
+ * supporting array usage ([#20](https://github.com/iotaledger/twin-data-processing/issues/20)) ([d2c68cd](https://github.com/iotaledger/twin-data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
34
48
 
35
49
 
36
50
  ### Dependencies
@@ -39,12 +53,12 @@
39
53
  * dependencies
40
54
  * @twin.org/data-processing-models bumped from 0.0.2-next.3 to 0.0.2-next.4
41
55
 
42
- ## [0.0.2-next.3](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.2...data-processing-extractors-v0.0.2-next.3) (2025-10-09)
56
+ ## [0.0.2-next.3](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.2...data-processing-extractors-v0.0.2-next.3) (2025-10-09)
43
57
 
44
58
 
45
59
  ### Features
46
60
 
47
- * add validate-locales ([9af0874](https://github.com/twinfoundation/data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
61
+ * add validate-locales ([9af0874](https://github.com/iotaledger/twin-data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
48
62
 
49
63
 
50
64
  ### Dependencies
@@ -53,12 +67,12 @@
53
67
  * dependencies
54
68
  * @twin.org/data-processing-models bumped from 0.0.2-next.2 to 0.0.2-next.3
55
69
 
56
- ## [0.0.2-next.2](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.1...data-processing-extractors-v0.0.2-next.2) (2025-08-29)
70
+ ## [0.0.2-next.2](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.1...data-processing-extractors-v0.0.2-next.2) (2025-08-29)
57
71
 
58
72
 
59
73
  ### Features
60
74
 
61
- * eslint migration to flat config ([a8138b0](https://github.com/twinfoundation/data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
75
+ * eslint migration to flat config ([a8138b0](https://github.com/iotaledger/twin-data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
62
76
 
63
77
 
64
78
  ### Dependencies
@@ -67,15 +81,15 @@
67
81
  * dependencies
68
82
  * @twin.org/data-processing-models bumped from 0.0.2-next.1 to 0.0.2-next.2
69
83
 
70
- ## [0.0.2-next.1](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.0...data-processing-extractors-v0.0.2-next.1) (2025-08-20)
84
+ ## [0.0.2-next.1](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.0...data-processing-extractors-v0.0.2-next.1) (2025-08-20)
71
85
 
72
86
 
73
87
  ### Features
74
88
 
75
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
76
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
77
- * update framework core ([96f6e80](https://github.com/twinfoundation/data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
78
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
89
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
90
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
91
+ * update framework core ([96f6e80](https://github.com/iotaledger/twin-data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
92
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
79
93
 
80
94
 
81
95
  ### Dependencies
@@ -89,10 +103,10 @@
89
103
 
90
104
  ### Features
91
105
 
92
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
93
- * release to production ([531ba89](https://github.com/twinfoundation/data-processing/commit/531ba89dea2deb2810870b72851dca3081f9d2ce))
94
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
95
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
106
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
107
+ * release to production ([531ba89](https://github.com/iotaledger/twin-data-processing/commit/531ba89dea2deb2810870b72851dca3081f9d2ce))
108
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
109
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
96
110
 
97
111
 
98
112
  ### Dependencies
@@ -101,7 +115,7 @@
101
115
  * dependencies
102
116
  * @twin.org/data-processing-models bumped from ^0.0.0 to ^0.0.1
103
117
 
104
- ## [0.0.1-next.16](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.15...data-processing-extractors-v0.0.1-next.16) (2025-06-20)
118
+ ## [0.0.1-next.16](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.15...data-processing-extractors-v0.0.1-next.16) (2025-06-20)
105
119
 
106
120
 
107
121
  ### Miscellaneous Chores
@@ -115,12 +129,12 @@
115
129
  * dependencies
116
130
  * @twin.org/data-processing-models bumped from 0.0.1-next.15 to 0.0.1-next.16
117
131
 
118
- ## [0.0.1-next.15](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.14...data-processing-extractors-v0.0.1-next.15) (2025-06-12)
132
+ ## [0.0.1-next.15](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.14...data-processing-extractors-v0.0.1-next.15) (2025-06-12)
119
133
 
120
134
 
121
135
  ### Features
122
136
 
123
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
137
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
124
138
 
125
139
 
126
140
  ### Dependencies
@@ -129,7 +143,7 @@
129
143
  * dependencies
130
144
  * @twin.org/data-processing-models bumped from 0.0.1-next.14 to 0.0.1-next.15
131
145
 
132
- ## [0.0.1-next.14](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.13...data-processing-extractors-v0.0.1-next.14) (2025-04-28)
146
+ ## [0.0.1-next.14](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.13...data-processing-extractors-v0.0.1-next.14) (2025-04-28)
133
147
 
134
148
 
135
149
  ### Miscellaneous Chores
@@ -143,7 +157,7 @@
143
157
  * dependencies
144
158
  * @twin.org/data-processing-models bumped from 0.0.1-next.13 to 0.0.1-next.14
145
159
 
146
- ## [0.0.1-next.13](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.12...data-processing-extractors-v0.0.1-next.13) (2025-04-24)
160
+ ## [0.0.1-next.13](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.12...data-processing-extractors-v0.0.1-next.13) (2025-04-24)
147
161
 
148
162
 
149
163
  ### Miscellaneous Chores
@@ -157,12 +171,12 @@
157
171
  * dependencies
158
172
  * @twin.org/data-processing-models bumped from 0.0.1-next.12 to 0.0.1-next.13
159
173
 
160
- ## [0.0.1-next.12](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.11...data-processing-extractors-v0.0.1-next.12) (2025-04-17)
174
+ ## [0.0.1-next.12](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.11...data-processing-extractors-v0.0.1-next.12) (2025-04-17)
161
175
 
162
176
 
163
177
  ### Features
164
178
 
165
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
179
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
166
180
 
167
181
 
168
182
  ### Dependencies
@@ -171,12 +185,12 @@
171
185
  * dependencies
172
186
  * @twin.org/data-processing-models bumped from 0.0.1-next.11 to 0.0.1-next.12
173
187
 
174
- ## [0.0.1-next.11](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.10...data-processing-extractors-v0.0.1-next.11) (2025-03-28)
188
+ ## [0.0.1-next.11](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.10...data-processing-extractors-v0.0.1-next.11) (2025-03-28)
175
189
 
176
190
 
177
191
  ### Features
178
192
 
179
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
193
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
180
194
 
181
195
 
182
196
  ### Dependencies
package/docs/examples.md CHANGED
@@ -1 +1,69 @@
1
- # @twin.org/data-processing-extractors - Examples
1
+ # Data Processing Extractors Examples
2
+
3
+ Use these snippets to map nested JSON data into flatter, application-focused shapes with JSONPath rules.
4
+
5
+ ## JsonPathExtractorConnector
6
+
7
+ ```typescript
8
+ import { JsonPathExtractorConnector } from '@twin.org/data-processing-extractors';
9
+
10
+ const extractor = new JsonPathExtractorConnector();
11
+
12
+ console.log(extractor.className()); // jsonPathExtractorConnector
13
+
14
+ const sourceData = {
15
+ users: [
16
+ { firstName: 'John', lastName: 'Doe', age: 30 },
17
+ { firstName: 'Jane', lastName: 'Doe', age: 25 }
18
+ ]
19
+ };
20
+
21
+ const extracted = await extractor.extract(sourceData, [
22
+ { source: '$.users[0].firstName', target: 'profile.givenName' },
23
+ { source: '$.users[1].lastName', target: 'profile.familyName' }
24
+ ]);
25
+
26
+ console.log(extracted); // { profile: { givenName: 'John', familyName: 'Doe' } }
27
+ ```
28
+
29
+ ```typescript
30
+ import { JsonPathExtractorConnector } from '@twin.org/data-processing-extractors';
31
+
32
+ const extractor = new JsonPathExtractorConnector();
33
+
34
+ const extracted = await extractor.extract(
35
+ {
36
+ store: {
37
+ books: [
38
+ { title: 'The Hobbit', author: 'J.R.R. Tolkien' },
39
+ { title: '1984', author: 'George Orwell' },
40
+ { title: 'Dune', author: 'Frank Herbert' }
41
+ ]
42
+ }
43
+ },
44
+ [{ source: '$.store.books[*].title', target: 'bookTitles' }]
45
+ );
46
+
47
+ console.log(extracted); // { bookTitles: ['The Hobbit', '1984', 'Dune'] }
48
+ ```
49
+
50
+ ```typescript
51
+ import { JsonPathExtractorConnector } from '@twin.org/data-processing-extractors';
52
+
53
+ const extractor = new JsonPathExtractorConnector();
54
+
55
+ const extracted = await extractor.extract(
56
+ {
57
+ users: {
58
+ user1: { firstName: 'John', lastName: 'Doe', age: 30 },
59
+ user2: { firstName: 'Jane', lastName: 'Doe', age: 25 }
60
+ }
61
+ },
62
+ [
63
+ { source: '$.users.*.firstName', target: 'selected', retainPathDepth: 2 },
64
+ { source: '$.users.*.lastName', target: 'selected', retainPathDepth: 2 }
65
+ ]
66
+ );
67
+
68
+ console.log(extracted); // { selected: { user1: { firstName: 'John', lastName: 'Doe' }, user2: { firstName: 'Jane', lastName: 'Doe' } } }
69
+ ```
package/docs/reference/classes/JsonPathExtractorConnector.md CHANGED
@@ -18,7 +18,7 @@ Class for extracting data from a JSON source.
18
18
 
19
19
  ## Properties
20
20
 
21
- ### NAMESPACE
21
+ ### NAMESPACE {#namespace}
22
22
 
23
23
  > `readonly` `static` **NAMESPACE**: `string` = `"json-path"`
24
24
 
@@ -26,7 +26,7 @@ The namespace supported by the data extractor connector.
26
26
 
27
27
  ***
28
28
 
29
- ### CLASS\_NAME
29
+ ### CLASS\_NAME {#class_name}
30
30
 
31
31
  > `readonly` `static` **CLASS\_NAME**: `string`
32
32
 
@@ -34,7 +34,7 @@ Runtime name for the class.
34
34
 
35
35
  ## Methods
36
36
 
37
- ### className()
37
+ ### className() {#classname}
38
38
 
39
39
  > **className**(): `string`
40
40
 
@@ -52,7 +52,7 @@ The class name of the component.
52
52
 
53
53
  ***
54
54
 
55
- ### extract()
55
+ ### extract() {#extract}
56
56
 
57
57
  > **extract**(`data`, `rules`): `Promise`\<`unknown`\>
58
58
 
package/package.json CHANGED
@@ -1,10 +1,10 @@
1
1
  {
2
2
  "name": "@twin.org/data-processing-extractors",
3
- "version": "0.0.3-next.1",
4
- "description": "Data extractor connector implementations",
3
+ "version": "0.0.3-next.2",
4
+ "description": "Connector implementations that extract data from external systems for processing",
5
5
  "repository": {
6
6
  "type": "git",
7
- "url": "git+https://github.com/twinfoundation/data-processing.git",
7
+ "url": "git+https://github.com/iotaledger/data-processing.git",
8
8
  "directory": "packages/data-processing-extractors"
9
9
  },
10
10
  "author": "martyn.janes@iota.org",
@@ -16,7 +16,7 @@
16
16
  "dependencies": {
17
17
  "@twin.org/core": "next",
18
18
  "@twin.org/data-json-path": "next",
19
- "@twin.org/data-processing-models": "0.0.3-next.1",
19
+ "@twin.org/data-processing-models": "0.0.3-next.2",
20
20
  "@twin.org/nameof": "next",
21
21
  "@twin.org/web": "next"
22
22
  },
@@ -49,7 +49,7 @@
49
49
  "conversion"
50
50
  ],
51
51
  "bugs": {
52
- "url": "git+https://github.com/twinfoundation/data-processing/issues"
52
+ "url": "git+https://github.com/iotaledger/data-processing/issues"
53
53
  },
54
54
  "homepage": "https://twindev.org"
55
55
  }