@twin.org/data-processing-extractors 0.0.3-next.1 → 0.0.3-next.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. package/README.md +1 -1
  2. package/docs/changelog.md +62 -34
  3. package/docs/examples.md +69 -1
  4. package/docs/reference/classes/JsonPathExtractorConnector.md +4 -4
  5. package/package.json +5 -5
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # TWIN Data Processing Extractors
2
2
 
3
- Data extractor connector implementations.
3
+ Connector implementations that extract data from external systems for processing.
4
4
 
5
5
  ## Installation
6
6
 
package/docs/changelog.md CHANGED
@@ -1,22 +1,50 @@
1
- # @twin.org/data-processing-extractors - Changelog
1
+ # Changelog
2
2
 
3
- ## [0.0.3-next.1](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.3-next.0...data-processing-extractors-v0.0.3-next.1) (2025-11-11)
3
+ ## [0.0.3-next.3](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.3-next.2...data-processing-extractors-v0.0.3-next.3) (2026-05-20)
4
4
 
5
5
 
6
6
  ### Features
7
7
 
8
- * add context id features ([#25](https://github.com/twinfoundation/data-processing/issues/25)) ([2a2a955](https://github.com/twinfoundation/data-processing/commit/2a2a9559d54e8c703e97e2c05996e678416ef560))
9
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
10
- * add validate-locales ([9af0874](https://github.com/twinfoundation/data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
11
- * eslint migration to flat config ([a8138b0](https://github.com/twinfoundation/data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
12
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
13
- * update framework core ([96f6e80](https://github.com/twinfoundation/data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
14
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
8
+ * update dependencies ([0b2e77d](https://github.com/iotaledger/twin-data-processing/commit/0b2e77d673ede6e92847ff0b50fdfcf4e9c08b3b))
9
+
10
+
11
+ ### Dependencies
12
+
13
+ * The following workspace dependencies were updated
14
+ * dependencies
15
+ * @twin.org/data-processing-models bumped from 0.0.3-next.2 to 0.0.3-next.3
16
+
17
+ ## [0.0.3-next.2](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.3-next.1...data-processing-extractors-v0.0.3-next.2) (2026-05-11)
18
+
19
+
20
+ ### Features
21
+
22
+ * typescript 6 update ([b67debb](https://github.com/iotaledger/twin-data-processing/commit/b67debb8aff83c0f3b179c6db038ed733f7308b1))
23
+
24
+
25
+ ### Dependencies
26
+
27
+ * The following workspace dependencies were updated
28
+ * dependencies
29
+ * @twin.org/data-processing-models bumped from 0.0.3-next.1 to 0.0.3-next.2
30
+
31
+ ## [0.0.3-next.1](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.3-next.0...data-processing-extractors-v0.0.3-next.1) (2025-11-11)
32
+
33
+
34
+ ### Features
35
+
36
+ * add context id features ([#25](https://github.com/iotaledger/twin-data-processing/issues/25)) ([2a2a955](https://github.com/iotaledger/twin-data-processing/commit/2a2a9559d54e8c703e97e2c05996e678416ef560))
37
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
38
+ * add validate-locales ([9af0874](https://github.com/iotaledger/twin-data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
39
+ * eslint migration to flat config ([a8138b0](https://github.com/iotaledger/twin-data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
40
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
41
+ * update framework core ([96f6e80](https://github.com/iotaledger/twin-data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
42
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
15
43
 
16
44
 
17
45
  ### Bug Fixes
18
46
 
19
- * supporting array usage ([#20](https://github.com/twinfoundation/data-processing/issues/20)) ([d2c68cd](https://github.com/twinfoundation/data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
47
+ * supporting array usage ([#20](https://github.com/iotaledger/twin-data-processing/issues/20)) ([d2c68cd](https://github.com/iotaledger/twin-data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
20
48
 
21
49
 
22
50
  ### Dependencies
@@ -25,12 +53,12 @@
25
53
  * dependencies
26
54
  * @twin.org/data-processing-models bumped from 0.0.3-next.0 to 0.0.3-next.1
27
55
 
28
- ## [0.0.2-next.4](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.3...data-processing-extractors-v0.0.2-next.4) (2025-10-22)
56
+ ## [0.0.2-next.4](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.3...data-processing-extractors-v0.0.2-next.4) (2025-10-22)
29
57
 
30
58
 
31
59
  ### Bug Fixes
32
60
 
33
- * supporting array usage ([#20](https://github.com/twinfoundation/data-processing/issues/20)) ([d2c68cd](https://github.com/twinfoundation/data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
61
+ * supporting array usage ([#20](https://github.com/iotaledger/twin-data-processing/issues/20)) ([d2c68cd](https://github.com/iotaledger/twin-data-processing/commit/d2c68cd3983622586cc073377545d6416b87a827))
34
62
 
35
63
 
36
64
  ### Dependencies
@@ -39,12 +67,12 @@
39
67
  * dependencies
40
68
  * @twin.org/data-processing-models bumped from 0.0.2-next.3 to 0.0.2-next.4
41
69
 
42
- ## [0.0.2-next.3](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.2...data-processing-extractors-v0.0.2-next.3) (2025-10-09)
70
+ ## [0.0.2-next.3](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.2...data-processing-extractors-v0.0.2-next.3) (2025-10-09)
43
71
 
44
72
 
45
73
  ### Features
46
74
 
47
- * add validate-locales ([9af0874](https://github.com/twinfoundation/data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
75
+ * add validate-locales ([9af0874](https://github.com/iotaledger/twin-data-processing/commit/9af0874cd3f44a4cfca514dde8380f002e62337e))
48
76
 
49
77
 
50
78
  ### Dependencies
@@ -53,12 +81,12 @@
53
81
  * dependencies
54
82
  * @twin.org/data-processing-models bumped from 0.0.2-next.2 to 0.0.2-next.3
55
83
 
56
- ## [0.0.2-next.2](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.1...data-processing-extractors-v0.0.2-next.2) (2025-08-29)
84
+ ## [0.0.2-next.2](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.1...data-processing-extractors-v0.0.2-next.2) (2025-08-29)
57
85
 
58
86
 
59
87
  ### Features
60
88
 
61
- * eslint migration to flat config ([a8138b0](https://github.com/twinfoundation/data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
89
+ * eslint migration to flat config ([a8138b0](https://github.com/iotaledger/twin-data-processing/commit/a8138b020ed58baeff162a822d4df207a6aeadc4))
62
90
 
63
91
 
64
92
  ### Dependencies
@@ -67,15 +95,15 @@
67
95
  * dependencies
68
96
  * @twin.org/data-processing-models bumped from 0.0.2-next.1 to 0.0.2-next.2
69
97
 
70
- ## [0.0.2-next.1](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.2-next.0...data-processing-extractors-v0.0.2-next.1) (2025-08-20)
98
+ ## [0.0.2-next.1](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.2-next.0...data-processing-extractors-v0.0.2-next.1) (2025-08-20)
71
99
 
72
100
 
73
101
  ### Features
74
102
 
75
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
76
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
77
- * update framework core ([96f6e80](https://github.com/twinfoundation/data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
78
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
103
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
104
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
105
+ * update framework core ([96f6e80](https://github.com/iotaledger/twin-data-processing/commit/96f6e8016a4ef0fb873f6775e750b4ab42e9f491))
106
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
79
107
 
80
108
 
81
109
  ### Dependencies
@@ -89,10 +117,10 @@
89
117
 
90
118
  ### Features
91
119
 
92
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
93
- * release to production ([531ba89](https://github.com/twinfoundation/data-processing/commit/531ba89dea2deb2810870b72851dca3081f9d2ce))
94
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
95
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
120
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
121
+ * release to production ([531ba89](https://github.com/iotaledger/twin-data-processing/commit/531ba89dea2deb2810870b72851dca3081f9d2ce))
122
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
123
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
96
124
 
97
125
 
98
126
  ### Dependencies
@@ -101,7 +129,7 @@
101
129
  * dependencies
102
130
  * @twin.org/data-processing-models bumped from ^0.0.0 to ^0.0.1
103
131
 
104
- ## [0.0.1-next.16](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.15...data-processing-extractors-v0.0.1-next.16) (2025-06-20)
132
+ ## [0.0.1-next.16](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.15...data-processing-extractors-v0.0.1-next.16) (2025-06-20)
105
133
 
106
134
 
107
135
  ### Miscellaneous Chores
@@ -115,12 +143,12 @@
115
143
  * dependencies
116
144
  * @twin.org/data-processing-models bumped from 0.0.1-next.15 to 0.0.1-next.16
117
145
 
118
- ## [0.0.1-next.15](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.14...data-processing-extractors-v0.0.1-next.15) (2025-06-12)
146
+ ## [0.0.1-next.15](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.14...data-processing-extractors-v0.0.1-next.15) (2025-06-12)
119
147
 
120
148
 
121
149
  ### Features
122
150
 
123
- * update dependencies ([21b8e80](https://github.com/twinfoundation/data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
151
+ * update dependencies ([21b8e80](https://github.com/iotaledger/twin-data-processing/commit/21b8e8007c87136a09f0a8e35ffde13a07ff4711))
124
152
 
125
153
 
126
154
  ### Dependencies
@@ -129,7 +157,7 @@
129
157
  * dependencies
130
158
  * @twin.org/data-processing-models bumped from 0.0.1-next.14 to 0.0.1-next.15
131
159
 
132
- ## [0.0.1-next.14](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.13...data-processing-extractors-v0.0.1-next.14) (2025-04-28)
160
+ ## [0.0.1-next.14](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.13...data-processing-extractors-v0.0.1-next.14) (2025-04-28)
133
161
 
134
162
 
135
163
  ### Miscellaneous Chores
@@ -143,7 +171,7 @@
143
171
  * dependencies
144
172
  * @twin.org/data-processing-models bumped from 0.0.1-next.13 to 0.0.1-next.14
145
173
 
146
- ## [0.0.1-next.13](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.12...data-processing-extractors-v0.0.1-next.13) (2025-04-24)
174
+ ## [0.0.1-next.13](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.12...data-processing-extractors-v0.0.1-next.13) (2025-04-24)
147
175
 
148
176
 
149
177
  ### Miscellaneous Chores
@@ -157,12 +185,12 @@
157
185
  * dependencies
158
186
  * @twin.org/data-processing-models bumped from 0.0.1-next.12 to 0.0.1-next.13
159
187
 
160
- ## [0.0.1-next.12](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.11...data-processing-extractors-v0.0.1-next.12) (2025-04-17)
188
+ ## [0.0.1-next.12](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.11...data-processing-extractors-v0.0.1-next.12) (2025-04-17)
161
189
 
162
190
 
163
191
  ### Features
164
192
 
165
- * use shared store mechanism ([#6](https://github.com/twinfoundation/data-processing/issues/6)) ([6009228](https://github.com/twinfoundation/data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
193
+ * use shared store mechanism ([#6](https://github.com/iotaledger/twin-data-processing/issues/6)) ([6009228](https://github.com/iotaledger/twin-data-processing/commit/600922880acef07cc2f818dee7645c342929108b))
166
194
 
167
195
 
168
196
  ### Dependencies
@@ -171,12 +199,12 @@
171
199
  * dependencies
172
200
  * @twin.org/data-processing-models bumped from 0.0.1-next.11 to 0.0.1-next.12
173
201
 
174
- ## [0.0.1-next.11](https://github.com/twinfoundation/data-processing/compare/data-processing-extractors-v0.0.1-next.10...data-processing-extractors-v0.0.1-next.11) (2025-03-28)
202
+ ## [0.0.1-next.11](https://github.com/iotaledger/twin-data-processing/compare/data-processing-extractors-v0.0.1-next.10...data-processing-extractors-v0.0.1-next.11) (2025-03-28)
175
203
 
176
204
 
177
205
  ### Features
178
206
 
179
- * Add rule group query functionality ([#2](https://github.com/twinfoundation/data-processing/issues/2)) ([0fbbfb0](https://github.com/twinfoundation/data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
207
+ * Add rule group query functionality ([#2](https://github.com/iotaledger/twin-data-processing/issues/2)) ([0fbbfb0](https://github.com/iotaledger/twin-data-processing/commit/0fbbfb065b6ecc293920b25f97ba011743105486))
180
208
 
181
209
 
182
210
  ### Dependencies
package/docs/examples.md CHANGED
@@ -1 +1,69 @@
1
- # @twin.org/data-processing-extractors - Examples
1
+ # Data Processing Extractors Examples
2
+
3
+ Use these snippets to map nested JSON data into flatter, application-focused shapes with JSONPath rules.
4
+
5
+ ## JsonPathExtractorConnector
6
+
7
+ ```typescript
8
+ import { JsonPathExtractorConnector } from '@twin.org/data-processing-extractors';
9
+
10
+ const extractor = new JsonPathExtractorConnector();
11
+
12
+ console.log(extractor.className()); // jsonPathExtractorConnector
13
+
14
+ const sourceData = {
15
+ users: [
16
+ { firstName: 'John', lastName: 'Doe', age: 30 },
17
+ { firstName: 'Jane', lastName: 'Doe', age: 25 }
18
+ ]
19
+ };
20
+
21
+ const extracted = await extractor.extract(sourceData, [
22
+ { source: '$.users[0].firstName', target: 'profile.givenName' },
23
+ { source: '$.users[1].lastName', target: 'profile.familyName' }
24
+ ]);
25
+
26
+ console.log(extracted); // { profile: { givenName: 'John', familyName: 'Doe' } }
27
+ ```
28
+
29
+ ```typescript
30
+ import { JsonPathExtractorConnector } from '@twin.org/data-processing-extractors';
31
+
32
+ const extractor = new JsonPathExtractorConnector();
33
+
34
+ const extracted = await extractor.extract(
35
+ {
36
+ store: {
37
+ books: [
38
+ { title: 'The Hobbit', author: 'J.R.R. Tolkien' },
39
+ { title: '1984', author: 'George Orwell' },
40
+ { title: 'Dune', author: 'Frank Herbert' }
41
+ ]
42
+ }
43
+ },
44
+ [{ source: '$.store.books[*].title', target: 'bookTitles' }]
45
+ );
46
+
47
+ console.log(extracted); // { bookTitles: ['The Hobbit', '1984', 'Dune'] }
48
+ ```
49
+
50
+ ```typescript
51
+ import { JsonPathExtractorConnector } from '@twin.org/data-processing-extractors';
52
+
53
+ const extractor = new JsonPathExtractorConnector();
54
+
55
+ const extracted = await extractor.extract(
56
+ {
57
+ users: {
58
+ user1: { firstName: 'John', lastName: 'Doe', age: 30 },
59
+ user2: { firstName: 'Jane', lastName: 'Doe', age: 25 }
60
+ }
61
+ },
62
+ [
63
+ { source: '$.users.*.firstName', target: 'selected', retainPathDepth: 2 },
64
+ { source: '$.users.*.lastName', target: 'selected', retainPathDepth: 2 }
65
+ ]
66
+ );
67
+
68
+ console.log(extracted); // { selected: { user1: { firstName: 'John', lastName: 'Doe' }, user2: { firstName: 'Jane', lastName: 'Doe' } } }
69
+ ```
package/docs/reference/classes/JsonPathExtractorConnector.md CHANGED
@@ -18,7 +18,7 @@ Class for extracting data from a JSON source.
18
18
 
19
19
  ## Properties
20
20
 
21
- ### NAMESPACE
21
+ ### NAMESPACE {#namespace}
22
22
 
23
23
  > `readonly` `static` **NAMESPACE**: `string` = `"json-path"`
24
24
 
@@ -26,7 +26,7 @@ The namespace supported by the data extractor connector.
26
26
 
27
27
  ***
28
28
 
29
- ### CLASS\_NAME
29
+ ### CLASS\_NAME {#class_name}
30
30
 
31
31
  > `readonly` `static` **CLASS\_NAME**: `string`
32
32
 
@@ -34,7 +34,7 @@ Runtime name for the class.
34
34
 
35
35
  ## Methods
36
36
 
37
- ### className()
37
+ ### className() {#classname}
38
38
 
39
39
  > **className**(): `string`
40
40
 
@@ -52,7 +52,7 @@ The class name of the component.
52
52
 
53
53
  ***
54
54
 
55
- ### extract()
55
+ ### extract() {#extract}
56
56
 
57
57
  > **extract**(`data`, `rules`): `Promise`\<`unknown`\>
58
58
 
package/package.json CHANGED
@@ -1,10 +1,10 @@
1
1
  {
2
2
  "name": "@twin.org/data-processing-extractors",
3
- "version": "0.0.3-next.1",
4
- "description": "Data extractor connector implementations",
3
+ "version": "0.0.3-next.3",
4
+ "description": "Connector implementations that extract data from external systems for processing",
5
5
  "repository": {
6
6
  "type": "git",
7
- "url": "git+https://github.com/twinfoundation/data-processing.git",
7
+ "url": "git+https://github.com/iotaledger/data-processing.git",
8
8
  "directory": "packages/data-processing-extractors"
9
9
  },
10
10
  "author": "martyn.janes@iota.org",
@@ -16,7 +16,7 @@
16
16
  "dependencies": {
17
17
  "@twin.org/core": "next",
18
18
  "@twin.org/data-json-path": "next",
19
- "@twin.org/data-processing-models": "0.0.3-next.1",
19
+ "@twin.org/data-processing-models": "0.0.3-next.3",
20
20
  "@twin.org/nameof": "next",
21
21
  "@twin.org/web": "next"
22
22
  },
@@ -49,7 +49,7 @@
49
49
  "conversion"
50
50
  ],
51
51
  "bugs": {
52
- "url": "git+https://github.com/twinfoundation/data-processing/issues"
52
+ "url": "git+https://github.com/iotaledger/data-processing/issues"
53
53
  },
54
54
  "homepage": "https://twindev.org"
55
55
  }