@amazon/vinyl-xml 1.1.0 → 1.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,214 @@
1
+ # @amazon/vinyl-xml
2
+
3
+ [![Website](https://img.shields.io/badge/website-amazonmusic.github.io%2Fvinyl-blue)](https://amazonmusic.github.io/vinyl)
4
+ [![npm](https://img.shields.io/npm/v/@amazon/vinyl-xml.svg)](https://www.npmjs.com/package/@amazon/vinyl-xml)
5
+ [![size](https://img.shields.io/bundlejs/size/@amazon/vinyl-xml.svg?label=size)](https://bundlejs.com/?q=@amazon/vinyl-xml)
6
+
7
+ A fast SAX XML parser that produces a type-safe object validated against an
8
+ XSD-derived rule set. Designed for parsing large schema-heavy documents like
9
+ DASH manifests with strong static typing and exact-source error reporting.
10
+
11
+ ## Install
12
+
13
+ ```shell
14
+ npm install @amazon/vinyl-xml
15
+ ```
16
+
17
+ ## Overview
18
+
19
+ To create a parser for a schema, first create the type definitions for the shape
20
+ of the document.
21
+
22
+ There are many ways to do this. For the DASH 2011 schema we used
23
+ [CXSD](https://github.com/charto/cxsd) (MIT Licensed); take the type definitions
24
+ and discard the generated parser.
25
+
26
+ Using the type definitions we build a rule set that follows the XSD. This is a
27
+ manual process; follow the XSD closely to match `min`/`max` occurrences,
28
+ defaults, and required attributes.
29
+
30
+ `MPDType` example, following the
31
+ [MPDType definition](https://github.com/Dash-Industry-Forum/MPEG-Conformance-and-reference-source/blob/master/conformance/MPDValidator/schemas/DASH-MPD.xsd):
32
+
33
+ ```typescript
34
+ const mpdType: XmlRules<MPDtype> = {
35
+ ProgramInformation: elements(programInformationType),
36
+ BaseURL: elements(baseURLType),
37
+ Location: elements({ _content: charactersString }),
38
+ Period: elements(periodType, { minOccurs: 1 }),
39
+ Metrics: elements(metricsType),
40
+ EssentialProperty: elements(descriptorType),
41
+ SupplementalProperty: elements(descriptorType),
42
+ UTCTiming: elements(descriptorType),
43
+ availabilityEndTime: attrDateTime,
44
+ availabilityStartTime: attrDateTime,
45
+ id: attrString,
46
+ maxSegmentDuration: attr(parseDuration, stringifyDuration),
47
+ maxSubsegmentDuration: attr(parseDuration, stringifyDuration),
48
+ mediaPresentationDuration: attr(parseDuration, stringifyDuration),
49
+ minBufferTime: attr(parseDuration, stringifyDuration, { required: true }),
50
+ minimumUpdatePeriod: attr(parseDuration, stringifyDuration),
51
+ profiles: attr(parseProfiles, stringifyProfiles, { required: true }),
52
+ publishTime: attrDateTime,
53
+ suggestedPresentationDelay: attr(parseDuration, stringifyDuration),
54
+ timeShiftBufferDepth: attr(parseDuration, stringifyDuration),
55
+ type: attr(parsePresentationTypeType, stringify, {
56
+ default: 'static',
57
+ } as const),
58
+ } as const
59
+ ```
60
+
61
+ _Style note — abbreviations should be cased as they are in the XSD; this follows
62
+ the exception noted in the
63
+ [TS style guide](https://google.github.io/styleguide/tsguide.html#identifiers-abbreviations)
64
+ we follow._
65
+
66
+ `XmlRules` definitions are meant to be human-readable and easy for the
67
+ TypeScript compiler to validate that the rule matches the type definition.
68
+ Before this rule set can be used by the parser or stringifier, it must be mapped
69
+ to a more performant machine-friendly version.
70
+
71
+ Map the rules using `mapXmlRules`:
72
+
73
+ ```typescript
74
+ const dashDrmRulesMapped = mapXmlRules(dashDrmRules)
75
+ ```
76
+
77
+ `XmlRules` maps XML content to a typed object using four rule types:
78
+
79
+ - `attr` — attributes
80
+ - `element` — elements with a max occurrence of 1
81
+ - `elements` — elements as an array
82
+ - `characters` — text nodes (concatenated)
83
+
84
+ Attribute and characters handlers provide parse and stringify functions to
85
+ serialize/deserialize the value from the XML.
86
+
87
+ From the rule set, create an XML handler for the SAX parser:
88
+
89
+ ```typescript
90
+ const dashManifestHandler = new ParseXmlHandlerImpl(dashDrmRulesMapped)
91
+ ```
92
+
93
+ With the handler we can parse XML into a validated, type-safe object that
94
+ matches our type definition:
95
+
96
+ ```typescript
97
+ export function parseDashManifest(manifest: string): DashDrmManifest {
98
+ return parseXml(manifest, dashManifestHandler)
99
+ }
100
+ ```
101
+
102
+ > **Concurrency note:** A handler is stateful while parsing a document, so if
103
+ > multiple workers parse documents simultaneously, create one handler per worker
104
+ > thread.
105
+
106
+ The overall process is as follows: `parseXml` is given an XML string and a
107
+ content handler. A content handler abstractly provides methods for opening and
108
+ closing XML nodes. Based on an XSD schema, we can build a content handler
109
+ `XmlRules` that knows how each node should be processed.
110
+
111
+ At a low level, `parseXml` uses `StringParser` to read XML content. As elements
112
+ are opened and closed and attributes or text nodes are read, the corresponding
113
+ content handler method is invoked. Validating against the schema rules as the
114
+ document is read gives the added benefit of knowing the exact XML source
115
+ location of an invalid document.
116
+
117
+ ## Namespaces
118
+
119
+ Namespaces are not optional; if an XSD schema uses a namespace, the namespace
120
+ must be defined in the rules.
121
+
122
+ ```typescript
123
+ export const dashNamespaceUri = 'urn:mpeg:dash:schema:mpd:2011'
124
+
125
+ export const dashManifestXmlRules: XmlRules<DashManifest> = {
126
+ MPD: element(mpdType, {
127
+ required: true,
128
+ namespaceUri: dashNamespaceUri,
129
+ }),
130
+ } as const
131
+ ```
132
+
133
+ Namespaces follow correct scoping and defaulting rules — see the
134
+ [W3C spec](https://www.w3.org/TR/REC-xml-names/).
135
+
136
+ For `XmlRules`, namespaces inherit the parent rule's namespace unless
137
+ overridden. When using `stringifyXml`, defining namespace prefixes is supported
138
+ but optional. If a prefix isn't specified, one is generated when the default
139
+ namespace isn't used.
140
+
141
+ ## Merging Schemas
142
+
143
+ In many cases, an XML document has multiple XSD definitions. These can be merged
144
+ using `mergeXmlRules`:
145
+
146
+ ```typescript
147
+ const dashDrmRules = mergeXmlRules(
148
+ dashManifestXmlRules,
149
+ createDashProtectionXmlRules(cencXmlRules),
150
+ createDashProtectionXmlRules(playreadyXmlRules)
151
+ )
152
+ ```
153
+
154
+ `mergeXmlRules` ensures the combined schema definitions are compatible. When the
155
+ rules represent separate XSD schemas with different namespace URIs, there
156
+ generally isn't overlap.
157
+
158
+ ## Stringify
159
+
160
+ `XmlRules` provides both deserializers and serializers, so the same mapped rule
161
+ set can be used in both the content handler for `parseXml` and `stringifyXml`:
162
+
163
+ ```typescript
164
+ stringifyXml(xmlString, dashDrmRulesMapped)
165
+ ```
166
+
167
+ ### Element Ordering
168
+
169
+ In an XML schema, order matters for `xs:sequence` definitions. Amazon Vinyl
170
+ `XmlRules` maps define properties in this order. Order doesn't matter for
171
+ parsing, but it does for serialization to pass schema validation. This means
172
+ serialization relies on the ES6 requirement that object keys are ordered by
173
+ insertion. Many ES5 browsers also follow this rule, but `stringifyXml` will not
174
+ be to spec for browsers that don't. XML serialization is not required for
175
+ playback — only deserialization.
176
+
177
+ ## Limitations
178
+
179
+ The parser currently ignores processing instructions and supports only the
180
+ default standard XML entities.
181
+
182
+ ## Parent References
183
+
184
+ Parsed XML nodes have a reference to their parent elements in the non-enumerable
185
+ `parent` attribute. References are correctly reassigned when using the `clone`
186
+ utility and are not included by `JSON.stringify`. Type definitions may include a
187
+ `parent` property; the type isn't enforced by the rule sets and is expected to
188
+ correspond to the type of parent element, or simply `any`.
189
+
190
+ ## Name Conflicts
191
+
192
+ In certain edge cases, an XSD may describe element or attribute rules that
193
+ cannot be mapped 1:1 with property names. Cases include:
194
+
195
+ - The child element or attribute name is an ES6 reserved word such as `public`,
196
+ `private`, `get`, `set`, etc.
197
+ - The name is an internally reserved word: `parent` or `clone`.
198
+ - A child element local name matches an attribute name. In these cases, choose a
199
+ different property name and provide a `localPart` value to the rule
200
+ configuration.
201
+
202
+ ```typescript
203
+ import { attrString } from '@amazon/vinyl-xml'
204
+
205
+ type MyType = { _parent: string }
206
+
207
+ export const myXmlRules: XmlRules<MyType> = {
208
+ _parent: attrString({ localPart: 'parent' }),
209
+ } as const
210
+ ```
211
+
212
+ ## License
213
+
214
+ Apache-2.0
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@amazon/vinyl-xml",
3
- "version": "1.1.0",
3
+ "version": "1.1.1",
4
4
  "description": "A SAX XML parser",
5
5
  "repository": {
6
6
  "type": "git",
@@ -69,15 +69,15 @@
69
69
  "watch:release": "tsx buildSrc/build.ts --watch --release"
70
70
  },
71
71
  "devDependencies": {
72
- "@amazon/vinyl-build-utils": "^1.1.0",
73
- "@amazon/vinyl-jasmine-wrapper": "^1.1.0"
72
+ "@amazon/vinyl-build-utils": "^1.1.1",
73
+ "@amazon/vinyl-jasmine-wrapper": "^1.1.1"
74
74
  },
75
75
  "dependencies": {
76
- "@amazon/vinyl-util": "^1.1.0",
77
- "@amazon/vinyl-validation": "^1.1.0"
76
+ "@amazon/vinyl-util": "^1.1.1",
77
+ "@amazon/vinyl-validation": "^1.1.1"
78
78
  },
79
79
  "browserslist": [
80
80
  "defaults, chrome 52, firefox 54, edge 15, safari 8"
81
81
  ],
82
- "gitHead": "ea13070fb52c1903723f08f31c356465d2f3279e"
82
+ "gitHead": "e5a3429d802564132b16808a5b8a8e15a023a548"
83
83
  }
@@ -1,210 +0,0 @@
1
- # XML Parsing
2
-
3
- Amazon Vinyl provides a fast way to parse XML documents into a type-safe object
4
- that is validated against an XSD schema.
5
-
6
- To create a parser for a schema, first create the Type definitions for the shape
7
- of the document.
8
-
9
- There are many ways to do this, for the Dash 2011 schema we used
10
- [CXSD](https://github.com/charto/cxsd) (MIT Licensed). Take the type
11
- definitions, and discard their generated parser.
12
-
13
- Using the type definitions we can build a ruleset following the XSD. This is a
14
- manual process; follow the XSD closely to match min/max occurrences, defaults,
15
- and required attributes.
16
-
17
- MPDType example, following the
18
- [MPDType definition](https://github.com/Dash-Industry-Forum/MPEG-Conformance-and-reference-source/blob/master/conformance/MPDValidator/schemas/DASH-MPD.xsd):
19
-
20
- ```typescript
21
- const mpdType: XmlRules<MPDtype> = {
22
- ProgramInformation: elements(programInformationType),
23
- BaseURL: elements(baseURLType),
24
- Location: elements({ _content: charactersString }),
25
- Period: elements(periodType, { minOccurs: 1 }),
26
- Metrics: elements(metricsType),
27
- EssentialProperty: elements(descriptorType),
28
- SupplementalProperty: elements(descriptorType),
29
- UTCTiming: elements(descriptorType),
30
- availabilityEndTime: attrDateTime,
31
- availabilityStartTime: attrDateTime,
32
- id: attrString,
33
- maxSegmentDuration: attr(parseDuration, stringifyDuration),
34
- maxSubsegmentDuration: attr(parseDuration, stringifyDuration),
35
- mediaPresentationDuration: attr(parseDuration, stringifyDuration),
36
- minBufferTime: attr(parseDuration, stringifyDuration, { required: true }),
37
- minimumUpdatePeriod: attr(parseDuration, stringifyDuration),
38
- profiles: attr(parseProfiles, stringifyProfiles, { required: true }),
39
- publishTime: attrDateTime,
40
- suggestedPresentationDelay: attr(parseDuration, stringifyDuration),
41
- timeShiftBufferDepth: attr(parseDuration, stringifyDuration),
42
- type: attr(parsePresentationTypeType, stringify, {
43
- default: 'static',
44
- } as const),
45
- } as const
46
- ```
47
-
48
- _Style note - abbreviations should be cased as they are in the XSD, this follows
49
- the exception noted in the
50
- [TS style guide](https://google.github.io/styleguide/tsguide.html#identifiers-abbreviations)
51
- we follow._
52
-
53
- `XmlRules` definitions are meant to be human-readable and easy for the
54
- TypeScript compiler to validate that the rule matches the type definition.
55
- Before this rule set can be used by the parser or stringifier, it must be mapped
56
- to a more performant machine-friendly version.
57
-
58
- Map the rules using `mapXmlRules`:
59
-
60
- ```typescript
61
- const dashDrmRulesMapped = mapXmlRules(dashDrmRules)
62
- ```
63
-
64
- XmlRules maps XML content to a typed object using the following four rules:
65
-
66
- - `attr` Attributes
67
- - `element` Elements with a max occurrence of 1.
68
- - `elements` Elements, as an array.
69
- - `characters` Text nodes (concatenated).
70
-
71
- Attribute and characters handlers provide parse and stringify functions to
72
- serialize/deserialize the value from the XML.
73
-
74
- From the ruleset, we can create an XML handler for the SAX parser:
75
-
76
- ```typescript
77
- const dashManifestHandler = new ParseXmlHandlerImpl(dashDrmRulesMapped)
78
- ```
79
-
80
- _Concurrency note: A handler is stateful while parsing a document, so if there
81
- are multiple workers parsing documents simultaneously, a handler per worker
82
- thread must be created._
83
-
84
- With the handler now we can parse XML into a validated type-safe object matching
85
- our type definition.
86
-
87
- ```typescript
88
- export function parseDashManifest(manifest: string): DashDrmManifest {
89
- return parseXml(manifest, dashManifestHandler)
90
- }
91
- ```
92
-
93
- The overall process is as follows:
94
-
95
- `parseXml` is given an XML string, and a content handler. A content handler
96
- abstractly provides methods for opening and closing XML nodes. Based on an XSD
97
- schema, we can build a content handler `XmlRules` that knows how each node
98
- should be processed.
99
-
100
- At a low level, `parseXml` uses `StringParser` to read XML content. As elements
101
- are opened and closed, and attributes or text nodes are read, the corresponding
102
- content handler method is invoked. Validating against the schema rules as the
103
- document is read gives the added benefit of knowing the exact XML source
104
- location of the invalid document.
105
-
106
- ## Namespaces
107
-
108
- Namespaces are not optional, if an XSD schema uses a namespace, the namespace
109
- must be defined in the rules.
110
-
111
- For example:
112
-
113
- ```typescript
114
- export const dashNamespaceUri = 'urn:mpeg:dash:schema:mpd:2011'
115
-
116
- export const dashManifestXmlRules: XmlRules<DashManifest> = {
117
- MPD: element(mpdType, {
118
- required: true,
119
- namespaceUri: dashNamespaceUri,
120
- }),
121
- } as const
122
- ```
123
-
124
- Namespaces follow correct scoping and defaulting rules.
125
-
126
- https://www.w3.org/TR/REC-xml-names/
127
-
128
- For XmlRules, namespaces inherit that of the parent rule unless overridden. When
129
- using `stringifyXml`, defining namespace prefixes is supported, but optional. If
130
- a prefix isn't specified, one will be generated if the default namespace isn't
131
- used.
132
-
133
- ## Merging Schemas
134
-
135
- In many cases, an XML document has multiple XSD definitions.
136
-
137
- These can be merged using `mergeXmlRules`.
138
-
139
- Example:
140
-
141
- ```typescript
142
- const dashDrmRules = mergeXmlRules(
143
- dashManifestXmlRules,
144
- createDashProtectionXmlRules(cencXmlRules),
145
- createDashProtectionXmlRules(playreadyXmlRules)
146
- )
147
- ```
148
-
149
- The `mergeXmlRules` utility ensures that the combined schema definitions are
150
- compatible. When the rules represent separate XSD schemas with different
151
- namespace uris, there generally isn't overlap.
152
-
153
- ## Stringify
154
-
155
- `XmlRules` provides both deserializers and serializers, so the same mapped
156
- ruleset can be used in both the content handler for `parseXml` and
157
- `stringifyXml`.
158
-
159
- ```typescript
160
- stringifyXml(xmlString, dashDrmRulesMapped)
161
- ```
162
-
163
- ### Element Ordering
164
-
165
- In an XML Schema, order matters for `xs:sequence` definitions. Amazon Vinyl
166
- `XmlRules` maps define properties in this order. This doesn't matter for
167
- parsing, but it does for serialization to pass schema. This means that
168
- serialization relies on the ES6 requirement that object keys are ordered by
169
- insertion. Many ES5 browsers also follow this rule, but this means that
170
- `xmlStringify` will not be to spec for browsers not following this requirement.
171
- XML serialization is not required for playback, only deserialization.
172
-
173
- ## Limitations
174
-
175
- The parser currently ignores processing instructions and supports only the
176
- default standard XML entities. This isn't a necessary feature for our purposes.
177
-
178
- ## Parent References
179
-
180
- Parsed XML nodes have a reference to their parent elements in the assigned
181
- attribute `parent`. These references are non-enumerable and will be correctly
182
- reassigned when using the `clone` utility. They will not be included if
183
- serialized using `JSON.stringify`. Type definitions may include a `parent`
184
- definition. The type is not enforced by the rule-sets and is expected to
185
- correspond to the type of parent to which the element may belong, or it may
186
- simply be typed as `any`.
187
-
188
- ## Name Conflicts
189
-
190
- In certain edge cases, an XSD may describe element or attribute rules that
191
- cannot be mapped 1:1 with property names. Cases include:
192
-
193
- - If the child element or attribute name is an ES6 reserved word such as
194
- `public`, `private`, `get`, `set`, etc.
195
- - If the name is an internally used reserved word: `parent` or `clone`.
196
- - If a child element local name matches an attribute name. In these cases, when
197
- defining an XML rule, choose a different property name, and provide a
198
- 'localPart' value to the rule configuration.
199
-
200
- For example:
201
-
202
- ```typescript
203
- import { attrString } from '@amazon/vinyl-xml'
204
-
205
- type MyType = { _parent: string }
206
-
207
- export const myXmlRules: XmlRules<MyType> = {
208
- _parent: attrString({ localPart: 'parent' }),
209
- } as const
210
- ```