sanitize 2.1.1 → 6.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of sanitize might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/HISTORY.md +520 -55
- data/LICENSE +1 -1
- data/README.md +438 -168
- data/lib/sanitize/config/basic.rb +12 -32
- data/lib/sanitize/config/default.rb +118 -0
- data/lib/sanitize/config/relaxed.rb +716 -53
- data/lib/sanitize/config/restricted.rb +3 -23
- data/lib/sanitize/config.rb +53 -79
- data/lib/sanitize/css.rb +348 -0
- data/lib/sanitize/transformers/clean_cdata.rb +3 -3
- data/lib/sanitize/transformers/clean_comment.rb +6 -3
- data/lib/sanitize/transformers/clean_css.rb +57 -0
- data/lib/sanitize/transformers/clean_doctype.rb +19 -0
- data/lib/sanitize/transformers/clean_element.rb +192 -124
- data/lib/sanitize/version.rb +3 -1
- data/lib/sanitize.rb +172 -143
- data/test/common.rb +3 -0
- data/test/test_clean_comment.rb +47 -0
- data/test/test_clean_css.rb +67 -0
- data/test/test_clean_doctype.rb +71 -0
- data/test/test_clean_element.rb +545 -0
- data/test/test_config.rb +65 -0
- data/test/test_malicious_css.rb +42 -0
- data/test/test_malicious_html.rb +235 -0
- data/test/test_parser.rb +75 -0
- data/test/test_sanitize.rb +151 -675
- data/test/test_sanitize_css.rb +424 -0
- data/test/test_transformers.rb +230 -0
- metadata +44 -41
data/README.md
CHANGED
@@ -1,20 +1,37 @@
|
|
1
1
|
Sanitize
|
2
2
|
========
|
3
3
|
|
4
|
-
Sanitize is
|
5
|
-
|
6
|
-
|
4
|
+
Sanitize is an allowlist-based HTML and CSS sanitizer. It removes all HTML
|
5
|
+
and/or CSS from a string except the elements, attributes, and properties you
|
6
|
+
choose to allow.
|
7
7
|
|
8
|
-
Using a simple configuration syntax, you can tell Sanitize to allow certain
|
8
|
+
Using a simple configuration syntax, you can tell Sanitize to allow certain HTML
|
9
9
|
elements, certain attributes within those elements, and even certain URL
|
10
|
-
protocols within attributes that contain URLs.
|
11
|
-
|
10
|
+
protocols within attributes that contain URLs. You can also allow specific CSS
|
11
|
+
properties, @ rules, and URL protocols in elements or attributes containing CSS.
|
12
|
+
Any HTML or CSS that you don't explicitly allow will be removed.
|
12
13
|
|
13
|
-
|
14
|
-
|
15
|
-
|
14
|
+
Sanitize is based on the [Nokogumbo HTML5 parser][nokogumbo], which parses HTML
|
15
|
+
exactly the same way modern browsers do, and [Crass][crass], which parses CSS
|
16
|
+
exactly the same way modern browsers do. As long as your allowlist config only
|
17
|
+
allows safe markup and CSS, even the most malformed or malicious input will be
|
18
|
+
transformed into safe output.
|
16
19
|
|
17
|
-
[](http://badge.fury.io/rb/sanitize)
|
21
|
+
[](https://github.com/rgrove/sanitize/actions?query=workflow%3ATests)
|
22
|
+
|
23
|
+
[crass]:https://github.com/rgrove/crass
|
24
|
+
[nokogumbo]:https://github.com/rubys/nokogumbo
|
25
|
+
|
26
|
+
Links
|
27
|
+
-----
|
28
|
+
|
29
|
+
* [Home](https://github.com/rgrove/sanitize/)
|
30
|
+
* [API Docs](http://rubydoc.info/github/rgrove/sanitize/master)
|
31
|
+
* [Issues](https://github.com/rgrove/sanitize/issues)
|
32
|
+
* [Release History](https://github.com/rgrove/sanitize/blob/master/HISTORY.md#sanitize-history)
|
33
|
+
* [Online Demo](https://sanitize.herokuapp.com/)
|
34
|
+
* [Biased comparison of Ruby HTML sanitization libraries](https://github.com/rgrove/sanitize/blob/master/COMPARISON.md)
|
18
35
|
|
19
36
|
Installation
|
20
37
|
-------------
|
@@ -23,63 +40,206 @@ Installation
|
|
23
40
|
gem install sanitize
|
24
41
|
```
|
25
42
|
|
43
|
+
Quick Start
|
44
|
+
-----------
|
45
|
+
|
46
|
+
```ruby
|
47
|
+
require 'sanitize'
|
48
|
+
|
49
|
+
# Clean up an HTML fragment using Sanitize's permissive but safe Relaxed config.
|
50
|
+
# This also sanitizes any CSS in `<style>` elements or `style` attributes.
|
51
|
+
Sanitize.fragment(html, Sanitize::Config::RELAXED)
|
52
|
+
|
53
|
+
# Clean up an HTML document using the Relaxed config.
|
54
|
+
Sanitize.document(html, Sanitize::Config::RELAXED)
|
55
|
+
|
56
|
+
# Clean up a standalone CSS stylesheet using the Relaxed config.
|
57
|
+
Sanitize::CSS.stylesheet(css, Sanitize::Config::RELAXED)
|
58
|
+
|
59
|
+
# Clean up some CSS properties using the Relaxed config.
|
60
|
+
Sanitize::CSS.properties(css, Sanitize::Config::RELAXED)
|
61
|
+
```
|
62
|
+
|
26
63
|
Usage
|
27
64
|
-----
|
28
65
|
|
66
|
+
Sanitize can sanitize the following types of input:
|
67
|
+
|
68
|
+
* HTML fragments
|
69
|
+
* HTML documents
|
70
|
+
* CSS stylesheets inside HTML `<style>` elements
|
71
|
+
* CSS properties inside HTML `style` attributes
|
72
|
+
* Standalone CSS stylesheets
|
73
|
+
* Standalone CSS properties
|
74
|
+
|
75
|
+
However, please note that Sanitize _cannot_ fully sanitize the contents of
|
76
|
+
`<math>` or `<svg>` elements, since these elements don't follow the same parsing
|
77
|
+
rules as the rest of HTML. If this is something you need, you may want to look
|
78
|
+
for another solution.
|
79
|
+
|
80
|
+
### HTML Fragments
|
81
|
+
|
82
|
+
A fragment is a snippet of HTML that doesn't contain a root-level `<html>`
|
83
|
+
element.
|
84
|
+
|
29
85
|
If you don't specify any configuration options, Sanitize will use its strictest
|
30
|
-
settings by default, which means it will strip all HTML and leave only text
|
86
|
+
settings by default, which means it will strip all HTML and leave only safe text
|
31
87
|
behind.
|
32
88
|
|
33
89
|
```ruby
|
34
|
-
|
35
|
-
|
90
|
+
html = '<b><a href="http://foo.com/">foo</a></b><img src="bar.jpg">'
|
91
|
+
Sanitize.fragment(html)
|
92
|
+
# => 'foo'
|
93
|
+
```
|
94
|
+
|
95
|
+
To keep certain elements, add them to the element allowlist.
|
96
|
+
|
97
|
+
```ruby
|
98
|
+
Sanitize.fragment(html, :elements => ['b'])
|
99
|
+
# => '<b>foo</b>'
|
100
|
+
```
|
36
101
|
|
37
|
-
|
102
|
+
### HTML Documents
|
38
103
|
|
39
|
-
|
104
|
+
When sanitizing a document, the `<html>` element must be allowlisted. You can
|
105
|
+
also set `:allow_doctype` to `true` to allow well-formed document type
|
106
|
+
definitions.
|
107
|
+
|
108
|
+
```ruby
|
109
|
+
html = %[
|
110
|
+
<!DOCTYPE html>
|
111
|
+
<html>
|
112
|
+
<b><a href="http://foo.com/">foo</a></b><img src="bar.jpg">
|
113
|
+
</html>
|
114
|
+
]
|
115
|
+
|
116
|
+
Sanitize.document(html,
|
117
|
+
:allow_doctype => true,
|
118
|
+
:elements => ['html']
|
119
|
+
)
|
120
|
+
# => %[
|
121
|
+
# <!DOCTYPE html>
|
122
|
+
# <html>foo
|
123
|
+
#
|
124
|
+
# </html>
|
125
|
+
# ]
|
126
|
+
```
|
127
|
+
|
128
|
+
### CSS in HTML
|
129
|
+
|
130
|
+
To sanitize CSS in an HTML fragment or document, first allowlist the `<style>`
|
131
|
+
element and/or the `style` attribute. Then allowlist the CSS properties,
|
132
|
+
@ rules, and URL protocols you wish to allow. You can also choose whether to
|
133
|
+
allow CSS comments or browser compatibility hacks.
|
134
|
+
|
135
|
+
```ruby
|
136
|
+
html = %[
|
137
|
+
<style>
|
138
|
+
div { color: green; width: 1024px; }
|
139
|
+
</style>
|
140
|
+
|
141
|
+
<div style="height: 100px; width: 100px;"></div>
|
142
|
+
<p>hello!</p>
|
143
|
+
]
|
144
|
+
|
145
|
+
Sanitize.fragment(html,
|
146
|
+
:elements => ['div', 'style'],
|
147
|
+
:attributes => {'div' => ['style']},
|
148
|
+
|
149
|
+
:css => {
|
150
|
+
:properties => ['width']
|
151
|
+
}
|
152
|
+
)
|
153
|
+
#=> %[
|
154
|
+
# <style>
|
155
|
+
# div { width: 1024px; }
|
156
|
+
# </style>
|
157
|
+
#
|
158
|
+
# <div style=" width: 100px;"></div>
|
159
|
+
# hello!
|
160
|
+
# ]
|
161
|
+
```
|
162
|
+
|
163
|
+
### Standalone CSS
|
164
|
+
|
165
|
+
Sanitize will happily clean up a standalone CSS stylesheet or property string
|
166
|
+
without needing to invoke the HTML parser.
|
167
|
+
|
168
|
+
```ruby
|
169
|
+
css = %[
|
170
|
+
@import url(evil.css);
|
171
|
+
|
172
|
+
a { text-decoration: none; }
|
173
|
+
|
174
|
+
a:hover {
|
175
|
+
left: expression(alert('xss!'));
|
176
|
+
text-decoration: underline;
|
177
|
+
}
|
178
|
+
]
|
179
|
+
|
180
|
+
Sanitize::CSS.stylesheet(css, Sanitize::Config::RELAXED)
|
181
|
+
# => %[
|
182
|
+
#
|
183
|
+
#
|
184
|
+
#
|
185
|
+
# a { text-decoration: none; }
|
186
|
+
#
|
187
|
+
# a:hover {
|
188
|
+
#
|
189
|
+
# text-decoration: underline;
|
190
|
+
# }
|
191
|
+
# ]
|
192
|
+
|
193
|
+
Sanitize::CSS.properties(%[
|
194
|
+
left: expression(alert('xss!'));
|
195
|
+
text-decoration: underline;
|
196
|
+
], Sanitize::Config::RELAXED)
|
197
|
+
# => %[
|
198
|
+
#
|
199
|
+
# text-decoration: underline;
|
200
|
+
# ]
|
40
201
|
|
41
|
-
# or sanitize an entire HTML document (example assumes _html_ is whitelisted)
|
42
|
-
html = '<!DOCTYPE html><html><b><a href="http://foo.com/">foo</a></b><img src="http://foo.com/bar.jpg"></html>'
|
43
|
-
Sanitize.clean_document(html) # => '<!DOCTYPE html>\n<html>foo</html>\n'
|
44
202
|
```
|
45
203
|
|
46
204
|
Configuration
|
47
205
|
-------------
|
48
206
|
|
49
207
|
In addition to the ultra-safe default settings, Sanitize comes with three other
|
50
|
-
built-in
|
208
|
+
built-in configurations that you can use out of the box or adapt to meet your
|
209
|
+
needs.
|
51
210
|
|
52
211
|
### Sanitize::Config::RESTRICTED
|
53
212
|
|
54
|
-
Allows only very simple inline
|
55
|
-
elements.
|
213
|
+
Allows only very simple inline markup. No links, images, or block elements.
|
56
214
|
|
57
215
|
```ruby
|
58
|
-
Sanitize.
|
216
|
+
Sanitize.fragment(html, Sanitize::Config::RESTRICTED)
|
217
|
+
# => '<b>foo</b>'
|
59
218
|
```
|
60
219
|
|
61
220
|
### Sanitize::Config::BASIC
|
62
221
|
|
63
|
-
Allows a variety of markup including formatting
|
64
|
-
|
65
|
-
|
222
|
+
Allows a variety of markup including formatting elements, links, and lists.
|
223
|
+
|
224
|
+
Images and tables are not allowed, links are limited to FTP, HTTP, HTTPS, and
|
225
|
+
mailto protocols, and a `rel="nofollow"` attribute is added to all links to
|
66
226
|
mitigate SEO spam.
|
67
227
|
|
68
228
|
```ruby
|
69
|
-
Sanitize.
|
229
|
+
Sanitize.fragment(html, Sanitize::Config::BASIC)
|
70
230
|
# => '<b><a href="http://foo.com/" rel="nofollow">foo</a></b>'
|
71
231
|
```
|
72
232
|
|
73
233
|
### Sanitize::Config::RELAXED
|
74
234
|
|
75
|
-
Allows an even wider variety of markup
|
76
|
-
Links are still limited to FTP, HTTP, HTTPS, and mailto protocols,
|
77
|
-
are limited to HTTP and HTTPS. In this mode, `rel="nofollow"` is
|
78
|
-
links.
|
235
|
+
Allows an even wider variety of markup, including images and tables, as well as
|
236
|
+
safe CSS. Links are still limited to FTP, HTTP, HTTPS, and mailto protocols,
|
237
|
+
while images are limited to HTTP and HTTPS. In this mode, `rel="nofollow"` is
|
238
|
+
not added to links.
|
79
239
|
|
80
240
|
```ruby
|
81
|
-
Sanitize.
|
82
|
-
# => '<b><a href="http://foo.com/">foo</a></b><img src="
|
241
|
+
Sanitize.fragment(html, Sanitize::Config::RELAXED)
|
242
|
+
# => '<b><a href="http://foo.com/">foo</a></b><img src="bar.jpg">'
|
83
243
|
```
|
84
244
|
|
85
245
|
### Custom Configuration
|
@@ -88,11 +248,51 @@ If the built-in modes don't meet your needs, you can easily specify a custom
|
|
88
248
|
configuration:
|
89
249
|
|
90
250
|
```ruby
|
91
|
-
Sanitize.
|
92
|
-
|
93
|
-
|
251
|
+
Sanitize.fragment(html,
|
252
|
+
:elements => ['a', 'span'],
|
253
|
+
|
254
|
+
:attributes => {
|
255
|
+
'a' => ['href', 'title'],
|
256
|
+
'span' => ['class']
|
257
|
+
},
|
258
|
+
|
259
|
+
:protocols => {
|
260
|
+
'a' => {'href' => ['http', 'https', 'mailto']}
|
261
|
+
}
|
262
|
+
)
|
263
|
+
```
|
264
|
+
|
265
|
+
You can also start with one of Sanitize's built-in configurations and then
|
266
|
+
customize it to meet your needs.
|
267
|
+
|
268
|
+
The built-in configs are deeply frozen to prevent people from modifying them
|
269
|
+
(either accidentally or maliciously). To customize a built-in config, create a
|
270
|
+
new copy using `Sanitize::Config.merge()`, like so:
|
271
|
+
|
272
|
+
```ruby
|
273
|
+
# Create a customized copy of the Basic config, adding <div> and <table> to the
|
274
|
+
# existing allowlisted elements.
|
275
|
+
Sanitize.fragment(html, Sanitize::Config.merge(Sanitize::Config::BASIC,
|
276
|
+
:elements => Sanitize::Config::BASIC[:elements] + ['div', 'table'],
|
277
|
+
:remove_contents => true
|
278
|
+
))
|
94
279
|
```
|
95
280
|
|
281
|
+
The example above adds the `<div>` and `<table>` elements to a copy of the
|
282
|
+
existing list of elements in `Sanitize::Config::BASIC`. If you instead want to
|
283
|
+
completely overwrite the elements array with your own, you can omit the `+`
|
284
|
+
operation:
|
285
|
+
|
286
|
+
```ruby
|
287
|
+
# Overwrite :elements instead of creating a copy with new entries.
|
288
|
+
Sanitize.fragment(html, Sanitize::Config.merge(Sanitize::Config::BASIC,
|
289
|
+
:elements => ['div', 'table'],
|
290
|
+
:remove_contents => true
|
291
|
+
))
|
292
|
+
```
|
293
|
+
|
294
|
+
### Config Settings
|
295
|
+
|
96
296
|
#### :add_attributes (Hash)
|
97
297
|
|
98
298
|
Attributes to add to specific elements. If the attribute already exists, it will
|
@@ -111,9 +311,15 @@ Whether or not to allow HTML comments. Allowing comments is strongly
|
|
111
311
|
discouraged, since IE allows script execution within conditional comments. The
|
112
312
|
default value is `false`.
|
113
313
|
|
314
|
+
#### :allow_doctype (boolean)
|
315
|
+
|
316
|
+
Whether or not to allow well-formed HTML doctype declarations such as "<!DOCTYPE
|
317
|
+
html>" when sanitizing a document. This setting is ignored when sanitizing
|
318
|
+
fragments. The default value is `false`.
|
319
|
+
|
114
320
|
#### :attributes (Hash)
|
115
321
|
|
116
|
-
Attributes to allow
|
322
|
+
Attributes to allow on specific elements. Specify all element names and
|
117
323
|
attributes in lowercase.
|
118
324
|
|
119
325
|
```ruby
|
@@ -124,8 +330,8 @@ attributes in lowercase.
|
|
124
330
|
}
|
125
331
|
```
|
126
332
|
|
127
|
-
If you'd like to allow certain attributes on all elements, use the symbol
|
128
|
-
|
333
|
+
If you'd like to allow certain attributes on all elements, use the symbol `:all`
|
334
|
+
instead of an element name.
|
129
335
|
|
130
336
|
```ruby
|
131
337
|
# Allow the class attribute on all elements.
|
@@ -135,8 +341,8 @@ If you'd like to allow certain attributes on all elements, use the symbol
|
|
135
341
|
}
|
136
342
|
```
|
137
343
|
|
138
|
-
To allow arbitrary HTML5 `data-*` attributes, use the symbol
|
139
|
-
|
344
|
+
To allow arbitrary HTML5 `data-*` attributes, use the symbol `:data` in place of
|
345
|
+
an attribute name.
|
140
346
|
|
141
347
|
```ruby
|
142
348
|
# Allow arbitrary HTML5 data-* attributes on <div> elements.
|
@@ -145,9 +351,67 @@ To allow arbitrary HTML5 `data-*` attributes, use the symbol
|
|
145
351
|
}
|
146
352
|
```
|
147
353
|
|
148
|
-
#### :
|
354
|
+
#### :css (Hash)
|
355
|
+
|
356
|
+
Hash of the following CSS config settings to be used when sanitizing CSS (either
|
357
|
+
standalone or embedded in HTML).
|
358
|
+
|
359
|
+
##### :css => :allow_comments (boolean)
|
360
|
+
|
361
|
+
Whether or not to allow CSS comments. The default value is `false`.
|
362
|
+
|
363
|
+
##### :css => :allow_hacks (boolean)
|
364
|
+
|
365
|
+
Whether or not to allow browser compatibility hacks such as the IE `*` and `_`
|
366
|
+
hacks. These are generally harmless, but technically result in invalid CSS. The
|
367
|
+
default is `false`.
|
368
|
+
|
369
|
+
##### :css => :at_rules (Array or Set)
|
370
|
+
|
371
|
+
Names of CSS [at-rules][at-rules] to allow that may not have associated blocks,
|
372
|
+
such as `import` or `charset`. Names should be specified in lowercase.
|
373
|
+
|
374
|
+
[at-rules]:https://developer.mozilla.org/en-US/docs/Web/CSS/At-rule
|
375
|
+
|
376
|
+
##### :css => :at_rules_with_properties (Array or Set)
|
377
|
+
|
378
|
+
Names of CSS [at-rules][at-rules] to allow that may have associated blocks
|
379
|
+
containing CSS properties. At-rules like `font-face` and `page` fall into this
|
380
|
+
category. Names should be specified in lowercase.
|
381
|
+
|
382
|
+
##### :css => :at_rules_with_styles (Array or Set)
|
383
|
+
|
384
|
+
Names of CSS [at-rules][at-rules] to allow that may have associated blocks
|
385
|
+
containing style rules. At-rules like `media` and `keyframes` fall into this
|
386
|
+
category. Names should be specified in lowercase.
|
387
|
+
|
388
|
+
##### :css => :import_url_validator
|
389
|
+
|
390
|
+
This is a `Proc` (or other callable object) that will be called and passed
|
391
|
+
the URL specified for any `@import` [at-rules][at-rules].
|
149
392
|
|
150
|
-
|
393
|
+
You can use this to limit what can be imported, for example something
|
394
|
+
like the following to limit `@import` to Google Fonts URLs:
|
395
|
+
|
396
|
+
```ruby
|
397
|
+
Proc.new { |url| url.start_with?("https://fonts.googleapis.com") }
|
398
|
+
```
|
399
|
+
|
400
|
+
##### :css => :properties (Array or Set)
|
401
|
+
|
402
|
+
List of CSS property names to allow. Names should be specified in lowercase.
|
403
|
+
|
404
|
+
##### :css => :protocols (Array or Set)
|
405
|
+
|
406
|
+
URL protocols to allow in CSS URLs. Should be specified in lowercase.
|
407
|
+
|
408
|
+
If you'd like to allow the use of relative URLs which don't have a protocol,
|
409
|
+
include the symbol `:relative` in the protocol array.
|
410
|
+
|
411
|
+
#### :elements (Array or Set)
|
412
|
+
|
413
|
+
Array of HTML element names to allow. Specify all names in lowercase. Any
|
414
|
+
elements not in this array will be removed.
|
151
415
|
|
152
416
|
```ruby
|
153
417
|
:elements => %w[
|
@@ -156,14 +420,22 @@ Array of element names to allow. Specify all names in lowercase.
|
|
156
420
|
]
|
157
421
|
```
|
158
422
|
|
159
|
-
|
423
|
+
**Warning:** Sanitize cannot fully sanitize the contents of `<math>` or `<svg>`
|
424
|
+
elements, since these elements don't follow the same parsing rules as the rest
|
425
|
+
of HTML. If you add `math` or `svg` to the allowlist, you must assume that any
|
426
|
+
content inside them will be allowed, even if that content would otherwise be
|
427
|
+
removed by Sanitize.
|
160
428
|
|
161
|
-
|
162
|
-
defaulting to `:html`.
|
429
|
+
#### :parser_options (Hash)
|
163
430
|
|
164
|
-
|
431
|
+
[Parsing options](https://github.com/rubys/nokogumbo/tree/master#parsing-options) to be supplied to `nokogumbo`.
|
165
432
|
|
166
|
-
|
433
|
+
```ruby
|
434
|
+
:parser_options => {
|
435
|
+
max_errors: -1,
|
436
|
+
max_tree_depth: -1
|
437
|
+
}
|
438
|
+
```
|
167
439
|
|
168
440
|
#### :protocols (Hash)
|
169
441
|
|
@@ -187,99 +459,106 @@ include the symbol `:relative` in the protocol array:
|
|
187
459
|
}
|
188
460
|
```
|
189
461
|
|
190
|
-
#### :remove_contents (boolean or Array)
|
462
|
+
#### :remove_contents (boolean or Array or Set)
|
191
463
|
|
192
|
-
If
|
464
|
+
If this is `true`, Sanitize will remove the contents of any non-allowlisted
|
193
465
|
elements in addition to the elements themselves. By default, Sanitize leaves the
|
194
466
|
safe parts of an element's contents behind when the element is removed.
|
195
467
|
|
196
|
-
If
|
197
|
-
elements (when filtered) will be removed, and the contents of all
|
198
|
-
elements will be left behind.
|
468
|
+
If this is an Array or Set of element names, then only the contents of the
|
469
|
+
specified elements (when filtered) will be removed, and the contents of all
|
470
|
+
other filtered elements will be left behind.
|
199
471
|
|
200
|
-
The default value is
|
472
|
+
The default value is `%w[iframe math noembed noframes noscript plaintext script style svg xmp]`.
|
201
473
|
|
202
|
-
#### :transformers
|
474
|
+
#### :transformers (Array or callable)
|
203
475
|
|
204
|
-
Custom transformer or array of custom transformers
|
205
|
-
|
476
|
+
Custom HTML transformer or array of custom transformers. See the Transformers
|
477
|
+
section below for details.
|
206
478
|
|
207
|
-
#### :
|
479
|
+
#### :whitespace_elements (Hash)
|
208
480
|
|
209
|
-
|
210
|
-
|
481
|
+
Hash of element names which, when removed, should have their contents surrounded
|
482
|
+
by whitespace to preserve readability.
|
211
483
|
|
212
|
-
|
484
|
+
Each element name is a key pointing to another Hash, which provides the specific
|
485
|
+
whitespace that should be inserted `:before` and `:after` the removed element's
|
486
|
+
position. The `:after` value will only be inserted if the removed element has
|
487
|
+
children, in which case it will be inserted after those children.
|
213
488
|
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
489
|
+
```ruby
|
490
|
+
:whitespace_elements => {
|
491
|
+
'br' => { :before => "\n", :after => "" },
|
492
|
+
'div' => { :before => "\n", :after => "\n" },
|
493
|
+
'p' => { :before => "\n", :after => "\n" }
|
494
|
+
}
|
495
|
+
```
|
218
496
|
|
219
|
-
|
220
|
-
`:whitespace_elements` array:
|
497
|
+
The default elements with whitespace added before and after are:
|
221
498
|
|
222
499
|
```
|
223
|
-
address article aside blockquote br dd div dl dt
|
224
|
-
h6 header hgroup hr li nav
|
500
|
+
address article aside blockquote br dd div dl dt
|
501
|
+
footer h1 h2 h3 h4 h5 h6 header hgroup hr li nav
|
502
|
+
ol p pre section ul
|
503
|
+
|
225
504
|
```
|
226
505
|
|
227
|
-
|
506
|
+
## Transformers
|
228
507
|
|
229
|
-
Transformers allow you to filter and modify nodes using your own custom
|
230
|
-
on top of (or instead of) Sanitize's core filter. A transformer is any
|
231
|
-
that responds to `call()` (such as a lambda or proc).
|
508
|
+
Transformers allow you to filter and modify HTML nodes using your own custom
|
509
|
+
logic, on top of (or instead of) Sanitize's core filter. A transformer is any
|
510
|
+
object that responds to `call()` (such as a lambda or proc).
|
232
511
|
|
233
|
-
To use one or more transformers, pass them to the `:transformers`
|
234
|
-
|
512
|
+
To use one or more transformers, pass them to the `:transformers` config
|
513
|
+
setting. You may pass a single transformer or an array of transformers.
|
235
514
|
|
236
515
|
```ruby
|
237
|
-
Sanitize.
|
516
|
+
Sanitize.fragment(html, :transformers => [
|
517
|
+
transformer_one,
|
518
|
+
transformer_two
|
519
|
+
])
|
238
520
|
```
|
239
521
|
|
240
|
-
|
522
|
+
### Input
|
241
523
|
|
242
|
-
Each
|
243
|
-
|
244
|
-
|
524
|
+
Each transformer's `call()` method will be called once for each node in the HTML
|
525
|
+
(including elements, text nodes, comments, etc.), and will receive as an
|
526
|
+
argument a Hash that contains the following items:
|
245
527
|
|
246
528
|
* **:config** - The current Sanitize configuration Hash.
|
247
529
|
|
248
|
-
* **:
|
530
|
+
* **:is_allowlisted** - `true` if the current node has been allowlisted by a
|
249
531
|
previous transformer, `false` otherwise. It's generally bad form to remove
|
250
|
-
a node that a previous transformer has
|
532
|
+
a node that a previous transformer has allowlisted.
|
251
533
|
|
252
534
|
* **:node** - A `Nokogiri::XML::Node` object representing an HTML node. The
|
253
535
|
node may be an element, a text node, a comment, a CDATA node, or a document
|
254
536
|
fragment. Use Nokogiri's inspection methods (`element?`, `text?`, etc.) to
|
255
537
|
selectively ignore node types you aren't interested in.
|
256
538
|
|
539
|
+
* **:node_allowlist** - Set of `Nokogiri::XML::Node` objects in the current
|
540
|
+
document that have been allowlisted by previous transformers, if any. It's
|
541
|
+
generally bad form to remove a node that a previous transformer has
|
542
|
+
allowlisted.
|
543
|
+
|
257
544
|
* **:node_name** - The name of the current HTML node, always lowercase (e.g.
|
258
545
|
"div" or "span"). For non-element nodes, the name will be something like
|
259
546
|
"text", "comment", "#cdata-section", "#document-fragment", etc.
|
260
547
|
|
261
|
-
|
262
|
-
document that have been whitelisted by previous transformers, if any. It's
|
263
|
-
generally bad form to remove a node that a previous transformer has
|
264
|
-
whitelisted.
|
265
|
-
|
266
|
-
* **:traversal_mode** - Current node traversal mode, either `:depth` for
|
267
|
-
depth-first (the default mode) or `:breadth` for breadth-first.
|
268
|
-
|
269
|
-
#### Output
|
548
|
+
### Output
|
270
549
|
|
271
550
|
A transformer doesn't have to return anything, but may optionally return a Hash,
|
272
551
|
which may contain the following items:
|
273
552
|
|
274
|
-
* **:
|
275
|
-
to add to the document's
|
276
|
-
These specific nodes and all their attributes will be
|
277
|
-
their children will not be.
|
553
|
+
* **:node_allowlist** - Array or Set of specific `Nokogiri::XML::Node`
|
554
|
+
objects to add to the document's allowlist, bypassing the current Sanitize
|
555
|
+
config. These specific nodes and all their attributes will be allowlisted,
|
556
|
+
but their children will not be.
|
278
557
|
|
279
558
|
If a transformer returns anything other than a Hash, the return value will be
|
280
559
|
ignored.
|
281
560
|
|
282
|
-
|
561
|
+
### Processing
|
283
562
|
|
284
563
|
Each transformer has full access to the `Nokogiri::XML::Node` that's passed into
|
285
564
|
it and to the rest of the document via the node's `document()` method. Any
|
@@ -288,58 +567,85 @@ in the document and passed on to subsequently called transformers and to
|
|
288
567
|
Sanitize itself. A transformer may even call Sanitize internally to perform
|
289
568
|
custom sanitization if needed.
|
290
569
|
|
291
|
-
Nodes are passed into transformers in the order in which they're traversed.
|
292
|
-
|
293
|
-
|
570
|
+
Nodes are passed into transformers in the order in which they're traversed.
|
571
|
+
Sanitize performs top-down traversal, meaning that nodes are traversed in the
|
572
|
+
same order you'd read them in the HTML, starting at the top node, then its first
|
573
|
+
child, and so on.
|
294
574
|
|
295
575
|
```ruby
|
296
|
-
html
|
297
|
-
|
298
|
-
|
299
|
-
|
300
|
-
|
301
|
-
|
302
|
-
|
303
|
-
|
304
|
-
|
576
|
+
html = %[
|
577
|
+
<header>
|
578
|
+
<span>
|
579
|
+
<strong>foo</strong>
|
580
|
+
</span>
|
581
|
+
<p>bar</p>
|
582
|
+
</header>
|
583
|
+
|
584
|
+
<footer></footer>
|
585
|
+
]
|
305
586
|
|
306
|
-
|
307
|
-
|
308
|
-
|
587
|
+
transformer = lambda do |env|
|
588
|
+
puts env[:node_name] if env[:node].element?
|
589
|
+
end
|
309
590
|
|
310
|
-
# Prints "
|
311
|
-
Sanitize.
|
591
|
+
# Prints "header", "span", "strong", "p", "footer".
|
592
|
+
Sanitize.fragment(html, :transformers => transformer)
|
312
593
|
```
|
313
594
|
|
314
595
|
Transformers have a tremendous amount of power, including the power to
|
315
596
|
completely bypass Sanitize's built-in filtering. Be careful! Your safety is in
|
316
597
|
your own hands.
|
317
598
|
|
318
|
-
|
599
|
+
### Example: Transformer to allow image URLs by domain
|
600
|
+
|
601
|
+
The following example demonstrates how to remove image elements unless they use
|
602
|
+
a relative URL or are hosted on a specific domain. It assumes that the `<img>`
|
603
|
+
element and its `src` attribute are already allowlisted.
|
604
|
+
|
605
|
+
```ruby
|
606
|
+
require 'uri'
|
607
|
+
|
608
|
+
image_allowlist_transformer = lambda do |env|
|
609
|
+
# Ignore everything except <img> elements.
|
610
|
+
return unless env[:node_name] == 'img'
|
611
|
+
|
612
|
+
node = env[:node]
|
613
|
+
image_uri = URI.parse(node['src'])
|
614
|
+
|
615
|
+
# Only allow relative URLs or URLs with the example.com domain. The
|
616
|
+
# image_uri.host.nil? check ensures that protocol-relative URLs like
|
617
|
+
# "//evil.com/foo.jpg".
|
618
|
+
unless image_uri.host == 'example.com' || (image_uri.host.nil? && image_uri.relative?)
|
619
|
+
node.unlink # `Nokogiri::XML::Node#unlink` removes a node from the document
|
620
|
+
end
|
621
|
+
end
|
622
|
+
```
|
623
|
+
|
624
|
+
### Example: Transformer to allow YouTube video embeds
|
319
625
|
|
320
|
-
The following example demonstrates how to create a
|
321
|
-
|
322
|
-
|
323
|
-
|
626
|
+
The following example demonstrates how to create a transformer that will safely
|
627
|
+
allow valid YouTube video embeds without having to allow other kinds of embedded
|
628
|
+
content, which would be the case if you tried to do this by just allowing all
|
629
|
+
`<iframe>` elements:
|
324
630
|
|
325
631
|
```ruby
|
326
|
-
lambda do |env|
|
632
|
+
youtube_transformer = lambda do |env|
|
327
633
|
node = env[:node]
|
328
634
|
node_name = env[:node_name]
|
329
635
|
|
330
|
-
# Don't continue if this node is already
|
331
|
-
return if env[:
|
636
|
+
# Don't continue if this node is already allowlisted or is not an element.
|
637
|
+
return if env[:is_allowlisted] || !node.element?
|
332
638
|
|
333
639
|
# Don't continue unless the node is an iframe.
|
334
640
|
return unless node_name == 'iframe'
|
335
641
|
|
336
642
|
# Verify that the video URL is actually a valid YouTube video URL.
|
337
|
-
return unless node['src'] =~
|
643
|
+
return unless node['src'] =~ %r|\A(?:https?:)?//(?:www\.)?youtube(?:-nocookie)?\.com/|
|
338
644
|
|
339
645
|
# We're now certain that this is a YouTube embed, but we still need to run
|
340
646
|
# it through a special Sanitize step to ensure that no unwanted elements or
|
341
647
|
# attributes that don't belong in a YouTube embed can sneak in.
|
342
|
-
Sanitize.
|
648
|
+
Sanitize.node!(node, {
|
343
649
|
:elements => %w[iframe],
|
344
650
|
|
345
651
|
:attributes => {
|
@@ -349,51 +655,15 @@ lambda do |env|
|
|
349
655
|
|
350
656
|
# Now that we're sure that this is a valid YouTube embed and that there are
|
351
657
|
# no unwanted elements or attributes hidden inside it, we can tell Sanitize
|
352
|
-
# to
|
353
|
-
{:
|
658
|
+
# to allowlist the current node.
|
659
|
+
{:node_allowlist => [node]}
|
354
660
|
end
|
355
|
-
```
|
356
661
|
|
357
|
-
|
358
|
-
|
359
|
-
|
360
|
-
|
361
|
-
|
362
|
-
|
363
|
-
|
364
|
-
|
365
|
-
* Wilson Bilkovich
|
366
|
-
* Peter Cooper
|
367
|
-
* Gabe da Silveira
|
368
|
-
* Nicholas Evans
|
369
|
-
* Nils Gemeinhardt
|
370
|
-
* Adam Hooper
|
371
|
-
* Mutwin Kraus
|
372
|
-
* Eaden McKee
|
373
|
-
* Dev Purkayastha
|
374
|
-
* David Reese
|
375
|
-
* Ardie Saeidi
|
376
|
-
* Rafael Souza
|
377
|
-
* Ben Wanicur
|
378
|
-
|
379
|
-
License
|
380
|
-
-------
|
381
|
-
|
382
|
-
Copyright (c) 2014 Ryan Grove (ryan@wonko.com)
|
383
|
-
|
384
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
385
|
-
this software and associated documentation files (the 'Software'), to deal in
|
386
|
-
the Software without restriction, including without limitation the rights to
|
387
|
-
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
388
|
-
the Software, and to permit persons to whom the Software is furnished to do so,
|
389
|
-
subject to the following conditions:
|
390
|
-
|
391
|
-
The above copyright notice and this permission notice shall be included in all
|
392
|
-
copies or substantial portions of the Software.
|
393
|
-
|
394
|
-
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
395
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
396
|
-
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
397
|
-
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
398
|
-
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
399
|
-
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
662
|
+
html = %[
|
663
|
+
<iframe width="420" height="315" src="//www.youtube.com/embed/dQw4w9WgXcQ"
|
664
|
+
frameborder="0" allowfullscreen></iframe>
|
665
|
+
]
|
666
|
+
|
667
|
+
Sanitize.fragment(html, :transformers => youtube_transformer)
|
668
|
+
# => '<iframe width="420" height="315" src="//www.youtube.com/embed/dQw4w9WgXcQ" frameborder="0" allowfullscreen=""></iframe>'
|
669
|
+
```
|