html-pipeline 2.14.3 → 3.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/FUNDING.yml +11 -3
- data/.github/dependabot.yml +27 -0
- data/.github/workflows/automerge.yml +13 -0
- data/.github/workflows/ci.yml +22 -0
- data/.github/workflows/lint.yml +23 -0
- data/.github/workflows/publish.yml +19 -0
- data/.rubocop.yml +17 -0
- data/.ruby-version +1 -0
- data/.vscode/settings.json +8 -0
- data/CHANGELOG.md +119 -2
- data/Gemfile +31 -15
- data/{LICENSE → LICENSE.txt} +2 -2
- data/README.md +241 -224
- data/Rakefile +14 -7
- data/UPGRADING.md +34 -0
- data/html-pipeline.gemspec +31 -21
- data/lib/html-pipeline.rb +3 -0
- data/lib/html_pipeline/convert_filter/markdown_filter.rb +26 -0
- data/lib/html_pipeline/convert_filter.rb +17 -0
- data/lib/html_pipeline/filter.rb +89 -0
- data/lib/html_pipeline/node_filter/absolute_source_filter.rb +54 -0
- data/lib/html_pipeline/node_filter/asset_proxy_filter.rb +86 -0
- data/lib/{html/pipeline → html_pipeline/node_filter}/emoji_filter.rb +58 -54
- data/lib/html_pipeline/node_filter/https_filter.rb +22 -0
- data/lib/html_pipeline/node_filter/image_max_width_filter.rb +40 -0
- data/lib/{html/pipeline/@mention_filter.rb → html_pipeline/node_filter/mention_filter.rb} +54 -68
- data/lib/html_pipeline/node_filter/syntax_highlight_filter.rb +62 -0
- data/lib/html_pipeline/node_filter/table_of_contents_filter.rb +70 -0
- data/lib/html_pipeline/node_filter/team_mention_filter.rb +105 -0
- data/lib/html_pipeline/node_filter.rb +31 -0
- data/lib/html_pipeline/sanitization_filter.rb +188 -0
- data/lib/{html/pipeline → html_pipeline/text_filter}/image_filter.rb +3 -3
- data/lib/{html/pipeline → html_pipeline/text_filter}/plain_text_input_filter.rb +3 -5
- data/lib/html_pipeline/text_filter.rb +21 -0
- data/lib/html_pipeline/version.rb +5 -0
- data/lib/html_pipeline.rb +281 -0
- metadata +58 -54
- data/.travis.yml +0 -43
- data/Appraisals +0 -19
- data/CONTRIBUTING.md +0 -60
- data/bin/html-pipeline +0 -78
- data/lib/html/pipeline/@team_mention_filter.rb +0 -99
- data/lib/html/pipeline/absolute_source_filter.rb +0 -52
- data/lib/html/pipeline/autolink_filter.rb +0 -34
- data/lib/html/pipeline/body_content.rb +0 -44
- data/lib/html/pipeline/camo_filter.rb +0 -105
- data/lib/html/pipeline/email_reply_filter.rb +0 -69
- data/lib/html/pipeline/filter.rb +0 -165
- data/lib/html/pipeline/https_filter.rb +0 -29
- data/lib/html/pipeline/image_max_width_filter.rb +0 -37
- data/lib/html/pipeline/markdown_filter.rb +0 -56
- data/lib/html/pipeline/sanitization_filter.rb +0 -144
- data/lib/html/pipeline/syntax_highlight_filter.rb +0 -50
- data/lib/html/pipeline/text_filter.rb +0 -16
- data/lib/html/pipeline/textile_filter.rb +0 -25
- data/lib/html/pipeline/toc_filter.rb +0 -69
- data/lib/html/pipeline/version.rb +0 -7
- data/lib/html/pipeline.rb +0 -210
data/README.md
CHANGED
@@ -1,23 +1,31 @@
|
|
1
|
-
# HTML
|
1
|
+
# HTML-Pipeline
|
2
2
|
|
3
|
-
|
4
|
-
|
3
|
+
> **Note**
|
4
|
+
> This README refers to the behavior in the new 3.0.0.pre gem.
|
5
|
+
|
6
|
+
HTML processing filters and utilities. This module is a small
|
7
|
+
framework for defining CSS-based content filters and applying them to user
|
5
8
|
provided content.
|
6
9
|
|
7
|
-
[
|
8
|
-
|
9
|
-
- [
|
10
|
-
- [
|
11
|
-
- [
|
12
|
-
- [
|
13
|
-
- [
|
14
|
-
- [
|
15
|
-
- [
|
16
|
-
|
17
|
-
- [
|
18
|
-
- [
|
19
|
-
- [
|
20
|
-
- [
|
10
|
+
[Although this project was started at GitHub](https://github.com/blog/1311-html-pipeline-chainable-content-filters), they no longer use it. This gem must be considered standalone and independent from GitHub.
|
11
|
+
|
12
|
+
- [HTML-Pipeline](#html-pipeline)
|
13
|
+
- [Installation](#installation)
|
14
|
+
- [Usage](#usage)
|
15
|
+
- [More Examples](#more-examples)
|
16
|
+
- [Filters](#filters)
|
17
|
+
- [TextFilters](#textfilters)
|
18
|
+
- [ConvertFilter](#convertfilter)
|
19
|
+
- [Sanitization](#sanitization)
|
20
|
+
- [NodeFilters](#nodefilters)
|
21
|
+
- [Dependencies](#dependencies)
|
22
|
+
- [Documentation](#documentation)
|
23
|
+
- [Instrumenting](#instrumenting)
|
24
|
+
- [Third Party Extensions](#third-party-extensions)
|
25
|
+
- [FAQ](#faq)
|
26
|
+
- [1. Why doesn't my pipeline work when there's no root element in the document?](#1-why-doesnt-my-pipeline-work-when-theres-no-root-element-in-the-document)
|
27
|
+
- [2. How do I customize an allowlist for `SanitizationFilter`s?](#2-how-do-i-customize-an-allowlist-for-sanitizationfilters)
|
28
|
+
- [Contributors](#contributors)
|
21
29
|
|
22
30
|
## Installation
|
23
31
|
|
@@ -42,220 +50,234 @@ $ gem install html-pipeline
|
|
42
50
|
## Usage
|
43
51
|
|
44
52
|
This library provides a handful of chainable HTML filters to transform user
|
45
|
-
content into markup.
|
46
|
-
|
47
|
-
outputs the result.
|
53
|
+
content into HTML markup. Each filter does some work, and then hands off the
|
54
|
+
results tothe next filter. A pipeline has several kinds of filters available to use:
|
48
55
|
|
49
|
-
|
56
|
+
- Multiple `TextFilter`s, which operate a UTF-8 string
|
57
|
+
- A `ConvertFilter` filter, which turns text into HTML (eg., Commonmark/Asciidoc -> HTML)
|
58
|
+
- A `SanitizationFilter`, which remove dangerous/unwanted HTML elements and attributes
|
59
|
+
- Multiple `NodeFilter`s, which operate on a UTF-8 HTML document
|
50
60
|
|
51
|
-
|
52
|
-
require 'html/pipeline'
|
61
|
+
You can assemble each sequence into a single pipeline, or choose to call each filter individually.
|
53
62
|
|
54
|
-
|
55
|
-
filter.call
|
56
|
-
```
|
63
|
+
As an example, suppose we want to transform Commonmark source text into Markdown HTML. With the content, we also want to:
|
57
64
|
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
65
|
+
- change every instance of `$NAME` to "`Johnny"
|
66
|
+
- strip undesired HTML
|
67
|
+
- linkify @mention
|
68
|
+
|
69
|
+
We can construct a pipeline to do all that like this:
|
62
70
|
|
63
71
|
```ruby
|
64
|
-
|
65
|
-
HTML::Pipeline::MarkdownFilter,
|
66
|
-
HTML::Pipeline::SyntaxHighlightFilter
|
67
|
-
]
|
68
|
-
result = pipeline.call <<-CODE
|
69
|
-
This is *great*:
|
72
|
+
require 'html_pipeline'
|
70
73
|
|
71
|
-
|
74
|
+
class HelloJohnnyFilter < HTMLPipelineFilter
|
75
|
+
def call
|
76
|
+
text.gsub("$NAME", "Johnny")
|
77
|
+
end
|
78
|
+
end
|
72
79
|
|
73
|
-
|
74
|
-
|
80
|
+
pipeline = HTMLPipeline.new(
|
81
|
+
text_filters: [HelloJohnnyFilter.new]
|
82
|
+
convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
|
83
|
+
# note: next line is not needed as sanitization occurs by default;
|
84
|
+
# see below for more info
|
85
|
+
sanitization_config: HTMLPipeline::SanitizationFilter::DEFAULT_CONFIG,
|
86
|
+
node_filters: [HTMLPipeline::NodeFilter::MentionFilter.new]
|
87
|
+
)
|
88
|
+
pipeline.call(user_supplied_text) # recommended: can call pipeline over and over
|
75
89
|
```
|
76
90
|
|
77
|
-
|
91
|
+
Filters can be custom ones you create (like `HelloJohnnyFilter`), and `HTMLPipeline` additionally provides several helpful ones (detailed below). If you only need a single filter, you can call one individually, too:
|
78
92
|
|
79
|
-
```
|
80
|
-
|
81
|
-
|
82
|
-
<pre><code>some_code(:first)
|
83
|
-
</code></pre>
|
93
|
+
```ruby
|
94
|
+
filter = HTMLPipeline::ConvertFilter::MarkdownFilter.new
|
95
|
+
filter.call(text)
|
84
96
|
```
|
85
97
|
|
86
|
-
|
98
|
+
Filters combine into a sequential pipeline, and each filter hands its
|
99
|
+
output to the next filter's input. Text filters are
|
100
|
+
processed first, then the convert filter, sanitization filter, and finally, the node filters.
|
87
101
|
|
88
|
-
Some filters take
|
102
|
+
Some filters take optional `context` and/or `result` hash(es). These are
|
89
103
|
used to pass around arguments and metadata between filters in a pipeline. For
|
90
|
-
example, if you
|
91
|
-
option in the context hash:
|
104
|
+
example, if you want to disable footnotes in the `MarkdownFilter`, you can pass an option in the context hash:
|
92
105
|
|
93
106
|
```ruby
|
94
|
-
|
95
|
-
filter.
|
107
|
+
context = { markdown: { extensions: { footnotes: false } } }
|
108
|
+
filter = HTMLPipeline::ConvertFilter::MarkdownFilter.new(context: context)
|
109
|
+
filter.call("Hi **world**!")
|
96
110
|
```
|
97
111
|
|
98
|
-
|
112
|
+
Please refer to the documentation for each filter to understand what configuration options are available.
|
113
|
+
|
114
|
+
### More Examples
|
99
115
|
|
100
|
-
|
116
|
+
Different pipelines can be defined for different parts of an app. Here are a few
|
101
117
|
paraphrased snippets to get you started:
|
102
118
|
|
103
119
|
```ruby
|
104
120
|
# The context hash is how you pass options between different filters.
|
105
121
|
# See individual filter source for explanation of options.
|
106
122
|
context = {
|
107
|
-
:
|
108
|
-
:
|
123
|
+
asset_root: "http://your-domain.com/where/your/images/live/icons",
|
124
|
+
base_url: "http://your-domain.com"
|
109
125
|
}
|
110
126
|
|
111
|
-
# Pipeline providing sanitization and image hijacking but no mention
|
112
|
-
# related features.
|
113
|
-
SimplePipeline = Pipeline.new [
|
114
|
-
SanitizationFilter,
|
115
|
-
TableOfContentsFilter, # add 'name' anchors to all headers and generate toc list
|
116
|
-
CamoFilter,
|
117
|
-
ImageMaxWidthFilter,
|
118
|
-
SyntaxHighlightFilter,
|
119
|
-
EmojiFilter,
|
120
|
-
AutolinkFilter
|
121
|
-
], context
|
122
|
-
|
123
127
|
# Pipeline used for user provided content on the web
|
124
|
-
MarkdownPipeline =
|
125
|
-
|
126
|
-
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
MentionFilter,
|
131
|
-
EmojiFilter,
|
132
|
-
SyntaxHighlightFilter
|
133
|
-
], context.merge(:gfm => true) # enable github formatted markdown
|
134
|
-
|
135
|
-
|
136
|
-
# Define a pipeline based on another pipeline's filters
|
137
|
-
NonGFMMarkdownPipeline = Pipeline.new(MarkdownPipeline.filters,
|
138
|
-
context.merge(:gfm => false))
|
128
|
+
MarkdownPipeline = HTMLPipeline.new (
|
129
|
+
text_filters: [HTMLPipeline::TextFilter::ImageFilter.new],
|
130
|
+
convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
|
131
|
+
node_filters: [
|
132
|
+
HTMLPipeline::NodeFilter::HttpsFilter.new,HTMLPipeline::NodeFilter::MentionFilter.new,
|
133
|
+
], context: context)
|
139
134
|
|
140
135
|
# Pipelines aren't limited to the web. You can use them for email
|
141
136
|
# processing also.
|
142
|
-
HtmlEmailPipeline =
|
143
|
-
|
144
|
-
|
145
|
-
|
146
|
-
|
147
|
-
# Just emoji.
|
148
|
-
EmojiPipeline = Pipeline.new [
|
149
|
-
PlainTextInputFilter,
|
150
|
-
EmojiFilter
|
151
|
-
], context
|
137
|
+
HtmlEmailPipeline = HTMLPipeline.new(
|
138
|
+
text_filters: [
|
139
|
+
PlainTextInputFilter.new,
|
140
|
+
ImageFilter.new
|
141
|
+
], {})
|
152
142
|
```
|
153
143
|
|
154
144
|
## Filters
|
155
145
|
|
156
|
-
|
157
|
-
* `TeamMentionFilter` - replace `@org/team` mentions with links
|
158
|
-
* `AbsoluteSourceFilter` - replace relative image urls with fully qualified versions
|
159
|
-
* `AutolinkFilter` - auto_linking urls in HTML
|
160
|
-
* `CamoFilter` - replace http image urls with [camo-fied](https://github.com/atmos/camo) https versions
|
161
|
-
* `EmailReplyFilter` - util filter for working with emails
|
162
|
-
* `EmojiFilter` - everyone loves [emoji](http://www.emoji-cheat-sheet.com/)!
|
163
|
-
* `HttpsFilter` - HTML Filter for replacing http github urls with https versions.
|
164
|
-
* `ImageMaxWidthFilter` - link to full size image for large images
|
165
|
-
* `MarkdownFilter` - convert markdown to html
|
166
|
-
* `PlainTextInputFilter` - html escape text and wrap the result in a div
|
167
|
-
* `SanitizationFilter` - allow sanitize user markup
|
168
|
-
* `SyntaxHighlightFilter` - code syntax highlighter
|
169
|
-
* `TextileFilter` - convert textile to html
|
170
|
-
* `TableOfContentsFilter` - anchor headings with name attributes and generate Table of Contents html unordered list linking headings
|
146
|
+
### TextFilters
|
171
147
|
|
172
|
-
|
148
|
+
`TextFilter`s must define a method named `call` which is called on the text. `@text`, `@config`, and `@result` are available to use, and any changes made to these ivars are passed on to the next filter.
|
149
|
+
|
150
|
+
- `ImageFilter` - converts image `url` into `<img>` tag
|
151
|
+
- `PlainTextInputFilter` - html escape text and wrap the result in a `<div>`
|
152
|
+
|
153
|
+
### ConvertFilter
|
154
|
+
|
155
|
+
The `ConvertFilter` takes text and turns it into HTML. `@text`, `@config`, and `@result` are available to use. `ConvertFilter` must defined a method named `call`, taking one argument, `text`. `call` must return a string representing the new HTML document.
|
173
156
|
|
174
|
-
|
175
|
-
|
176
|
-
|
177
|
-
|
178
|
-
|
157
|
+
- `MarkdownFilter` - creates HTML from text using [Commonmarker](https://www.github.com/gjtorikian/commonmarker)
|
158
|
+
|
159
|
+
### Sanitization
|
160
|
+
|
161
|
+
Because the web can be a scary place, HTML is automatically sanitized after the `ConvertFilter` runs and before the `NodeFilter`s are processed. This is to prevent malicious or unexpected input from entering the pipeline.
|
162
|
+
|
163
|
+
The sanitization process takes a hash configuration of settings. See the [Selma](https://www.github.com/gjtorikian/selma) documentation for more information on how to configure these settings.
|
164
|
+
|
165
|
+
A default sanitization config is provided by this library (`HTMLPipeline::SanitizationFilter::DEFAULT_CONFIG`). A sample custom sanitization allowlist might look like this:
|
179
166
|
|
180
167
|
```ruby
|
181
|
-
|
168
|
+
ALLOWLIST = {
|
169
|
+
elements: ["p", "pre", "code"]
|
170
|
+
}
|
171
|
+
|
172
|
+
pipeline = HTMLPipeline.new \
|
173
|
+
text_filters: [
|
174
|
+
HTMLPipeline::MarkdownFilter,
|
175
|
+
],
|
176
|
+
convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
|
177
|
+
sanitization_config: ALLOWLIST
|
178
|
+
|
179
|
+
result = pipeline.call <<-CODE
|
180
|
+
This is *great*:
|
181
|
+
|
182
|
+
some_code(:first)
|
183
|
+
|
184
|
+
CODE
|
185
|
+
result[:output].to_s
|
182
186
|
```
|
183
187
|
|
184
|
-
|
185
|
-
* `EmailReplyFilter` - `escape_utils`, `email_reply_parser`
|
186
|
-
* `EmojiFilter` - `gemoji`
|
187
|
-
* `MarkdownFilter` - `commonmarker`
|
188
|
-
* `PlainTextInputFilter` - `escape_utils`
|
189
|
-
* `SanitizationFilter` - `sanitize`
|
190
|
-
* `SyntaxHighlightFilter` - `rouge`
|
191
|
-
* `TableOfContentsFilter` - `escape_utils`
|
192
|
-
* `TextileFilter` - `RedCloth`
|
188
|
+
This would print:
|
193
189
|
|
194
|
-
|
190
|
+
```html
|
191
|
+
<p>This is great:</p>
|
192
|
+
<pre><code>some_code(:first)
|
193
|
+
</code></pre>
|
194
|
+
```
|
195
195
|
|
196
|
-
|
196
|
+
Sanitization can be disabled if and only if `nil` is explicitly passed as
|
197
|
+
the config:
|
197
198
|
|
198
|
-
|
199
|
+
```ruby
|
200
|
+
pipeline = HTMLPipeline.new \
|
201
|
+
text_filters: [
|
202
|
+
HTMLPipeline::MarkdownFilter,
|
203
|
+
],
|
204
|
+
convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
|
205
|
+
sanitization_config: nil
|
206
|
+
```
|
207
|
+
|
208
|
+
For more examples of customizing the sanitization process to include the tags you want, check out [the tests](test/sanitization_filter_test.rb) and [the FAQ](#faq).
|
199
209
|
|
200
|
-
|
201
|
-
To write a custom filter, you need a class with a `call` method that inherits
|
202
|
-
from `HTML::Pipeline::Filter`.
|
210
|
+
### NodeFilters
|
203
211
|
|
204
|
-
|
212
|
+
`NodeFilters`s can operate either on HTML elements or text nodes using CSS selectors. Each `NodeFilter` must define a method named `selector` which provides an instance of `Selma::Selector`. If elements are being manipulated, `handle_element` must be defined, taking one argument, `element`; if text nodes are being manipulated, `handle_text_chunk` must be defined, taking one argument, `text_chunk`. `@config`, and `@result` are available to use, and any changes made to these ivars are passed on to the next filter.
|
213
|
+
|
214
|
+
`NodeFilter` also has an optional method, `after_initialize`, which is run after the filter initializes. This can be useful in setting up a custom state for `result` to take advantage of.
|
215
|
+
|
216
|
+
Here's an example `NodeFilter` that adds a base url to images that are root relative:
|
205
217
|
|
206
218
|
```ruby
|
207
219
|
require 'uri'
|
208
220
|
|
209
|
-
class RootRelativeFilter <
|
221
|
+
class RootRelativeFilter < HTMLPipeline::NodeFilter
|
210
222
|
|
211
|
-
|
212
|
-
|
213
|
-
|
214
|
-
|
215
|
-
if src.start_with? '/'
|
216
|
-
img["src"] = URI.join(context[:base_url], src).to_s
|
217
|
-
end
|
218
|
-
end
|
219
|
-
doc
|
223
|
+
SELECTOR = Selma::Selector.new(match_element: "img")
|
224
|
+
|
225
|
+
def selector
|
226
|
+
SELECTOR
|
220
227
|
end
|
221
228
|
|
229
|
+
def handle_element(img)
|
230
|
+
next if img['src'].nil?
|
231
|
+
src = img['src'].strip
|
232
|
+
if src.start_with? '/'
|
233
|
+
img["src"] = URI.join(context[:base_url], src).to_s
|
234
|
+
end
|
235
|
+
end
|
222
236
|
end
|
223
237
|
```
|
224
238
|
|
225
|
-
|
239
|
+
For more information on how to write effective `NodeFilter`s, refer to the provided filters, and see the underlying lib, [Selma](https://www.github.com/gjtorikian/selma) for more information.
|
240
|
+
|
241
|
+
- `AbsoluteSourceFilter`: replace relative image urls with fully qualified versions
|
242
|
+
- `AssetProxyFilter`: replace image links with an encoded link to an asset server
|
243
|
+
- `EmojiFilter`: converts `:<emoji>:` to [emoji](http://www.emoji-cheat-sheet.com/)
|
244
|
+
- (Note: the included `MarkdownFilter` will already convert emoji)
|
245
|
+
- `HttpsFilter`: Replacing http urls with https versions
|
246
|
+
- `ImageMaxWidthFilter`: link to full size image for large images
|
247
|
+
- `MentionFilter`: replace `@user` mentions with links
|
248
|
+
- `SanitizationFilter`: allow sanitize user markup
|
249
|
+
- `SyntaxHighlightFilter`: applies syntax highlighting to `pre` blocks
|
250
|
+
- (Note: the included `MarkdownFilter` will already apply highlighting)
|
251
|
+
- `TableOfContentsFilter`: anchor headings with name attributes and generate Table of Contents html unordered list linking headings
|
252
|
+
- `TeamMentionFilter`: replace `@org/team` mentions with links
|
253
|
+
|
254
|
+
## Dependencies
|
255
|
+
|
256
|
+
Since filters can be customized to your heart's content, gem dependencies are _not_ bundled; this project doesn't know which of the default filters you might use, and as such, you must bundle each filter's gem dependencies yourself.
|
257
|
+
|
258
|
+
For example, `SyntaxHighlightFilter` uses [rouge](https://github.com/jneen/rouge)
|
259
|
+
to detect and highlight languages; to use the `SyntaxHighlightFilter`, you must add the following to your Gemfile:
|
226
260
|
|
227
261
|
```ruby
|
228
|
-
|
262
|
+
gem "rouge"
|
229
263
|
```
|
230
264
|
|
231
|
-
|
265
|
+
> **Note**
|
266
|
+
> See the [Gemfile](/Gemfile) `:test` group for any version requirements.
|
232
267
|
|
233
|
-
|
234
|
-
[an issue](https://github.com/jch/html-pipeline/issues) first. This allows us discuss
|
235
|
-
whether the filter is a common enough use case to belong in this gem, or should be
|
236
|
-
built as an external gem.
|
268
|
+
When developing a custom filter, call `HTMLPipeline.require_dependency` at the start to ensure that the local machine has the necessary dependency. You can also use `HTMLPipeline.require_dependencies` to provide a list of dependencies to check.
|
237
269
|
|
238
|
-
|
270
|
+
On a similar note, you must manually require whichever filters you desire:
|
239
271
|
|
240
|
-
|
241
|
-
|
242
|
-
|
243
|
-
|
244
|
-
|
245
|
-
* [tilt-html-pipeline](https://github.com/bradgessler/tilt-html-pipeline)
|
246
|
-
* [html-pipeline-wiki-link'](https://github.com/lifted-studios/html-pipeline-wiki-link) - WikiMedia-style wiki links
|
247
|
-
* [task_list](https://github.com/github/task_list) - GitHub flavor Markdown Task List
|
248
|
-
* [html-pipeline-nico_link](https://github.com/rutan/html-pipeline-nico_link) - An HTML::Pipeline filter for [niconico](http://www.nicovideo.jp) description links
|
249
|
-
* [html-pipeline-gitlab](https://gitlab.com/gitlab-org/html-pipeline-gitlab) - This gem implements various filters for html-pipeline used by GitLab
|
250
|
-
* [html-pipeline-youtube](https://github.com/st0012/html-pipeline-youtube) - An HTML::Pipeline filter for YouTube links
|
251
|
-
* [html-pipeline-flickr](https://github.com/st0012/html-pipeline-flickr) - An HTML::Pipeline filter for Flickr links
|
252
|
-
* [html-pipeline-vimeo](https://github.com/dlackty/html-pipeline-vimeo) - An HTML::Pipeline filter for Vimeo links
|
253
|
-
* [html-pipeline-hashtag](https://github.com/mr-dxdy/html-pipeline-hashtag) - An HTML::Pipeline filter for hashtags
|
254
|
-
* [html-pipeline-linkify_github](https://github.com/jollygoodcode/html-pipeline-linkify_github) - An HTML::Pipeline filter to autolink GitHub urls
|
255
|
-
* [html-pipeline-redcarpet_filter](https://github.com/bmikol/html-pipeline-redcarpet_filter) - Render Markdown source text into Markdown HTML using Redcarpet
|
256
|
-
* [html-pipeline-typogruby_filter](https://github.com/bmikol/html-pipeline-typogruby_filter) - Add Typogruby text filters to your HTML::Pipeline
|
257
|
-
* [korgi](https://github.com/jodeci/korgi) - HTML::Pipeline filters for links to Rails resources
|
272
|
+
```ruby
|
273
|
+
require "html_pipeline" # must be included
|
274
|
+
require "html_pipeline/convert_filter/markdown_filter" # included because you want to use this filter
|
275
|
+
require "html_pipeline/node_filter/mention_filter" # included because you want to use this filter
|
276
|
+
```
|
258
277
|
|
278
|
+
## Documentation
|
279
|
+
|
280
|
+
Full reference documentation can be [found here](http://rubydoc.info/gems/html-pipeline/frames).
|
259
281
|
|
260
282
|
## Instrumenting
|
261
283
|
|
@@ -263,107 +285,102 @@ Filters and Pipelines can be set up to be instrumented when called. The pipeline
|
|
263
285
|
must be setup with an
|
264
286
|
[ActiveSupport::Notifications](http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html)
|
265
287
|
compatible service object and a name. New pipeline objects will default to the
|
266
|
-
`
|
288
|
+
`HTMLPipeline.default_instrumentation_service` object.
|
267
289
|
|
268
|
-
```
|
290
|
+
```ruby
|
269
291
|
# the AS::Notifications-compatible service object
|
270
292
|
service = ActiveSupport::Notifications
|
271
293
|
|
272
294
|
# instrument a specific pipeline
|
273
|
-
pipeline =
|
295
|
+
pipeline = HTMLPipeline.new [MarkdownFilter], context
|
274
296
|
pipeline.setup_instrumentation "MarkdownPipeline", service
|
275
297
|
|
276
298
|
# or set default instrumentation service for all new pipelines
|
277
|
-
|
278
|
-
pipeline =
|
299
|
+
HTMLPipeline.default_instrumentation_service = service
|
300
|
+
pipeline = HTMLPipeline.new [MarkdownFilter], context
|
279
301
|
pipeline.setup_instrumentation "MarkdownPipeline"
|
280
302
|
```
|
281
303
|
|
282
304
|
Filters are instrumented when they are run through the pipeline. A
|
283
|
-
`call_filter.html_pipeline` event is published once
|
284
|
-
`
|
305
|
+
`call_filter.html_pipeline` event is published once any filter finishes; `call_text_filters`
|
306
|
+
and `call_node_filters` is published when all of the text and node filters are finished, respectively.
|
307
|
+
The `payload` should include the `filter` name. Each filter will trigger its own
|
285
308
|
instrumentation call.
|
286
309
|
|
287
|
-
```
|
310
|
+
```ruby
|
288
311
|
service.subscribe "call_filter.html_pipeline" do |event, start, ending, transaction_id, payload|
|
289
312
|
payload[:pipeline] #=> "MarkdownPipeline", set with `setup_instrumentation`
|
290
313
|
payload[:filter] #=> "MarkdownFilter"
|
291
314
|
payload[:context] #=> context Hash
|
292
315
|
payload[:result] #=> instance of result class
|
293
|
-
payload[:result][:output] #=> output HTML String
|
316
|
+
payload[:result][:output] #=> output HTML String
|
294
317
|
end
|
295
318
|
```
|
296
319
|
|
297
320
|
The full pipeline is also instrumented:
|
298
321
|
|
299
|
-
```
|
300
|
-
service.subscribe "
|
322
|
+
```ruby
|
323
|
+
service.subscribe "call_text_filters.html_pipeline" do |event, start, ending, transaction_id, payload|
|
301
324
|
payload[:pipeline] #=> "MarkdownPipeline", set with `setup_instrumentation`
|
302
325
|
payload[:filters] #=> ["MarkdownFilter"]
|
303
|
-
payload[:doc] #=> HTML String
|
326
|
+
payload[:doc] #=> HTML String
|
304
327
|
payload[:context] #=> context Hash
|
305
328
|
payload[:result] #=> instance of result class
|
306
|
-
payload[:result][:output] #=> output HTML String
|
329
|
+
payload[:result][:output] #=> output HTML String
|
307
330
|
end
|
308
331
|
```
|
309
332
|
|
333
|
+
## Third Party Extensions
|
334
|
+
|
335
|
+
If you have an idea for a filter, propose it as
|
336
|
+
[an issue](https://github.com/gjtorikian/html-pipeline/issues) first. This allows us to discuss
|
337
|
+
whether the filter is a common enough use case to belong in this gem, or should be
|
338
|
+
built as an external gem.
|
339
|
+
|
340
|
+
Here are some extensions people have built:
|
341
|
+
|
342
|
+
- [html-pipeline-asciidoc_filter](https://github.com/asciidoctor/html-pipeline-asciidoc_filter)
|
343
|
+
- [jekyll-html-pipeline](https://github.com/gjtorikian/jekyll-html-pipeline)
|
344
|
+
- [nanoc-html-pipeline](https://github.com/burnto/nanoc-html-pipeline)
|
345
|
+
- [html-pipeline-bitly](https://github.com/dewski/html-pipeline-bitly)
|
346
|
+
- [html-pipeline-cite](https://github.com/lifted-studios/html-pipeline-cite)
|
347
|
+
- [tilt-html-pipeline](https://github.com/bradgessler/tilt-html-pipeline)
|
348
|
+
- [html-pipeline-wiki-link'](https://github.com/lifted-studios/html-pipeline-wiki-link) - WikiMedia-style wiki links
|
349
|
+
- [task_list](https://github.com/github/task_list) - GitHub flavor Markdown Task List
|
350
|
+
- [html-pipeline-nico_link](https://github.com/rutan/html-pipeline-nico_link) - An HTMLPipeline filter for [niconico](http://www.nicovideo.jp) description links
|
351
|
+
- [html-pipeline-gitlab](https://gitlab.com/gitlab-org/html-pipeline-gitlab) - This gem implements various filters for html-pipeline used by GitLab
|
352
|
+
- [html-pipeline-youtube](https://github.com/st0012/html-pipeline-youtube) - An HTMLPipeline filter for YouTube links
|
353
|
+
- [html-pipeline-flickr](https://github.com/st0012/html-pipeline-flickr) - An HTMLPipeline filter for Flickr links
|
354
|
+
- [html-pipeline-vimeo](https://github.com/dlackty/html-pipeline-vimeo) - An HTMLPipeline filter for Vimeo links
|
355
|
+
- [html-pipeline-hashtag](https://github.com/mr-dxdy/html-pipeline-hashtag) - An HTMLPipeline filter for hashtags
|
356
|
+
- [html-pipeline-linkify_github](https://github.com/jollygoodcode/html-pipeline-linkify_github) - An HTMLPipeline filter to autolink GitHub urls
|
357
|
+
- [html-pipeline-redcarpet_filter](https://github.com/bmikol/html-pipeline-redcarpet_filter) - Render Markdown source text into Markdown HTML using Redcarpet
|
358
|
+
- [html-pipeline-typogruby_filter](https://github.com/bmikol/html-pipeline-typogruby_filter) - Add Typogruby text filters to your HTMLPipeline
|
359
|
+
- [korgi](https://github.com/jodeci/korgi) - HTMLPipeline filters for links to Rails resources
|
360
|
+
|
310
361
|
## FAQ
|
311
362
|
|
312
363
|
### 1. Why doesn't my pipeline work when there's no root element in the document?
|
313
364
|
|
314
365
|
To make a pipeline work on a plain text document, put the `PlainTextInputFilter`
|
315
|
-
at the
|
316
|
-
filters have a root element to work with. If you're passing in an HTML fragment,
|
366
|
+
at the end of your `text_filter`s config . This will wrap the content in a `div` so the filters have a root element to work with. If you're passing in an HTML fragment,
|
317
367
|
but it doesn't have a root element, you can wrap the content in a `div`
|
318
|
-
yourself.
|
319
|
-
|
320
|
-
```ruby
|
321
|
-
EmojiPipeline = Pipeline.new [
|
322
|
-
PlainTextInputFilter, # <- Wraps input in a div and escapes html tags
|
323
|
-
EmojiFilter
|
324
|
-
], context
|
325
|
-
|
326
|
-
plain_text = "Gutentag! :wave:"
|
327
|
-
EmojiPipeline.call(plain_text)
|
328
|
-
|
329
|
-
html_fragment = "This is outside of an html element, but <strong>this isn't. :+1:</strong>"
|
330
|
-
EmojiPipeline.call("<div>#{html_fragment}</div>") # <- Wrap your own html fragments to avoid escaping
|
331
|
-
```
|
368
|
+
yourself.
|
332
369
|
|
333
370
|
### 2. How do I customize an allowlist for `SanitizationFilter`s?
|
334
371
|
|
335
|
-
`SanitizationFilter::ALLOWLIST` is the default allowlist used if no
|
336
|
-
argument is given
|
372
|
+
`HTMLPipeline::SanitizationFilter::ALLOWLIST` is the default allowlist used if no `sanitization_config`
|
373
|
+
argument is given. The default is a good starting template for
|
337
374
|
you to add additional elements. You can either modify the constant's value, or
|
338
|
-
re-define your own
|
339
|
-
|
340
|
-
## Contributing
|
375
|
+
re-define your own config and pass that in, such as:
|
341
376
|
|
342
|
-
|
343
|
-
|
344
|
-
|
345
|
-
|
346
|
-
3. Commit your changes (`git commit -am 'Added some feature'`)
|
347
|
-
4. Push to the branch (`git push origin my-new-feature`)
|
348
|
-
5. Create new [Pull Request](https://help.github.com/articles/using-pull-requests)
|
349
|
-
|
350
|
-
To see what has changed in recent versions, see the [CHANGELOG](https://github.com/jch/html-pipeline/blob/master/CHANGELOG.md).
|
377
|
+
```ruby
|
378
|
+
config = HTMLPipeline::SanitizerFilter::DEFAULT_CONFIG.dup
|
379
|
+
config[:elements] << "iframe" # sure, whatever you want
|
380
|
+
```
|
351
381
|
|
352
382
|
### Contributors
|
353
383
|
|
354
|
-
Thanks to all of [these contributors](https://github.com/
|
355
|
-
|
356
|
-
Project is a member of the [OSS Manifesto](http://ossmanifesto.org/).
|
357
|
-
|
358
|
-
The current maintainer is @gjtorikian
|
359
|
-
|
360
|
-
### Releasing A New Version
|
361
|
-
|
362
|
-
This section is for gem maintainers to cut a new version of the gem.
|
384
|
+
Thanks to all of [these contributors](https://github.com/gjtorikian/html-pipeline/graphs/contributors).
|
363
385
|
|
364
|
-
|
365
|
-
* update lib/html/pipeline/version.rb to next version number X.X.X
|
366
|
-
* update CHANGELOG.md. Prepare a draft with `script/changelog`
|
367
|
-
* push branch and create a new pull request
|
368
|
-
* after tests are green, merge to master
|
369
|
-
* on the master branch, run `script/release`
|
386
|
+
This project is a member of the [OSS Manifesto](http://ossmanifesto.org/).
|
data/Rakefile
CHANGED
@@ -1,17 +1,24 @@
|
|
1
1
|
#!/usr/bin/env rake
|
2
2
|
# frozen_string_literal: true
|
3
3
|
|
4
|
-
require
|
5
|
-
require
|
6
|
-
|
7
|
-
require 'bundler/gem_tasks'
|
8
|
-
require 'rake/testtask'
|
4
|
+
require "bundler/gem_tasks"
|
5
|
+
require "rubygems/package_task"
|
6
|
+
require "rake/testtask"
|
9
7
|
|
10
8
|
Rake::TestTask.new do |t|
|
11
|
-
t.libs <<
|
12
|
-
t.test_files = FileList[
|
9
|
+
t.libs << "test"
|
10
|
+
t.test_files = FileList["test/**/*_test.rb"]
|
13
11
|
t.verbose = true
|
14
12
|
t.warning = false
|
15
13
|
end
|
16
14
|
|
17
15
|
task default: :test
|
16
|
+
|
17
|
+
require "rubocop/rake_task"
|
18
|
+
|
19
|
+
RuboCop::RakeTask.new(:rubocop)
|
20
|
+
|
21
|
+
GEMSPEC = Bundler.load_gemspec("html-pipeline.gemspec")
|
22
|
+
gem_path = Gem::PackageTask.new(GEMSPEC).define
|
23
|
+
desc "Package the ruby gem"
|
24
|
+
task "package" => [gem_path]
|