html-pipeline 2.14.3 → 3.0.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (59) hide show
  1. checksums.yaml +4 -4
  2. data/.github/FUNDING.yml +11 -3
  3. data/.github/dependabot.yml +27 -0
  4. data/.github/workflows/automerge.yml +13 -0
  5. data/.github/workflows/ci.yml +22 -0
  6. data/.github/workflows/lint.yml +23 -0
  7. data/.github/workflows/publish.yml +19 -0
  8. data/.rubocop.yml +17 -0
  9. data/.ruby-version +1 -0
  10. data/.vscode/settings.json +8 -0
  11. data/CHANGELOG.md +119 -2
  12. data/Gemfile +31 -15
  13. data/{LICENSE → LICENSE.txt} +2 -2
  14. data/README.md +241 -224
  15. data/Rakefile +14 -7
  16. data/UPGRADING.md +34 -0
  17. data/html-pipeline.gemspec +31 -21
  18. data/lib/html-pipeline.rb +3 -0
  19. data/lib/html_pipeline/convert_filter/markdown_filter.rb +26 -0
  20. data/lib/html_pipeline/convert_filter.rb +17 -0
  21. data/lib/html_pipeline/filter.rb +89 -0
  22. data/lib/html_pipeline/node_filter/absolute_source_filter.rb +54 -0
  23. data/lib/html_pipeline/node_filter/asset_proxy_filter.rb +86 -0
  24. data/lib/{html/pipeline → html_pipeline/node_filter}/emoji_filter.rb +58 -54
  25. data/lib/html_pipeline/node_filter/https_filter.rb +22 -0
  26. data/lib/html_pipeline/node_filter/image_max_width_filter.rb +40 -0
  27. data/lib/{html/pipeline/@mention_filter.rb → html_pipeline/node_filter/mention_filter.rb} +54 -68
  28. data/lib/html_pipeline/node_filter/syntax_highlight_filter.rb +62 -0
  29. data/lib/html_pipeline/node_filter/table_of_contents_filter.rb +70 -0
  30. data/lib/html_pipeline/node_filter/team_mention_filter.rb +105 -0
  31. data/lib/html_pipeline/node_filter.rb +31 -0
  32. data/lib/html_pipeline/sanitization_filter.rb +188 -0
  33. data/lib/{html/pipeline → html_pipeline/text_filter}/image_filter.rb +3 -3
  34. data/lib/{html/pipeline → html_pipeline/text_filter}/plain_text_input_filter.rb +3 -5
  35. data/lib/html_pipeline/text_filter.rb +21 -0
  36. data/lib/html_pipeline/version.rb +5 -0
  37. data/lib/html_pipeline.rb +281 -0
  38. metadata +58 -54
  39. data/.travis.yml +0 -43
  40. data/Appraisals +0 -19
  41. data/CONTRIBUTING.md +0 -60
  42. data/bin/html-pipeline +0 -78
  43. data/lib/html/pipeline/@team_mention_filter.rb +0 -99
  44. data/lib/html/pipeline/absolute_source_filter.rb +0 -52
  45. data/lib/html/pipeline/autolink_filter.rb +0 -34
  46. data/lib/html/pipeline/body_content.rb +0 -44
  47. data/lib/html/pipeline/camo_filter.rb +0 -105
  48. data/lib/html/pipeline/email_reply_filter.rb +0 -69
  49. data/lib/html/pipeline/filter.rb +0 -165
  50. data/lib/html/pipeline/https_filter.rb +0 -29
  51. data/lib/html/pipeline/image_max_width_filter.rb +0 -37
  52. data/lib/html/pipeline/markdown_filter.rb +0 -56
  53. data/lib/html/pipeline/sanitization_filter.rb +0 -144
  54. data/lib/html/pipeline/syntax_highlight_filter.rb +0 -50
  55. data/lib/html/pipeline/text_filter.rb +0 -16
  56. data/lib/html/pipeline/textile_filter.rb +0 -25
  57. data/lib/html/pipeline/toc_filter.rb +0 -69
  58. data/lib/html/pipeline/version.rb +0 -7
  59. data/lib/html/pipeline.rb +0 -210
data/README.md CHANGED
@@ -1,23 +1,31 @@
1
- # HTML::Pipeline [![Build Status](https://travis-ci.org/jch/html-pipeline.svg?branch=master)](https://travis-ci.org/jch/html-pipeline)
1
+ # HTML-Pipeline
2
2
 
3
- HTML processing filters and utilities. This module includes a small
4
- framework for defining DOM based content filters and applying them to user
3
+ > **Note**
4
+ > This README refers to the behavior in the new 3.0.0.pre gem.
5
+
6
+ HTML processing filters and utilities. This module is a small
7
+ framework for defining CSS-based content filters and applying them to user
5
8
  provided content.
6
9
 
7
- [This project was started at GitHub](https://github.com/blog/1311-html-pipeline-chainable-content-filters). While GitHub still uses a similar design and pattern for rendering content, this gem should be considered standalone and independent from GitHub.
8
-
9
- - [Installation](#installation)
10
- - [Usage](#usage)
11
- - [Examples](#examples)
12
- - [Filters](#filters)
13
- - [Dependencies](#dependencies)
14
- - [Documentation](#documentation)
15
- - [Extending](#extending)
16
- - [3rd Party Extensions](#3rd-party-extensions)
17
- - [Instrumenting](#instrumenting)
18
- - [Contributing](#contributing)
19
- - [Contributors](#contributors)
20
- - [Releasing A New Version](#releasing-a-new-version)
10
+ [Although this project was started at GitHub](https://github.com/blog/1311-html-pipeline-chainable-content-filters), they no longer use it. This gem must be considered standalone and independent from GitHub.
11
+
12
+ - [HTML-Pipeline](#html-pipeline)
13
+ - [Installation](#installation)
14
+ - [Usage](#usage)
15
+ - [More Examples](#more-examples)
16
+ - [Filters](#filters)
17
+ - [TextFilters](#textfilters)
18
+ - [ConvertFilter](#convertfilter)
19
+ - [Sanitization](#sanitization)
20
+ - [NodeFilters](#nodefilters)
21
+ - [Dependencies](#dependencies)
22
+ - [Documentation](#documentation)
23
+ - [Instrumenting](#instrumenting)
24
+ - [Third Party Extensions](#third-party-extensions)
25
+ - [FAQ](#faq)
26
+ - [1. Why doesn't my pipeline work when there's no root element in the document?](#1-why-doesnt-my-pipeline-work-when-theres-no-root-element-in-the-document)
27
+ - [2. How do I customize an allowlist for `SanitizationFilter`s?](#2-how-do-i-customize-an-allowlist-for-sanitizationfilters)
28
+ - [Contributors](#contributors)
21
29
 
22
30
  ## Installation
23
31
 
@@ -42,220 +50,234 @@ $ gem install html-pipeline
42
50
  ## Usage
43
51
 
44
52
  This library provides a handful of chainable HTML filters to transform user
45
- content into markup. A filter takes an HTML string or
46
- `Nokogiri::HTML::DocumentFragment`, optionally manipulates it, and then
47
- outputs the result.
53
+ content into HTML markup. Each filter does some work, and then hands off the
54
+ results tothe next filter. A pipeline has several kinds of filters available to use:
48
55
 
49
- For example, to transform Markdown source into Markdown HTML:
56
+ - Multiple `TextFilter`s, which operate a UTF-8 string
57
+ - A `ConvertFilter` filter, which turns text into HTML (eg., Commonmark/Asciidoc -> HTML)
58
+ - A `SanitizationFilter`, which remove dangerous/unwanted HTML elements and attributes
59
+ - Multiple `NodeFilter`s, which operate on a UTF-8 HTML document
50
60
 
51
- ```ruby
52
- require 'html/pipeline'
61
+ You can assemble each sequence into a single pipeline, or choose to call each filter individually.
53
62
 
54
- filter = HTML::Pipeline::MarkdownFilter.new("Hi **world**!")
55
- filter.call
56
- ```
63
+ As an example, suppose we want to transform Commonmark source text into Markdown HTML. With the content, we also want to:
57
64
 
58
- Filters can be combined into a pipeline which causes each filter to hand its
59
- output to the next filter's input. So if you wanted to have content be
60
- filtered through Markdown and be syntax highlighted, you can create the
61
- following pipeline:
65
+ - change every instance of `$NAME` to "`Johnny"
66
+ - strip undesired HTML
67
+ - linkify @mention
68
+
69
+ We can construct a pipeline to do all that like this:
62
70
 
63
71
  ```ruby
64
- pipeline = HTML::Pipeline.new [
65
- HTML::Pipeline::MarkdownFilter,
66
- HTML::Pipeline::SyntaxHighlightFilter
67
- ]
68
- result = pipeline.call <<-CODE
69
- This is *great*:
72
+ require 'html_pipeline'
70
73
 
71
- some_code(:first)
74
+ class HelloJohnnyFilter < HTMLPipelineFilter
75
+ def call
76
+ text.gsub("$NAME", "Johnny")
77
+ end
78
+ end
72
79
 
73
- CODE
74
- result[:output].to_s
80
+ pipeline = HTMLPipeline.new(
81
+ text_filters: [HelloJohnnyFilter.new]
82
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
83
+ # note: next line is not needed as sanitization occurs by default;
84
+ # see below for more info
85
+ sanitization_config: HTMLPipeline::SanitizationFilter::DEFAULT_CONFIG,
86
+ node_filters: [HTMLPipeline::NodeFilter::MentionFilter.new]
87
+ )
88
+ pipeline.call(user_supplied_text) # recommended: can call pipeline over and over
75
89
  ```
76
90
 
77
- Prints:
91
+ Filters can be custom ones you create (like `HelloJohnnyFilter`), and `HTMLPipeline` additionally provides several helpful ones (detailed below). If you only need a single filter, you can call one individually, too:
78
92
 
79
- ```html
80
- <p>This is <em>great</em>:</p>
81
-
82
- <pre><code>some_code(:first)
83
- </code></pre>
93
+ ```ruby
94
+ filter = HTMLPipeline::ConvertFilter::MarkdownFilter.new
95
+ filter.call(text)
84
96
  ```
85
97
 
86
- To generate CSS for HTML formatted code, use the [Rouge CSS Theme](https://github.com/rouge-ruby/rouge#css-options) `#css` method. `rouge` is a dependency of the `SyntaxHighlightFilter`.
98
+ Filters combine into a sequential pipeline, and each filter hands its
99
+ output to the next filter's input. Text filters are
100
+ processed first, then the convert filter, sanitization filter, and finally, the node filters.
87
101
 
88
- Some filters take an optional **context** and/or **result** hash. These are
102
+ Some filters take optional `context` and/or `result` hash(es). These are
89
103
  used to pass around arguments and metadata between filters in a pipeline. For
90
- example, if you don't want to use GitHub formatted Markdown, you can pass an
91
- option in the context hash:
104
+ example, if you want to disable footnotes in the `MarkdownFilter`, you can pass an option in the context hash:
92
105
 
93
106
  ```ruby
94
- filter = HTML::Pipeline::MarkdownFilter.new("Hi **world**!", :gfm => false)
95
- filter.call
107
+ context = { markdown: { extensions: { footnotes: false } } }
108
+ filter = HTMLPipeline::ConvertFilter::MarkdownFilter.new(context: context)
109
+ filter.call("Hi **world**!")
96
110
  ```
97
111
 
98
- ### Examples
112
+ Please refer to the documentation for each filter to understand what configuration options are available.
113
+
114
+ ### More Examples
99
115
 
100
- We define different pipelines for different parts of our app. Here are a few
116
+ Different pipelines can be defined for different parts of an app. Here are a few
101
117
  paraphrased snippets to get you started:
102
118
 
103
119
  ```ruby
104
120
  # The context hash is how you pass options between different filters.
105
121
  # See individual filter source for explanation of options.
106
122
  context = {
107
- :asset_root => "http://your-domain.com/where/your/images/live/icons",
108
- :base_url => "http://your-domain.com"
123
+ asset_root: "http://your-domain.com/where/your/images/live/icons",
124
+ base_url: "http://your-domain.com"
109
125
  }
110
126
 
111
- # Pipeline providing sanitization and image hijacking but no mention
112
- # related features.
113
- SimplePipeline = Pipeline.new [
114
- SanitizationFilter,
115
- TableOfContentsFilter, # add 'name' anchors to all headers and generate toc list
116
- CamoFilter,
117
- ImageMaxWidthFilter,
118
- SyntaxHighlightFilter,
119
- EmojiFilter,
120
- AutolinkFilter
121
- ], context
122
-
123
127
  # Pipeline used for user provided content on the web
124
- MarkdownPipeline = Pipeline.new [
125
- MarkdownFilter,
126
- SanitizationFilter,
127
- CamoFilter,
128
- ImageMaxWidthFilter,
129
- HttpsFilter,
130
- MentionFilter,
131
- EmojiFilter,
132
- SyntaxHighlightFilter
133
- ], context.merge(:gfm => true) # enable github formatted markdown
134
-
135
-
136
- # Define a pipeline based on another pipeline's filters
137
- NonGFMMarkdownPipeline = Pipeline.new(MarkdownPipeline.filters,
138
- context.merge(:gfm => false))
128
+ MarkdownPipeline = HTMLPipeline.new (
129
+ text_filters: [HTMLPipeline::TextFilter::ImageFilter.new],
130
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
131
+ node_filters: [
132
+ HTMLPipeline::NodeFilter::HttpsFilter.new,HTMLPipeline::NodeFilter::MentionFilter.new,
133
+ ], context: context)
139
134
 
140
135
  # Pipelines aren't limited to the web. You can use them for email
141
136
  # processing also.
142
- HtmlEmailPipeline = Pipeline.new [
143
- PlainTextInputFilter,
144
- ImageMaxWidthFilter
145
- ], {}
146
-
147
- # Just emoji.
148
- EmojiPipeline = Pipeline.new [
149
- PlainTextInputFilter,
150
- EmojiFilter
151
- ], context
137
+ HtmlEmailPipeline = HTMLPipeline.new(
138
+ text_filters: [
139
+ PlainTextInputFilter.new,
140
+ ImageFilter.new
141
+ ], {})
152
142
  ```
153
143
 
154
144
  ## Filters
155
145
 
156
- * `MentionFilter` - replace `@user` mentions with links
157
- * `TeamMentionFilter` - replace `@org/team` mentions with links
158
- * `AbsoluteSourceFilter` - replace relative image urls with fully qualified versions
159
- * `AutolinkFilter` - auto_linking urls in HTML
160
- * `CamoFilter` - replace http image urls with [camo-fied](https://github.com/atmos/camo) https versions
161
- * `EmailReplyFilter` - util filter for working with emails
162
- * `EmojiFilter` - everyone loves [emoji](http://www.emoji-cheat-sheet.com/)!
163
- * `HttpsFilter` - HTML Filter for replacing http github urls with https versions.
164
- * `ImageMaxWidthFilter` - link to full size image for large images
165
- * `MarkdownFilter` - convert markdown to html
166
- * `PlainTextInputFilter` - html escape text and wrap the result in a div
167
- * `SanitizationFilter` - allow sanitize user markup
168
- * `SyntaxHighlightFilter` - code syntax highlighter
169
- * `TextileFilter` - convert textile to html
170
- * `TableOfContentsFilter` - anchor headings with name attributes and generate Table of Contents html unordered list linking headings
146
+ ### TextFilters
171
147
 
172
- ## Dependencies
148
+ `TextFilter`s must define a method named `call` which is called on the text. `@text`, `@config`, and `@result` are available to use, and any changes made to these ivars are passed on to the next filter.
149
+
150
+ - `ImageFilter` - converts image `url` into `<img>` tag
151
+ - `PlainTextInputFilter` - html escape text and wrap the result in a `<div>`
152
+
153
+ ### ConvertFilter
154
+
155
+ The `ConvertFilter` takes text and turns it into HTML. `@text`, `@config`, and `@result` are available to use. `ConvertFilter` must defined a method named `call`, taking one argument, `text`. `call` must return a string representing the new HTML document.
173
156
 
174
- Filter gem dependencies are not bundled; you must bundle the filter's gem
175
- dependencies. The below list details filters with dependencies. For example,
176
- `SyntaxHighlightFilter` uses [rouge](https://github.com/jneen/rouge)
177
- to detect and highlight languages. For example, to use the `SyntaxHighlightFilter`,
178
- add the following to your Gemfile:
157
+ - `MarkdownFilter` - creates HTML from text using [Commonmarker](https://www.github.com/gjtorikian/commonmarker)
158
+
159
+ ### Sanitization
160
+
161
+ Because the web can be a scary place, HTML is automatically sanitized after the `ConvertFilter` runs and before the `NodeFilter`s are processed. This is to prevent malicious or unexpected input from entering the pipeline.
162
+
163
+ The sanitization process takes a hash configuration of settings. See the [Selma](https://www.github.com/gjtorikian/selma) documentation for more information on how to configure these settings.
164
+
165
+ A default sanitization config is provided by this library (`HTMLPipeline::SanitizationFilter::DEFAULT_CONFIG`). A sample custom sanitization allowlist might look like this:
179
166
 
180
167
  ```ruby
181
- gem 'rouge'
168
+ ALLOWLIST = {
169
+ elements: ["p", "pre", "code"]
170
+ }
171
+
172
+ pipeline = HTMLPipeline.new \
173
+ text_filters: [
174
+ HTMLPipeline::MarkdownFilter,
175
+ ],
176
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
177
+ sanitization_config: ALLOWLIST
178
+
179
+ result = pipeline.call <<-CODE
180
+ This is *great*:
181
+
182
+ some_code(:first)
183
+
184
+ CODE
185
+ result[:output].to_s
182
186
  ```
183
187
 
184
- * `AutolinkFilter` - `rinku`
185
- * `EmailReplyFilter` - `escape_utils`, `email_reply_parser`
186
- * `EmojiFilter` - `gemoji`
187
- * `MarkdownFilter` - `commonmarker`
188
- * `PlainTextInputFilter` - `escape_utils`
189
- * `SanitizationFilter` - `sanitize`
190
- * `SyntaxHighlightFilter` - `rouge`
191
- * `TableOfContentsFilter` - `escape_utils`
192
- * `TextileFilter` - `RedCloth`
188
+ This would print:
193
189
 
194
- _Note:_ See [Gemfile](/Gemfile) `:test` block for version requirements.
190
+ ```html
191
+ <p>This is great:</p>
192
+ <pre><code>some_code(:first)
193
+ </code></pre>
194
+ ```
195
195
 
196
- ## Documentation
196
+ Sanitization can be disabled if and only if `nil` is explicitly passed as
197
+ the config:
197
198
 
198
- Full reference documentation can be [found here](http://rubydoc.info/gems/html-pipeline/frames).
199
+ ```ruby
200
+ pipeline = HTMLPipeline.new \
201
+ text_filters: [
202
+ HTMLPipeline::MarkdownFilter,
203
+ ],
204
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
205
+ sanitization_config: nil
206
+ ```
207
+
208
+ For more examples of customizing the sanitization process to include the tags you want, check out [the tests](test/sanitization_filter_test.rb) and [the FAQ](#faq).
199
209
 
200
- ## Extending
201
- To write a custom filter, you need a class with a `call` method that inherits
202
- from `HTML::Pipeline::Filter`.
210
+ ### NodeFilters
203
211
 
204
- For example this filter adds a base url to images that are root relative:
212
+ `NodeFilters`s can operate either on HTML elements or text nodes using CSS selectors. Each `NodeFilter` must define a method named `selector` which provides an instance of `Selma::Selector`. If elements are being manipulated, `handle_element` must be defined, taking one argument, `element`; if text nodes are being manipulated, `handle_text_chunk` must be defined, taking one argument, `text_chunk`. `@config`, and `@result` are available to use, and any changes made to these ivars are passed on to the next filter.
213
+
214
+ `NodeFilter` also has an optional method, `after_initialize`, which is run after the filter initializes. This can be useful in setting up a custom state for `result` to take advantage of.
215
+
216
+ Here's an example `NodeFilter` that adds a base url to images that are root relative:
205
217
 
206
218
  ```ruby
207
219
  require 'uri'
208
220
 
209
- class RootRelativeFilter < HTML::Pipeline::Filter
221
+ class RootRelativeFilter < HTMLPipeline::NodeFilter
210
222
 
211
- def call
212
- doc.search("img").each do |img|
213
- next if img['src'].nil?
214
- src = img['src'].strip
215
- if src.start_with? '/'
216
- img["src"] = URI.join(context[:base_url], src).to_s
217
- end
218
- end
219
- doc
223
+ SELECTOR = Selma::Selector.new(match_element: "img")
224
+
225
+ def selector
226
+ SELECTOR
220
227
  end
221
228
 
229
+ def handle_element(img)
230
+ next if img['src'].nil?
231
+ src = img['src'].strip
232
+ if src.start_with? '/'
233
+ img["src"] = URI.join(context[:base_url], src).to_s
234
+ end
235
+ end
222
236
  end
223
237
  ```
224
238
 
225
- Now this filter can be used in a pipeline:
239
+ For more information on how to write effective `NodeFilter`s, refer to the provided filters, and see the underlying lib, [Selma](https://www.github.com/gjtorikian/selma) for more information.
240
+
241
+ - `AbsoluteSourceFilter`: replace relative image urls with fully qualified versions
242
+ - `AssetProxyFilter`: replace image links with an encoded link to an asset server
243
+ - `EmojiFilter`: converts `:<emoji>:` to [emoji](http://www.emoji-cheat-sheet.com/)
244
+ - (Note: the included `MarkdownFilter` will already convert emoji)
245
+ - `HttpsFilter`: Replacing http urls with https versions
246
+ - `ImageMaxWidthFilter`: link to full size image for large images
247
+ - `MentionFilter`: replace `@user` mentions with links
248
+ - `SanitizationFilter`: allow sanitize user markup
249
+ - `SyntaxHighlightFilter`: applies syntax highlighting to `pre` blocks
250
+ - (Note: the included `MarkdownFilter` will already apply highlighting)
251
+ - `TableOfContentsFilter`: anchor headings with name attributes and generate Table of Contents html unordered list linking headings
252
+ - `TeamMentionFilter`: replace `@org/team` mentions with links
253
+
254
+ ## Dependencies
255
+
256
+ Since filters can be customized to your heart's content, gem dependencies are _not_ bundled; this project doesn't know which of the default filters you might use, and as such, you must bundle each filter's gem dependencies yourself.
257
+
258
+ For example, `SyntaxHighlightFilter` uses [rouge](https://github.com/jneen/rouge)
259
+ to detect and highlight languages; to use the `SyntaxHighlightFilter`, you must add the following to your Gemfile:
226
260
 
227
261
  ```ruby
228
- Pipeline.new [ RootRelativeFilter ], { :base_url => 'http://somehost.com' }
262
+ gem "rouge"
229
263
  ```
230
264
 
231
- ### 3rd Party Extensions
265
+ > **Note**
266
+ > See the [Gemfile](/Gemfile) `:test` group for any version requirements.
232
267
 
233
- If you have an idea for a filter, propose it as
234
- [an issue](https://github.com/jch/html-pipeline/issues) first. This allows us discuss
235
- whether the filter is a common enough use case to belong in this gem, or should be
236
- built as an external gem.
268
+ When developing a custom filter, call `HTMLPipeline.require_dependency` at the start to ensure that the local machine has the necessary dependency. You can also use `HTMLPipeline.require_dependencies` to provide a list of dependencies to check.
237
269
 
238
- Here are some extensions people have built:
270
+ On a similar note, you must manually require whichever filters you desire:
239
271
 
240
- * [html-pipeline-asciidoc_filter](https://github.com/asciidoctor/html-pipeline-asciidoc_filter)
241
- * [jekyll-html-pipeline](https://github.com/gjtorikian/jekyll-html-pipeline)
242
- * [nanoc-html-pipeline](https://github.com/burnto/nanoc-html-pipeline)
243
- * [html-pipeline-bitly](https://github.com/dewski/html-pipeline-bitly)
244
- * [html-pipeline-cite](https://github.com/lifted-studios/html-pipeline-cite)
245
- * [tilt-html-pipeline](https://github.com/bradgessler/tilt-html-pipeline)
246
- * [html-pipeline-wiki-link'](https://github.com/lifted-studios/html-pipeline-wiki-link) - WikiMedia-style wiki links
247
- * [task_list](https://github.com/github/task_list) - GitHub flavor Markdown Task List
248
- * [html-pipeline-nico_link](https://github.com/rutan/html-pipeline-nico_link) - An HTML::Pipeline filter for [niconico](http://www.nicovideo.jp) description links
249
- * [html-pipeline-gitlab](https://gitlab.com/gitlab-org/html-pipeline-gitlab) - This gem implements various filters for html-pipeline used by GitLab
250
- * [html-pipeline-youtube](https://github.com/st0012/html-pipeline-youtube) - An HTML::Pipeline filter for YouTube links
251
- * [html-pipeline-flickr](https://github.com/st0012/html-pipeline-flickr) - An HTML::Pipeline filter for Flickr links
252
- * [html-pipeline-vimeo](https://github.com/dlackty/html-pipeline-vimeo) - An HTML::Pipeline filter for Vimeo links
253
- * [html-pipeline-hashtag](https://github.com/mr-dxdy/html-pipeline-hashtag) - An HTML::Pipeline filter for hashtags
254
- * [html-pipeline-linkify_github](https://github.com/jollygoodcode/html-pipeline-linkify_github) - An HTML::Pipeline filter to autolink GitHub urls
255
- * [html-pipeline-redcarpet_filter](https://github.com/bmikol/html-pipeline-redcarpet_filter) - Render Markdown source text into Markdown HTML using Redcarpet
256
- * [html-pipeline-typogruby_filter](https://github.com/bmikol/html-pipeline-typogruby_filter) - Add Typogruby text filters to your HTML::Pipeline
257
- * [korgi](https://github.com/jodeci/korgi) - HTML::Pipeline filters for links to Rails resources
272
+ ```ruby
273
+ require "html_pipeline" # must be included
274
+ require "html_pipeline/convert_filter/markdown_filter" # included because you want to use this filter
275
+ require "html_pipeline/node_filter/mention_filter" # included because you want to use this filter
276
+ ```
258
277
 
278
+ ## Documentation
279
+
280
+ Full reference documentation can be [found here](http://rubydoc.info/gems/html-pipeline/frames).
259
281
 
260
282
  ## Instrumenting
261
283
 
@@ -263,107 +285,102 @@ Filters and Pipelines can be set up to be instrumented when called. The pipeline
263
285
  must be setup with an
264
286
  [ActiveSupport::Notifications](http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html)
265
287
  compatible service object and a name. New pipeline objects will default to the
266
- `HTML::Pipeline.default_instrumentation_service` object.
288
+ `HTMLPipeline.default_instrumentation_service` object.
267
289
 
268
- ``` ruby
290
+ ```ruby
269
291
  # the AS::Notifications-compatible service object
270
292
  service = ActiveSupport::Notifications
271
293
 
272
294
  # instrument a specific pipeline
273
- pipeline = HTML::Pipeline.new [MarkdownFilter], context
295
+ pipeline = HTMLPipeline.new [MarkdownFilter], context
274
296
  pipeline.setup_instrumentation "MarkdownPipeline", service
275
297
 
276
298
  # or set default instrumentation service for all new pipelines
277
- HTML::Pipeline.default_instrumentation_service = service
278
- pipeline = HTML::Pipeline.new [MarkdownFilter], context
299
+ HTMLPipeline.default_instrumentation_service = service
300
+ pipeline = HTMLPipeline.new [MarkdownFilter], context
279
301
  pipeline.setup_instrumentation "MarkdownPipeline"
280
302
  ```
281
303
 
282
304
  Filters are instrumented when they are run through the pipeline. A
283
- `call_filter.html_pipeline` event is published once the filter finishes. The
284
- `payload` should include the `filter` name. Each filter will trigger its own
305
+ `call_filter.html_pipeline` event is published once any filter finishes; `call_text_filters`
306
+ and `call_node_filters` is published when all of the text and node filters are finished, respectively.
307
+ The `payload` should include the `filter` name. Each filter will trigger its own
285
308
  instrumentation call.
286
309
 
287
- ``` ruby
310
+ ```ruby
288
311
  service.subscribe "call_filter.html_pipeline" do |event, start, ending, transaction_id, payload|
289
312
  payload[:pipeline] #=> "MarkdownPipeline", set with `setup_instrumentation`
290
313
  payload[:filter] #=> "MarkdownFilter"
291
314
  payload[:context] #=> context Hash
292
315
  payload[:result] #=> instance of result class
293
- payload[:result][:output] #=> output HTML String or Nokogiri::DocumentFragment
316
+ payload[:result][:output] #=> output HTML String
294
317
  end
295
318
  ```
296
319
 
297
320
  The full pipeline is also instrumented:
298
321
 
299
- ``` ruby
300
- service.subscribe "call_pipeline.html_pipeline" do |event, start, ending, transaction_id, payload|
322
+ ```ruby
323
+ service.subscribe "call_text_filters.html_pipeline" do |event, start, ending, transaction_id, payload|
301
324
  payload[:pipeline] #=> "MarkdownPipeline", set with `setup_instrumentation`
302
325
  payload[:filters] #=> ["MarkdownFilter"]
303
- payload[:doc] #=> HTML String or Nokogiri::DocumentFragment
326
+ payload[:doc] #=> HTML String
304
327
  payload[:context] #=> context Hash
305
328
  payload[:result] #=> instance of result class
306
- payload[:result][:output] #=> output HTML String or Nokogiri::DocumentFragment
329
+ payload[:result][:output] #=> output HTML String
307
330
  end
308
331
  ```
309
332
 
333
+ ## Third Party Extensions
334
+
335
+ If you have an idea for a filter, propose it as
336
+ [an issue](https://github.com/gjtorikian/html-pipeline/issues) first. This allows us to discuss
337
+ whether the filter is a common enough use case to belong in this gem, or should be
338
+ built as an external gem.
339
+
340
+ Here are some extensions people have built:
341
+
342
+ - [html-pipeline-asciidoc_filter](https://github.com/asciidoctor/html-pipeline-asciidoc_filter)
343
+ - [jekyll-html-pipeline](https://github.com/gjtorikian/jekyll-html-pipeline)
344
+ - [nanoc-html-pipeline](https://github.com/burnto/nanoc-html-pipeline)
345
+ - [html-pipeline-bitly](https://github.com/dewski/html-pipeline-bitly)
346
+ - [html-pipeline-cite](https://github.com/lifted-studios/html-pipeline-cite)
347
+ - [tilt-html-pipeline](https://github.com/bradgessler/tilt-html-pipeline)
348
+ - [html-pipeline-wiki-link'](https://github.com/lifted-studios/html-pipeline-wiki-link) - WikiMedia-style wiki links
349
+ - [task_list](https://github.com/github/task_list) - GitHub flavor Markdown Task List
350
+ - [html-pipeline-nico_link](https://github.com/rutan/html-pipeline-nico_link) - An HTMLPipeline filter for [niconico](http://www.nicovideo.jp) description links
351
+ - [html-pipeline-gitlab](https://gitlab.com/gitlab-org/html-pipeline-gitlab) - This gem implements various filters for html-pipeline used by GitLab
352
+ - [html-pipeline-youtube](https://github.com/st0012/html-pipeline-youtube) - An HTMLPipeline filter for YouTube links
353
+ - [html-pipeline-flickr](https://github.com/st0012/html-pipeline-flickr) - An HTMLPipeline filter for Flickr links
354
+ - [html-pipeline-vimeo](https://github.com/dlackty/html-pipeline-vimeo) - An HTMLPipeline filter for Vimeo links
355
+ - [html-pipeline-hashtag](https://github.com/mr-dxdy/html-pipeline-hashtag) - An HTMLPipeline filter for hashtags
356
+ - [html-pipeline-linkify_github](https://github.com/jollygoodcode/html-pipeline-linkify_github) - An HTMLPipeline filter to autolink GitHub urls
357
+ - [html-pipeline-redcarpet_filter](https://github.com/bmikol/html-pipeline-redcarpet_filter) - Render Markdown source text into Markdown HTML using Redcarpet
358
+ - [html-pipeline-typogruby_filter](https://github.com/bmikol/html-pipeline-typogruby_filter) - Add Typogruby text filters to your HTMLPipeline
359
+ - [korgi](https://github.com/jodeci/korgi) - HTMLPipeline filters for links to Rails resources
360
+
310
361
  ## FAQ
311
362
 
312
363
  ### 1. Why doesn't my pipeline work when there's no root element in the document?
313
364
 
314
365
  To make a pipeline work on a plain text document, put the `PlainTextInputFilter`
315
- at the beginning of your pipeline. This will wrap the content in a `div` so the
316
- filters have a root element to work with. If you're passing in an HTML fragment,
366
+ at the end of your `text_filter`s config . This will wrap the content in a `div` so the filters have a root element to work with. If you're passing in an HTML fragment,
317
367
  but it doesn't have a root element, you can wrap the content in a `div`
318
- yourself. For example:
319
-
320
- ```ruby
321
- EmojiPipeline = Pipeline.new [
322
- PlainTextInputFilter, # <- Wraps input in a div and escapes html tags
323
- EmojiFilter
324
- ], context
325
-
326
- plain_text = "Gutentag! :wave:"
327
- EmojiPipeline.call(plain_text)
328
-
329
- html_fragment = "This is outside of an html element, but <strong>this isn't. :+1:</strong>"
330
- EmojiPipeline.call("<div>#{html_fragment}</div>") # <- Wrap your own html fragments to avoid escaping
331
- ```
368
+ yourself.
332
369
 
333
370
  ### 2. How do I customize an allowlist for `SanitizationFilter`s?
334
371
 
335
- `SanitizationFilter::ALLOWLIST` is the default allowlist used if no `:allowlist`
336
- argument is given in the context. The default is a good starting template for
372
+ `HTMLPipeline::SanitizationFilter::ALLOWLIST` is the default allowlist used if no `sanitization_config`
373
+ argument is given. The default is a good starting template for
337
374
  you to add additional elements. You can either modify the constant's value, or
338
- re-define your own constant and pass that in via the context.
339
-
340
- ## Contributing
375
+ re-define your own config and pass that in, such as:
341
376
 
342
- Please review the [Contributing Guide](https://github.com/jch/html-pipeline/blob/master/CONTRIBUTING.md).
343
-
344
- 1. [Fork it](https://help.github.com/articles/fork-a-repo)
345
- 2. Create your feature branch (`git checkout -b my-new-feature`)
346
- 3. Commit your changes (`git commit -am 'Added some feature'`)
347
- 4. Push to the branch (`git push origin my-new-feature`)
348
- 5. Create new [Pull Request](https://help.github.com/articles/using-pull-requests)
349
-
350
- To see what has changed in recent versions, see the [CHANGELOG](https://github.com/jch/html-pipeline/blob/master/CHANGELOG.md).
377
+ ```ruby
378
+ config = HTMLPipeline::SanitizerFilter::DEFAULT_CONFIG.dup
379
+ config[:elements] << "iframe" # sure, whatever you want
380
+ ```
351
381
 
352
382
  ### Contributors
353
383
 
354
- Thanks to all of [these contributors](https://github.com/jch/html-pipeline/graphs/contributors).
355
-
356
- Project is a member of the [OSS Manifesto](http://ossmanifesto.org/).
357
-
358
- The current maintainer is @gjtorikian
359
-
360
- ### Releasing A New Version
361
-
362
- This section is for gem maintainers to cut a new version of the gem.
384
+ Thanks to all of [these contributors](https://github.com/gjtorikian/html-pipeline/graphs/contributors).
363
385
 
364
- * create a new branch named `release-x.y.z` where `x.y.z` follows [semver](http://semver.org)
365
- * update lib/html/pipeline/version.rb to next version number X.X.X
366
- * update CHANGELOG.md. Prepare a draft with `script/changelog`
367
- * push branch and create a new pull request
368
- * after tests are green, merge to master
369
- * on the master branch, run `script/release`
386
+ This project is a member of the [OSS Manifesto](http://ossmanifesto.org/).
data/Rakefile CHANGED
@@ -1,17 +1,24 @@
1
1
  #!/usr/bin/env rake
2
2
  # frozen_string_literal: true
3
3
 
4
- require 'rubygems'
5
- require 'bundler/setup'
6
-
7
- require 'bundler/gem_tasks'
8
- require 'rake/testtask'
4
+ require "bundler/gem_tasks"
5
+ require "rubygems/package_task"
6
+ require "rake/testtask"
9
7
 
10
8
  Rake::TestTask.new do |t|
11
- t.libs << 'test'
12
- t.test_files = FileList['test/**/*_test.rb']
9
+ t.libs << "test"
10
+ t.test_files = FileList["test/**/*_test.rb"]
13
11
  t.verbose = true
14
12
  t.warning = false
15
13
  end
16
14
 
17
15
  task default: :test
16
+
17
+ require "rubocop/rake_task"
18
+
19
+ RuboCop::RakeTask.new(:rubocop)
20
+
21
+ GEMSPEC = Bundler.load_gemspec("html-pipeline.gemspec")
22
+ gem_path = Gem::PackageTask.new(GEMSPEC).define
23
+ desc "Package the ruby gem"
24
+ task "package" => [gem_path]