html-pipeline 2.14.3 → 3.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (59) hide show
  1. checksums.yaml +4 -4
  2. data/.github/FUNDING.yml +11 -3
  3. data/.github/dependabot.yml +27 -0
  4. data/.github/workflows/automerge.yml +13 -0
  5. data/.github/workflows/ci.yml +22 -0
  6. data/.github/workflows/lint.yml +23 -0
  7. data/.github/workflows/publish.yml +19 -0
  8. data/.rubocop.yml +17 -0
  9. data/.ruby-version +1 -0
  10. data/.vscode/settings.json +8 -0
  11. data/CHANGELOG.md +119 -2
  12. data/Gemfile +31 -15
  13. data/{LICENSE → LICENSE.txt} +2 -2
  14. data/README.md +241 -224
  15. data/Rakefile +14 -7
  16. data/UPGRADING.md +34 -0
  17. data/html-pipeline.gemspec +31 -21
  18. data/lib/html-pipeline.rb +3 -0
  19. data/lib/html_pipeline/convert_filter/markdown_filter.rb +26 -0
  20. data/lib/html_pipeline/convert_filter.rb +17 -0
  21. data/lib/html_pipeline/filter.rb +89 -0
  22. data/lib/html_pipeline/node_filter/absolute_source_filter.rb +54 -0
  23. data/lib/html_pipeline/node_filter/asset_proxy_filter.rb +86 -0
  24. data/lib/{html/pipeline → html_pipeline/node_filter}/emoji_filter.rb +58 -54
  25. data/lib/html_pipeline/node_filter/https_filter.rb +22 -0
  26. data/lib/html_pipeline/node_filter/image_max_width_filter.rb +40 -0
  27. data/lib/{html/pipeline/@mention_filter.rb → html_pipeline/node_filter/mention_filter.rb} +54 -68
  28. data/lib/html_pipeline/node_filter/syntax_highlight_filter.rb +62 -0
  29. data/lib/html_pipeline/node_filter/table_of_contents_filter.rb +70 -0
  30. data/lib/html_pipeline/node_filter/team_mention_filter.rb +105 -0
  31. data/lib/html_pipeline/node_filter.rb +31 -0
  32. data/lib/html_pipeline/sanitization_filter.rb +188 -0
  33. data/lib/{html/pipeline → html_pipeline/text_filter}/image_filter.rb +3 -3
  34. data/lib/{html/pipeline → html_pipeline/text_filter}/plain_text_input_filter.rb +3 -5
  35. data/lib/html_pipeline/text_filter.rb +21 -0
  36. data/lib/html_pipeline/version.rb +5 -0
  37. data/lib/html_pipeline.rb +281 -0
  38. metadata +58 -54
  39. data/.travis.yml +0 -43
  40. data/Appraisals +0 -19
  41. data/CONTRIBUTING.md +0 -60
  42. data/bin/html-pipeline +0 -78
  43. data/lib/html/pipeline/@team_mention_filter.rb +0 -99
  44. data/lib/html/pipeline/absolute_source_filter.rb +0 -52
  45. data/lib/html/pipeline/autolink_filter.rb +0 -34
  46. data/lib/html/pipeline/body_content.rb +0 -44
  47. data/lib/html/pipeline/camo_filter.rb +0 -105
  48. data/lib/html/pipeline/email_reply_filter.rb +0 -69
  49. data/lib/html/pipeline/filter.rb +0 -165
  50. data/lib/html/pipeline/https_filter.rb +0 -29
  51. data/lib/html/pipeline/image_max_width_filter.rb +0 -37
  52. data/lib/html/pipeline/markdown_filter.rb +0 -56
  53. data/lib/html/pipeline/sanitization_filter.rb +0 -144
  54. data/lib/html/pipeline/syntax_highlight_filter.rb +0 -50
  55. data/lib/html/pipeline/text_filter.rb +0 -16
  56. data/lib/html/pipeline/textile_filter.rb +0 -25
  57. data/lib/html/pipeline/toc_filter.rb +0 -69
  58. data/lib/html/pipeline/version.rb +0 -7
  59. data/lib/html/pipeline.rb +0 -210
data/README.md CHANGED
@@ -1,23 +1,31 @@
1
- # HTML::Pipeline [![Build Status](https://travis-ci.org/jch/html-pipeline.svg?branch=master)](https://travis-ci.org/jch/html-pipeline)
1
+ # HTML-Pipeline
2
2
 
3
- HTML processing filters and utilities. This module includes a small
4
- framework for defining DOM based content filters and applying them to user
3
+ > **Note**
4
+ > This README refers to the behavior in the new 3.0.0.pre gem.
5
+
6
+ HTML processing filters and utilities. This module is a small
7
+ framework for defining CSS-based content filters and applying them to user
5
8
  provided content.
6
9
 
7
- [This project was started at GitHub](https://github.com/blog/1311-html-pipeline-chainable-content-filters). While GitHub still uses a similar design and pattern for rendering content, this gem should be considered standalone and independent from GitHub.
8
-
9
- - [Installation](#installation)
10
- - [Usage](#usage)
11
- - [Examples](#examples)
12
- - [Filters](#filters)
13
- - [Dependencies](#dependencies)
14
- - [Documentation](#documentation)
15
- - [Extending](#extending)
16
- - [3rd Party Extensions](#3rd-party-extensions)
17
- - [Instrumenting](#instrumenting)
18
- - [Contributing](#contributing)
19
- - [Contributors](#contributors)
20
- - [Releasing A New Version](#releasing-a-new-version)
10
+ [Although this project was started at GitHub](https://github.com/blog/1311-html-pipeline-chainable-content-filters), they no longer use it. This gem must be considered standalone and independent from GitHub.
11
+
12
+ - [HTML-Pipeline](#html-pipeline)
13
+ - [Installation](#installation)
14
+ - [Usage](#usage)
15
+ - [More Examples](#more-examples)
16
+ - [Filters](#filters)
17
+ - [TextFilters](#textfilters)
18
+ - [ConvertFilter](#convertfilter)
19
+ - [Sanitization](#sanitization)
20
+ - [NodeFilters](#nodefilters)
21
+ - [Dependencies](#dependencies)
22
+ - [Documentation](#documentation)
23
+ - [Instrumenting](#instrumenting)
24
+ - [Third Party Extensions](#third-party-extensions)
25
+ - [FAQ](#faq)
26
+ - [1. Why doesn't my pipeline work when there's no root element in the document?](#1-why-doesnt-my-pipeline-work-when-theres-no-root-element-in-the-document)
27
+ - [2. How do I customize an allowlist for `SanitizationFilter`s?](#2-how-do-i-customize-an-allowlist-for-sanitizationfilters)
28
+ - [Contributors](#contributors)
21
29
 
22
30
  ## Installation
23
31
 
@@ -42,220 +50,234 @@ $ gem install html-pipeline
42
50
  ## Usage
43
51
 
44
52
  This library provides a handful of chainable HTML filters to transform user
45
- content into markup. A filter takes an HTML string or
46
- `Nokogiri::HTML::DocumentFragment`, optionally manipulates it, and then
47
- outputs the result.
53
+ content into HTML markup. Each filter does some work, and then hands off the
54
+ results tothe next filter. A pipeline has several kinds of filters available to use:
48
55
 
49
- For example, to transform Markdown source into Markdown HTML:
56
+ - Multiple `TextFilter`s, which operate a UTF-8 string
57
+ - A `ConvertFilter` filter, which turns text into HTML (eg., Commonmark/Asciidoc -> HTML)
58
+ - A `SanitizationFilter`, which remove dangerous/unwanted HTML elements and attributes
59
+ - Multiple `NodeFilter`s, which operate on a UTF-8 HTML document
50
60
 
51
- ```ruby
52
- require 'html/pipeline'
61
+ You can assemble each sequence into a single pipeline, or choose to call each filter individually.
53
62
 
54
- filter = HTML::Pipeline::MarkdownFilter.new("Hi **world**!")
55
- filter.call
56
- ```
63
+ As an example, suppose we want to transform Commonmark source text into Markdown HTML. With the content, we also want to:
57
64
 
58
- Filters can be combined into a pipeline which causes each filter to hand its
59
- output to the next filter's input. So if you wanted to have content be
60
- filtered through Markdown and be syntax highlighted, you can create the
61
- following pipeline:
65
+ - change every instance of `$NAME` to "`Johnny"
66
+ - strip undesired HTML
67
+ - linkify @mention
68
+
69
+ We can construct a pipeline to do all that like this:
62
70
 
63
71
  ```ruby
64
- pipeline = HTML::Pipeline.new [
65
- HTML::Pipeline::MarkdownFilter,
66
- HTML::Pipeline::SyntaxHighlightFilter
67
- ]
68
- result = pipeline.call <<-CODE
69
- This is *great*:
72
+ require 'html_pipeline'
70
73
 
71
- some_code(:first)
74
+ class HelloJohnnyFilter < HTMLPipelineFilter
75
+ def call
76
+ text.gsub("$NAME", "Johnny")
77
+ end
78
+ end
72
79
 
73
- CODE
74
- result[:output].to_s
80
+ pipeline = HTMLPipeline.new(
81
+ text_filters: [HelloJohnnyFilter.new]
82
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
83
+ # note: next line is not needed as sanitization occurs by default;
84
+ # see below for more info
85
+ sanitization_config: HTMLPipeline::SanitizationFilter::DEFAULT_CONFIG,
86
+ node_filters: [HTMLPipeline::NodeFilter::MentionFilter.new]
87
+ )
88
+ pipeline.call(user_supplied_text) # recommended: can call pipeline over and over
75
89
  ```
76
90
 
77
- Prints:
91
+ Filters can be custom ones you create (like `HelloJohnnyFilter`), and `HTMLPipeline` additionally provides several helpful ones (detailed below). If you only need a single filter, you can call one individually, too:
78
92
 
79
- ```html
80
- <p>This is <em>great</em>:</p>
81
-
82
- <pre><code>some_code(:first)
83
- </code></pre>
93
+ ```ruby
94
+ filter = HTMLPipeline::ConvertFilter::MarkdownFilter.new
95
+ filter.call(text)
84
96
  ```
85
97
 
86
- To generate CSS for HTML formatted code, use the [Rouge CSS Theme](https://github.com/rouge-ruby/rouge#css-options) `#css` method. `rouge` is a dependency of the `SyntaxHighlightFilter`.
98
+ Filters combine into a sequential pipeline, and each filter hands its
99
+ output to the next filter's input. Text filters are
100
+ processed first, then the convert filter, sanitization filter, and finally, the node filters.
87
101
 
88
- Some filters take an optional **context** and/or **result** hash. These are
102
+ Some filters take optional `context` and/or `result` hash(es). These are
89
103
  used to pass around arguments and metadata between filters in a pipeline. For
90
- example, if you don't want to use GitHub formatted Markdown, you can pass an
91
- option in the context hash:
104
+ example, if you want to disable footnotes in the `MarkdownFilter`, you can pass an option in the context hash:
92
105
 
93
106
  ```ruby
94
- filter = HTML::Pipeline::MarkdownFilter.new("Hi **world**!", :gfm => false)
95
- filter.call
107
+ context = { markdown: { extensions: { footnotes: false } } }
108
+ filter = HTMLPipeline::ConvertFilter::MarkdownFilter.new(context: context)
109
+ filter.call("Hi **world**!")
96
110
  ```
97
111
 
98
- ### Examples
112
+ Please refer to the documentation for each filter to understand what configuration options are available.
113
+
114
+ ### More Examples
99
115
 
100
- We define different pipelines for different parts of our app. Here are a few
116
+ Different pipelines can be defined for different parts of an app. Here are a few
101
117
  paraphrased snippets to get you started:
102
118
 
103
119
  ```ruby
104
120
  # The context hash is how you pass options between different filters.
105
121
  # See individual filter source for explanation of options.
106
122
  context = {
107
- :asset_root => "http://your-domain.com/where/your/images/live/icons",
108
- :base_url => "http://your-domain.com"
123
+ asset_root: "http://your-domain.com/where/your/images/live/icons",
124
+ base_url: "http://your-domain.com"
109
125
  }
110
126
 
111
- # Pipeline providing sanitization and image hijacking but no mention
112
- # related features.
113
- SimplePipeline = Pipeline.new [
114
- SanitizationFilter,
115
- TableOfContentsFilter, # add 'name' anchors to all headers and generate toc list
116
- CamoFilter,
117
- ImageMaxWidthFilter,
118
- SyntaxHighlightFilter,
119
- EmojiFilter,
120
- AutolinkFilter
121
- ], context
122
-
123
127
  # Pipeline used for user provided content on the web
124
- MarkdownPipeline = Pipeline.new [
125
- MarkdownFilter,
126
- SanitizationFilter,
127
- CamoFilter,
128
- ImageMaxWidthFilter,
129
- HttpsFilter,
130
- MentionFilter,
131
- EmojiFilter,
132
- SyntaxHighlightFilter
133
- ], context.merge(:gfm => true) # enable github formatted markdown
134
-
135
-
136
- # Define a pipeline based on another pipeline's filters
137
- NonGFMMarkdownPipeline = Pipeline.new(MarkdownPipeline.filters,
138
- context.merge(:gfm => false))
128
+ MarkdownPipeline = HTMLPipeline.new (
129
+ text_filters: [HTMLPipeline::TextFilter::ImageFilter.new],
130
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
131
+ node_filters: [
132
+ HTMLPipeline::NodeFilter::HttpsFilter.new,HTMLPipeline::NodeFilter::MentionFilter.new,
133
+ ], context: context)
139
134
 
140
135
  # Pipelines aren't limited to the web. You can use them for email
141
136
  # processing also.
142
- HtmlEmailPipeline = Pipeline.new [
143
- PlainTextInputFilter,
144
- ImageMaxWidthFilter
145
- ], {}
146
-
147
- # Just emoji.
148
- EmojiPipeline = Pipeline.new [
149
- PlainTextInputFilter,
150
- EmojiFilter
151
- ], context
137
+ HtmlEmailPipeline = HTMLPipeline.new(
138
+ text_filters: [
139
+ PlainTextInputFilter.new,
140
+ ImageFilter.new
141
+ ], {})
152
142
  ```
153
143
 
154
144
  ## Filters
155
145
 
156
- * `MentionFilter` - replace `@user` mentions with links
157
- * `TeamMentionFilter` - replace `@org/team` mentions with links
158
- * `AbsoluteSourceFilter` - replace relative image urls with fully qualified versions
159
- * `AutolinkFilter` - auto_linking urls in HTML
160
- * `CamoFilter` - replace http image urls with [camo-fied](https://github.com/atmos/camo) https versions
161
- * `EmailReplyFilter` - util filter for working with emails
162
- * `EmojiFilter` - everyone loves [emoji](http://www.emoji-cheat-sheet.com/)!
163
- * `HttpsFilter` - HTML Filter for replacing http github urls with https versions.
164
- * `ImageMaxWidthFilter` - link to full size image for large images
165
- * `MarkdownFilter` - convert markdown to html
166
- * `PlainTextInputFilter` - html escape text and wrap the result in a div
167
- * `SanitizationFilter` - allow sanitize user markup
168
- * `SyntaxHighlightFilter` - code syntax highlighter
169
- * `TextileFilter` - convert textile to html
170
- * `TableOfContentsFilter` - anchor headings with name attributes and generate Table of Contents html unordered list linking headings
146
+ ### TextFilters
171
147
 
172
- ## Dependencies
148
+ `TextFilter`s must define a method named `call` which is called on the text. `@text`, `@config`, and `@result` are available to use, and any changes made to these ivars are passed on to the next filter.
149
+
150
+ - `ImageFilter` - converts image `url` into `<img>` tag
151
+ - `PlainTextInputFilter` - html escape text and wrap the result in a `<div>`
152
+
153
+ ### ConvertFilter
154
+
155
+ The `ConvertFilter` takes text and turns it into HTML. `@text`, `@config`, and `@result` are available to use. `ConvertFilter` must defined a method named `call`, taking one argument, `text`. `call` must return a string representing the new HTML document.
173
156
 
174
- Filter gem dependencies are not bundled; you must bundle the filter's gem
175
- dependencies. The below list details filters with dependencies. For example,
176
- `SyntaxHighlightFilter` uses [rouge](https://github.com/jneen/rouge)
177
- to detect and highlight languages. For example, to use the `SyntaxHighlightFilter`,
178
- add the following to your Gemfile:
157
+ - `MarkdownFilter` - creates HTML from text using [Commonmarker](https://www.github.com/gjtorikian/commonmarker)
158
+
159
+ ### Sanitization
160
+
161
+ Because the web can be a scary place, HTML is automatically sanitized after the `ConvertFilter` runs and before the `NodeFilter`s are processed. This is to prevent malicious or unexpected input from entering the pipeline.
162
+
163
+ The sanitization process takes a hash configuration of settings. See the [Selma](https://www.github.com/gjtorikian/selma) documentation for more information on how to configure these settings.
164
+
165
+ A default sanitization config is provided by this library (`HTMLPipeline::SanitizationFilter::DEFAULT_CONFIG`). A sample custom sanitization allowlist might look like this:
179
166
 
180
167
  ```ruby
181
- gem 'rouge'
168
+ ALLOWLIST = {
169
+ elements: ["p", "pre", "code"]
170
+ }
171
+
172
+ pipeline = HTMLPipeline.new \
173
+ text_filters: [
174
+ HTMLPipeline::MarkdownFilter,
175
+ ],
176
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
177
+ sanitization_config: ALLOWLIST
178
+
179
+ result = pipeline.call <<-CODE
180
+ This is *great*:
181
+
182
+ some_code(:first)
183
+
184
+ CODE
185
+ result[:output].to_s
182
186
  ```
183
187
 
184
- * `AutolinkFilter` - `rinku`
185
- * `EmailReplyFilter` - `escape_utils`, `email_reply_parser`
186
- * `EmojiFilter` - `gemoji`
187
- * `MarkdownFilter` - `commonmarker`
188
- * `PlainTextInputFilter` - `escape_utils`
189
- * `SanitizationFilter` - `sanitize`
190
- * `SyntaxHighlightFilter` - `rouge`
191
- * `TableOfContentsFilter` - `escape_utils`
192
- * `TextileFilter` - `RedCloth`
188
+ This would print:
193
189
 
194
- _Note:_ See [Gemfile](/Gemfile) `:test` block for version requirements.
190
+ ```html
191
+ <p>This is great:</p>
192
+ <pre><code>some_code(:first)
193
+ </code></pre>
194
+ ```
195
195
 
196
- ## Documentation
196
+ Sanitization can be disabled if and only if `nil` is explicitly passed as
197
+ the config:
197
198
 
198
- Full reference documentation can be [found here](http://rubydoc.info/gems/html-pipeline/frames).
199
+ ```ruby
200
+ pipeline = HTMLPipeline.new \
201
+ text_filters: [
202
+ HTMLPipeline::MarkdownFilter,
203
+ ],
204
+ convert_filter: HTMLPipeline::ConvertFilter::MarkdownFilter.new,
205
+ sanitization_config: nil
206
+ ```
207
+
208
+ For more examples of customizing the sanitization process to include the tags you want, check out [the tests](test/sanitization_filter_test.rb) and [the FAQ](#faq).
199
209
 
200
- ## Extending
201
- To write a custom filter, you need a class with a `call` method that inherits
202
- from `HTML::Pipeline::Filter`.
210
+ ### NodeFilters
203
211
 
204
- For example this filter adds a base url to images that are root relative:
212
+ `NodeFilters`s can operate either on HTML elements or text nodes using CSS selectors. Each `NodeFilter` must define a method named `selector` which provides an instance of `Selma::Selector`. If elements are being manipulated, `handle_element` must be defined, taking one argument, `element`; if text nodes are being manipulated, `handle_text_chunk` must be defined, taking one argument, `text_chunk`. `@config`, and `@result` are available to use, and any changes made to these ivars are passed on to the next filter.
213
+
214
+ `NodeFilter` also has an optional method, `after_initialize`, which is run after the filter initializes. This can be useful in setting up a custom state for `result` to take advantage of.
215
+
216
+ Here's an example `NodeFilter` that adds a base url to images that are root relative:
205
217
 
206
218
  ```ruby
207
219
  require 'uri'
208
220
 
209
- class RootRelativeFilter < HTML::Pipeline::Filter
221
+ class RootRelativeFilter < HTMLPipeline::NodeFilter
210
222
 
211
- def call
212
- doc.search("img").each do |img|
213
- next if img['src'].nil?
214
- src = img['src'].strip
215
- if src.start_with? '/'
216
- img["src"] = URI.join(context[:base_url], src).to_s
217
- end
218
- end
219
- doc
223
+ SELECTOR = Selma::Selector.new(match_element: "img")
224
+
225
+ def selector
226
+ SELECTOR
220
227
  end
221
228
 
229
+ def handle_element(img)
230
+ next if img['src'].nil?
231
+ src = img['src'].strip
232
+ if src.start_with? '/'
233
+ img["src"] = URI.join(context[:base_url], src).to_s
234
+ end
235
+ end
222
236
  end
223
237
  ```
224
238
 
225
- Now this filter can be used in a pipeline:
239
+ For more information on how to write effective `NodeFilter`s, refer to the provided filters, and see the underlying lib, [Selma](https://www.github.com/gjtorikian/selma) for more information.
240
+
241
+ - `AbsoluteSourceFilter`: replace relative image urls with fully qualified versions
242
+ - `AssetProxyFilter`: replace image links with an encoded link to an asset server
243
+ - `EmojiFilter`: converts `:<emoji>:` to [emoji](http://www.emoji-cheat-sheet.com/)
244
+ - (Note: the included `MarkdownFilter` will already convert emoji)
245
+ - `HttpsFilter`: Replacing http urls with https versions
246
+ - `ImageMaxWidthFilter`: link to full size image for large images
247
+ - `MentionFilter`: replace `@user` mentions with links
248
+ - `SanitizationFilter`: allow sanitize user markup
249
+ - `SyntaxHighlightFilter`: applies syntax highlighting to `pre` blocks
250
+ - (Note: the included `MarkdownFilter` will already apply highlighting)
251
+ - `TableOfContentsFilter`: anchor headings with name attributes and generate Table of Contents html unordered list linking headings
252
+ - `TeamMentionFilter`: replace `@org/team` mentions with links
253
+
254
+ ## Dependencies
255
+
256
+ Since filters can be customized to your heart's content, gem dependencies are _not_ bundled; this project doesn't know which of the default filters you might use, and as such, you must bundle each filter's gem dependencies yourself.
257
+
258
+ For example, `SyntaxHighlightFilter` uses [rouge](https://github.com/jneen/rouge)
259
+ to detect and highlight languages; to use the `SyntaxHighlightFilter`, you must add the following to your Gemfile:
226
260
 
227
261
  ```ruby
228
- Pipeline.new [ RootRelativeFilter ], { :base_url => 'http://somehost.com' }
262
+ gem "rouge"
229
263
  ```
230
264
 
231
- ### 3rd Party Extensions
265
+ > **Note**
266
+ > See the [Gemfile](/Gemfile) `:test` group for any version requirements.
232
267
 
233
- If you have an idea for a filter, propose it as
234
- [an issue](https://github.com/jch/html-pipeline/issues) first. This allows us discuss
235
- whether the filter is a common enough use case to belong in this gem, or should be
236
- built as an external gem.
268
+ When developing a custom filter, call `HTMLPipeline.require_dependency` at the start to ensure that the local machine has the necessary dependency. You can also use `HTMLPipeline.require_dependencies` to provide a list of dependencies to check.
237
269
 
238
- Here are some extensions people have built:
270
+ On a similar note, you must manually require whichever filters you desire:
239
271
 
240
- * [html-pipeline-asciidoc_filter](https://github.com/asciidoctor/html-pipeline-asciidoc_filter)
241
- * [jekyll-html-pipeline](https://github.com/gjtorikian/jekyll-html-pipeline)
242
- * [nanoc-html-pipeline](https://github.com/burnto/nanoc-html-pipeline)
243
- * [html-pipeline-bitly](https://github.com/dewski/html-pipeline-bitly)
244
- * [html-pipeline-cite](https://github.com/lifted-studios/html-pipeline-cite)
245
- * [tilt-html-pipeline](https://github.com/bradgessler/tilt-html-pipeline)
246
- * [html-pipeline-wiki-link'](https://github.com/lifted-studios/html-pipeline-wiki-link) - WikiMedia-style wiki links
247
- * [task_list](https://github.com/github/task_list) - GitHub flavor Markdown Task List
248
- * [html-pipeline-nico_link](https://github.com/rutan/html-pipeline-nico_link) - An HTML::Pipeline filter for [niconico](http://www.nicovideo.jp) description links
249
- * [html-pipeline-gitlab](https://gitlab.com/gitlab-org/html-pipeline-gitlab) - This gem implements various filters for html-pipeline used by GitLab
250
- * [html-pipeline-youtube](https://github.com/st0012/html-pipeline-youtube) - An HTML::Pipeline filter for YouTube links
251
- * [html-pipeline-flickr](https://github.com/st0012/html-pipeline-flickr) - An HTML::Pipeline filter for Flickr links
252
- * [html-pipeline-vimeo](https://github.com/dlackty/html-pipeline-vimeo) - An HTML::Pipeline filter for Vimeo links
253
- * [html-pipeline-hashtag](https://github.com/mr-dxdy/html-pipeline-hashtag) - An HTML::Pipeline filter for hashtags
254
- * [html-pipeline-linkify_github](https://github.com/jollygoodcode/html-pipeline-linkify_github) - An HTML::Pipeline filter to autolink GitHub urls
255
- * [html-pipeline-redcarpet_filter](https://github.com/bmikol/html-pipeline-redcarpet_filter) - Render Markdown source text into Markdown HTML using Redcarpet
256
- * [html-pipeline-typogruby_filter](https://github.com/bmikol/html-pipeline-typogruby_filter) - Add Typogruby text filters to your HTML::Pipeline
257
- * [korgi](https://github.com/jodeci/korgi) - HTML::Pipeline filters for links to Rails resources
272
+ ```ruby
273
+ require "html_pipeline" # must be included
274
+ require "html_pipeline/convert_filter/markdown_filter" # included because you want to use this filter
275
+ require "html_pipeline/node_filter/mention_filter" # included because you want to use this filter
276
+ ```
258
277
 
278
+ ## Documentation
279
+
280
+ Full reference documentation can be [found here](http://rubydoc.info/gems/html-pipeline/frames).
259
281
 
260
282
  ## Instrumenting
261
283
 
@@ -263,107 +285,102 @@ Filters and Pipelines can be set up to be instrumented when called. The pipeline
263
285
  must be setup with an
264
286
  [ActiveSupport::Notifications](http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html)
265
287
  compatible service object and a name. New pipeline objects will default to the
266
- `HTML::Pipeline.default_instrumentation_service` object.
288
+ `HTMLPipeline.default_instrumentation_service` object.
267
289
 
268
- ``` ruby
290
+ ```ruby
269
291
  # the AS::Notifications-compatible service object
270
292
  service = ActiveSupport::Notifications
271
293
 
272
294
  # instrument a specific pipeline
273
- pipeline = HTML::Pipeline.new [MarkdownFilter], context
295
+ pipeline = HTMLPipeline.new [MarkdownFilter], context
274
296
  pipeline.setup_instrumentation "MarkdownPipeline", service
275
297
 
276
298
  # or set default instrumentation service for all new pipelines
277
- HTML::Pipeline.default_instrumentation_service = service
278
- pipeline = HTML::Pipeline.new [MarkdownFilter], context
299
+ HTMLPipeline.default_instrumentation_service = service
300
+ pipeline = HTMLPipeline.new [MarkdownFilter], context
279
301
  pipeline.setup_instrumentation "MarkdownPipeline"
280
302
  ```
281
303
 
282
304
  Filters are instrumented when they are run through the pipeline. A
283
- `call_filter.html_pipeline` event is published once the filter finishes. The
284
- `payload` should include the `filter` name. Each filter will trigger its own
305
+ `call_filter.html_pipeline` event is published once any filter finishes; `call_text_filters`
306
+ and `call_node_filters` is published when all of the text and node filters are finished, respectively.
307
+ The `payload` should include the `filter` name. Each filter will trigger its own
285
308
  instrumentation call.
286
309
 
287
- ``` ruby
310
+ ```ruby
288
311
  service.subscribe "call_filter.html_pipeline" do |event, start, ending, transaction_id, payload|
289
312
  payload[:pipeline] #=> "MarkdownPipeline", set with `setup_instrumentation`
290
313
  payload[:filter] #=> "MarkdownFilter"
291
314
  payload[:context] #=> context Hash
292
315
  payload[:result] #=> instance of result class
293
- payload[:result][:output] #=> output HTML String or Nokogiri::DocumentFragment
316
+ payload[:result][:output] #=> output HTML String
294
317
  end
295
318
  ```
296
319
 
297
320
  The full pipeline is also instrumented:
298
321
 
299
- ``` ruby
300
- service.subscribe "call_pipeline.html_pipeline" do |event, start, ending, transaction_id, payload|
322
+ ```ruby
323
+ service.subscribe "call_text_filters.html_pipeline" do |event, start, ending, transaction_id, payload|
301
324
  payload[:pipeline] #=> "MarkdownPipeline", set with `setup_instrumentation`
302
325
  payload[:filters] #=> ["MarkdownFilter"]
303
- payload[:doc] #=> HTML String or Nokogiri::DocumentFragment
326
+ payload[:doc] #=> HTML String
304
327
  payload[:context] #=> context Hash
305
328
  payload[:result] #=> instance of result class
306
- payload[:result][:output] #=> output HTML String or Nokogiri::DocumentFragment
329
+ payload[:result][:output] #=> output HTML String
307
330
  end
308
331
  ```
309
332
 
333
+ ## Third Party Extensions
334
+
335
+ If you have an idea for a filter, propose it as
336
+ [an issue](https://github.com/gjtorikian/html-pipeline/issues) first. This allows us to discuss
337
+ whether the filter is a common enough use case to belong in this gem, or should be
338
+ built as an external gem.
339
+
340
+ Here are some extensions people have built:
341
+
342
+ - [html-pipeline-asciidoc_filter](https://github.com/asciidoctor/html-pipeline-asciidoc_filter)
343
+ - [jekyll-html-pipeline](https://github.com/gjtorikian/jekyll-html-pipeline)
344
+ - [nanoc-html-pipeline](https://github.com/burnto/nanoc-html-pipeline)
345
+ - [html-pipeline-bitly](https://github.com/dewski/html-pipeline-bitly)
346
+ - [html-pipeline-cite](https://github.com/lifted-studios/html-pipeline-cite)
347
+ - [tilt-html-pipeline](https://github.com/bradgessler/tilt-html-pipeline)
348
+ - [html-pipeline-wiki-link'](https://github.com/lifted-studios/html-pipeline-wiki-link) - WikiMedia-style wiki links
349
+ - [task_list](https://github.com/github/task_list) - GitHub flavor Markdown Task List
350
+ - [html-pipeline-nico_link](https://github.com/rutan/html-pipeline-nico_link) - An HTMLPipeline filter for [niconico](http://www.nicovideo.jp) description links
351
+ - [html-pipeline-gitlab](https://gitlab.com/gitlab-org/html-pipeline-gitlab) - This gem implements various filters for html-pipeline used by GitLab
352
+ - [html-pipeline-youtube](https://github.com/st0012/html-pipeline-youtube) - An HTMLPipeline filter for YouTube links
353
+ - [html-pipeline-flickr](https://github.com/st0012/html-pipeline-flickr) - An HTMLPipeline filter for Flickr links
354
+ - [html-pipeline-vimeo](https://github.com/dlackty/html-pipeline-vimeo) - An HTMLPipeline filter for Vimeo links
355
+ - [html-pipeline-hashtag](https://github.com/mr-dxdy/html-pipeline-hashtag) - An HTMLPipeline filter for hashtags
356
+ - [html-pipeline-linkify_github](https://github.com/jollygoodcode/html-pipeline-linkify_github) - An HTMLPipeline filter to autolink GitHub urls
357
+ - [html-pipeline-redcarpet_filter](https://github.com/bmikol/html-pipeline-redcarpet_filter) - Render Markdown source text into Markdown HTML using Redcarpet
358
+ - [html-pipeline-typogruby_filter](https://github.com/bmikol/html-pipeline-typogruby_filter) - Add Typogruby text filters to your HTMLPipeline
359
+ - [korgi](https://github.com/jodeci/korgi) - HTMLPipeline filters for links to Rails resources
360
+
310
361
  ## FAQ
311
362
 
312
363
  ### 1. Why doesn't my pipeline work when there's no root element in the document?
313
364
 
314
365
  To make a pipeline work on a plain text document, put the `PlainTextInputFilter`
315
- at the beginning of your pipeline. This will wrap the content in a `div` so the
316
- filters have a root element to work with. If you're passing in an HTML fragment,
366
+ at the end of your `text_filter`s config . This will wrap the content in a `div` so the filters have a root element to work with. If you're passing in an HTML fragment,
317
367
  but it doesn't have a root element, you can wrap the content in a `div`
318
- yourself. For example:
319
-
320
- ```ruby
321
- EmojiPipeline = Pipeline.new [
322
- PlainTextInputFilter, # <- Wraps input in a div and escapes html tags
323
- EmojiFilter
324
- ], context
325
-
326
- plain_text = "Gutentag! :wave:"
327
- EmojiPipeline.call(plain_text)
328
-
329
- html_fragment = "This is outside of an html element, but <strong>this isn't. :+1:</strong>"
330
- EmojiPipeline.call("<div>#{html_fragment}</div>") # <- Wrap your own html fragments to avoid escaping
331
- ```
368
+ yourself.
332
369
 
333
370
  ### 2. How do I customize an allowlist for `SanitizationFilter`s?
334
371
 
335
- `SanitizationFilter::ALLOWLIST` is the default allowlist used if no `:allowlist`
336
- argument is given in the context. The default is a good starting template for
372
+ `HTMLPipeline::SanitizationFilter::ALLOWLIST` is the default allowlist used if no `sanitization_config`
373
+ argument is given. The default is a good starting template for
337
374
  you to add additional elements. You can either modify the constant's value, or
338
- re-define your own constant and pass that in via the context.
339
-
340
- ## Contributing
375
+ re-define your own config and pass that in, such as:
341
376
 
342
- Please review the [Contributing Guide](https://github.com/jch/html-pipeline/blob/master/CONTRIBUTING.md).
343
-
344
- 1. [Fork it](https://help.github.com/articles/fork-a-repo)
345
- 2. Create your feature branch (`git checkout -b my-new-feature`)
346
- 3. Commit your changes (`git commit -am 'Added some feature'`)
347
- 4. Push to the branch (`git push origin my-new-feature`)
348
- 5. Create new [Pull Request](https://help.github.com/articles/using-pull-requests)
349
-
350
- To see what has changed in recent versions, see the [CHANGELOG](https://github.com/jch/html-pipeline/blob/master/CHANGELOG.md).
377
+ ```ruby
378
+ config = HTMLPipeline::SanitizerFilter::DEFAULT_CONFIG.dup
379
+ config[:elements] << "iframe" # sure, whatever you want
380
+ ```
351
381
 
352
382
  ### Contributors
353
383
 
354
- Thanks to all of [these contributors](https://github.com/jch/html-pipeline/graphs/contributors).
355
-
356
- Project is a member of the [OSS Manifesto](http://ossmanifesto.org/).
357
-
358
- The current maintainer is @gjtorikian
359
-
360
- ### Releasing A New Version
361
-
362
- This section is for gem maintainers to cut a new version of the gem.
384
+ Thanks to all of [these contributors](https://github.com/gjtorikian/html-pipeline/graphs/contributors).
363
385
 
364
- * create a new branch named `release-x.y.z` where `x.y.z` follows [semver](http://semver.org)
365
- * update lib/html/pipeline/version.rb to next version number X.X.X
366
- * update CHANGELOG.md. Prepare a draft with `script/changelog`
367
- * push branch and create a new pull request
368
- * after tests are green, merge to master
369
- * on the master branch, run `script/release`
386
+ This project is a member of the [OSS Manifesto](http://ossmanifesto.org/).
data/Rakefile CHANGED
@@ -1,17 +1,24 @@
1
1
  #!/usr/bin/env rake
2
2
  # frozen_string_literal: true
3
3
 
4
- require 'rubygems'
5
- require 'bundler/setup'
6
-
7
- require 'bundler/gem_tasks'
8
- require 'rake/testtask'
4
+ require "bundler/gem_tasks"
5
+ require "rubygems/package_task"
6
+ require "rake/testtask"
9
7
 
10
8
  Rake::TestTask.new do |t|
11
- t.libs << 'test'
12
- t.test_files = FileList['test/**/*_test.rb']
9
+ t.libs << "test"
10
+ t.test_files = FileList["test/**/*_test.rb"]
13
11
  t.verbose = true
14
12
  t.warning = false
15
13
  end
16
14
 
17
15
  task default: :test
16
+
17
+ require "rubocop/rake_task"
18
+
19
+ RuboCop::RakeTask.new(:rubocop)
20
+
21
+ GEMSPEC = Bundler.load_gemspec("html-pipeline.gemspec")
22
+ gem_path = Gem::PackageTask.new(GEMSPEC).define
23
+ desc "Package the ruby gem"
24
+ task "package" => [gem_path]