jekyll-devto 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +8 -0
- data/LICENSE +21 -0
- data/README.md +129 -0
- data/examples/devto-publish.yml +50 -0
- data/exe/jekyll-devto +59 -0
- data/lib/jekyll/devto/client.rb +60 -0
- data/lib/jekyll/devto/feed.xml +33 -0
- data/lib/jekyll/devto/generator.rb +43 -0
- data/lib/jekyll/devto/html.rb +70 -0
- data/lib/jekyll/devto/publisher.rb +135 -0
- data/lib/jekyll/devto/version.rb +7 -0
- data/lib/jekyll-devto.rb +6 -0
- metadata +93 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 5de7210b54c1eeda3f3930fd529ab2acb06d08578b098be5b0bcc9389f2decbb
|
|
4
|
+
data.tar.gz: af15eb800bd515a2554b93c431a054edabc161883108301e6f542d10eefa0d43
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 6bf236df31a78ac816447e7fc62f5424cf369e4e1f94287b618e18b16ca15e18e6f629b11b62313e046f8e808ffe4d45bf792ae61979c5c64e96345229ce27d0
|
|
7
|
+
data.tar.gz: e30ab831596ed0880e33bac1eeec3874adf4da9fe2670a67cbc48dc215366fc34955f315f6d92d48ec7175424106d7d4450a41a81591b59efbad465ec74b03a5
|
data/CHANGELOG.md
ADDED
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Juan Vásquez
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
# jekyll-devto
|
|
2
|
+
|
|
3
|
+
Cross-post a Jekyll blog to [dev.to](https://dev.to), complete, through dev.to's own RSS import.
|
|
4
|
+
|
|
5
|
+
dev.to can import posts from your feed, but with a typical Jekyll feed you get:
|
|
6
|
+
|
|
7
|
+
- **Cut-off posts.** Feeds that carry a summary, or an empty `<content src="...">` link (the
|
|
8
|
+
[Chirpy](https://github.com/cotes2020/jekyll-theme-chirpy) theme's feed does this), import as the
|
|
9
|
+
summary only.
|
|
10
|
+
- **Line numbers inside the code.** Rouge with `line_numbers: true`, or a `{% highlight ruby linenos %}` tag, renders the code as a table with a gutter, and dev.to's HTML to Markdown conversion keeps the numbers as code.
|
|
11
|
+
- **Broken links and images.** Root-relative URLs point nowhere once the post lives on dev.to.
|
|
12
|
+
- **Drafts you publish by hand.** dev.to always imports as drafts.
|
|
13
|
+
|
|
14
|
+
This gem fixes all four:
|
|
15
|
+
|
|
16
|
+
1. A generator that adds `/devto.xml`, an RSS feed with the full rendered post, plain code blocks and absolute URLs.
|
|
17
|
+
2. A `jekyll-devto publish` command that publishes the imported drafts once their post is live on your site.
|
|
18
|
+
|
|
19
|
+
It works from the HTML Jekyll already rendered, so anything your theme and Kramdown support comes through. It never edits your posts.
|
|
20
|
+
|
|
21
|
+
## Install
|
|
22
|
+
|
|
23
|
+
```ruby
|
|
24
|
+
# Gemfile
|
|
25
|
+
gem 'jekyll-devto'
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
```yaml
|
|
29
|
+
# _config.yml
|
|
30
|
+
url: "https://example.com" # required: links in the feed are absolute
|
|
31
|
+
plugins:
|
|
32
|
+
- jekyll-devto
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Build, and the feed is at `https://example.com/devto.xml`.
|
|
36
|
+
|
|
37
|
+
On dev.to, go to **Settings → Extensions → Publishing to DEV Community from RSS**, set the feed URL
|
|
38
|
+
to your `devto.xml`, and turn on **Mark the RSS source as canonical URL by default** so search
|
|
39
|
+
engines treat your site as the original.
|
|
40
|
+
|
|
41
|
+
## Configuration
|
|
42
|
+
|
|
43
|
+
All optional.
|
|
44
|
+
|
|
45
|
+
```yaml
|
|
46
|
+
devto:
|
|
47
|
+
path: "/devto.xml" # where the feed is written
|
|
48
|
+
limit: 20 # newest posts to include (default: all)
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
To keep a post off dev.to, set `devto: false` in its front matter.
|
|
52
|
+
|
|
53
|
+
## Publishing the drafts
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
export DEVTO_API_KEY=... # https://dev.to/settings/extensions, "DEV Community API Keys"
|
|
57
|
+
|
|
58
|
+
bundle exec jekyll-devto publish # dry run: prints what it would publish
|
|
59
|
+
bundle exec jekyll-devto publish --publish # publishes
|
|
60
|
+
bundle exec jekyll-devto publish --days 14 --feed https://example.com/devto.xml
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
It reads your **live** feed (`url` + `devto.path` from `_config.yml`, or `--feed`), keeps the posts
|
|
64
|
+
published in the last `--days` days (7 by default, never future-dated ones), finds the draft dev.to
|
|
65
|
+
imported from each one, and publishes it.
|
|
66
|
+
|
|
67
|
+
- Drafts are matched by `canonical_url` first, then by title.
|
|
68
|
+
- Older drafts are left alone on purpose. The feed carries your whole archive, and publishing every
|
|
69
|
+
match would push years of old posts to dev.to at once.
|
|
70
|
+
- A post dev.to rejects is reported and the run moves on. Afterwards the drafts are listed again, and
|
|
71
|
+
any post still among them fails the run. The command exits 1 if anything failed.
|
|
72
|
+
|
|
73
|
+
### On a schedule with GitHub Actions
|
|
74
|
+
|
|
75
|
+
[`examples/devto-publish.yml`](examples/devto-publish.yml) runs it after each deploy and once a day.
|
|
76
|
+
Add the key as a repository secret:
|
|
77
|
+
|
|
78
|
+
```sh
|
|
79
|
+
gh secret set DEVTO_API_KEY
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## dev.to behaviour worth knowing
|
|
83
|
+
|
|
84
|
+
Checked against [Forem's source](https://github.com/forem/forem), the software dev.to runs:
|
|
85
|
+
|
|
86
|
+
- **Drafts match on title or link.** dev.to skips a feed entry when you already have an article with
|
|
87
|
+
the same title or link (`Feeds::CheckItemPreviouslyImported`). A deleted draft is imported again on
|
|
88
|
+
the next fetch while its post is still in the feed.
|
|
89
|
+
- **The imported body says `published: false` in its own front matter**, and that wins over the
|
|
90
|
+
API's `published` field (`Article#evaluate_front_matter`). The publish command flips it inside the
|
|
91
|
+
body. A plain `published: true` request leaves the post a draft.
|
|
92
|
+
- **Code blocks arrive without a language.** dev.to removes every `class` attribute before
|
|
93
|
+
converting (`Feeds::CleanHtml`), so syntax highlighting is lost on import whatever the feed says.
|
|
94
|
+
- **Only the first four tags are kept**, stripped to letters and digits.
|
|
95
|
+
- **"Replace self-referential links with DEV-specific links"** rewrites links between your posts to
|
|
96
|
+
their dev.to articles at import time, drafts included. Publish the linked post first, or leave
|
|
97
|
+
that option off.
|
|
98
|
+
- **The one-time "Import from XML" box** takes at most 25 entries and 500 KB
|
|
99
|
+
(`Feeds::ImportFromXml`). Use `devto.limit` if you need it.
|
|
100
|
+
|
|
101
|
+
## Development
|
|
102
|
+
|
|
103
|
+
```sh
|
|
104
|
+
bundle install
|
|
105
|
+
bundle exec rake test
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
The feed tests build a fixture site and replay dev.to's import (Feedjira, Forem's class stripping,
|
|
109
|
+
ReverseMarkdown) to check what dev.to would store.
|
|
110
|
+
|
|
111
|
+
## Releases
|
|
112
|
+
|
|
113
|
+
`jekyll-devto` follows [Semantic Versioning](https://semver.org), and releases are automated with [release-please](https://github.com/googleapis/release-please) from [Conventional Commits](https://www.conventionalcommits.org):
|
|
114
|
+
|
|
115
|
+
- `fix:` bumps the **PATCH** version, and so do `perf:`, `refactor:` and `docs:`
|
|
116
|
+
- `feat:` bumps the **MINOR** version
|
|
117
|
+
- `BREAKING CHANGE:` in the commit footer bumps the **MAJOR** version (the MINOR one while the version is `0.x`)
|
|
118
|
+
- `test:`, `ci:` and `chore:` are left out of the CHANGELOG and do not trigger a release on their own
|
|
119
|
+
|
|
120
|
+
### Steps to release a new version
|
|
121
|
+
|
|
122
|
+
1. Merge pull requests to `main` with Conventional Commit titles (`fix(html): ...`, `feat(publisher): ...`)
|
|
123
|
+
2. release-please keeps a `chore(main): release x.y.z` pull request open with the version bump in `lib/jekyll/devto/version.rb` and the new `CHANGELOG.md` entries
|
|
124
|
+
3. Review and merge that pull request
|
|
125
|
+
4. The `Release` workflow tags `vx.y.z`, creates the GitHub release, runs the tests, and pushes the gem to RubyGems with [trusted publishing](https://guides.rubygems.org/trusted-publishing/), so no API key or MFA code is needed
|
|
126
|
+
|
|
127
|
+
## License
|
|
128
|
+
|
|
129
|
+
MIT
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Copy to .github/workflows/devto-publish.yml and set the DEVTO_API_KEY
|
|
2
|
+
# repository secret (https://dev.to/settings/extensions).
|
|
3
|
+
name: "Publish to dev.to"
|
|
4
|
+
on:
|
|
5
|
+
# Right after a deploy, so a post goes out on dev.to the day it goes live.
|
|
6
|
+
# Use the `name:` of your deploy workflow.
|
|
7
|
+
workflow_run:
|
|
8
|
+
workflows: ["Build and Deploy"]
|
|
9
|
+
types: [completed]
|
|
10
|
+
branches: [main]
|
|
11
|
+
|
|
12
|
+
# dev.to imports the feed on its own schedule, so the draft may not exist
|
|
13
|
+
# yet when the deploy finishes. Retry once a day; posts already published
|
|
14
|
+
# are skipped, so repeated runs change nothing.
|
|
15
|
+
schedule:
|
|
16
|
+
- cron: "0 21 * * *"
|
|
17
|
+
|
|
18
|
+
workflow_dispatch:
|
|
19
|
+
inputs:
|
|
20
|
+
days:
|
|
21
|
+
description: "Publish drafts of posts published in the last N days"
|
|
22
|
+
default: "7"
|
|
23
|
+
|
|
24
|
+
permissions:
|
|
25
|
+
contents: read
|
|
26
|
+
|
|
27
|
+
concurrency:
|
|
28
|
+
group: "devto-publish"
|
|
29
|
+
cancel-in-progress: false
|
|
30
|
+
|
|
31
|
+
jobs:
|
|
32
|
+
publish:
|
|
33
|
+
if: github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success'
|
|
34
|
+
runs-on: ubuntu-latest
|
|
35
|
+
env:
|
|
36
|
+
DEVTO_API_KEY: ${{ secrets.DEVTO_API_KEY }}
|
|
37
|
+
# Passed through env, never interpolated into the script: a value typed
|
|
38
|
+
# into the manual trigger would otherwise run as shell code.
|
|
39
|
+
DAYS: ${{ inputs.days || '7' }}
|
|
40
|
+
|
|
41
|
+
steps:
|
|
42
|
+
- uses: actions/checkout@v5
|
|
43
|
+
|
|
44
|
+
- uses: ruby/setup-ruby@v1
|
|
45
|
+
with:
|
|
46
|
+
bundler-cache: true
|
|
47
|
+
|
|
48
|
+
- name: Publish dev.to drafts
|
|
49
|
+
if: env.DEVTO_API_KEY != ''
|
|
50
|
+
run: bundle exec jekyll-devto publish --publish --days "$DAYS"
|
data/exe/jekyll-devto
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
# Publishes the dev.to drafts of posts that are live on the site.
|
|
5
|
+
#
|
|
6
|
+
# jekyll-devto publish # dry run, prints the plan
|
|
7
|
+
# jekyll-devto publish --publish # publishes
|
|
8
|
+
# jekyll-devto publish --days 14 --feed https://example.com/devto.xml
|
|
9
|
+
#
|
|
10
|
+
# The API key comes from the DEVTO_API_KEY environment variable
|
|
11
|
+
# (https://dev.to/settings/extensions).
|
|
12
|
+
|
|
13
|
+
require 'optparse'
|
|
14
|
+
require 'yaml'
|
|
15
|
+
require_relative '../lib/jekyll/devto/version'
|
|
16
|
+
require_relative '../lib/jekyll/devto/client'
|
|
17
|
+
require_relative '../lib/jekyll/devto/publisher'
|
|
18
|
+
|
|
19
|
+
def default_feed
|
|
20
|
+
config = File.exist?('_config.yml') ? YAML.safe_load_file('_config.yml', aliases: true) || {} : {}
|
|
21
|
+
url = config['url'].to_s
|
|
22
|
+
return nil if url.empty?
|
|
23
|
+
|
|
24
|
+
path = config.dig('devto', 'path') if config['devto'].is_a?(Hash)
|
|
25
|
+
"#{url.chomp('/')}#{config['baseurl']}/#{(path || 'devto.xml').sub(%r{\A/}, '')}"
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
options = { days: 7, publish: false, feed: nil }
|
|
29
|
+
parser = OptionParser.new do |o|
|
|
30
|
+
o.banner = 'Usage: jekyll-devto publish [options]'
|
|
31
|
+
o.on('--publish', 'Publish the drafts (default: dry run)') { options[:publish] = true }
|
|
32
|
+
o.on('--days N', Integer, 'Only posts published in the last N days (default: 7)') { |n| options[:days] = n }
|
|
33
|
+
o.on('--feed URL_OR_PATH', 'The dev.to feed (default: url + devto.xml from _config.yml)') { |f| options[:feed] = f }
|
|
34
|
+
o.on('-v', '--version', 'Print the version') { puts Jekyll::Devto::VERSION; exit }
|
|
35
|
+
end
|
|
36
|
+
begin
|
|
37
|
+
parser.parse!
|
|
38
|
+
rescue OptionParser::ParseError => e
|
|
39
|
+
abort "#{e.message}\n#{parser}"
|
|
40
|
+
end
|
|
41
|
+
command = ARGV.shift
|
|
42
|
+
|
|
43
|
+
abort parser.banner unless command == 'publish'
|
|
44
|
+
|
|
45
|
+
feed = options[:feed] || default_feed
|
|
46
|
+
abort 'No feed: pass --feed, or run from a site whose _config.yml sets url.' unless feed
|
|
47
|
+
|
|
48
|
+
begin
|
|
49
|
+
failures = Jekyll::Devto::Publisher.new(
|
|
50
|
+
feed: feed,
|
|
51
|
+
client: Jekyll::Devto::Client.new(ENV.fetch('DEVTO_API_KEY', nil)),
|
|
52
|
+
days: options[:days],
|
|
53
|
+
publish: options[:publish]
|
|
54
|
+
).run
|
|
55
|
+
rescue Jekyll::Devto::Client::Error => e
|
|
56
|
+
abort e.message
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
exit(failures.zero? ? 0 : 1)
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
require 'net/http'
|
|
5
|
+
require 'openssl'
|
|
6
|
+
require_relative 'version'
|
|
7
|
+
|
|
8
|
+
module Jekyll
|
|
9
|
+
module Devto
|
|
10
|
+
# Minimal dev.to (Forem) API client: the two calls the publisher needs.
|
|
11
|
+
class Client
|
|
12
|
+
API = URI('https://dev.to/api/')
|
|
13
|
+
PER_PAGE = 1000
|
|
14
|
+
|
|
15
|
+
Error = Class.new(StandardError)
|
|
16
|
+
|
|
17
|
+
# What a request can raise before there is a response to look at. Each
|
|
18
|
+
# is turned into Error, so the CLI reports it instead of crashing.
|
|
19
|
+
NETWORK_ERRORS = [SystemCallError, SocketError, IOError, Timeout::Error,
|
|
20
|
+
OpenSSL::SSL::SSLError, Net::HTTPBadResponse, Net::ProtocolError].freeze
|
|
21
|
+
|
|
22
|
+
def initialize(api_key)
|
|
23
|
+
raise Error, 'DEVTO_API_KEY is not set' if api_key.to_s.empty?
|
|
24
|
+
|
|
25
|
+
@api_key = api_key
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def drafts
|
|
29
|
+
(1..).each_with_object([]) do |page, all|
|
|
30
|
+
batch = request(Net::HTTP::Get, "articles/me/unpublished?per_page=#{PER_PAGE}&page=#{page}")
|
|
31
|
+
all.concat(batch)
|
|
32
|
+
break all if batch.size < PER_PAGE
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def update(id, article)
|
|
37
|
+
request(Net::HTTP::Put, "articles/#{id}", { article: article })
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
private
|
|
41
|
+
|
|
42
|
+
def request(verb, path, body = nil)
|
|
43
|
+
uri = API + path
|
|
44
|
+
req = verb.new(uri)
|
|
45
|
+
req['api-key'] = @api_key
|
|
46
|
+
req['Accept'] = 'application/vnd.forem.api-v1+json'
|
|
47
|
+
req['Content-Type'] = 'application/json'
|
|
48
|
+
req['User-Agent'] = "jekyll-devto/#{VERSION}"
|
|
49
|
+
req.body = JSON.generate(body) if body
|
|
50
|
+
|
|
51
|
+
res = Net::HTTP.start(uri.host, uri.port, use_ssl: true) { |http| http.request(req) }
|
|
52
|
+
raise Error, "#{req.method} #{uri.path} failed: #{res.code} #{res.body}" unless res.is_a?(Net::HTTPSuccess)
|
|
53
|
+
|
|
54
|
+
JSON.parse(res.body)
|
|
55
|
+
rescue *NETWORK_ERRORS, JSON::ParserError => e
|
|
56
|
+
raise Error, "#{verb::METHOD} #{uri.path} failed: #{e.message}"
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
end
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
<?xml version="1.0" encoding="UTF-8"?>
|
|
2
|
+
{%- assign devto = site.devto | default: empty -%}
|
|
3
|
+
{%- comment -%} Not site.url + baseurl: links in content already carry baseurl (relative_url adds it). {%- endcomment -%}
|
|
4
|
+
{%- assign site_url = site.url -%}
|
|
5
|
+
{%- assign posts = site.posts | where_exp: "post", "post.devto != false" -%}
|
|
6
|
+
{%- if devto.limit %}{% assign posts = posts | slice: 0, devto.limit %}{% endif %}
|
|
7
|
+
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom">
|
|
8
|
+
<channel>
|
|
9
|
+
<title>{{ site.title | default: site.name | xml_escape }}</title>
|
|
10
|
+
<link>{{ "/" | absolute_url }}</link>
|
|
11
|
+
<description>{{ site.description | xml_escape }}</description>
|
|
12
|
+
{%- if site.lang %}
|
|
13
|
+
<language>{{ site.lang }}</language>
|
|
14
|
+
{%- endif %}
|
|
15
|
+
<lastBuildDate>{{ site.time | date_to_rfc822 }}</lastBuildDate>
|
|
16
|
+
<atom:link href="{{ page.url | absolute_url }}" rel="self" type="application/rss+xml" />
|
|
17
|
+
{%- for post in posts %}
|
|
18
|
+
{%- assign post_url = post.url | absolute_url %}
|
|
19
|
+
<item>
|
|
20
|
+
<title>{{ post.title | strip_html | strip | xml_escape }}</title>
|
|
21
|
+
<link>{{ post_url }}</link>
|
|
22
|
+
<guid isPermaLink="true">{{ post_url }}</guid>
|
|
23
|
+
<pubDate>{{ post.date | date_to_rfc822 }}</pubDate>
|
|
24
|
+
{%- assign tags = post.tags | default: post.categories %}
|
|
25
|
+
{%- for tag in tags %}
|
|
26
|
+
<category>{{ tag | xml_escape }}</category>
|
|
27
|
+
{%- endfor %}
|
|
28
|
+
<description>{{ post.description | default: post.excerpt | strip_html | strip_newlines | truncatewords: 60 | xml_escape }}</description>
|
|
29
|
+
<content:encoded>{{ post.content | devto_html: site_url | xml_escape }}</content:encoded>
|
|
30
|
+
</item>
|
|
31
|
+
{%- endfor %}
|
|
32
|
+
</channel>
|
|
33
|
+
</rss>
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Jekyll
|
|
4
|
+
module Devto
|
|
5
|
+
# Adds the dev.to feed to the site as a page, the same way jekyll-feed adds
|
|
6
|
+
# its own. Posts render before pages, so `post.content` in the template is
|
|
7
|
+
# the final HTML.
|
|
8
|
+
class Generator < Jekyll::Generator
|
|
9
|
+
safe true
|
|
10
|
+
priority :lowest
|
|
11
|
+
|
|
12
|
+
DEFAULT_PATH = 'devto.xml'
|
|
13
|
+
TEMPLATE = File.expand_path('feed.xml', __dir__)
|
|
14
|
+
|
|
15
|
+
def generate(site)
|
|
16
|
+
path = feed_path(site)
|
|
17
|
+
if site.pages.any? { |page| page.url == "/#{path}" }
|
|
18
|
+
Jekyll.logger.warn 'jekyll-devto:', "a page already lives at /#{path}, not generating the feed"
|
|
19
|
+
return
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
site.pages << feed_page(site, path)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
private
|
|
26
|
+
|
|
27
|
+
def feed_path(site)
|
|
28
|
+
config = site.config['devto']
|
|
29
|
+
path = config.is_a?(Hash) ? config['path'] : nil
|
|
30
|
+
(path || DEFAULT_PATH).sub(%r{\A/}, '')
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def feed_page(site, path)
|
|
34
|
+
PageWithoutAFile.new(site, __dir__, File.dirname(path).sub(/\A\.\z/, ''), File.basename(path)).tap do |page|
|
|
35
|
+
page.content = File.read(TEMPLATE)
|
|
36
|
+
page.data['layout'] = nil
|
|
37
|
+
page.data['sitemap'] = false
|
|
38
|
+
page.data['permalink'] = "/#{path}"
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Jekyll
|
|
4
|
+
module Devto
|
|
5
|
+
# Turns a rendered post body into HTML that survives dev.to's RSS import.
|
|
6
|
+
#
|
|
7
|
+
# dev.to strips every class attribute and converts the HTML to Markdown
|
|
8
|
+
# (Forem's Feeds::CleanHtml, then ReverseMarkdown). Two things break on the
|
|
9
|
+
# way:
|
|
10
|
+
#
|
|
11
|
+
# - Rouge with `line_numbers: true`, or a `{% highlight lang linenos %}` tag,
|
|
12
|
+
# renders the code as a table with a gutter, and the gutter numbers end
|
|
13
|
+
# up inside the code.
|
|
14
|
+
# - Root-relative links and images point nowhere once the post lives on
|
|
15
|
+
# dev.to.
|
|
16
|
+
module HTML
|
|
17
|
+
# Kramdown block options such as {: .nolineno } or {: file="..." } add
|
|
18
|
+
# classes and attributes to the wrapper, so the class is matched anywhere
|
|
19
|
+
# in the tag.
|
|
20
|
+
ROUGE_BLOCK = %r{
|
|
21
|
+
<div\b[^>]*\bclass="[^"]*\bhighlighter-rouge\b[^"]*"[^>]*>\s*
|
|
22
|
+
<div\ class="highlight">\s*<pre\ class="highlight"><code>
|
|
23
|
+
(?<body>.*?)
|
|
24
|
+
</code></pre>\s*</div>\s*</div>
|
|
25
|
+
}mx
|
|
26
|
+
|
|
27
|
+
# What the {% highlight %} Liquid tag renders.
|
|
28
|
+
HIGHLIGHT_TAG = %r{
|
|
29
|
+
<figure\ class="highlight"><pre><code\b[^>]*>
|
|
30
|
+
(?<body>.*?)
|
|
31
|
+
</code></pre></figure>
|
|
32
|
+
}mx
|
|
33
|
+
|
|
34
|
+
# Kramdown names the cells rouge-gutter and rouge-code; the highlight tag
|
|
35
|
+
# names them gutter and code.
|
|
36
|
+
GUTTER = %r{<table class="rouge-table">.*?<td class="(?:rouge-)?code"><pre>(?<code>.*?)</pre>}m
|
|
37
|
+
|
|
38
|
+
# Real tags only. Code samples reach the HTML escaped (<img src="/x">),
|
|
39
|
+
# so they never match and are left exactly as written.
|
|
40
|
+
TAG = /<[a-zA-Z][^>]*>/
|
|
41
|
+
ROOT_RELATIVE = %r{\b(src|href)="/(?!/)}
|
|
42
|
+
|
|
43
|
+
module_function
|
|
44
|
+
|
|
45
|
+
def convert(html, base_url)
|
|
46
|
+
html.to_s
|
|
47
|
+
.gsub(ROUGE_BLOCK) { plain_code(Regexp.last_match[:body]) }
|
|
48
|
+
.gsub(HIGHLIGHT_TAG) { plain_code(Regexp.last_match[:body]) }
|
|
49
|
+
.gsub(TAG) { |tag| tag.gsub(ROOT_RELATIVE, %(\\1="#{base_url.to_s.chomp('/')}/)) }
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# The language is not kept: dev.to strips every class before converting,
|
|
53
|
+
# so it would never arrive.
|
|
54
|
+
def plain_code(body)
|
|
55
|
+
body = Regexp.last_match[:code] if body =~ GUTTER
|
|
56
|
+
|
|
57
|
+
"<pre><code>#{body.gsub(/<[^>]+>/, '')}</code></pre>"
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# Liquid filter for the feed template.
|
|
62
|
+
module Filters
|
|
63
|
+
def devto_html(html, base_url)
|
|
64
|
+
HTML.convert(html, base_url)
|
|
65
|
+
end
|
|
66
|
+
end
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
Liquid::Template.register_filter(Jekyll::Devto::Filters)
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'net/http'
|
|
4
|
+
require 'rexml/document'
|
|
5
|
+
require 'time'
|
|
6
|
+
require_relative 'client'
|
|
7
|
+
|
|
8
|
+
module Jekyll
|
|
9
|
+
module Devto
|
|
10
|
+
# Publishes the dev.to drafts of posts that are live on the site.
|
|
11
|
+
#
|
|
12
|
+
# dev.to's RSS import always creates drafts. This reads the site's dev.to
|
|
13
|
+
# feed, keeps the posts published in the last `days` days, finds the draft
|
|
14
|
+
# imported from each one, and publishes it. Older drafts are left alone on
|
|
15
|
+
# purpose: the feed carries the whole archive, and publishing every match
|
|
16
|
+
# would push years of old posts to dev.to at once.
|
|
17
|
+
class Publisher
|
|
18
|
+
MAX_REDIRECTS = 5
|
|
19
|
+
|
|
20
|
+
Post = Struct.new(:title, :link, :date, keyword_init: true)
|
|
21
|
+
|
|
22
|
+
def initialize(feed:, client:, days: 7, publish: false, now: Time.now, out: $stdout, err: $stderr)
|
|
23
|
+
@feed = feed
|
|
24
|
+
@client = client
|
|
25
|
+
@days = days
|
|
26
|
+
@publish = publish
|
|
27
|
+
@now = now
|
|
28
|
+
@out = out
|
|
29
|
+
@err = err
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
# Returns the number of posts that failed to publish.
|
|
33
|
+
def run
|
|
34
|
+
posts = due_posts
|
|
35
|
+
@out.puts "Posts live in the last #{@days} days: #{posts.size}"
|
|
36
|
+
return 0 if posts.empty?
|
|
37
|
+
|
|
38
|
+
drafts = @client.drafts
|
|
39
|
+
@out.puts "Drafts on dev.to: #{drafts.size}"
|
|
40
|
+
|
|
41
|
+
sent = {}
|
|
42
|
+
failures = []
|
|
43
|
+
posts.each do |post|
|
|
44
|
+
draft = find_draft(drafts, post)
|
|
45
|
+
next report_missing(post, drafts) unless draft
|
|
46
|
+
|
|
47
|
+
unless @publish
|
|
48
|
+
@out.puts " would publish #{post.title.inspect} -> dev.to draft #{draft['id']}"
|
|
49
|
+
next
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# One rejected post must not stop the rest, or every retry would stop at it.
|
|
53
|
+
begin
|
|
54
|
+
result = @client.update(draft['id'], published: true, body_markdown: self.class.published_body(draft['body_markdown']))
|
|
55
|
+
sent[draft['id']] = post
|
|
56
|
+
@out.puts " sent #{post.title.inspect} -> #{result['url']}"
|
|
57
|
+
rescue StandardError => e
|
|
58
|
+
failures << post
|
|
59
|
+
@err.puts " FAILED #{post.title.inspect}: #{e.message}"
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
failures.concat(still_drafts(sent))
|
|
64
|
+
@err.puts "#{failures.size} post(s) failed to publish" if failures.any?
|
|
65
|
+
failures.size
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# The imported body carries `published: false` in its own front matter,
|
|
69
|
+
# and dev.to applies that over the request's `published` field, so it is
|
|
70
|
+
# flipped inside the body as well.
|
|
71
|
+
def self.published_body(markdown)
|
|
72
|
+
# Only inside the front matter: up to the first closing ---, so a
|
|
73
|
+
# "published: false" line in the post body is left alone.
|
|
74
|
+
markdown.to_s.sub(/\A---\r?\n.*?^---[ \t]*\r?$/m) do |front_matter|
|
|
75
|
+
front_matter.sub(/^published:[ \t]*false[ \t]*(?=\r?$)/, 'published: true')
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def due_posts
|
|
80
|
+
REXML::Document.new(read_feed).get_elements('//item').filter_map do |item|
|
|
81
|
+
date = Time.rfc2822(item.elements['pubDate'].text)
|
|
82
|
+
next if date > @now || date < @now - (@days * 86_400)
|
|
83
|
+
|
|
84
|
+
Post.new(title: item.elements['title'].text.to_s.strip, link: item.elements['link'].text.to_s.strip, date: date)
|
|
85
|
+
end
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
private
|
|
89
|
+
|
|
90
|
+
# Follows redirects (http to https, apex to www), which Net::HTTP does not.
|
|
91
|
+
def read_feed
|
|
92
|
+
return File.read(@feed) unless @feed.match?(%r{\Ahttps?://})
|
|
93
|
+
|
|
94
|
+
uri = URI(@feed)
|
|
95
|
+
MAX_REDIRECTS.succ.times do
|
|
96
|
+
res = Net::HTTP.get_response(uri)
|
|
97
|
+
return res.body if res.is_a?(Net::HTTPSuccess)
|
|
98
|
+
raise Client::Error, "could not read #{@feed}: #{res.code}" unless res.is_a?(Net::HTTPRedirection)
|
|
99
|
+
|
|
100
|
+
uri = URI.join(uri, res['location'])
|
|
101
|
+
end
|
|
102
|
+
raise Client::Error, "could not read #{@feed}: more than #{MAX_REDIRECTS} redirects"
|
|
103
|
+
rescue *Client::NETWORK_ERRORS => e
|
|
104
|
+
raise Client::Error, "could not read #{@feed}: #{e.message}"
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
# canonical_url first: dev.to sets it to the post's link when the feed
|
|
108
|
+
# source has "Mark the RSS source as canonical URL" on. Title otherwise.
|
|
109
|
+
def find_draft(drafts, post)
|
|
110
|
+
drafts.find { |d| d['canonical_url'].to_s.chomp('/') == post.link.chomp('/') } ||
|
|
111
|
+
drafts.find { |d| d['title'].to_s.strip == post.title }
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
def report_missing(post, drafts)
|
|
115
|
+
@out.puts " skip #{post.title.inspect}: no dev.to draft (not imported yet, or already published)"
|
|
116
|
+
near = drafts.map { |d| d['title'].to_s }.find { |t| t.downcase.include?(post.title.downcase[0, 20]) }
|
|
117
|
+
@out.puts " closest draft title: #{near.inspect}" if near
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
# The PUT response does not say whether the article is published, so ask
|
|
121
|
+
# dev.to again: anything still in the drafts list did not go out.
|
|
122
|
+
def still_drafts(sent)
|
|
123
|
+
return [] if sent.empty?
|
|
124
|
+
|
|
125
|
+
ids = @client.drafts.map { |d| d['id'] } & sent.keys
|
|
126
|
+
ids.map { |id| sent[id] }.each { |post| @err.puts " STILL A DRAFT: #{post.title.inspect}" }
|
|
127
|
+
rescue Client::Error => e
|
|
128
|
+
# The PUTs already went out, so say which posts could not be confirmed
|
|
129
|
+
# rather than losing the summary.
|
|
130
|
+
@err.puts " could not confirm #{sent.size} post(s) went out: #{e.message}"
|
|
131
|
+
sent.values
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
end
|
|
135
|
+
end
|
data/lib/jekyll-devto.rb
ADDED
metadata
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: jekyll-devto
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Juan Vásquez
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: jekyll
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '4.0'
|
|
19
|
+
- - "<"
|
|
20
|
+
- !ruby/object:Gem::Version
|
|
21
|
+
version: '5'
|
|
22
|
+
type: :runtime
|
|
23
|
+
prerelease: false
|
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
25
|
+
requirements:
|
|
26
|
+
- - ">="
|
|
27
|
+
- !ruby/object:Gem::Version
|
|
28
|
+
version: '4.0'
|
|
29
|
+
- - "<"
|
|
30
|
+
- !ruby/object:Gem::Version
|
|
31
|
+
version: '5'
|
|
32
|
+
- !ruby/object:Gem::Dependency
|
|
33
|
+
name: rexml
|
|
34
|
+
requirement: !ruby/object:Gem::Requirement
|
|
35
|
+
requirements:
|
|
36
|
+
- - "~>"
|
|
37
|
+
- !ruby/object:Gem::Version
|
|
38
|
+
version: '3.2'
|
|
39
|
+
type: :runtime
|
|
40
|
+
prerelease: false
|
|
41
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
42
|
+
requirements:
|
|
43
|
+
- - "~>"
|
|
44
|
+
- !ruby/object:Gem::Version
|
|
45
|
+
version: '3.2'
|
|
46
|
+
description: |
|
|
47
|
+
Generates a feed dev.to imports with the full post (code blocks without
|
|
48
|
+
Rouge line numbers, absolute links and images), and a command that
|
|
49
|
+
publishes the imported drafts once their post is live on your site.
|
|
50
|
+
email:
|
|
51
|
+
- juan@ombulabs.com
|
|
52
|
+
executables:
|
|
53
|
+
- jekyll-devto
|
|
54
|
+
extensions: []
|
|
55
|
+
extra_rdoc_files: []
|
|
56
|
+
files:
|
|
57
|
+
- CHANGELOG.md
|
|
58
|
+
- LICENSE
|
|
59
|
+
- README.md
|
|
60
|
+
- examples/devto-publish.yml
|
|
61
|
+
- exe/jekyll-devto
|
|
62
|
+
- lib/jekyll-devto.rb
|
|
63
|
+
- lib/jekyll/devto/client.rb
|
|
64
|
+
- lib/jekyll/devto/feed.xml
|
|
65
|
+
- lib/jekyll/devto/generator.rb
|
|
66
|
+
- lib/jekyll/devto/html.rb
|
|
67
|
+
- lib/jekyll/devto/publisher.rb
|
|
68
|
+
- lib/jekyll/devto/version.rb
|
|
69
|
+
homepage: https://github.com/JuanVqz/jekyll-devto
|
|
70
|
+
licenses:
|
|
71
|
+
- MIT
|
|
72
|
+
metadata:
|
|
73
|
+
source_code_uri: https://github.com/JuanVqz/jekyll-devto
|
|
74
|
+
changelog_uri: https://github.com/JuanVqz/jekyll-devto/blob/main/CHANGELOG.md
|
|
75
|
+
rubygems_mfa_required: 'true'
|
|
76
|
+
rdoc_options: []
|
|
77
|
+
require_paths:
|
|
78
|
+
- lib
|
|
79
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
80
|
+
requirements:
|
|
81
|
+
- - ">="
|
|
82
|
+
- !ruby/object:Gem::Version
|
|
83
|
+
version: '3.1'
|
|
84
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
85
|
+
requirements:
|
|
86
|
+
- - ">="
|
|
87
|
+
- !ruby/object:Gem::Version
|
|
88
|
+
version: '0'
|
|
89
|
+
requirements: []
|
|
90
|
+
rubygems_version: 4.0.20
|
|
91
|
+
specification_version: 4
|
|
92
|
+
summary: Cross-post a Jekyll blog to dev.to, complete, through its RSS import.
|
|
93
|
+
test_files: []
|