trellodon 0.1.0 → 0.3.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: bf1b8715b6cbad2c1bd3ab4fce72198a442380cf4f2d7bb39c55928ef72b032f
4
- data.tar.gz: 2caa2a3a24fe463cc1cb730a3aad68928323ec0fea04fea8e3a0f10ebf051ea7
3
+ metadata.gz: a61911e92509a77f13aa90f9f56c14d038dbe443b4e1063d28f66b6745b04432
4
+ data.tar.gz: '079609a773107e786a7d795fe7d23564ded2aa66e8d587a6fd22b4e288e15478'
5
5
  SHA512:
6
- metadata.gz: b50b3279f7a9e787b2f2e718109902a9cda88729f7d7b330aa5db9db944f913af33d6f26fab7b1f6c4d2866a1241d62d99c4dbe989ea468499bcaaa4908a153f
7
- data.tar.gz: 52d86ebd0b43ad0b064facbfb66332623422b2c688eff732188b7ea2ce8baba2842a331a941e1aafb410f114c31aef3ec80f7588b7f6a07808070019f9eb12ba
6
+ metadata.gz: 617119b48c1f2385c87d57ffd2683933097ed1aae11a51577f65e16012ab1f58b65187f57884123ed04ba522797925826e3cf202e7d4dad9d1fd63a3d736bcef
7
+ data.tar.gz: 30c1f1bcb2df2c69a3c8d0cebc93a597f16d2207d33bef979670134d306bd0173cc97948f38f739cf1d6b5eef0a7a602112fb0de40021d078d0188280ca4d039
data/CHANGELOG.md CHANGED
@@ -1,5 +1,31 @@
1
- ## [Unreleased]
1
+ # Change log
2
+
3
+ ## [master]
4
+
5
+ ## [0.3.0] - 2022-04-12 🚀🚀🚀
6
+
7
+ - Handle missing members. ([@palkan][])
8
+
9
+ - Use inline scheduler by default. ([@palkan][])
10
+
11
+ - Replace `--sync` with `--concurrency=0`. ([@palkan][])
12
+
13
+ ## [0.2.1] - 2022-04-12 🚀
14
+
15
+ - Misc changes.
16
+
17
+ ## [0.2.0] - 2022-04-12
18
+
19
+ - Store Trello API creds locally for avoid asking it before execution ([@fargelus][]).
20
+ - Download attachments, human names in comments, rename index.md to README.md ([@rinasergeeva][]).
21
+ - Added minimal project README.md ([@rinasergeeva][], [@fargelus][]).
22
+ - --sync param for switch to `Schedulers::Inline` in CLI ([@fargelus][]).
23
+ - Concurrent scheduler with customizable thread pool size ([@fargelus][]).
24
+ - Checklists support ([@rinasergeeva][]).
2
25
 
3
26
  ## [0.1.0] - 2022-04-07
4
27
 
5
28
  - Initial release
29
+
30
+ [@fargelus]: https://github.com/fargelus
31
+ [@rinasergeeva]: https://github.com/rinasergeeva
data/README.md CHANGED
@@ -1,3 +1,92 @@
1
- # trellodon
1
+ ![Build](https://github.com/evilmartians/trellodon/workflows/Build/badge.svg)
2
+ [![Gem Version](https://badge.fury.io/rb/trellodon.svg)](https://rubygems.org/gems/trellodon)
2
3
 
3
- Dump Trello boards to your file system
4
+ # Trellodon
5
+
6
+ A Ruby tool to export a Trello board and convert it into a set of folders and markdown files, corresponding to lists and cards on the board. For each card, you’ll get the basic details such as:
7
+
8
+ * Name
9
+ * Description
10
+ * Labels
11
+ * Comments
12
+ * Attachments
13
+
14
+ <img src="https://evilmartians.com/badges/sponsored-by-evil-martians.svg" alt="Sponsored by Evil Martians" width="236" height="54"></a>
15
+
16
+ ## Installation
17
+
18
+ ```
19
+ gem install trellodon
20
+ ```
21
+
22
+ ## Prerequisites
23
+
24
+ Trellodon needs two secret codes to download your boards:
25
+
26
+ * Trello API key
27
+ * Authentication token
28
+
29
+ To generate them, go to the [trello.com/app-key](https://trello.com/app-key) page and follow the instructions.
30
+
31
+ ## Usage
32
+
33
+ ```sh
34
+ Usage: trellodon dump [options]
35
+ Options:
36
+ --board VALUE Board URL or ID
37
+ --out VALUE Destination folder path
38
+ --concurrency VALUE Amount of processing threads (default: 4). Set to 0 to execute API requests in-line
39
+ --clear-auth Remove saved api credentials
40
+ -v, --version Print current version
41
+ -h, --help Print help
42
+ ```
43
+
44
+ ### Detailed example
45
+
46
+ Suppose we have a Trello board called `projects` with lists `Brainstorm`, `TODO`, `DOING`, `DONE`. Lists contains own cards.<br>
47
+ All Trello boards have its own id this id requires Trellodon to fetch board via Trello API.
48
+ You can find this id in two ways:
49
+
50
+ * In URL `https://trello.com/b/{id}/projects` your Trello board id will be placed as `{id}` in example;
51
+ * Put `.json` in your board URL like this `https://trello.com/b/{id}/projects.json` id field in json output is what you need;
52
+
53
+ You can also paste whole board URL and Trellodon will parse it correctly.<br>
54
+ After launch Trellodon creates follow output in specified folder:
55
+
56
+ ```sh
57
+ projects/
58
+ Brainstorm/
59
+ first_card_title/
60
+ README.md
61
+ attachments/
62
+ image.png
63
+ TODO/
64
+ first_card_title/
65
+ README.md
66
+ attachments/
67
+ report.docx
68
+ ...
69
+ ```
70
+
71
+ Each card has follow output format:
72
+
73
+ ```md
74
+ ---
75
+ title: card_title
76
+ last_updated_at: 2022-03-16 16:28:39 UTC
77
+ labels: Test
78
+ ---
79
+
80
+ # card_title
81
+
82
+ ## Description
83
+ Some card description
84
+
85
+ ## Comments
86
+ ** John Doe @doe at 2022-03-16 16:28:39 UTC**
87
+
88
+ ## Attachments
89
+ ### Image.png
90
+ **date**: 2022-03-16
91
+ **url**: https://trello.com/1/cards/#{card_id}/attachments/#{attachment.id}/download/image.png
92
+ ```
@@ -1,10 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "trellodon/api_client"
4
- require "trellodon/config"
5
-
6
3
  module Trellodon
7
- class APIExecutor
4
+ class Executor
8
5
  BOARD_ID_REGEX = /\/b\/([^\/]+)\//
9
6
  private_constant :BOARD_ID_REGEX
10
7
 
@@ -16,7 +13,7 @@ module Trellodon
16
13
  @scheduler = scheduler
17
14
  check_credentials!
18
15
 
19
- @api_client = APIClient.new(api_key: @api_key, api_token: @api_token)
16
+ @client = Client.new(api_key: @api_key, api_token: @api_token)
20
17
  end
21
18
 
22
19
  def download(board_pointer)
@@ -25,13 +22,13 @@ module Trellodon
25
22
  startup_time = Time.now
26
23
  logger.debug "Fetching board 🚀️️"
27
24
  board = scheduler.post do
28
- api_client.fetch_board(board_id).tap { formatter.board_added(_1) }
25
+ client.fetch_board(board_id).tap { |_1| formatter.board_added(_1) }
29
26
  end.value
30
27
 
31
- logger.debug "Fetching cards in board with comments and attachments 🐢"
28
+ logger.debug "Fetching #{board.card_ids.size} cards from the board with comments and attachments 🐢"
32
29
  board.card_ids.map do |card_id|
33
30
  scheduler.post do
34
- api_client.fetch_card(card_id).tap { formatter.card_added(_1) }
31
+ client.fetch_card(card_id).tap { |_1| formatter.card_added(_1) }
35
32
  end
36
33
  end.map(&:value)
37
34
 
@@ -41,7 +38,7 @@ module Trellodon
41
38
 
42
39
  private
43
40
 
44
- attr_reader :api_client, :board_id, :logger, :scheduler, :formatter
41
+ attr_reader :client, :board_id, :logger, :scheduler, :formatter
45
42
 
46
43
  def check_credentials!
47
44
  return if @api_key.to_s.size.positive? && @api_token.to_s.size.positive?
@@ -0,0 +1,149 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "trello"
4
+ require "trellodon/entities"
5
+
6
+ module Trellodon
7
+ class Client
8
+ class DeletedMember < Struct.new(:id)
9
+ def full_name ; "[DELETED]"; end
10
+
11
+ def username ; "__deleted__"; end
12
+ end
13
+
14
+ def initialize(api_key:, api_token:, logger: Config.logger)
15
+ @logger = logger
16
+ @client = Trello::Client.new(
17
+ developer_public_key: api_key,
18
+ member_token: api_token
19
+ )
20
+ @members = {}
21
+ end
22
+
23
+ def fetch_board(board_id)
24
+ retrying do
25
+ board = @client.find(:boards, board_id)
26
+
27
+ Board.new(
28
+ id: board.id,
29
+ short_id: board.short_url.chars.last(8).join,
30
+ name: board.name,
31
+ lists: lists_from(board),
32
+ card_ids: @client.find_many(Trello::Card, "/boards/#{board_id}/cards", fields: "id").map(&:id),
33
+ last_activity_date: board.last_activity_date
34
+ )
35
+ end
36
+ end
37
+
38
+ def fetch_card(card_id)
39
+ retrying do
40
+ card = @client.find(:cards, card_id)
41
+
42
+ Card.new(
43
+ id: card.id,
44
+ short_id: card.short_url.chars.last(8).join,
45
+ name: card.name,
46
+ desc: card.desc,
47
+ labels: card.labels.map(&:name),
48
+ list_id: card.list_id,
49
+ comments: comments_from(card),
50
+ attachments: attachments_from(card),
51
+ checklists: checklists_from(card),
52
+ last_activity_date: card.last_activity_date
53
+ )
54
+ end
55
+ end
56
+
57
+ private
58
+
59
+ attr_reader :logger
60
+
61
+ def retrying(&block)
62
+ attempt = 0
63
+ begin
64
+ block.call
65
+ rescue Trello::Error => err
66
+ raise unless err.status_code == 429
67
+ attempt += 1
68
+ cooldown = 2**attempt + rand(2**attempt) - 2**(attempt - 1)
69
+ logger.warn "API limit exceeded, cool down for #{cooldown}s"
70
+ sleep cooldown
71
+ retry
72
+ end
73
+ end
74
+
75
+ def comments_from(card)
76
+ card.comments.map do |comment|
77
+ Comment.new(
78
+ text: comment.data["text"],
79
+ date: comment.date,
80
+ creator: member_from(comment)
81
+ )
82
+ end
83
+ end
84
+
85
+ def attachments_from(card)
86
+ card.attachments.map do |attach|
87
+ Attachment.new(
88
+ file_name: attach.file_name,
89
+ mime_type: attach.mime_type,
90
+ bytes: attach.bytes,
91
+ date: attach.date,
92
+ name: attach.name,
93
+ is_upload: attach.is_upload,
94
+ headers: attach.is_upload ? {"Authorization" => "OAuth oauth_consumer_key=\"#{@client.configuration.developer_public_key}\", oauth_token=\"#{@client.configuration.member_token}\""} : {},
95
+ url: attach.url
96
+ )
97
+ end
98
+ end
99
+
100
+ def lists_from(board)
101
+ board.lists.map do |list|
102
+ List.new(
103
+ id: list.id,
104
+ name: list.name,
105
+ short_id: board.short_url.chars.last(8).join
106
+ )
107
+ end
108
+ end
109
+
110
+ def checklists_from(card)
111
+ card.checklists.map do |checklist|
112
+ Checklist.new(
113
+ id: checklist.id,
114
+ name: checklist.name,
115
+ items: checklist_items_from(checklist)
116
+ )
117
+ end
118
+ end
119
+
120
+ def checklist_items_from(checklist)
121
+ checklist.items.map do |item|
122
+ ChecklistItem.new(
123
+ id: item.id,
124
+ name: item.name,
125
+ state: item.state
126
+ )
127
+ end
128
+ end
129
+
130
+ def member_from(comment)
131
+ return @members[comment.creator_id] if @members.has_key? comment.creator_id
132
+
133
+ member =
134
+ begin
135
+ @client.find(:members, comment.creator_id)
136
+ rescue Trello::Error => err
137
+ raise unless err.status_code == 404
138
+
139
+ DeletedMember.new(comment.creator_id)
140
+ end
141
+
142
+ @members[comment.creator_id] = Member.new(
143
+ id: member.id,
144
+ full_name: member.full_name,
145
+ username: member.username
146
+ )
147
+ end
148
+ end
149
+ end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "trellodon/schedulers/base"
4
+ require "concurrent"
5
+
6
+ module Trellodon
7
+ module Schedulers
8
+ class Concurrent < Base
9
+ using RubyNext
10
+
11
+ DEFAULT_MAX_THREADS = 4
12
+
13
+ attr_reader :max_threads
14
+
15
+ def initialize(threads_count = DEFAULT_MAX_THREADS)
16
+ @max_threads = threads_count
17
+ end
18
+
19
+ def post(&block)
20
+ ::Concurrent::Future.execute(executor: executor, &block)
21
+ end
22
+
23
+ private
24
+
25
+ def executor
26
+ ::Concurrent::FixedThreadPool.new(max_threads)
27
+ end
28
+ end
29
+ end
30
+ end
data/lib/trellodon/cli.rb CHANGED
@@ -1,36 +1,34 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "trellodon"
4
-
5
4
  require "tty-prompt"
5
+ require "fileutils"
6
6
 
7
7
  module Trellodon
8
8
  class CLI
9
- class Prompt
10
- def initialize
11
- @prompt = TTY::Prompt.new
12
- end
9
+ class Prompt < TTY::Prompt
10
+ include Singleton
13
11
 
14
12
  def ask_api_key
15
- @prompt.mask(
13
+ mask(
16
14
  "Provide your Developer API Key (see https://trello.com/app-key):"
17
15
  ) { |q| q.required true }
18
16
  end
19
17
 
20
18
  def ask_api_token
21
- @prompt.mask(
19
+ mask(
22
20
  "Provide your API token:"
23
21
  ) { |q| q.required true }
24
22
  end
25
23
 
26
24
  def ask_board
27
- @prompt.ask("Which board would you like to dump? (URL or ID)") do |q|
25
+ ask("Which board would you like to dump? (URL or ID)") do |q|
28
26
  q.required true
29
27
  end
30
28
  end
31
29
 
32
30
  def ask_folder
33
- @prompt.ask("Destination folder?", default: "./")
31
+ ask("Destination folder?", default: "./")
34
32
  end
35
33
  end
36
34
 
@@ -44,10 +42,15 @@ module Trellodon
44
42
  out: "Destination folder path"
45
43
  )
46
44
 
47
- extend_options do |opts, _|
48
- opts.banner = "Usage: trellodon dump\n"\
45
+ extend_options do |opts, config|
46
+ opts.banner = "Usage: trellodon dump \e[34m[options]\e[0m\n"\
49
47
  "Options:\n"
50
48
 
49
+ opts.on("--concurrency VALUE", "Amount of processing threads (default: 4)")
50
+ opts.on("--clear-auth", "Remove saved api credentials") do
51
+ config.reload(api_token: nil, api_key: nil)
52
+ end
53
+
51
54
  opts.on_tail("-v", "--version", "Print current version") do
52
55
  puts Trellodon::VERSION
53
56
  exit
@@ -59,8 +62,25 @@ module Trellodon
59
62
  end
60
63
  end
61
64
 
65
+ def resolve_config_path(_, _)
66
+ config_path
67
+ end
68
+
69
+ def config_path
70
+ File.join(Dir.home, ".config", "#{config_name}.yml")
71
+ end
72
+
73
+ def update_config_file!
74
+ dir = File.dirname(config_path)
75
+ FileUtils.mkdir_p(dir) unless File.exist?(dir)
76
+
77
+ yml = "api_token: #{api_token}\n"\
78
+ "api_key: #{api_key}"
79
+ File.write(config_path, yml)
80
+ end
81
+
62
82
  private def prompt
63
- @prompt ||= Prompt.new
83
+ Prompt.instance
64
84
  end
65
85
 
66
86
  def api_key
@@ -82,34 +102,65 @@ module Trellodon
82
102
 
83
103
  def initialize
84
104
  @config = Config.new
105
+ @scheduler = Schedulers::Inline.new
85
106
  end
86
107
 
87
108
  def run
88
109
  check_command!
89
110
 
90
- executor = Trellodon::APIExecutor.new(
111
+ need_to_fill_config = config.to_h[:api_key].nil?
112
+
113
+ executor = Trellodon::Executor.new(
91
114
  api_key: config.api_key,
92
115
  api_token: config.api_token,
93
116
  formatter: Trellodon::Formatters::Markdown.new(output_dir: config.out),
94
- scheduler: Schedulers::Threaded.new
117
+ scheduler: scheduler
95
118
  )
119
+
120
+ ask_save_credentials if need_to_fill_config
121
+
96
122
  executor.download(config.board)
97
123
  end
98
124
 
99
125
  private
100
126
 
101
- attr_reader :config
127
+ attr_reader :config, :scheduler
128
+
129
+ def prompt
130
+ Prompt.instance
131
+ end
102
132
 
103
133
  def check_command!
104
- command, = config.option_parser.permute!(ARGV)
134
+ params = {}
135
+ command, = config.option_parser.parse!(ARGV, into: params)
105
136
  unless command
106
137
  puts config.option_parser.help
107
138
  exit
108
139
  end
109
140
 
110
- return if command == "dump"
141
+ if command == "dump"
142
+ change_scheduler(params) if params.key?(:sync) || params.key?(:concurrency)
143
+ return
144
+ end
111
145
 
112
146
  raise "Unknown command: #{command}. Available commands: dump"
113
147
  end
148
+
149
+ def ask_save_credentials
150
+ answer = prompt.yes?("Would you like to save api creds so next time trellodon won't asking it?", default: true)
151
+ config.update_config_file! if answer
152
+ end
153
+
154
+ def change_scheduler(params)
155
+ return unless params[:concurrency]
156
+
157
+ threads = Integer(params[:concurrency])
158
+
159
+ return @scheduler = Schedulers::Inline.new if threads.zero?
160
+
161
+ raise "Value of concurrency option must be integer and greater than zero" unless threads.positive?
162
+
163
+ @scheduler = Schedulers::Concurrent.new(threads)
164
+ end
114
165
  end
115
166
  end
@@ -1,17 +1,23 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "trello"
4
-
5
4
  require "trellodon/entities"
6
5
 
7
6
  module Trellodon
8
- class APIClient
7
+ class Client
8
+ class DeletedMember < Struct.new(:id)
9
+ def full_name = "[DELETED]"
10
+
11
+ def username = "__deleted__"
12
+ end
13
+
9
14
  def initialize(api_key:, api_token:, logger: Config.logger)
10
15
  @logger = logger
11
16
  @client = Trello::Client.new(
12
17
  developer_public_key: api_key,
13
18
  member_token: api_token
14
19
  )
20
+ @members = {}
15
21
  end
16
22
 
17
23
  def fetch_board(board_id)
@@ -42,6 +48,7 @@ module Trellodon
42
48
  list_id: card.list_id,
43
49
  comments: comments_from(card),
44
50
  attachments: attachments_from(card),
51
+ checklists: checklists_from(card),
45
52
  last_activity_date: card.last_activity_date
46
53
  )
47
54
  end
@@ -57,15 +64,10 @@ module Trellodon
57
64
  block.call
58
65
  rescue Trello::Error => err
59
66
  raise unless err.status_code == 429
60
-
61
67
  attempt += 1
62
-
63
68
  cooldown = 2**attempt + rand(2**attempt) - 2**(attempt - 1)
64
-
65
69
  logger.warn "API limit exceeded, cool down for #{cooldown}s"
66
-
67
70
  sleep cooldown
68
-
69
71
  retry
70
72
  end
71
73
  end
@@ -75,7 +77,7 @@ module Trellodon
75
77
  Comment.new(
76
78
  text: comment.data["text"],
77
79
  date: comment.date,
78
- creator_id: comment.creator_id
80
+ creator: member_from(comment)
79
81
  )
80
82
  end
81
83
  end
@@ -86,6 +88,10 @@ module Trellodon
86
88
  file_name: attach.file_name,
87
89
  mime_type: attach.mime_type,
88
90
  bytes: attach.bytes,
91
+ date: attach.date,
92
+ name: attach.name,
93
+ is_upload: attach.is_upload,
94
+ headers: attach.is_upload ? {"Authorization" => "OAuth oauth_consumer_key=\"#{@client.configuration.developer_public_key}\", oauth_token=\"#{@client.configuration.member_token}\""} : {},
89
95
  url: attach.url
90
96
  )
91
97
  end
@@ -100,5 +106,44 @@ module Trellodon
100
106
  )
101
107
  end
102
108
  end
109
+
110
+ def checklists_from(card)
111
+ card.checklists.map do |checklist|
112
+ Checklist.new(
113
+ id: checklist.id,
114
+ name: checklist.name,
115
+ items: checklist_items_from(checklist)
116
+ )
117
+ end
118
+ end
119
+
120
+ def checklist_items_from(checklist)
121
+ checklist.items.map do |item|
122
+ ChecklistItem.new(
123
+ id: item.id,
124
+ name: item.name,
125
+ state: item.state
126
+ )
127
+ end
128
+ end
129
+
130
+ def member_from(comment)
131
+ return @members[comment.creator_id] if @members.has_key? comment.creator_id
132
+
133
+ member =
134
+ begin
135
+ @client.find(:members, comment.creator_id)
136
+ rescue Trello::Error => err
137
+ raise unless err.status_code == 404
138
+
139
+ DeletedMember.new(comment.creator_id)
140
+ end
141
+
142
+ @members[comment.creator_id] = Member.new(
143
+ id: member.id,
144
+ full_name: member.full_name,
145
+ username: member.username
146
+ )
147
+ end
103
148
  end
104
149
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- Attachment = Struct.new("Attachment", :file_name, :mime_type, :bytes, :url, keyword_init: true)
4
+ Attachment = Struct.new("Attachment", :file_name, :mime_type, :bytes, :date, :name, :is_upload, :headers, :url, keyword_init: true)
5
5
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- Card = Struct.new("Card", :id, :short_id, :name, :desc, :list_id, :labels, :comments, :attachments, :last_activity_date, keyword_init: true)
4
+ Card = Struct.new("Card", :id, :short_id, :name, :desc, :list_id, :labels, :comments, :attachments, :checklists, :last_activity_date, keyword_init: true)
5
5
  end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ Checklist = Struct.new("Checklist", :id, :name, :items, keyword_init: true)
5
+ end
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ ChecklistItem = Struct.new("ChecklistItem", :id, :name, :state, keyword_init: true) do
5
+ def checked?
6
+ state == "complete"
7
+ end
8
+ end
9
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- Comment = Struct.new("Comment", :text, :date, :creator_id, keyword_init: true)
4
+ Comment = Struct.new("Comment", :text, :date, :creator, keyword_init: true)
5
5
  end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ Member = Struct.new("Member", :id, :full_name, :username, keyword_init: true)
5
+ end
@@ -5,3 +5,6 @@ require "trellodon/entities/card"
5
5
  require "trellodon/entities/list"
6
6
  require "trellodon/entities/comment"
7
7
  require "trellodon/entities/attachment"
8
+ require "trellodon/entities/checklist"
9
+ require "trellodon/entities/checklist_item"
10
+ require "trellodon/entities/member"
@@ -0,0 +1,65 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ class Executor
5
+ BOARD_ID_REGEX = /\/b\/([^\/]+)\//
6
+ private_constant :BOARD_ID_REGEX
7
+
8
+ def initialize(api_key:, api_token:, scheduler:, formatter:, logger: Config.logger)
9
+ @api_key = api_key
10
+ @api_token = api_token
11
+ @formatter = formatter
12
+ @logger = logger
13
+ @scheduler = scheduler
14
+ check_credentials!
15
+
16
+ @client = Client.new(api_key: @api_key, api_token: @api_token)
17
+ end
18
+
19
+ def download(board_pointer)
20
+ extract_board_id(board_pointer)
21
+
22
+ startup_time = Time.now
23
+ logger.debug "Fetching board 🚀️️"
24
+ board = scheduler.post do
25
+ client.fetch_board(board_id).tap { formatter.board_added(_1) }
26
+ end.value
27
+
28
+ logger.debug "Fetching #{board.card_ids.size} cards from the board with comments and attachments 🐢"
29
+ board.card_ids.map do |card_id|
30
+ scheduler.post do
31
+ client.fetch_card(card_id).tap { formatter.card_added(_1) }
32
+ end
33
+ end.map(&:value)
34
+
35
+ formatter.finish
36
+ logger.debug "All Trello API requests finished in #{(Time.now - startup_time).to_i} seconds ⌛"
37
+ end
38
+
39
+ private
40
+
41
+ attr_reader :client, :board_id, :logger, :scheduler, :formatter
42
+
43
+ def check_credentials!
44
+ return if @api_key.to_s.size.positive? && @api_token.to_s.size.positive?
45
+
46
+ raise ArgumentError, "Missing credentials. Please fill out both api_key, api_token first."
47
+ end
48
+
49
+ def extract_board_id(board_pointer)
50
+ @board_id = if URI::DEFAULT_PARSER.make_regexp.match?(board_pointer)
51
+ parse_board_url(board_pointer)
52
+ else
53
+ board_pointer
54
+ end
55
+ end
56
+
57
+ def parse_board_url(board_url)
58
+ rel_path = URI.parse(board_url).request_uri
59
+ match_data = rel_path.match(BOARD_ID_REGEX)
60
+ raise ArgumentError, "Wrong trello board url" if match_data.nil? || match_data.size != 2
61
+
62
+ match_data[1]
63
+ end
64
+ end
65
+ end
@@ -9,13 +9,16 @@ module Trellodon
9
9
  class Markdown < Base
10
10
  attr_reader :output_dir
11
11
 
12
+ MD_FILENAME = "README.md"
13
+ ATTACHMENTS_DIRNAME = "attachments"
14
+
12
15
  def initialize(output_dir:, **opts)
13
16
  super(**opts)
14
17
  @output_dir = output_dir
15
18
  end
16
19
 
17
20
  def card_added(card)
18
- card_mdfile = File.join(card_path(card), "index.md")
21
+ card_mdfile = File.join(card_path(card), MD_FILENAME)
19
22
  raise "File #{card_mdfile} already exists" if File.exist?(card_mdfile)
20
23
 
21
24
  File.write(card_mdfile, format_card(card))
@@ -29,7 +32,7 @@ module Trellodon
29
32
  private
30
33
 
31
34
  def card_path(card)
32
- raise "Board is undefined" if @board.nil?
35
+ raise "Board is undefined" unless @board
33
36
  raise "List id is undefined" if card.list_id.nil? || card.list_id.empty?
34
37
  list = @board.get_list(card.list_id)
35
38
  raise "List #{card.list_id} is not found" if list.nil?
@@ -43,17 +46,16 @@ module Trellodon
43
46
  end
44
47
 
45
48
  def card_attachments_path(card)
46
- attachments_dir = File.join(card_path(card), "attachments")
49
+ attachments_dir = File.join(card_path(card), ATTACHMENTS_DIRNAME)
47
50
  FileUtils.mkdir_p(attachments_dir) unless File.directory?(attachments_dir)
48
51
  attachments_dir
49
52
  end
50
53
 
51
54
  def download_attachments(card)
52
- return # FIXME: 401 Unauthorized ???
53
- return if card.attachments.nil? || card.attachments.empty? # rubocop:disable Lint/UnreachableCode
55
+ return if card.attachments.nil? || card.attachments.empty?
54
56
  attachments_path = card_attachments_path(card)
55
- card.attachments.each do |att|
56
- IO.copy_stream(URI.parse(att.url).open, File.join(attachments_path, att.file_name))
57
+ card.attachments.select(&:is_upload).each do |att|
58
+ IO.copy_stream(URI.parse(att.url).open(att.headers), File.join(attachments_path, att.file_name))
57
59
  end
58
60
  end
59
61
 
@@ -61,7 +63,9 @@ module Trellodon
61
63
  create_card_header(card) +
62
64
  create_card_title(card) +
63
65
  create_card_description(card) +
64
- create_card_comments(card)
66
+ create_card_checklists(card) +
67
+ create_card_comments(card) +
68
+ create_card_attachments(card)
65
69
  end
66
70
 
67
71
  def create_card_header(card)
@@ -85,13 +89,35 @@ module Trellodon
85
89
  <<~EOS
86
90
 
87
91
  ## Description
92
+
88
93
  #{card.desc}
89
94
  EOS
90
95
  end
91
96
 
92
97
  def create_card_labels(card)
93
98
  return "" if card.labels.nil? || card.labels.empty?
94
- card.labels.map { |label| "\n - " + label }.reduce(:+)
99
+ card.labels.map { |label| "\n - " + label }.join
100
+ end
101
+
102
+ def create_card_checklists(card)
103
+ return "" if card.checklists.nil? || card.checklists.empty?
104
+ <<~EOS
105
+
106
+ ## Checklists
107
+ #{card.checklists.map { |checklist| create_card_checklist(checklist) }.join}
108
+ EOS
109
+ end
110
+
111
+ def create_card_checklist(checklist)
112
+ <<~EOS
113
+
114
+ ### #{checklist.name}
115
+ #{checklist.items&.map { |item| create_card_checklist_item(item) }&.join}
116
+ EOS
117
+ end
118
+
119
+ def create_card_checklist_item(item)
120
+ "\n- [#{item.checked? ? "x" : " "}] #{item.name}"
95
121
  end
96
122
 
97
123
  def create_card_comments(card)
@@ -99,17 +125,42 @@ module Trellodon
99
125
  <<~EOS
100
126
 
101
127
  ## Comments
102
- #{card.comments.map { |comment| create_card_comment(comment) }.reduce(:+)}
128
+ #{card.comments.map { |comment| create_card_comment(comment) }.join}
103
129
  EOS
104
130
  end
105
131
 
106
132
  def create_card_comment(comment)
107
133
  <<~EOS
108
134
 
109
- **#{comment.creator_id} @ #{comment.date}**
135
+ **#{comment.creator.full_name} (@#{comment.creator.username}) at #{comment.date}**
110
136
  #{comment.text}
111
137
  EOS
112
138
  end
139
+
140
+ def create_card_attachments(card)
141
+ return "" if card.attachments.nil? || card.attachments.empty?
142
+ <<~EOS
143
+
144
+ ## Attachments
145
+ #{card.attachments.map { |attachment| create_card_attachment(attachment) }.join}
146
+ EOS
147
+ end
148
+
149
+ def create_card_attachment(attachment)
150
+ <<~EOS
151
+
152
+ ### #{attachment.name}
153
+
154
+ **date**: #{attachment.date}
155
+ **url**: #{attachment.url}
156
+ **local copy**: #{create_attachment_local_link(attachment)}
157
+ EOS
158
+ end
159
+
160
+ def create_attachment_local_link(attachment)
161
+ return "-" unless attachment.is_upload
162
+ "[#{attachment.file_name}](#{File.join("./", ATTACHMENTS_DIRNAME, attachment.file_name)})"
163
+ end
113
164
  end
114
165
  end
115
166
  end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "trellodon/schedulers/base"
4
+ require "concurrent"
5
+
6
+ module Trellodon
7
+ module Schedulers
8
+ class Concurrent < Base
9
+ using RubyNext
10
+
11
+ DEFAULT_MAX_THREADS = 4
12
+
13
+ attr_reader :max_threads
14
+
15
+ def initialize(threads_count = DEFAULT_MAX_THREADS)
16
+ @max_threads = threads_count
17
+ end
18
+
19
+ def post(&block)
20
+ ::Concurrent::Future.execute(executor:, &block)
21
+ end
22
+
23
+ private
24
+
25
+ def executor
26
+ ::Concurrent::FixedThreadPool.new(max_threads)
27
+ end
28
+ end
29
+ end
30
+ end
@@ -3,3 +3,4 @@
3
3
  require "trellodon/schedulers/base"
4
4
  require "trellodon/schedulers/inline"
5
5
  require "trellodon/schedulers/threaded"
6
+ require "trellodon/schedulers/concurrent"
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- VERSION = "0.1.0"
4
+ VERSION = "0.3.0"
5
5
  end
data/lib/trellodon.rb CHANGED
@@ -1,8 +1,14 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "ruby-next"
4
+
5
+ require "ruby-next/language/setup"
6
+ RubyNext::Language.setup_gem_load_path(transpile: true)
7
+
3
8
  require "trellodon/version"
4
9
  require "trellodon/config"
5
- require "trellodon/api_executor"
10
+ require "trellodon/client"
11
+ require "trellodon/executor"
6
12
 
7
13
  require "trellodon/entities"
8
14
  require "trellodon/schedulers"
metadata CHANGED
@@ -1,15 +1,16 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: trellodon
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - fargelus
8
8
  - rinasergeeva
9
+ - palkan
9
10
  autorequire:
10
11
  bindir: bin
11
12
  cert_chain: []
12
- date: 2022-04-08 00:00:00.000000000 Z
13
+ date: 2022-04-12 00:00:00.000000000 Z
13
14
  dependencies:
14
15
  - !ruby/object:Gem::Dependency
15
16
  name: anyway_config
@@ -53,12 +54,41 @@ dependencies:
53
54
  - - ">="
54
55
  - !ruby/object:Gem::Version
55
56
  version: '0'
57
+ - !ruby/object:Gem::Dependency
58
+ name: concurrent-ruby
59
+ requirement: !ruby/object:Gem::Requirement
60
+ requirements:
61
+ - - ">="
62
+ - !ruby/object:Gem::Version
63
+ version: '0'
64
+ type: :runtime
65
+ prerelease: false
66
+ version_requirements: !ruby/object:Gem::Requirement
67
+ requirements:
68
+ - - ">="
69
+ - !ruby/object:Gem::Version
70
+ version: '0'
71
+ - !ruby/object:Gem::Dependency
72
+ name: ruby-next-core
73
+ requirement: !ruby/object:Gem::Requirement
74
+ requirements:
75
+ - - ">="
76
+ - !ruby/object:Gem::Version
77
+ version: 0.15.0
78
+ type: :runtime
79
+ prerelease: false
80
+ version_requirements: !ruby/object:Gem::Requirement
81
+ requirements:
82
+ - - ">="
83
+ - !ruby/object:Gem::Version
84
+ version: 0.15.0
56
85
  description: "\n The main purpose of Trellodon is to make it possible to backup
57
86
  Trello boards to file system in a human-readable form (e.g., Markdown files).\n
58
87
  \ "
59
88
  email:
60
89
  - ddraudred@gmail.com
61
90
  - catherine.sergeeva@gmail.com
91
+ - dementiev.vm@gmail.com
62
92
  executables:
63
93
  - trellodon
64
94
  extensions: []
@@ -70,22 +100,29 @@ files:
70
100
  - bin/console
71
101
  - bin/setup
72
102
  - bin/trellodon
103
+ - lib/.rbnext/2.7/trellodon/executor.rb
104
+ - lib/.rbnext/3.0/trellodon/client.rb
105
+ - lib/.rbnext/3.1/trellodon/schedulers/concurrent.rb
73
106
  - lib/trellodon.rb
74
- - lib/trellodon/api_client.rb
75
- - lib/trellodon/api_executor.rb
76
107
  - lib/trellodon/cli.rb
108
+ - lib/trellodon/client.rb
77
109
  - lib/trellodon/config.rb
78
110
  - lib/trellodon/entities.rb
79
111
  - lib/trellodon/entities/attachment.rb
80
112
  - lib/trellodon/entities/board.rb
81
113
  - lib/trellodon/entities/card.rb
114
+ - lib/trellodon/entities/checklist.rb
115
+ - lib/trellodon/entities/checklist_item.rb
82
116
  - lib/trellodon/entities/comment.rb
83
117
  - lib/trellodon/entities/list.rb
118
+ - lib/trellodon/entities/member.rb
119
+ - lib/trellodon/executor.rb
84
120
  - lib/trellodon/formatters.rb
85
121
  - lib/trellodon/formatters/base.rb
86
122
  - lib/trellodon/formatters/markdown.rb
87
123
  - lib/trellodon/schedulers.rb
88
124
  - lib/trellodon/schedulers/base.rb
125
+ - lib/trellodon/schedulers/concurrent.rb
89
126
  - lib/trellodon/schedulers/inline.rb
90
127
  - lib/trellodon/schedulers/threaded.rb
91
128
  - lib/trellodon/version.rb