trellodon 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: bf1b8715b6cbad2c1bd3ab4fce72198a442380cf4f2d7bb39c55928ef72b032f
4
- data.tar.gz: 2caa2a3a24fe463cc1cb730a3aad68928323ec0fea04fea8e3a0f10ebf051ea7
3
+ metadata.gz: a61911e92509a77f13aa90f9f56c14d038dbe443b4e1063d28f66b6745b04432
4
+ data.tar.gz: '079609a773107e786a7d795fe7d23564ded2aa66e8d587a6fd22b4e288e15478'
5
5
  SHA512:
6
- metadata.gz: b50b3279f7a9e787b2f2e718109902a9cda88729f7d7b330aa5db9db944f913af33d6f26fab7b1f6c4d2866a1241d62d99c4dbe989ea468499bcaaa4908a153f
7
- data.tar.gz: 52d86ebd0b43ad0b064facbfb66332623422b2c688eff732188b7ea2ce8baba2842a331a941e1aafb410f114c31aef3ec80f7588b7f6a07808070019f9eb12ba
6
+ metadata.gz: 617119b48c1f2385c87d57ffd2683933097ed1aae11a51577f65e16012ab1f58b65187f57884123ed04ba522797925826e3cf202e7d4dad9d1fd63a3d736bcef
7
+ data.tar.gz: 30c1f1bcb2df2c69a3c8d0cebc93a597f16d2207d33bef979670134d306bd0173cc97948f38f739cf1d6b5eef0a7a602112fb0de40021d078d0188280ca4d039
data/CHANGELOG.md CHANGED
@@ -1,5 +1,31 @@
1
- ## [Unreleased]
1
+ # Change log
2
+
3
+ ## [master]
4
+
5
+ ## [0.3.0] - 2022-04-12 🚀🚀🚀
6
+
7
+ - Handle missing members. ([@palkan][])
8
+
9
+ - Use inline scheduler by default. ([@palkan][])
10
+
11
+ - Replace `--sync` with `--concurrency=0`. ([@palkan][])
12
+
13
+ ## [0.2.1] - 2022-04-12 🚀
14
+
15
+ - Misc changes.
16
+
17
+ ## [0.2.0] - 2022-04-12
18
+
19
+ - Store Trello API creds locally for avoid asking it before execution ([@fargelus][]).
20
+ - Download attachments, human names in comments, rename index.md to README.md ([@rinasergeeva][]).
21
+ - Added minimal project README.md ([@rinasergeeva][], [@fargelus][]).
22
+ - --sync param for switch to `Schedulers::Inline` in CLI ([@fargelus][]).
23
+ - Concurrent scheduler with customizable thread pool size ([@fargelus][]).
24
+ - Checklists support ([@rinasergeeva][]).
2
25
 
3
26
  ## [0.1.0] - 2022-04-07
4
27
 
5
28
  - Initial release
29
+
30
+ [@fargelus]: https://github.com/fargelus
31
+ [@rinasergeeva]: https://github.com/rinasergeeva
data/README.md CHANGED
@@ -1,3 +1,92 @@
1
- # trellodon
1
+ ![Build](https://github.com/evilmartians/trellodon/workflows/Build/badge.svg)
2
+ [![Gem Version](https://badge.fury.io/rb/trellodon.svg)](https://rubygems.org/gems/trellodon)
2
3
 
3
- Dump Trello boards to your file system
4
+ # Trellodon
5
+
6
+ A Ruby tool to export a Trello board and convert it into a set of folders and markdown files, corresponding to lists and cards on the board. For each card, you’ll get the basic details such as:
7
+
8
+ * Name
9
+ * Description
10
+ * Labels
11
+ * Comments
12
+ * Attachments
13
+
14
+ <img src="https://evilmartians.com/badges/sponsored-by-evil-martians.svg" alt="Sponsored by Evil Martians" width="236" height="54"></a>
15
+
16
+ ## Installation
17
+
18
+ ```
19
+ gem install trellodon
20
+ ```
21
+
22
+ ## Prerequisites
23
+
24
+ Trellodon needs two secret codes to download your boards:
25
+
26
+ * Trello API key
27
+ * Authentication token
28
+
29
+ To generate them, go to the [trello.com/app-key](https://trello.com/app-key) page and follow the instructions.
30
+
31
+ ## Usage
32
+
33
+ ```sh
34
+ Usage: trellodon dump [options]
35
+ Options:
36
+ --board VALUE Board URL or ID
37
+ --out VALUE Destination folder path
38
+ --concurrency VALUE Amount of processing threads (default: 4). Set to 0 to execute API requests in-line
39
+ --clear-auth Remove saved api credentials
40
+ -v, --version Print current version
41
+ -h, --help Print help
42
+ ```
43
+
44
+ ### Detailed example
45
+
46
+ Suppose we have a Trello board called `projects` with lists `Brainstorm`, `TODO`, `DOING`, `DONE`. Lists contains own cards.<br>
47
+ All Trello boards have its own id this id requires Trellodon to fetch board via Trello API.
48
+ You can find this id in two ways:
49
+
50
+ * In URL `https://trello.com/b/{id}/projects` your Trello board id will be placed as `{id}` in example;
51
+ * Put `.json` in your board URL like this `https://trello.com/b/{id}/projects.json` id field in json output is what you need;
52
+
53
+ You can also paste whole board URL and Trellodon will parse it correctly.<br>
54
+ After launch Trellodon creates follow output in specified folder:
55
+
56
+ ```sh
57
+ projects/
58
+ Brainstorm/
59
+ first_card_title/
60
+ README.md
61
+ attachments/
62
+ image.png
63
+ TODO/
64
+ first_card_title/
65
+ README.md
66
+ attachments/
67
+ report.docx
68
+ ...
69
+ ```
70
+
71
+ Each card has follow output format:
72
+
73
+ ```md
74
+ ---
75
+ title: card_title
76
+ last_updated_at: 2022-03-16 16:28:39 UTC
77
+ labels: Test
78
+ ---
79
+
80
+ # card_title
81
+
82
+ ## Description
83
+ Some card description
84
+
85
+ ## Comments
86
+ ** John Doe @doe at 2022-03-16 16:28:39 UTC**
87
+
88
+ ## Attachments
89
+ ### Image.png
90
+ **date**: 2022-03-16
91
+ **url**: https://trello.com/1/cards/#{card_id}/attachments/#{attachment.id}/download/image.png
92
+ ```
@@ -1,10 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "trellodon/api_client"
4
- require "trellodon/config"
5
-
6
3
  module Trellodon
7
- class APIExecutor
4
+ class Executor
8
5
  BOARD_ID_REGEX = /\/b\/([^\/]+)\//
9
6
  private_constant :BOARD_ID_REGEX
10
7
 
@@ -16,7 +13,7 @@ module Trellodon
16
13
  @scheduler = scheduler
17
14
  check_credentials!
18
15
 
19
- @api_client = APIClient.new(api_key: @api_key, api_token: @api_token)
16
+ @client = Client.new(api_key: @api_key, api_token: @api_token)
20
17
  end
21
18
 
22
19
  def download(board_pointer)
@@ -25,13 +22,13 @@ module Trellodon
25
22
  startup_time = Time.now
26
23
  logger.debug "Fetching board 🚀️️"
27
24
  board = scheduler.post do
28
- api_client.fetch_board(board_id).tap { formatter.board_added(_1) }
25
+ client.fetch_board(board_id).tap { |_1| formatter.board_added(_1) }
29
26
  end.value
30
27
 
31
- logger.debug "Fetching cards in board with comments and attachments 🐢"
28
+ logger.debug "Fetching #{board.card_ids.size} cards from the board with comments and attachments 🐢"
32
29
  board.card_ids.map do |card_id|
33
30
  scheduler.post do
34
- api_client.fetch_card(card_id).tap { formatter.card_added(_1) }
31
+ client.fetch_card(card_id).tap { |_1| formatter.card_added(_1) }
35
32
  end
36
33
  end.map(&:value)
37
34
 
@@ -41,7 +38,7 @@ module Trellodon
41
38
 
42
39
  private
43
40
 
44
- attr_reader :api_client, :board_id, :logger, :scheduler, :formatter
41
+ attr_reader :client, :board_id, :logger, :scheduler, :formatter
45
42
 
46
43
  def check_credentials!
47
44
  return if @api_key.to_s.size.positive? && @api_token.to_s.size.positive?
@@ -0,0 +1,149 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "trello"
4
+ require "trellodon/entities"
5
+
6
+ module Trellodon
7
+ class Client
8
+ class DeletedMember < Struct.new(:id)
9
+ def full_name ; "[DELETED]"; end
10
+
11
+ def username ; "__deleted__"; end
12
+ end
13
+
14
+ def initialize(api_key:, api_token:, logger: Config.logger)
15
+ @logger = logger
16
+ @client = Trello::Client.new(
17
+ developer_public_key: api_key,
18
+ member_token: api_token
19
+ )
20
+ @members = {}
21
+ end
22
+
23
+ def fetch_board(board_id)
24
+ retrying do
25
+ board = @client.find(:boards, board_id)
26
+
27
+ Board.new(
28
+ id: board.id,
29
+ short_id: board.short_url.chars.last(8).join,
30
+ name: board.name,
31
+ lists: lists_from(board),
32
+ card_ids: @client.find_many(Trello::Card, "/boards/#{board_id}/cards", fields: "id").map(&:id),
33
+ last_activity_date: board.last_activity_date
34
+ )
35
+ end
36
+ end
37
+
38
+ def fetch_card(card_id)
39
+ retrying do
40
+ card = @client.find(:cards, card_id)
41
+
42
+ Card.new(
43
+ id: card.id,
44
+ short_id: card.short_url.chars.last(8).join,
45
+ name: card.name,
46
+ desc: card.desc,
47
+ labels: card.labels.map(&:name),
48
+ list_id: card.list_id,
49
+ comments: comments_from(card),
50
+ attachments: attachments_from(card),
51
+ checklists: checklists_from(card),
52
+ last_activity_date: card.last_activity_date
53
+ )
54
+ end
55
+ end
56
+
57
+ private
58
+
59
+ attr_reader :logger
60
+
61
+ def retrying(&block)
62
+ attempt = 0
63
+ begin
64
+ block.call
65
+ rescue Trello::Error => err
66
+ raise unless err.status_code == 429
67
+ attempt += 1
68
+ cooldown = 2**attempt + rand(2**attempt) - 2**(attempt - 1)
69
+ logger.warn "API limit exceeded, cool down for #{cooldown}s"
70
+ sleep cooldown
71
+ retry
72
+ end
73
+ end
74
+
75
+ def comments_from(card)
76
+ card.comments.map do |comment|
77
+ Comment.new(
78
+ text: comment.data["text"],
79
+ date: comment.date,
80
+ creator: member_from(comment)
81
+ )
82
+ end
83
+ end
84
+
85
+ def attachments_from(card)
86
+ card.attachments.map do |attach|
87
+ Attachment.new(
88
+ file_name: attach.file_name,
89
+ mime_type: attach.mime_type,
90
+ bytes: attach.bytes,
91
+ date: attach.date,
92
+ name: attach.name,
93
+ is_upload: attach.is_upload,
94
+ headers: attach.is_upload ? {"Authorization" => "OAuth oauth_consumer_key=\"#{@client.configuration.developer_public_key}\", oauth_token=\"#{@client.configuration.member_token}\""} : {},
95
+ url: attach.url
96
+ )
97
+ end
98
+ end
99
+
100
+ def lists_from(board)
101
+ board.lists.map do |list|
102
+ List.new(
103
+ id: list.id,
104
+ name: list.name,
105
+ short_id: board.short_url.chars.last(8).join
106
+ )
107
+ end
108
+ end
109
+
110
+ def checklists_from(card)
111
+ card.checklists.map do |checklist|
112
+ Checklist.new(
113
+ id: checklist.id,
114
+ name: checklist.name,
115
+ items: checklist_items_from(checklist)
116
+ )
117
+ end
118
+ end
119
+
120
+ def checklist_items_from(checklist)
121
+ checklist.items.map do |item|
122
+ ChecklistItem.new(
123
+ id: item.id,
124
+ name: item.name,
125
+ state: item.state
126
+ )
127
+ end
128
+ end
129
+
130
+ def member_from(comment)
131
+ return @members[comment.creator_id] if @members.has_key? comment.creator_id
132
+
133
+ member =
134
+ begin
135
+ @client.find(:members, comment.creator_id)
136
+ rescue Trello::Error => err
137
+ raise unless err.status_code == 404
138
+
139
+ DeletedMember.new(comment.creator_id)
140
+ end
141
+
142
+ @members[comment.creator_id] = Member.new(
143
+ id: member.id,
144
+ full_name: member.full_name,
145
+ username: member.username
146
+ )
147
+ end
148
+ end
149
+ end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "trellodon/schedulers/base"
4
+ require "concurrent"
5
+
6
+ module Trellodon
7
+ module Schedulers
8
+ class Concurrent < Base
9
+ using RubyNext
10
+
11
+ DEFAULT_MAX_THREADS = 4
12
+
13
+ attr_reader :max_threads
14
+
15
+ def initialize(threads_count = DEFAULT_MAX_THREADS)
16
+ @max_threads = threads_count
17
+ end
18
+
19
+ def post(&block)
20
+ ::Concurrent::Future.execute(executor: executor, &block)
21
+ end
22
+
23
+ private
24
+
25
+ def executor
26
+ ::Concurrent::FixedThreadPool.new(max_threads)
27
+ end
28
+ end
29
+ end
30
+ end
data/lib/trellodon/cli.rb CHANGED
@@ -1,36 +1,34 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "trellodon"
4
-
5
4
  require "tty-prompt"
5
+ require "fileutils"
6
6
 
7
7
  module Trellodon
8
8
  class CLI
9
- class Prompt
10
- def initialize
11
- @prompt = TTY::Prompt.new
12
- end
9
+ class Prompt < TTY::Prompt
10
+ include Singleton
13
11
 
14
12
  def ask_api_key
15
- @prompt.mask(
13
+ mask(
16
14
  "Provide your Developer API Key (see https://trello.com/app-key):"
17
15
  ) { |q| q.required true }
18
16
  end
19
17
 
20
18
  def ask_api_token
21
- @prompt.mask(
19
+ mask(
22
20
  "Provide your API token:"
23
21
  ) { |q| q.required true }
24
22
  end
25
23
 
26
24
  def ask_board
27
- @prompt.ask("Which board would you like to dump? (URL or ID)") do |q|
25
+ ask("Which board would you like to dump? (URL or ID)") do |q|
28
26
  q.required true
29
27
  end
30
28
  end
31
29
 
32
30
  def ask_folder
33
- @prompt.ask("Destination folder?", default: "./")
31
+ ask("Destination folder?", default: "./")
34
32
  end
35
33
  end
36
34
 
@@ -44,10 +42,15 @@ module Trellodon
44
42
  out: "Destination folder path"
45
43
  )
46
44
 
47
- extend_options do |opts, _|
48
- opts.banner = "Usage: trellodon dump\n"\
45
+ extend_options do |opts, config|
46
+ opts.banner = "Usage: trellodon dump \e[34m[options]\e[0m\n"\
49
47
  "Options:\n"
50
48
 
49
+ opts.on("--concurrency VALUE", "Amount of processing threads (default: 4)")
50
+ opts.on("--clear-auth", "Remove saved api credentials") do
51
+ config.reload(api_token: nil, api_key: nil)
52
+ end
53
+
51
54
  opts.on_tail("-v", "--version", "Print current version") do
52
55
  puts Trellodon::VERSION
53
56
  exit
@@ -59,8 +62,25 @@ module Trellodon
59
62
  end
60
63
  end
61
64
 
65
+ def resolve_config_path(_, _)
66
+ config_path
67
+ end
68
+
69
+ def config_path
70
+ File.join(Dir.home, ".config", "#{config_name}.yml")
71
+ end
72
+
73
+ def update_config_file!
74
+ dir = File.dirname(config_path)
75
+ FileUtils.mkdir_p(dir) unless File.exist?(dir)
76
+
77
+ yml = "api_token: #{api_token}\n"\
78
+ "api_key: #{api_key}"
79
+ File.write(config_path, yml)
80
+ end
81
+
62
82
  private def prompt
63
- @prompt ||= Prompt.new
83
+ Prompt.instance
64
84
  end
65
85
 
66
86
  def api_key
@@ -82,34 +102,65 @@ module Trellodon
82
102
 
83
103
  def initialize
84
104
  @config = Config.new
105
+ @scheduler = Schedulers::Inline.new
85
106
  end
86
107
 
87
108
  def run
88
109
  check_command!
89
110
 
90
- executor = Trellodon::APIExecutor.new(
111
+ need_to_fill_config = config.to_h[:api_key].nil?
112
+
113
+ executor = Trellodon::Executor.new(
91
114
  api_key: config.api_key,
92
115
  api_token: config.api_token,
93
116
  formatter: Trellodon::Formatters::Markdown.new(output_dir: config.out),
94
- scheduler: Schedulers::Threaded.new
117
+ scheduler: scheduler
95
118
  )
119
+
120
+ ask_save_credentials if need_to_fill_config
121
+
96
122
  executor.download(config.board)
97
123
  end
98
124
 
99
125
  private
100
126
 
101
- attr_reader :config
127
+ attr_reader :config, :scheduler
128
+
129
+ def prompt
130
+ Prompt.instance
131
+ end
102
132
 
103
133
  def check_command!
104
- command, = config.option_parser.permute!(ARGV)
134
+ params = {}
135
+ command, = config.option_parser.parse!(ARGV, into: params)
105
136
  unless command
106
137
  puts config.option_parser.help
107
138
  exit
108
139
  end
109
140
 
110
- return if command == "dump"
141
+ if command == "dump"
142
+ change_scheduler(params) if params.key?(:sync) || params.key?(:concurrency)
143
+ return
144
+ end
111
145
 
112
146
  raise "Unknown command: #{command}. Available commands: dump"
113
147
  end
148
+
149
+ def ask_save_credentials
150
+ answer = prompt.yes?("Would you like to save api creds so next time trellodon won't asking it?", default: true)
151
+ config.update_config_file! if answer
152
+ end
153
+
154
+ def change_scheduler(params)
155
+ return unless params[:concurrency]
156
+
157
+ threads = Integer(params[:concurrency])
158
+
159
+ return @scheduler = Schedulers::Inline.new if threads.zero?
160
+
161
+ raise "Value of concurrency option must be integer and greater than zero" unless threads.positive?
162
+
163
+ @scheduler = Schedulers::Concurrent.new(threads)
164
+ end
114
165
  end
115
166
  end
@@ -1,17 +1,23 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "trello"
4
-
5
4
  require "trellodon/entities"
6
5
 
7
6
  module Trellodon
8
- class APIClient
7
+ class Client
8
+ class DeletedMember < Struct.new(:id)
9
+ def full_name = "[DELETED]"
10
+
11
+ def username = "__deleted__"
12
+ end
13
+
9
14
  def initialize(api_key:, api_token:, logger: Config.logger)
10
15
  @logger = logger
11
16
  @client = Trello::Client.new(
12
17
  developer_public_key: api_key,
13
18
  member_token: api_token
14
19
  )
20
+ @members = {}
15
21
  end
16
22
 
17
23
  def fetch_board(board_id)
@@ -42,6 +48,7 @@ module Trellodon
42
48
  list_id: card.list_id,
43
49
  comments: comments_from(card),
44
50
  attachments: attachments_from(card),
51
+ checklists: checklists_from(card),
45
52
  last_activity_date: card.last_activity_date
46
53
  )
47
54
  end
@@ -57,15 +64,10 @@ module Trellodon
57
64
  block.call
58
65
  rescue Trello::Error => err
59
66
  raise unless err.status_code == 429
60
-
61
67
  attempt += 1
62
-
63
68
  cooldown = 2**attempt + rand(2**attempt) - 2**(attempt - 1)
64
-
65
69
  logger.warn "API limit exceeded, cool down for #{cooldown}s"
66
-
67
70
  sleep cooldown
68
-
69
71
  retry
70
72
  end
71
73
  end
@@ -75,7 +77,7 @@ module Trellodon
75
77
  Comment.new(
76
78
  text: comment.data["text"],
77
79
  date: comment.date,
78
- creator_id: comment.creator_id
80
+ creator: member_from(comment)
79
81
  )
80
82
  end
81
83
  end
@@ -86,6 +88,10 @@ module Trellodon
86
88
  file_name: attach.file_name,
87
89
  mime_type: attach.mime_type,
88
90
  bytes: attach.bytes,
91
+ date: attach.date,
92
+ name: attach.name,
93
+ is_upload: attach.is_upload,
94
+ headers: attach.is_upload ? {"Authorization" => "OAuth oauth_consumer_key=\"#{@client.configuration.developer_public_key}\", oauth_token=\"#{@client.configuration.member_token}\""} : {},
89
95
  url: attach.url
90
96
  )
91
97
  end
@@ -100,5 +106,44 @@ module Trellodon
100
106
  )
101
107
  end
102
108
  end
109
+
110
+ def checklists_from(card)
111
+ card.checklists.map do |checklist|
112
+ Checklist.new(
113
+ id: checklist.id,
114
+ name: checklist.name,
115
+ items: checklist_items_from(checklist)
116
+ )
117
+ end
118
+ end
119
+
120
+ def checklist_items_from(checklist)
121
+ checklist.items.map do |item|
122
+ ChecklistItem.new(
123
+ id: item.id,
124
+ name: item.name,
125
+ state: item.state
126
+ )
127
+ end
128
+ end
129
+
130
+ def member_from(comment)
131
+ return @members[comment.creator_id] if @members.has_key? comment.creator_id
132
+
133
+ member =
134
+ begin
135
+ @client.find(:members, comment.creator_id)
136
+ rescue Trello::Error => err
137
+ raise unless err.status_code == 404
138
+
139
+ DeletedMember.new(comment.creator_id)
140
+ end
141
+
142
+ @members[comment.creator_id] = Member.new(
143
+ id: member.id,
144
+ full_name: member.full_name,
145
+ username: member.username
146
+ )
147
+ end
103
148
  end
104
149
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- Attachment = Struct.new("Attachment", :file_name, :mime_type, :bytes, :url, keyword_init: true)
4
+ Attachment = Struct.new("Attachment", :file_name, :mime_type, :bytes, :date, :name, :is_upload, :headers, :url, keyword_init: true)
5
5
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- Card = Struct.new("Card", :id, :short_id, :name, :desc, :list_id, :labels, :comments, :attachments, :last_activity_date, keyword_init: true)
4
+ Card = Struct.new("Card", :id, :short_id, :name, :desc, :list_id, :labels, :comments, :attachments, :checklists, :last_activity_date, keyword_init: true)
5
5
  end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ Checklist = Struct.new("Checklist", :id, :name, :items, keyword_init: true)
5
+ end
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ ChecklistItem = Struct.new("ChecklistItem", :id, :name, :state, keyword_init: true) do
5
+ def checked?
6
+ state == "complete"
7
+ end
8
+ end
9
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- Comment = Struct.new("Comment", :text, :date, :creator_id, keyword_init: true)
4
+ Comment = Struct.new("Comment", :text, :date, :creator, keyword_init: true)
5
5
  end
@@ -0,0 +1,5 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ Member = Struct.new("Member", :id, :full_name, :username, keyword_init: true)
5
+ end
@@ -5,3 +5,6 @@ require "trellodon/entities/card"
5
5
  require "trellodon/entities/list"
6
6
  require "trellodon/entities/comment"
7
7
  require "trellodon/entities/attachment"
8
+ require "trellodon/entities/checklist"
9
+ require "trellodon/entities/checklist_item"
10
+ require "trellodon/entities/member"
@@ -0,0 +1,65 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Trellodon
4
+ class Executor
5
+ BOARD_ID_REGEX = /\/b\/([^\/]+)\//
6
+ private_constant :BOARD_ID_REGEX
7
+
8
+ def initialize(api_key:, api_token:, scheduler:, formatter:, logger: Config.logger)
9
+ @api_key = api_key
10
+ @api_token = api_token
11
+ @formatter = formatter
12
+ @logger = logger
13
+ @scheduler = scheduler
14
+ check_credentials!
15
+
16
+ @client = Client.new(api_key: @api_key, api_token: @api_token)
17
+ end
18
+
19
+ def download(board_pointer)
20
+ extract_board_id(board_pointer)
21
+
22
+ startup_time = Time.now
23
+ logger.debug "Fetching board 🚀️️"
24
+ board = scheduler.post do
25
+ client.fetch_board(board_id).tap { formatter.board_added(_1) }
26
+ end.value
27
+
28
+ logger.debug "Fetching #{board.card_ids.size} cards from the board with comments and attachments 🐢"
29
+ board.card_ids.map do |card_id|
30
+ scheduler.post do
31
+ client.fetch_card(card_id).tap { formatter.card_added(_1) }
32
+ end
33
+ end.map(&:value)
34
+
35
+ formatter.finish
36
+ logger.debug "All Trello API requests finished in #{(Time.now - startup_time).to_i} seconds ⌛"
37
+ end
38
+
39
+ private
40
+
41
+ attr_reader :client, :board_id, :logger, :scheduler, :formatter
42
+
43
+ def check_credentials!
44
+ return if @api_key.to_s.size.positive? && @api_token.to_s.size.positive?
45
+
46
+ raise ArgumentError, "Missing credentials. Please fill out both api_key, api_token first."
47
+ end
48
+
49
+ def extract_board_id(board_pointer)
50
+ @board_id = if URI::DEFAULT_PARSER.make_regexp.match?(board_pointer)
51
+ parse_board_url(board_pointer)
52
+ else
53
+ board_pointer
54
+ end
55
+ end
56
+
57
+ def parse_board_url(board_url)
58
+ rel_path = URI.parse(board_url).request_uri
59
+ match_data = rel_path.match(BOARD_ID_REGEX)
60
+ raise ArgumentError, "Wrong trello board url" if match_data.nil? || match_data.size != 2
61
+
62
+ match_data[1]
63
+ end
64
+ end
65
+ end
@@ -9,13 +9,16 @@ module Trellodon
9
9
  class Markdown < Base
10
10
  attr_reader :output_dir
11
11
 
12
+ MD_FILENAME = "README.md"
13
+ ATTACHMENTS_DIRNAME = "attachments"
14
+
12
15
  def initialize(output_dir:, **opts)
13
16
  super(**opts)
14
17
  @output_dir = output_dir
15
18
  end
16
19
 
17
20
  def card_added(card)
18
- card_mdfile = File.join(card_path(card), "index.md")
21
+ card_mdfile = File.join(card_path(card), MD_FILENAME)
19
22
  raise "File #{card_mdfile} already exists" if File.exist?(card_mdfile)
20
23
 
21
24
  File.write(card_mdfile, format_card(card))
@@ -29,7 +32,7 @@ module Trellodon
29
32
  private
30
33
 
31
34
  def card_path(card)
32
- raise "Board is undefined" if @board.nil?
35
+ raise "Board is undefined" unless @board
33
36
  raise "List id is undefined" if card.list_id.nil? || card.list_id.empty?
34
37
  list = @board.get_list(card.list_id)
35
38
  raise "List #{card.list_id} is not found" if list.nil?
@@ -43,17 +46,16 @@ module Trellodon
43
46
  end
44
47
 
45
48
  def card_attachments_path(card)
46
- attachments_dir = File.join(card_path(card), "attachments")
49
+ attachments_dir = File.join(card_path(card), ATTACHMENTS_DIRNAME)
47
50
  FileUtils.mkdir_p(attachments_dir) unless File.directory?(attachments_dir)
48
51
  attachments_dir
49
52
  end
50
53
 
51
54
  def download_attachments(card)
52
- return # FIXME: 401 Unauthorized ???
53
- return if card.attachments.nil? || card.attachments.empty? # rubocop:disable Lint/UnreachableCode
55
+ return if card.attachments.nil? || card.attachments.empty?
54
56
  attachments_path = card_attachments_path(card)
55
- card.attachments.each do |att|
56
- IO.copy_stream(URI.parse(att.url).open, File.join(attachments_path, att.file_name))
57
+ card.attachments.select(&:is_upload).each do |att|
58
+ IO.copy_stream(URI.parse(att.url).open(att.headers), File.join(attachments_path, att.file_name))
57
59
  end
58
60
  end
59
61
 
@@ -61,7 +63,9 @@ module Trellodon
61
63
  create_card_header(card) +
62
64
  create_card_title(card) +
63
65
  create_card_description(card) +
64
- create_card_comments(card)
66
+ create_card_checklists(card) +
67
+ create_card_comments(card) +
68
+ create_card_attachments(card)
65
69
  end
66
70
 
67
71
  def create_card_header(card)
@@ -85,13 +89,35 @@ module Trellodon
85
89
  <<~EOS
86
90
 
87
91
  ## Description
92
+
88
93
  #{card.desc}
89
94
  EOS
90
95
  end
91
96
 
92
97
  def create_card_labels(card)
93
98
  return "" if card.labels.nil? || card.labels.empty?
94
- card.labels.map { |label| "\n - " + label }.reduce(:+)
99
+ card.labels.map { |label| "\n - " + label }.join
100
+ end
101
+
102
+ def create_card_checklists(card)
103
+ return "" if card.checklists.nil? || card.checklists.empty?
104
+ <<~EOS
105
+
106
+ ## Checklists
107
+ #{card.checklists.map { |checklist| create_card_checklist(checklist) }.join}
108
+ EOS
109
+ end
110
+
111
+ def create_card_checklist(checklist)
112
+ <<~EOS
113
+
114
+ ### #{checklist.name}
115
+ #{checklist.items&.map { |item| create_card_checklist_item(item) }&.join}
116
+ EOS
117
+ end
118
+
119
+ def create_card_checklist_item(item)
120
+ "\n- [#{item.checked? ? "x" : " "}] #{item.name}"
95
121
  end
96
122
 
97
123
  def create_card_comments(card)
@@ -99,17 +125,42 @@ module Trellodon
99
125
  <<~EOS
100
126
 
101
127
  ## Comments
102
- #{card.comments.map { |comment| create_card_comment(comment) }.reduce(:+)}
128
+ #{card.comments.map { |comment| create_card_comment(comment) }.join}
103
129
  EOS
104
130
  end
105
131
 
106
132
  def create_card_comment(comment)
107
133
  <<~EOS
108
134
 
109
- **#{comment.creator_id} @ #{comment.date}**
135
+ **#{comment.creator.full_name} (@#{comment.creator.username}) at #{comment.date}**
110
136
  #{comment.text}
111
137
  EOS
112
138
  end
139
+
140
+ def create_card_attachments(card)
141
+ return "" if card.attachments.nil? || card.attachments.empty?
142
+ <<~EOS
143
+
144
+ ## Attachments
145
+ #{card.attachments.map { |attachment| create_card_attachment(attachment) }.join}
146
+ EOS
147
+ end
148
+
149
+ def create_card_attachment(attachment)
150
+ <<~EOS
151
+
152
+ ### #{attachment.name}
153
+
154
+ **date**: #{attachment.date}
155
+ **url**: #{attachment.url}
156
+ **local copy**: #{create_attachment_local_link(attachment)}
157
+ EOS
158
+ end
159
+
160
+ def create_attachment_local_link(attachment)
161
+ return "-" unless attachment.is_upload
162
+ "[#{attachment.file_name}](#{File.join("./", ATTACHMENTS_DIRNAME, attachment.file_name)})"
163
+ end
113
164
  end
114
165
  end
115
166
  end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "trellodon/schedulers/base"
4
+ require "concurrent"
5
+
6
+ module Trellodon
7
+ module Schedulers
8
+ class Concurrent < Base
9
+ using RubyNext
10
+
11
+ DEFAULT_MAX_THREADS = 4
12
+
13
+ attr_reader :max_threads
14
+
15
+ def initialize(threads_count = DEFAULT_MAX_THREADS)
16
+ @max_threads = threads_count
17
+ end
18
+
19
+ def post(&block)
20
+ ::Concurrent::Future.execute(executor:, &block)
21
+ end
22
+
23
+ private
24
+
25
+ def executor
26
+ ::Concurrent::FixedThreadPool.new(max_threads)
27
+ end
28
+ end
29
+ end
30
+ end
@@ -3,3 +3,4 @@
3
3
  require "trellodon/schedulers/base"
4
4
  require "trellodon/schedulers/inline"
5
5
  require "trellodon/schedulers/threaded"
6
+ require "trellodon/schedulers/concurrent"
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Trellodon
4
- VERSION = "0.1.0"
4
+ VERSION = "0.3.0"
5
5
  end
data/lib/trellodon.rb CHANGED
@@ -1,8 +1,14 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "ruby-next"
4
+
5
+ require "ruby-next/language/setup"
6
+ RubyNext::Language.setup_gem_load_path(transpile: true)
7
+
3
8
  require "trellodon/version"
4
9
  require "trellodon/config"
5
- require "trellodon/api_executor"
10
+ require "trellodon/client"
11
+ require "trellodon/executor"
6
12
 
7
13
  require "trellodon/entities"
8
14
  require "trellodon/schedulers"
metadata CHANGED
@@ -1,15 +1,16 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: trellodon
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - fargelus
8
8
  - rinasergeeva
9
+ - palkan
9
10
  autorequire:
10
11
  bindir: bin
11
12
  cert_chain: []
12
- date: 2022-04-08 00:00:00.000000000 Z
13
+ date: 2022-04-12 00:00:00.000000000 Z
13
14
  dependencies:
14
15
  - !ruby/object:Gem::Dependency
15
16
  name: anyway_config
@@ -53,12 +54,41 @@ dependencies:
53
54
  - - ">="
54
55
  - !ruby/object:Gem::Version
55
56
  version: '0'
57
+ - !ruby/object:Gem::Dependency
58
+ name: concurrent-ruby
59
+ requirement: !ruby/object:Gem::Requirement
60
+ requirements:
61
+ - - ">="
62
+ - !ruby/object:Gem::Version
63
+ version: '0'
64
+ type: :runtime
65
+ prerelease: false
66
+ version_requirements: !ruby/object:Gem::Requirement
67
+ requirements:
68
+ - - ">="
69
+ - !ruby/object:Gem::Version
70
+ version: '0'
71
+ - !ruby/object:Gem::Dependency
72
+ name: ruby-next-core
73
+ requirement: !ruby/object:Gem::Requirement
74
+ requirements:
75
+ - - ">="
76
+ - !ruby/object:Gem::Version
77
+ version: 0.15.0
78
+ type: :runtime
79
+ prerelease: false
80
+ version_requirements: !ruby/object:Gem::Requirement
81
+ requirements:
82
+ - - ">="
83
+ - !ruby/object:Gem::Version
84
+ version: 0.15.0
56
85
  description: "\n The main purpose of Trellodon is to make it possible to backup
57
86
  Trello boards to file system in a human-readable form (e.g., Markdown files).\n
58
87
  \ "
59
88
  email:
60
89
  - ddraudred@gmail.com
61
90
  - catherine.sergeeva@gmail.com
91
+ - dementiev.vm@gmail.com
62
92
  executables:
63
93
  - trellodon
64
94
  extensions: []
@@ -70,22 +100,29 @@ files:
70
100
  - bin/console
71
101
  - bin/setup
72
102
  - bin/trellodon
103
+ - lib/.rbnext/2.7/trellodon/executor.rb
104
+ - lib/.rbnext/3.0/trellodon/client.rb
105
+ - lib/.rbnext/3.1/trellodon/schedulers/concurrent.rb
73
106
  - lib/trellodon.rb
74
- - lib/trellodon/api_client.rb
75
- - lib/trellodon/api_executor.rb
76
107
  - lib/trellodon/cli.rb
108
+ - lib/trellodon/client.rb
77
109
  - lib/trellodon/config.rb
78
110
  - lib/trellodon/entities.rb
79
111
  - lib/trellodon/entities/attachment.rb
80
112
  - lib/trellodon/entities/board.rb
81
113
  - lib/trellodon/entities/card.rb
114
+ - lib/trellodon/entities/checklist.rb
115
+ - lib/trellodon/entities/checklist_item.rb
82
116
  - lib/trellodon/entities/comment.rb
83
117
  - lib/trellodon/entities/list.rb
118
+ - lib/trellodon/entities/member.rb
119
+ - lib/trellodon/executor.rb
84
120
  - lib/trellodon/formatters.rb
85
121
  - lib/trellodon/formatters/base.rb
86
122
  - lib/trellodon/formatters/markdown.rb
87
123
  - lib/trellodon/schedulers.rb
88
124
  - lib/trellodon/schedulers/base.rb
125
+ - lib/trellodon/schedulers/concurrent.rb
89
126
  - lib/trellodon/schedulers/inline.rb
90
127
  - lib/trellodon/schedulers/threaded.rb
91
128
  - lib/trellodon/version.rb