fyodor 0.2.1 → 0.2.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: f14c74a8744a5b6ecc59dbada1fb738913a965fac085dbdd6b1179fda4c4aaa1
4
- data.tar.gz: dbdea2a6e45f4c8a90cf435cc879f5408838590308e56f540efbe566de8434dd
3
+ metadata.gz: dae694865dbdb0187906ad21b610ac7f9486819141f86392816a41fbe835f71c
4
+ data.tar.gz: 97c07e5ba51877015426221aed2158a20436f36d8afbca165b8433c20050e02a
5
5
  SHA512:
6
- metadata.gz: 7e98fb7efcd144c55cebf5608d63143d6fc4f0e7cf734a482f918d653aa33a8368193ac50c860001c78006f2fb6332a35daa5647746a87b8d268a34318cf6fac
7
- data.tar.gz: d8fb7b506b1f911be98005be170e3db68c526a7490462eb213a0f1d132d21f88aecd5374e529730c914d1fa84a4080e81eed5ff824bd9114821e9be5e0cbf415
6
+ metadata.gz: ec0b9492639f2b77ba2523106db931b3363c42731cab3b456f83e6c7e91ccc6ec196d4cbe9d8798f62a5c452eed391fec521d7e8da56c537be51a224e93963bc
7
+ data.tar.gz: 359197fe0aeb85d76f1d025c2e3517a7fb02c60712989f79dd14be2004f50f3892a312b4114a3d29245d26a8f2e72f59dd4a2925c7c800f3dc63b435d63e4952
data/README.md CHANGED
@@ -6,7 +6,7 @@ Convert your Amazon Kindle highlights, notes and bookmarks into markdown files.
6
6
 
7
7
  This application parses `My Clippings.txt` from your Kindle and generates a markdown file for each book/document, in the format `#{Author} - #{Title}.md`. This way, your annotations are conveniently stored and easily managed.
8
8
 
9
- [For samples of the output, click here.](samples/)
9
+ [For samples of the output, click here.](docs/output_demo)
10
10
 
11
11
  To read more about the motivation and what problem it tries to solve, [check this blog post](http://rafaelc.org/blog/export-all-your-kindle-highlights-and-notes/).
12
12
 
@@ -30,18 +30,26 @@ Install Ruby and run:
30
30
  $ gem install fyodor
31
31
  ```
32
32
 
33
+ ## Updating
34
+
35
+ Run:
36
+
37
+ ```
38
+ $ gem update fyodor
39
+ ```
40
+
33
41
  ## Configuration
34
42
 
35
- Fyodor has an optional configuration file, which is used for the following features.
43
+ Fyodor has an optional configuration file, which is used for the following.
36
44
 
37
45
  ### Languages
38
46
 
39
- If your Kindle is not in English, you should tell Fyodor how some things are called by your `My Clippings.txt` (e.g. highlights, pages, etc). _Note that basic parsing should still work without configuration, but you won't take advantage of many features, resulting in a dirtier output._
47
+ If your Kindle is not in English, you should tell Fyodor how some things are called by your `My Clippings.txt` (e.g. highlights, pages, etc). _Fyodor should still work without configuration, but you won't take advantage of many features, resulting in a dirtier output._
40
48
 
41
49
  1. Download the sample config to `~/.config/fyodor.toml` or `$XDG_CONFIG_HOME/fyodor.toml`:
42
50
 
43
51
  ```
44
- $ wget https://raw.githubusercontent.com/rccavalcanti/fyodor/master/fyodor.toml.sample -O ~/.config/fyodor.toml
52
+ $ wget https://raw.githubusercontent.com/rccavalcanti/fyodor/master/docs/fyodor.toml.sample -O ~/.config/fyodor.toml
45
53
  ```
46
54
 
47
55
  2. Open both the configuration and your `My Clippings.txt` in your preferred editor. Change the values in the `[parser]` section to mirror what you get in `My Clippings.txt`.
@@ -74,8 +82,8 @@ Where:
74
82
  - `CLIPPINGS_FILE` is the path for `My Clippings.txt`.
75
83
  - `OUTPUT_DIR` is the directory to write the markdown files. If none is supplied, this will be `fyodor_output` under the current directory.
76
84
 
77
- ## LICENSE
85
+ ## License
78
86
 
79
- Released under [GNU GPL v3](LICENSE).
87
+ Licensed under [GPLv3](LICENSE)
80
88
 
81
- Copyright 2019 Rafael Cavalcanti <hi@rafaelc.org>
89
+ Copyright (C) 2019-2020 [Rafael Cavalcanti](https://rafaelc.org/)
data/bin/fyodor CHANGED
@@ -1,15 +1,59 @@
1
1
  #!/usr/bin/env ruby
2
2
 
3
- require "fyodor/cli"
4
-
5
- # Pass --trace to get the stack trace
6
- if ARGV.include?("--trace")
7
- ARGV.delete("--trace")
8
- Fyodor::CLI.new.main
9
- else
10
- begin
11
- Fyodor::CLI.new.main
12
- rescue StandardError => e
13
- abort(e.message)
3
+ require "fyodor"
4
+ require "fyodor/config_getter"
5
+ require "fyodor/stats_printer"
6
+ require "fyodor/version"
7
+ require "optimist"
8
+ require "pathname"
9
+
10
+ module Fyodor
11
+ class CLI
12
+ def main
13
+ parse_args
14
+ @config = ConfigGetter.new.config
15
+
16
+ if @cli_opts.trace
17
+ run
18
+ else
19
+ begin
20
+ run
21
+ rescue StandardError => e
22
+ abort(e.message)
23
+ end
24
+ end
25
+ end
26
+
27
+ private
28
+
29
+ def get_path(path)
30
+ Pathname.new(path).expand_path
31
+ end
32
+
33
+ def default_output_dir
34
+ Pathname.new(Dir.pwd) + "fyodor_output"
35
+ end
36
+
37
+ def run
38
+ library = Library.new
39
+ ClippingsParser.new(@clippings_path, @config["parser"]).parse(library)
40
+ StatsPrinter.new(library).print
41
+ OutputWriter.new(library, @output_dir, @config["output"]).write_all
42
+ end
43
+
44
+ def parse_args
45
+ @cli_opts = Optimist::options do
46
+ version "Fyodor v#{Fyodor::VERSION}"
47
+ synopsis "Usage: fyodor [options] my_clippings_path [output_dir]"
48
+
49
+ opt :trace, "Show backtrace", :default => false
50
+ end
51
+
52
+ Optimist::die "Wrong number of arguments." if ARGV.count != 1 && ARGV.count != 2
53
+ @clippings_path = get_path(ARGV[0])
54
+ @output_dir = ARGV[1].nil? ? default_output_dir : get_path(ARGV[1])
55
+ end
14
56
  end
15
57
  end
58
+
59
+ Fyodor::CLI.new.main
@@ -3,7 +3,6 @@ require_relative "entry_parser"
3
3
  module Fyodor
4
4
  class ClippingsParser
5
5
  SEPARATOR = /^==========\r?\n$/
6
- ENTRY_SIZE = 5
7
6
 
8
7
  def initialize(clippings_path, parser_config)
9
8
  @path = clippings_path
@@ -25,9 +24,7 @@ module Fyodor
25
24
  private
26
25
 
27
26
  def end_entry?(lines)
28
- return false if lines.size < ENTRY_SIZE
29
- return true if lines.size == ENTRY_SIZE && lines.last =~ SEPARATOR
30
- raise "MyClippings is badly formatted"
27
+ lines.last =~ SEPARATOR
31
28
  end
32
29
 
33
30
  def parse_entry(lines)
@@ -1,23 +1,35 @@
1
1
  module Fyodor
2
2
  class Entry
3
- TYPE = { note: "note",
4
- highlight: "highlight",
5
- bookmark: "bookmark",
6
- clip: "clip" }
7
-
8
- attr_reader :book_title, :book_author, :text, :desc, :type, :loc, :loc_start, :page, :time
9
-
10
- def initialize(attrs)
11
- @book_title = attrs[:book_title]
12
- @book_author = attrs[:book_author]
13
- @text = attrs[:text]
14
- @desc = attrs[:desc]
15
- @type = attrs[:type]
16
- @loc = attrs[:loc]
17
- # This is our comparable, we need it as a number.
18
- @loc_start = attrs[:loc_start].to_i
19
- @page = attrs[:page]
20
- @time = attrs[:time]
3
+ TYPE = {
4
+ note: "note",
5
+ highlight: "highlight",
6
+ bookmark: "bookmark",
7
+ clip: "clip"
8
+ }
9
+
10
+ ATTRS = [
11
+ :book_title,
12
+ :book_author,
13
+ :text,
14
+ :desc,
15
+ :type,
16
+ :loc,
17
+ :loc_start,
18
+ :page,
19
+ :page_start,
20
+ :time
21
+ ]
22
+
23
+ attr_reader *ATTRS
24
+
25
+ def initialize(args)
26
+ ATTRS.each do |attr|
27
+ instance_variable_set("@#{attr}", args[attr])
28
+ end
29
+
30
+ # These are our comparables. Let's make sure they are numbers.
31
+ @loc_start = @loc_start.to_i
32
+ @page_start = @page_start.to_i
21
33
 
22
34
  raise ArgumentError, "Invalid Entry type" unless TYPE.value?(@type) || @type.nil?
23
35
  end
@@ -31,11 +43,13 @@ module Fyodor
31
43
  end
32
44
 
33
45
  def desc_parsed?
34
- @loc_start != 0 && ! @type.nil?
46
+ ! @type.nil? && (@loc_start != 0 || @page_start != 0)
35
47
  end
36
48
 
37
49
  # Override this method to use a SortedSet.
38
50
  def <=>(other)
51
+ return @page_start <=> other.page_start if @loc_start == 0
52
+
39
53
  @loc_start <=> other.loc_start
40
54
  end
41
55
 
@@ -44,7 +58,7 @@ module Fyodor
44
58
  return false if @type != other.type || @text != other.text
45
59
 
46
60
  if desc_parsed? && other.desc_parsed?
47
- @loc == other.loc
61
+ @loc == other.loc && @page == other.page
48
62
  else
49
63
  @desc == other.desc
50
64
  end
@@ -54,7 +68,7 @@ module Fyodor
54
68
 
55
69
  def hash
56
70
  if desc_parsed?
57
- @text.hash ^ @type.hash ^ @loc.hash
71
+ @text.hash ^ @type.hash ^ @loc.hash ^ @page.hash
58
72
  else
59
73
  @text.hash ^ @desc.hash
60
74
  end
@@ -9,26 +9,37 @@ module Fyodor
9
9
  end
10
10
 
11
11
  def entry
12
- Entry.new({book_title: book[:title],
13
- book_author: book[:author],
14
- text: text,
15
- desc: desc,
16
- type: type,
17
- loc: loc,
18
- loc_start: loc_start,
19
- page: page,
20
- time: time})
12
+ Entry.new({
13
+ book_title: book[:title],
14
+ book_author: book[:author],
15
+ text: text,
16
+ desc: desc,
17
+ type: type,
18
+ loc: loc,
19
+ loc_start: loc_start,
20
+ page: page,
21
+ page_start: page_start,
22
+ time: time
23
+ })
21
24
  end
22
25
 
23
26
 
24
27
  private
25
28
 
26
29
  def book
27
- title, author = @lines[0].scan(regex_cap(:title_author)).first
30
+ regex = /^(.*) \((.*)\)\r?\n$/
31
+
32
+ title, author = @lines[0].scan(regex).first
28
33
  # If book has no author, regex fails.
29
- title = @lines[0] if title.nil?
34
+ if title.nil?
35
+ title = @lines[0]
36
+ author = ""
37
+ end
38
+
39
+ title = rm_zero_width_chars(title).strip
40
+ author = rm_zero_width_chars(author).strip
30
41
 
31
- {title: title.strip, author: author.to_s.strip}
42
+ {title: title, author: author}
32
43
  end
33
44
 
34
45
  def desc
@@ -36,58 +47,61 @@ module Fyodor
36
47
  end
37
48
 
38
49
  def type
39
- Entry::TYPE.values.find { |type| @lines[1] =~ regex_type(type) }
50
+ Entry::TYPE.values.find do |type|
51
+ keyword = Regexp.quote(@config[type])
52
+ regex = /^- #{keyword}/i
53
+
54
+ @lines[1] =~ regex
55
+ end
40
56
  end
41
57
 
42
58
  def loc
43
- @lines[1][regex_cap(:loc), 1]
59
+ keyword = Regexp.quote(@config["loc"])
60
+ regex = /#{keyword} (\S+)/i
61
+
62
+ @lines[1][regex, 1]
44
63
  end
45
64
 
46
65
  def loc_start
47
- @lines[1][regex_cap(:loc_start), 1].to_i
66
+ keyword = Regexp.quote(@config["loc"])
67
+ regex = /#{keyword} (\d+)(-\d+)?/i
68
+
69
+ @lines[1][regex, 1].to_i
48
70
  end
49
71
 
50
72
  def page
51
- @lines[1][regex_cap(:page), 1]
52
- end
73
+ keyword = Regexp.quote(@config["page"])
74
+ regex = /#{keyword} (\S+)/i
53
75
 
54
- def time
55
- @lines[1][regex_cap(:time), 1]
76
+ @lines[1][regex, 1]
56
77
  end
57
78
 
58
- def text
59
- @lines[3].strip
79
+ def page_start
80
+ keyword = Regexp.quote(@config["page"])
81
+ regex = /#{keyword} (\d+)(-\d+)?/i
82
+
83
+ @lines[1][regex, 1].to_i
60
84
  end
61
85
 
62
- def regex_type(type)
63
- s = Regexp.quote(@config[type])
64
- /^- #{s}/
86
+ def time
87
+ keyword = Regexp.quote(@config["time"])
88
+ regex = /#{keyword} (.*)\r?\n$/i
89
+
90
+ @lines[1][regex, 1]
65
91
  end
66
92
 
67
- def regex_cap(item)
68
- case item
69
- when :title_author
70
- /^(.*) \((.*)\)\r?\n$/
71
- when :loc
72
- s = Regexp.quote(@config["loc"])
73
- /#{s} (\S+)/
74
- when :loc_start
75
- s = Regexp.quote(@config["loc"])
76
- /#{s} (\d+)(-\d+)?/
77
- when :page
78
- s = Regexp.quote(@config["page"])
79
- /#{s} (\S+)/
80
- when :time
81
- s = Regexp.quote(@config["time"])
82
- /#{s} (.*)\r?\n$/
83
- end
93
+ def text
94
+ @lines[3..-2].join.strip
84
95
  end
85
96
 
86
97
  def format_check
87
- raise "Entry must have five lines" unless @lines.size == 5
88
98
  raise "Entry is badly formatted" if @lines[0].strip.empty?
89
99
  raise "Entry is badly formatted" if @lines[1].strip.empty?
90
100
  raise "Entry is badly formatted" unless @lines[2].strip.empty?
91
101
  end
102
+
103
+ def rm_zero_width_chars(str)
104
+ str.gsub(/[\u200B-\u200D\uFEFF]/, '')
105
+ end
92
106
  end
93
107
  end
@@ -52,25 +52,24 @@ module Fyodor
52
52
  output = "---\n\n"
53
53
  output += "## #{title}\n\n" unless title.nil?
54
54
  entries.each do |entry|
55
- output += "#{entry_text(entry)}\n\n"
56
- output += "<p style=\"text-align: right;\"><sup>#{entry_desc(entry)}</sup></p>\n\n"
55
+ output += "#{item_text(entry)}\n\n"
56
+ output += "<p style=\"text-align: right;\"><sup>#{item_desc(entry)}</sup></p>\n\n"
57
57
  end
58
58
  output
59
59
  end
60
60
 
61
- def entry_text(entry)
61
+ def item_text(entry)
62
62
  case entry.type
63
63
  when Entry::TYPE[:bookmark]
64
64
  "* #{page(entry)}"
65
65
  when Entry::TYPE[:note]
66
-
67
- "> _#{text(entry)}_"
66
+ "* _Note:_\n#{entry.text.strip}"
68
67
  else
69
- "> #{text(entry)}"
68
+ "* #{entry.text.strip}"
70
69
  end
71
70
  end
72
71
 
73
- def entry_desc(entry)
72
+ def item_desc(entry)
74
73
  return entry.desc unless entry.desc_parsed?
75
74
 
76
75
  case entry.type
@@ -93,10 +92,5 @@ module Fyodor
93
92
  def type(entry)
94
93
  SINGULAR[entry.type]
95
94
  end
96
-
97
- def text(entry)
98
- # Markdown needs no white space between text and formatters
99
- entry.text.strip
100
- end
101
95
  end
102
96
  end
@@ -1,3 +1,3 @@
1
1
  module Fyodor
2
- VERSION = "0.2.1".freeze
2
+ VERSION = "0.2.6".freeze
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: fyodor
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.1
4
+ version: 0.2.6
5
5
  platform: ruby
6
6
  authors:
7
7
  - Rafael Cavalcanti
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-01-08 00:00:00.000000000 Z
11
+ date: 2020-06-30 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: toml
@@ -36,7 +36,6 @@ files:
36
36
  - bin/fyodor
37
37
  - lib/fyodor.rb
38
38
  - lib/fyodor/book.rb
39
- - lib/fyodor/cli.rb
40
39
  - lib/fyodor/clippings_parser.rb
41
40
  - lib/fyodor/config_getter.rb
42
41
  - lib/fyodor/core_extensions/hash/merging.rb
@@ -68,7 +67,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
68
67
  - !ruby/object:Gem::Version
69
68
  version: '0'
70
69
  requirements: []
71
- rubygems_version: 3.0.6
70
+ rubygems_version: 3.1.2
72
71
  signing_key:
73
72
  specification_version: 4
74
73
  summary: Kindle clippings parser
@@ -1,42 +0,0 @@
1
- require "fyodor/config_getter"
2
- require "fyodor/stats_printer"
3
- require "fyodor/library"
4
- require "fyodor/clippings_parser"
5
- require "fyodor/output_writer"
6
- require "pathname"
7
-
8
- module Fyodor
9
- class CLI
10
- USAGE = "Usage: #{File.basename($0)} my_clippings_path [output_dir]"
11
-
12
- def initialize
13
- get_args
14
- @config = ConfigGetter.new.config
15
- end
16
-
17
- def main
18
- library = Library.new
19
- ClippingsParser.new(@clippings_path, @config["parser"]).parse(library)
20
- StatsPrinter.new(library).print
21
- OutputWriter.new(library, @output_dir, @config["output"]).write_all
22
- end
23
-
24
-
25
- private
26
-
27
- def get_args
28
- abort(USAGE) if ARGV.count != 1 && ARGV.count != 2
29
-
30
- @clippings_path = get_path(ARGV[0])
31
- @output_dir = ARGV[1].nil? ? default_output_dir : get_path(ARGV[1])
32
- end
33
-
34
- def get_path(path)
35
- Pathname.new(path).expand_path
36
- end
37
-
38
- def default_output_dir
39
- Pathname.new(Dir.pwd) + "fyodor_output"
40
- end
41
- end
42
- end