fyodor 0.2.7 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 88110cfdbb8c0f625d97c460e6b087f1f00b0efa2bc54f244254e9e90dd17102
-  data.tar.gz: '097449b93162d2be858b05942e41b05904dfdf464c7f5fbc1fc70fa725fbb992'
+  metadata.gz: 928a21f90c52229824428fa62a99e0ccd1b3319c954aa9f89696833933920315
+  data.tar.gz: ceedbb0b474de8f63f72fb4388f1e4eaf2a540644a61e26ed5a7616c0def2dbc
 SHA512:
-  metadata.gz: 265349bddd3c8c4445d8d65684eaacb24cc49d2089842708cfb9d92b748fd3c226e62509d17e2b374cc2d1cbc1682d58be1770bf10a738fa4defeb4a1928bab9
-  data.tar.gz: e6dd20487302828751669be11669f75bb536439321c292b72f12f390f9c388f3c3ba90ebbab0ce659a6179a5342fc95fbda20feeb3fa832e376c732b35472a65
+  metadata.gz: 44a7b6488f71988cdbfdb0b35083ae3e156e66899eee560188fc11da7127ff190edc60e8d9f22b3fab97cc96462efdc7732dbf905ce22b6cf87af14f5b41e61e
+  data.tar.gz: 53e882dd8ea580dd5657d088b064a00b45f86b145b8fbf403c0c84bbd9de5d7549933345b8291a7a2bc73260d5a3a5e98f711fd7b235953a2f554f394022f89d

data/README.md

@@ -8,7 +8,7 @@ This application parses `My Clippings.txt` from your Kindle and generates a mark
 
 [For samples of the output, click here.](docs/output_demo)
 
-To read more about the motivation and what problem it tries to solve, [check this blog post](http://rafaelc.org/blog/export-all-your-kindle-highlights-and-notes/).
+To read more about the motivation and what problem it tries to solve, [check this blog post](https://rafaelc.org/posts/export-all-your-kindle-highlights-and-notes/).
 
 ## Features
 
@@ -22,6 +22,13 @@ To read more about the motivation and what problem it tries to solve, [check thi
 
 This program is based on the clippings file generated by Kindle 2019, but should work with other models.
 
+## Limitations
+
+We are limited by the data Kindle makes available through `My Clippings.txt`. This means:
+
+- We don't have chapter information.
+- We can’t guess all entries that were deleted.
+
 ## Installation
 
 Install Ruby and run:
@@ -46,10 +53,10 @@ Fyodor has an optional configuration file, which is used for the following.
 
 If your Kindle is not in English, you should tell Fyodor how some things are called by your `My Clippings.txt` (e.g. highlights, pages, etc). _Fyodor should still work without configuration, but you won't take advantage of many features, resulting in a dirtier output._
 
-1. Download the sample config to `~/.config/fyodor.toml` or `$XDG_CONFIG_HOME/fyodor.toml`:
+1. Download the sample config to `~/.config/fyodor/fyodor.toml` or `$XDG_CONFIG_HOME/fyodor/fyodor.toml`:
 
-```
-$ wget https://raw.githubusercontent.com/rccavalcanti/fyodor/master/docs/fyodor.toml.sample -O ~/.config/fyodor.toml
+```sh
+$ curl https://raw.githubusercontent.com/rc2dev/fyodor/master/docs/fyodor.toml.sample --create-dirs -o ~/.config/fyodor/fyodor.toml
 ```
 
 2. Open both the configuration and your `My Clippings.txt` in your preferred editor. Change the values in the `[parser]` section to mirror what you get in `My Clippings.txt`.
@@ -71,6 +78,24 @@ time = "Adicionado:"
 
 In the configuration file you can also set whether to print the time of each entry. On `[output]`, set `time` to `true` or `false`.
 
+### Templating
+
+If you want to change the structure of the files output by Fyodor, you can use custom templates.
+
+It should be a ERB template and placed at `~/.config/fyodor/template.erb` or `$XDG_CONFIG_HOME/fyodor/template.erb`. Fyodor will find and use it automatically.
+
+You can find a sample close to the default [here](docs/template.erb.sample). You can use any method or attribute available [at this class](lib/fyodor/output_generator.rb).
+
+
+### Extension
+
+If you want to change the extension of the output files - typically after changing the template -, set `extension` on fyodor.toml. For example, to change it to HTML:
+
+```toml
+[output]
+extension = "html"
+```
+
 ## Usage
 
 ```
@@ -82,14 +107,20 @@ Where:
 - `CLIPPINGS_FILE` is the path for `My Clippings.txt`.
 - `OUTPUT_DIR` is the directory to write the markdown files. If none is supplied, this will be `fyodor_output` under the current directory.
 
+## PSA: HTML to markdown
+
+Did you export your annotations to HTML using the Kindle mobile app?
+
+You can convert it to markdown with [a script I wrote specifically for it](https://rafaelc.org/k/kindle2md).
+
 ## Buy me a coffee
 
 If you like Fyodor, you can show your support here:
 
-<a href="https://www.buymeacoffee.com/rafaelc" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" width="200" ></a>
+<a href="https://rafaelc.org/coffee" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" width="200" ></a>
 
 ## License
 
 Licensed under [GPLv3](LICENSE)
 
-Copyright (C) 2019-2020 [Rafael Cavalcanti](https://rafaelc.org/)
+Copyright (C) 2019-2022 [Rafael Cavalcanti](https://rafaelc.org/dev)

data/lib/fyodor/book.rb

@@ -26,8 +26,8 @@ module Fyodor
     end
 
     def basename
-      base = @author.to_s.empty? ? @title : "#{@author} - #{@title}"
-      base.strip.gsub(/[?*:|\/"<>]/,"_")
+      base = @author.to_s.empty? ? "Author N/A - #{@title}" : "#{@author} - #{@title}"
+      base.strip
     end
 
     def count_types

data/lib/fyodor/config_getter.rb

@@ -4,7 +4,10 @@ require "toml"
 
 module Fyodor
   class ConfigGetter
-    DEFAULT = {
+    CONFIG_NAME = "fyodor.toml"
+    TEMPLATE_NAME = "template.erb"
+
+    DEFAULTS = {
       "parser" => {
         "highlight" => "Your Highlight",
         "note" => "Your Note",
@@ -15,7 +18,8 @@ module Fyodor
         "time" => "Added on"
       },
       "output" => {
-        "time" => false
+        "time" => false,
+        "extension" => "md"
       }
     }
 
@@ -28,29 +32,49 @@ module Fyodor
 
     def get_config
       Hash.include CoreExtensions::Hash::Merging
-      print_path
 
-      user_config = path.nil? ? {} : TOML.load_file(path)
-      DEFAULT.deep_merge(user_config)
+      @config_path = get_path(CONFIG_NAME)
+      print_config_path
+      user_config = @config_path.nil? ? {} : TOML.load_file(@config_path)
+
+      config = DEFAULTS.deep_merge(user_config)
+      @template_path = get_path(TEMPLATE_NAME)
+      config["output"]["template_path"] = @template_path
+      print_template_path
+
+      config
     end
 
-    def path
-      @path ||= paths.find { |path| path.exist? }
+    def get_path(name)
+      possible_dirs.each do |d|
+        path = d + name
+        return path if path.exist?
+      end
+
+      return nil
     end
 
-    def paths
-      return @paths unless @paths.nil?
+    def possible_dirs
+      return @possible_dirs unless @possible_dirs.nil?
 
-      @paths = []
-      @paths << Pathname.new(ENV["XDG_CONFIG_HOME"]) + "fyodor.toml" unless ENV["XDG_CONFIG_HOME"].nil?
-      @paths << Pathname.new("~/.config/fyodor.toml").expand_path
+      @possible_dirs = []
+      @possible_dirs << Pathname.new(ENV["XDG_CONFIG_HOME"]) + "fyodor" unless ENV["XDG_CONFIG_HOME"].nil?
+      @possible_dirs << Pathname.new("~/.config/fyodor").expand_path
+    end
+
+    def print_config_path
+      if @config_path.nil?
+        puts "No config found: using defaults.\n"
+      else
+        puts "Using config at #{@config_path}.\n"
+      end
     end
 
-    def print_path
-      if path.nil?
-        puts "No config found: using defaults.\n\n"
+    def print_template_path
+      if @template_path.nil?
+        puts "No template found: using default.\n\n"
       else
-        puts "Using config at #{path}\n\n"
+        puts "Using template at #{@template_path}.\n\n"
       end
     end
   end

data/lib/fyodor/output_generator.rb

@@ -0,0 +1,93 @@
+require "fyodor/strings"
+require "erb"
+
+module Fyodor
+  class OutputGenerator
+    include Strings
+
+    # The use of <% - 1 %> is a workaround for trimming indentation before <%=.
+    DEFAULT_TEMPLATE = %q{<% -%>
+      <%- 1 %><%= "# #{@book.basename}" %>
+      <% if regular_entries.size > 0 %>
+        <%- 1 %><%= "## Highlights and notes" %>
+
+        <%- 1 %><%= render_entries(regular_entries) %>
+      <% end -%>
+      <% if bookmarks.size > 0 %>
+        <%- 1 %><%= "## Bookmarks" %>
+
+        <%- 1 %><%= render_entries(bookmarks) %>
+      <% end -%>
+    }
+
+    def initialize(book, config)
+      @book = book
+      @config = config
+    end
+
+    def content
+      ERB.new(template, nil, '-').result(binding)
+    end
+
+
+    private
+
+    def template
+      return DEFAULT_TEMPLATE if @config["template_path"].to_s.empty?
+
+      File.read(@config["template_path"])
+    end
+
+    def regular_entries
+      @book.reject { |entry| entry.type == Entry::TYPE[:bookmark] }
+    end
+
+    def bookmarks
+      @book.select { |entry| entry.type == Entry::TYPE[:bookmark] }
+    end
+
+    def render_entries(entries)
+      output = ""
+      entries.each do |entry|
+        output += "- #{item_text(entry)}\n\n"
+        output += "  #{item_desc(entry)}\n\n"
+      end
+      output
+    end
+
+    def item_text(entry)
+      case entry.type
+      when Entry::TYPE[:bookmark]
+        "#{page(entry)}"
+      when Entry::TYPE[:note]
+        "_Note:_ #{entry.text.strip}"
+      else
+        "#{entry.text.strip}"
+      end
+    end
+
+    def item_desc(entry)
+      return entry.desc unless entry.desc_parsed?
+
+      case entry.type
+      when Entry::TYPE[:bookmark]
+        time(entry)
+      else
+        (type(entry) + " @ " + page(entry) + " " + time(entry)).strip
+      end
+    end
+
+    def page(entry)
+      ((entry.page.nil? ? "" : "page #{entry.page}, ") +
+        (entry.loc.nil? ? "" : "loc. #{entry.loc}")).delete_suffix(", ")
+    end
+
+    def time(entry)
+      @config["time"] ? "[#{entry.time}]" : ""
+    end
+
+    def type(entry)
+      SINGULAR[entry.type]
+    end
+  end
+end

data/lib/fyodor/output_writer.rb

@@ -1,4 +1,4 @@
-require "fyodor/md_generator"
+require "fyodor/output_generator"
 
 module Fyodor
   class OutputWriter
@@ -12,7 +12,7 @@ module Fyodor
     def write_all
       puts "\nWriting to #{@output_dir}..." unless @library.empty?
       @library.each do |book|
-        content = MdGenerator.new(book, @config).content
+        content = OutputGenerator.new(book, @config).content
         File.open(path(book), "w") { |f| f.puts(content) }
       end
     end
@@ -21,11 +21,13 @@ module Fyodor
     private
 
     def path(book)
-      path = @output_dir + "#{book.basename}.md"
+      basename = book.basename.gsub(/[?*:|\/"<>]/,"_")
+      extension = @config["extension"]
+      path = @output_dir + "#{basename}.#{extension}"
 
       i = 2
       while(path.exist?)
-        path = @output_dir + "#{book.basename} - #{i}.md"
+        path = @output_dir + "${basename - #{i}.#{extension}"
         i += 1
       end
 

data/lib/fyodor/version.rb

@@ -1,3 +1,3 @@
 module Fyodor
-  VERSION = "0.2.7".freeze
+  VERSION = "0.3.0".freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fyodor
 version: !ruby/object:Gem::Version
-  version: 0.2.7
+  version: 0.3.0
 platform: ruby
 authors:
 - Rafael Cavalcanti
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-07-24 00:00:00.000000000 Z
+date: 2022-02-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: toml
@@ -39,7 +39,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '3.0'
 description: Parse Kindle clippings into markdown files
-email: code@rafaelc.org
+email: dev@rafaelc.org
 executables:
 - fyodor
 extensions: []
@@ -56,16 +56,16 @@ files:
 - lib/fyodor/entry.rb
 - lib/fyodor/entry_parser.rb
 - lib/fyodor/library.rb
-- lib/fyodor/md_generator.rb
+- lib/fyodor/output_generator.rb
 - lib/fyodor/output_writer.rb
 - lib/fyodor/stats_printer.rb
 - lib/fyodor/strings.rb
 - lib/fyodor/version.rb
-homepage: https://github.com/rccavalcanti/fyodor
+homepage: https://rafaelc.org/fyodor
 licenses:
 - GPL-3.0-only
 metadata:
-  source_code_uri: https://github.com/rccavalcanti/fyodor
+  source_code_uri: https://rafaelc.org/fyodor
 post_install_message: 
 rdoc_options: []
 require_paths:
@@ -75,6 +75,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
   - - ">="
     - !ruby/object:Gem::Version
       version: '2.5'
+  - - "<"
+    - !ruby/object:Gem::Version
+      version: '3'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="

data/lib/fyodor/md_generator.rb

@@ -1,96 +0,0 @@
-require "fyodor/strings"
-
-module Fyodor
-  class MdGenerator
-    include Strings
-
-    def initialize(book, config)
-      @book = book
-      @config = config
-    end
-
-    def content
-      header + body + bookmarks
-    end
-
-
-    private
-
-    def header
-      return <<~EOF
-      # #{@book.title}
-      #{"by #{@book.author}" unless @book.author.to_s.empty?}
-
-      #{header_counts}
-
-      EOF
-    end
-
-    def header_counts
-      output = ""
-      @book.count_types.each do |type, n|
-        output += "#{n} #{pluralize(type, n)}, " if n > 0
-      end
-      output.delete_suffix(", ")
-    end
-
-    def pluralize(type, n)
-      n == 1 ? SINGULAR[type] : PLURAL[type]
-    end
-
-    def body
-      entries = @book.reject { |entry| entry.type == Entry::TYPE[:bookmark] }
-      entries.size == 0 ? "" : entries_render(entries)
-    end
-
-    def bookmarks
-      bookmarks = @book.select { |entry| entry.type == Entry::TYPE[:bookmark] }
-      bookmarks.size == 0 ? "" : entries_render(bookmarks, "Bookmarks")
-    end
-
-    def entries_render(entries, title=nil)
-      output = "---\n\n"
-      output += "## #{title}\n\n" unless title.nil?
-      entries.each do |entry|
-        output += "#{item_text(entry)}\n\n"
-        output += "<p style=\"text-align: right;\"><sup>#{item_desc(entry)}</sup></p>\n\n"
-      end
-      output
-    end
-
-    def item_text(entry)
-      case entry.type
-      when Entry::TYPE[:bookmark]
-        "* #{page(entry)}"
-      when Entry::TYPE[:note]
-        "* _Note:_\n#{entry.text.strip}"
-      else
-        "* #{entry.text.strip}"
-      end
-    end
-
-    def item_desc(entry)
-      return entry.desc unless entry.desc_parsed?
-
-      case entry.type
-      when Entry::TYPE[:bookmark]
-        time(entry)
-      else
-        (type(entry) + " @ " + page(entry) + " " + time(entry)).strip
-      end
-    end
-
-    def page(entry)
-      ((entry.page.nil? ? "" : "page #{entry.page}, ") +
-        (entry.loc.nil? ? "" : "loc. #{entry.loc}")).delete_suffix(", ")
-    end
-
-    def time(entry)
-      @config["time"] ? "[#{entry.time}]" : ""
-    end
-
-    def type(entry)
-      SINGULAR[entry.type]
-    end
-  end
-end