fyodor 0.2.10 → 0.3.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 02a743dcc50a2c528b31aa2d4208cb4f220cb7ee5ee7a1ab19d36ee701c38de5
-  data.tar.gz: 005baf6cf829812dc8f9ab1d9f5d2761ff3a610e607e11abed66e507f6eee649
+  metadata.gz: 7fa1b1e78cc0ab15159e69a7f63c51ff94d549a618059395d29401ddc16ff45c
+  data.tar.gz: 325bfb536a54858e628a81094b707f7cd3f0be83d2cf613cf6b8b8de79b33015
 SHA512:
-  metadata.gz: d6b6b7784323ac398eb36a4321d23a558c632246d2a5199dec38a28bd22a60fc77b93b76a31f95586ce27aac34538c74da6332fc96b84d61ad1b8f104eba56de
-  data.tar.gz: 0b6f4ab38b109fc7d64914d8b90ba45d32185b625c66f0099abb95299c38c639d788ff552c6b412c11f6ddf656d7a6cd6e9bb1109d5a6a22b25b7a6ad9021bc5
+  metadata.gz: 4e5f8277ece0aa866131788f2702a1dc137cea14d86795e2c71c74947d9c532482b2bc0a89e34d7500af69a15642a774ce3a815882f835b4ce34cd155ec96af2
+  data.tar.gz: d8a7b83be1154a5385f8870e94f8973de8f8b744ffb09122d55dae9282b074ede788fba4ca1eccb9d1263786ad478c16d6d02a6bd1466e42aab603ed4fe22d53

data/README.md

@@ -1,6 +1,6 @@
 # Fyodor
 
-Convert your Amazon Kindle highlights, notes and bookmarks into markdown files.
+Convert your Amazon Kindle highlights, notes and bookmarks into markdown (or any format).
 
 ## What is Fyodor
 
@@ -17,8 +17,9 @@ To read more about the motivation and what problem it tries to solve, [check thi
 - Orders your entries by location/page on each book (the clippings file is ordered by time).
 - Easily configurable for your language, allowing you to get all features and beautiful output.
 - This software goes some length to be locale agnostic: basic parsing should work without configuration for any language. It should also work even if your clippings file has multiple locales.
-- Bookmarks are printed together and notes are formatted differently, for better visualization.
-- Output in a format that is clean and easy to edit/fiddle around: markdown.
+- By default, output in a format that is clean and easy to edit/fiddle around: markdown.
+- Bookmarks are placed together and notes are formatted differently, for better visualization.
+- Entirely customizable output, in the format you prefer, through templates.
 
 This program is based on the clippings file generated by Kindle 2019, but should work with other models.
 
@@ -33,7 +34,7 @@ We are limited by the data Kindle makes available through `My Clippings.txt`. Th
 
 Install Ruby and run:
 
-```
+```shell-session
 $ gem install fyodor
 ```
 
@@ -41,29 +42,29 @@ $ gem install fyodor
 
 Run:
 
-```
+```shell-session
 $ gem update fyodor
 ```
 
 ## Configuration
 
-Fyodor has an optional configuration file, which is used for the following.
-
-### Languages
-
-If your Kindle is not in English, you should tell Fyodor how some things are called by your `My Clippings.txt` (e.g. highlights, pages, etc). _Fyodor should still work without configuration, but you won't take advantage of many features, resulting in a dirtier output._
+Fyodor reads an optional configuration file at `~/.config/fyodor/fyodor.toml` or `$XDG_CONFIG_HOME/fyodor/fyodor.toml`. This section describes the available parameters (none is required).
 
-1. Download the sample config to `~/.config/fyodor/fyodor.toml` or `$XDG_CONFIG_HOME/fyodor/fyodor.toml`:
+To download the [sample configuration](docs/fyodor.toml.sample):
 
-```sh
+```shell-session
 $ curl https://raw.githubusercontent.com/rc2dev/fyodor/master/docs/fyodor.toml.sample --create-dirs -o ~/.config/fyodor/fyodor.toml
 ```
 
-2. Open both the configuration and your `My Clippings.txt` in your preferred editor. Change the values in the `[parser]` section to mirror what you get in `My Clippings.txt`.
+### Languages
+
+If your Kindle device is not set to English (US), you should tell Fyodor how some things are named on your `My Clippings.txt` (highlights, pages, etc). I went through some length to make Fyodor work regardless of this step. However, if you don't set this correctly, you won't take advantage of many features, resulting in a dirtier output.
+
+Open both `fyodor.toml` and your `My Clippings.txt` in your preferred editor. Change the values in the `[parser]` section to mirror what you get in `My Clippings.txt`.
 
 For example, this is the configuration for Brazilian Portuguese:
 
-```
+```toml
 [parser]
 highlight = "Seu destaque"
 note = "Sua nota"
@@ -76,18 +77,41 @@ time = "Adicionado:"
 
 ### Showing the time
 
-In the configuration file you can also set whether to print the time of each entry. On `[output]`, set `time` to `true` or `false`.
+You can also set whether to print the time of each entry. Under `[output]`, set `time` to `true` or `false`:
 
-## Usage
+```toml
+[output]
+time = true
+```
 
+### Extension
+
+If you want to change the extension of the output files (typically after changing the template), set `extension` under `[output]`. For example, to change it to HTML:
+
+```toml
+[output]
+extension = "html"
 ```
+
+## Templating
+
+You may change the structure of the files output by Fyodor by providing your own template in any format you wish.
+
+To do that, place a ERB template at `~/.config/fyodor/template.erb` or `$XDG_CONFIG_HOME/fyodor/template.erb` and Fyodor will use it automatically.
+
+The default template can be found [here](share/template.erb). You can use any method or attribute available [on this class](lib/fyodor/output_generator.rb).
+
+
+## Usage
+
+```shell-session
 $ fyodor CLIPPINGS_FILE [OUTPUT_DIR]
 ```
 
 Where:
 
 - `CLIPPINGS_FILE` is the path for `My Clippings.txt`.
-- `OUTPUT_DIR` is the directory to write the markdown files. If none is supplied, this will be `fyodor_output` under the current directory.
+- `OUTPUT_DIR` is the directory to write the output files. If none is supplied, this will be `fyodor_output` under the current directory.
 
 ## PSA: HTML to markdown
 
@@ -105,4 +129,4 @@ If you like Fyodor, you can show your support here:
 
 Licensed under [GPLv3](LICENSE)
 
-Copyright (C) 2019-2020 [Rafael Cavalcanti](https://rafaelc.org/)
+Copyright (C) 2019-2022 [Rafael Cavalcanti](https://rafaelc.org/dev)

data/lib/fyodor/book.rb

@@ -15,7 +15,7 @@ module Fyodor
 
       @title = title
       @author = author
-      @entries = SortedSet.new
+      @entries = Set.new
       @rej_dup = 0
     end
 
@@ -26,8 +26,8 @@ module Fyodor
     end
 
     def basename
-      base = @author.to_s.empty? ? @title : "#{@author} - #{@title}"
-      base.strip.gsub(/[?*:|\/"<>]/,"_")
+      base = @author.to_s.empty? ? "Author N/A - #{@title}" : "#{@author} - #{@title}"
+      base.strip
     end
 
     def count_types

data/lib/fyodor/config_getter.rb

@@ -4,7 +4,9 @@ require "toml"
 
 module Fyodor
   class ConfigGetter
-    DEFAULT = {
+    DEFAULT_TEMPLATE_PATH = File.dirname(__FILE__) + "/../../share/template.erb"
+
+    DEFAULT_CONFIG = {
       "parser" => {
         "highlight" => "Your Highlight",
         "note" => "Your Note",
@@ -15,43 +17,54 @@ module Fyodor
         "time" => "Added on"
       },
       "output" => {
-        "time" => false
+        "time" => false,
+        "extension" => "md"
       }
     }
 
     def config
-      @config ||= get_config
+      return @config if defined?(@config)
+
+      Hash.include CoreExtensions::Hash::Merging
+      config = DEFAULT_CONFIG.deep_merge(user_config)
+      config["output"]["template"] = template
+
+      @config = config
     end
 
 
     private
 
-    def get_config
-      Hash.include CoreExtensions::Hash::Merging
-      print_path
+    def config_dir
+      @config_dir ||= Pathname.new(ENV["XDG_CONFIG_HOME"] || "~/.config").expand_path + "fyodor"
+    end
 
-      user_config = path.nil? ? {} : TOML.load_file(path)
-      DEFAULT.deep_merge(user_config)
+    def user_config_path
+      config_dir + "fyodor.toml"
     end
 
-    def path
-      @path ||= paths.find { |path| path.exist? }
+    def user_template_path
+      config_dir + "template.erb"
     end
 
-    def paths
-      return @paths unless @paths.nil?
+    def user_config
+      if user_config_path.exist?
+        puts "Using config at #{user_config_path}.\n"
+        return TOML.load_file(user_config_path)
+      end
 
-      @paths = []
-      @paths << Pathname.new(ENV["XDG_CONFIG_HOME"]) + "fyodor/fyodor.toml" unless ENV["XDG_CONFIG_HOME"].nil?
-      @paths << Pathname.new("~/.config/fyodor/fyodor.toml").expand_path
+      puts "No config found: using defaults.\n"
+      {}
     end
 
-    def print_path
-      if path.nil?
-        puts "No config found: using defaults.\n\n"
-      else
-        puts "Using config at #{path}\n\n"
+    def template
+      if user_template_path.exist?
+        puts "Using custom template at #{user_template_path}.\n\n"
+        return File.read(user_template_path)
       end
+
+      puts "No custom template found: using default.\n\n"
+      File.read(DEFAULT_TEMPLATE_PATH)
     end
   end
 end

data/lib/fyodor/entry.rb

@@ -48,20 +48,18 @@ module Fyodor
 
     # Override this method to use a SortedSet.
     def <=>(other)
-      return @page_start <=> other.page_start if @loc_start == 0
+      return (@page_start <=> other.page_start) if @loc_start == 0
 
       @loc_start <=> other.loc_start
     end
 
     # Override the following methods for deduplication.
     def ==(other)
-      return false if @type != other.type || @text != other.text
+      return false if (@type != other.type || @text != other.text)
 
-      if desc_parsed? && other.desc_parsed?
-        @loc == other.loc && @page == other.page
-      else
-        @desc == other.desc
-      end
+      return (@loc == other.loc && @page == other.page) if desc_parsed? && other.desc_parsed?
+
+      @desc == other.desc
     end
 
     alias eql? ==

data/lib/fyodor/output_generator.rb

@@ -0,0 +1,72 @@
+require "fyodor/strings"
+require "erb"
+
+module Fyodor
+  class OutputGenerator
+    include Strings
+
+    def initialize(book, config)
+      @book = book
+      @config = config
+    end
+
+    def content
+      ERB.new(@config["template"], nil, '-').result(binding)
+    end
+
+
+    private
+    
+    def regular_entries
+      @book.reject { |entry| entry.type == Entry::TYPE[:bookmark] }
+    end
+
+    def bookmarks
+      @book.select { |entry| entry.type == Entry::TYPE[:bookmark] }
+    end
+
+    def render_entries(entries)
+      output = ""
+      entries.sort.each do |entry|
+        output += "- #{item_text(entry)}\n\n"
+        output += "  #{item_desc(entry)}\n\n" unless item_desc(entry).empty?
+      end
+      output.strip
+    end
+
+    def item_text(entry)
+      case entry.type
+      when Entry::TYPE[:bookmark]
+        "#{page(entry)}"
+      when Entry::TYPE[:note]
+        "_Note:_ #{entry.text.strip}"
+      else
+        "#{entry.text.strip}"
+      end
+    end
+
+    def item_desc(entry)
+      return entry.desc unless entry.desc_parsed?
+
+      case entry.type
+      when Entry::TYPE[:bookmark]
+        time(entry)
+      else
+        (type(entry) + " @ " + page(entry) + " " + time(entry)).strip
+      end
+    end
+
+    def page(entry)
+      ((entry.page.nil? ? "" : "page #{entry.page}, ") +
+        (entry.loc.nil? ? "" : "loc. #{entry.loc}")).delete_suffix(", ")
+    end
+
+    def time(entry)
+      @config["time"] ? "[#{entry.time}]" : ""
+    end
+
+    def type(entry)
+      SINGULAR[entry.type]
+    end
+  end
+end

data/lib/fyodor/output_writer.rb

@@ -1,4 +1,4 @@
-require "fyodor/md_generator"
+require "fyodor/output_generator"
 
 module Fyodor
   class OutputWriter
@@ -12,7 +12,7 @@ module Fyodor
     def write_all
       puts "\nWriting to #{@output_dir}..." unless @library.empty?
       @library.each do |book|
-        content = MdGenerator.new(book, @config).content
+        content = OutputGenerator.new(book, @config).content
         File.open(path(book), "w") { |f| f.puts(content) }
       end
     end
@@ -21,11 +21,13 @@ module Fyodor
     private
 
     def path(book)
-      path = @output_dir + "#{book.basename}.md"
+      basename = book.basename.gsub(/[?*:|\/"<>]/,"_")
+      extension = @config["extension"]
+      path = @output_dir + "#{basename}.#{extension}"
 
       i = 2
       while(path.exist?)
-        path = @output_dir + "#{book.basename} - #{i}.md"
+        path = @output_dir + "#{basename} - #{i}.#{extension}"
         i += 1
       end
 

data/lib/fyodor/version.rb

@@ -1,3 +1,3 @@
 module Fyodor
-  VERSION = "0.2.10".freeze
+  VERSION = "0.3.3".freeze
 end

data/share/template.erb

@@ -0,0 +1,16 @@
+<%# Fyodor's default template. -%>
+# <%= @book.basename %>
+
+## Highlights and notes
+
+<% if regular_entries.size > 0 -%>
+<%= render_entries(regular_entries) %>
+<% else -%>
+None.
+<% end -%>
+
+<% if bookmarks.size > 0 -%>
+## Bookmarks
+
+<%= render_entries(bookmarks) %>
+<% end -%>

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fyodor
 version: !ruby/object:Gem::Version
-  version: 0.2.10
+  version: 0.3.3
 platform: ruby
 authors:
 - Rafael Cavalcanti
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2021-11-23 00:00:00.000000000 Z
+date: 2022-02-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: toml
@@ -38,7 +38,7 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.0'
-description: Parse Kindle clippings into markdown files
+description: Parse Kindle clippings into markdown (or any format)
 email: dev@rafaelc.org
 executables:
 - fyodor
@@ -56,11 +56,12 @@ files:
 - lib/fyodor/entry.rb
 - lib/fyodor/entry_parser.rb
 - lib/fyodor/library.rb
-- lib/fyodor/md_generator.rb
+- lib/fyodor/output_generator.rb
 - lib/fyodor/output_writer.rb
 - lib/fyodor/stats_printer.rb
 - lib/fyodor/strings.rb
 - lib/fyodor/version.rb
+- share/template.erb
 homepage: https://rafaelc.org/fyodor
 licenses:
 - GPL-3.0-only
@@ -77,14 +78,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '2.5'
   - - "<"
     - !ruby/object:Gem::Version
-      version: '3'
+      version: '4'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.1.2
+rubygems_version: 3.2.3
 signing_key: 
 specification_version: 4
 summary: Kindle clippings parser

data/lib/fyodor/md_generator.rb

@@ -1,96 +0,0 @@
-require "fyodor/strings"
-
-module Fyodor
-  class MdGenerator
-    include Strings
-
-    def initialize(book, config)
-      @book = book
-      @config = config
-    end
-
-    def content
-      header + body + bookmarks
-    end
-
-
-    private
-
-    def header
-      return <<~EOF
-      # #{@book.title}
-      #{"by #{@book.author}" unless @book.author.to_s.empty?}
-
-      #{header_counts}
-
-      EOF
-    end
-
-    def header_counts
-      output = ""
-      @book.count_types.each do |type, n|
-        output += "#{n} #{pluralize(type, n)}, " if n > 0
-      end
-      output.delete_suffix(", ")
-    end
-
-    def pluralize(type, n)
-      n == 1 ? SINGULAR[type] : PLURAL[type]
-    end
-
-    def body
-      entries = @book.reject { |entry| entry.type == Entry::TYPE[:bookmark] }
-      entries.size == 0 ? "" : entries_render(entries)
-    end
-
-    def bookmarks
-      bookmarks = @book.select { |entry| entry.type == Entry::TYPE[:bookmark] }
-      bookmarks.size == 0 ? "" : entries_render(bookmarks, "Bookmarks")
-    end
-
-    def entries_render(entries, title=nil)
-      output = "---\n\n"
-      output += "## #{title}\n\n" unless title.nil?
-      entries.each do |entry|
-        output += "#{item_text(entry)}\n\n"
-        output += "<p style=\"text-align: right;\"><sup>#{item_desc(entry)}</sup></p>\n\n"
-      end
-      output
-    end
-
-    def item_text(entry)
-      case entry.type
-      when Entry::TYPE[:bookmark]
-        "* #{page(entry)}"
-      when Entry::TYPE[:note]
-        "* _Note:_\n#{entry.text.strip}"
-      else
-        "* #{entry.text.strip}"
-      end
-    end
-
-    def item_desc(entry)
-      return entry.desc unless entry.desc_parsed?
-
-      case entry.type
-      when Entry::TYPE[:bookmark]
-        time(entry)
-      else
-        (type(entry) + " @ " + page(entry) + " " + time(entry)).strip
-      end
-    end
-
-    def page(entry)
-      ((entry.page.nil? ? "" : "page #{entry.page}, ") +
-        (entry.loc.nil? ? "" : "loc. #{entry.loc}")).delete_suffix(", ")
-    end
-
-    def time(entry)
-      @config["time"] ? "[#{entry.time}]" : ""
-    end
-
-    def type(entry)
-      SINGULAR[entry.type]
-    end
-  end
-end