pirka 0.1.0

data/Gemfile

@@ -0,0 +1,14 @@
+source 'https://rubygems.org'
+
+gem 'epub-parser', path: '../epub-parser'
+gem 'epub-maker', path: '../epub-maker'
+
+gemspec
+
+group :development do
+  gem 'kramdown'
+end
+
+if RUBY_PLATFORM.match /darwin/
+  gem 'terminal-notifier'
+end

data/README.md

@@ -0,0 +1,201 @@
+Pirka
+=====
+
+* [Homepage](https://gitlab.com/KitaitiMakoto/pirka)
+* [Documentation](http://www.rubydoc.info/gems/pirka)
+* [Email](mailto:KitaitiMakoto at gmail.com)
+
+Description
+-----------
+
+Pirka highlights source code syntax in EPUB books
+
+Features
+--------
+
+* Highlights synatax in EPUB files
+* Extracts `<code>` elements from EPUB files and asks you which programing language they are, and then uses them to highlight
+* Downloads shared code and language information from Git repository
+
+Examples
+--------
+
+### Highlighting source code syntax in EPUB books ###
+
+    $ pirka path/to/book.epub
+
+It's a short cut to:
+
+    $ pirka highlight path/to/book.epub
+
+### Detecting source code from EPUB books ###
+
+    $ pirka detect path/to/book.epub
+    Detecting code from "Book Title"
+    Library file was saved to:
+    path/to/library.yaml
+
+`library.yaml` here includes:
+
+* location of `<code>` element(expressed by [EPUB CFI][] but you don't need to undarstand it.)
+* `language` field with blank value
+* file path in EPUB file(zip archive)
+* source code
+
+Example:
+
+      epubcfi(/6/64!/4/2/30/2):              # location
+        language:                            # language name. blank at first
+        item: OEBPS/text/p-003-003.xhtml     # file path in zip archive
+        code: |                              # source code
+          f1 = open("|cat", "w")
+          f2 = open("|sed 's/a/b/'", "w")
+          f1.print "Hello\n"
+          f2.print "abc\n"
+          f1.close
+          f2.close
+
+In Pirka context, the file is called *library*.
+
+`pirka highlight` command determines programming languages of source code according to this file.
+Read source code, determine languages, write it at `language` field, remove `code` field, and then
+you can highlight the EPUB file by `pirka highlight`.
+
+You also determine languages interactively. Set `-i`(`--interactive`) option:
+
+    $ pirka detect -i path/to/book.epub
+
+[EPUB CFI]: http://www.idpf.org/epub/linking/cfi/
+
+### Updating libraries ###
+
+    $ pirka update
+
+Pirka provides official library files for some EPUB books as Git repository((https://gitlab.com/KitaitiMakoto/pirka-library)[https://gitlab.com/KitaitiMakoto/pirka-library]). `pirka update` command fethes the files from the repository and you benefit from it.
+
+Additionally, you can host library files by your own and make Pirka recognizes it by configuration file. See later section for that.
+
+### Listing supported books ###
+
+    $ pirka lib
+
+`pirka lib` command lists books Pirka can highlight with:
+
+* Release Identifier(special identifier for EPUB books)
+* location of library file
+* title
+* some other metadata
+
+### Configuration ###
+
+Pirka can be configured by environment variables, config file and command-line options.
+
+#### Environment variables ####
+
+XDG_DATA_HOME
+: Affects directory to save library files.
+: Library files are saved to `$XDG_DATA_HOME/pirka/local`
+: The directory is used to search library, too.
+: Default: `$HOME/.local/share`
+
+XDG_DATA_DIRS
+: Affects directory to save library files.
+: You can specify multiple directory by seperating with a colon like `XDG_DATA_DIRS=/dir1:/dir2`.
+: `/dir1/pirka/local` and `/dir2/pirka/local` are used to search library, for example.
+: Default: `/usr/local/share:/usr/share`
+
+XDG_CONFIG_HOME
+: Affects directory to search and save config file.
+: `$XDG_CONFIG_DIRS/pirka.yaml` is recognized as config file.
+: Default: `$HOME/.config`
+
+XDG_CONFIG_DIRS
+: Affects directory to search config file.
+: You can specify multiple directory by seperating with a colon like `XDG_CONFIG_DIRS=/dir1:/dir2`.
+: `/dir1/pirka.yaml` and `/dir2/pirka.yaml` are searched as config file.
+: Default: `/etc/xdg`
+
+#### Config file ####
+
+Config file is a YAML file. Properties below are recognized:
+
+data_home
+: Directory to save and search library files.
+: Default: `$XDG_CONFIG_HOME/pirka/local`
+
+additional_directories
+: Directories to search library files.
+: Expressed by sequence(array).
+: Default: `[]`
+
+library_repositories
+: Git repository URIs used by `pirka lib` command.
+: Expressed by sequence(array).
+: Default: `[]`
+
+#### Command-line options ####
+
+You can configure Pirka by `pirka` command's global options:
+
+`-c`, `--config=FILE`
+: Path to config file.
+: Default: /Users/ikeda/.config/pirka.yaml
+
+`-s`, `--data-home=DIRECTORY`
+: Same to config file's `data_home` property.
+
+`-d`, `--directory=DIRECTORY`
+: Same to config file's `additional_directories` property.
+: Able to multilpe times.
+
+You can also see help by
+
+    $ pirka --help
+    Pirka highlights source code syntax in EPUB files
+    
+    Usage: pirka [global options] [<command>] [options]
+    
+    Global options:
+        -c, --config=FILE                Config file. Defaults to /Users/ikeda/.config/pirka.yaml
+        -s, --data-home=DIRECTORY        Directory to *SAVE* library data
+        -d, --directory=DIRECTORY        Directory to *SEARCH* library data.
+                                         Specify multiple times to add multiple directories.
+    
+    Commands:
+        highlight        Highlights source code in EPUB file
+        detect           Detects source code from EPUB file and generate library file
+        update           Update library files by remote files
+        lib              Show library infomation
+    If command is ommitted, highlight is used with no option
+
+Requirements
+------------
+
+* Ruby 2.2 or later
+* C compiler to compile [Nokogiri][] gem
+
+[Nokogiri]: http://www.nokogiri.org/
+
+Install
+-------
+
+    $ gem install pirka
+
+### Make faster ###
+
+By default, Pirka uses [archive-zip][] gem, a pure Ruby implementation, for zip archive but you can make command execution faster by using [Zip/Ruby][] gem, a C implementation. Just install Zip/Ruby:
+
+    $ gem install zipruby
+
+Pirka, actually internally-used [EPUB Parser][], tries to load Zip/Ruby and use it if available.
+
+[archive-zip]: https://github.com/javanthropus/archive-zip
+[Zip/Ruby]: https://bitbucket.org/winebarrel/zip-ruby/wiki/Home
+[EPUB Parser]: http://www.rubydoc.info/gems/epub-parser/file/docs/Home.markdown
+
+Copyright
+---------
+
+Copyright (c) 2017 KITAITI Makoto
+
+See {file:COPYING.txt} for details.

data/Rakefile

@@ -0,0 +1,27 @@
+# encoding: utf-8
+
+require 'rubygems'
+
+begin
+  require 'bundler/setup'
+rescue LoadError => e
+  abort e.message
+end
+
+require 'rake'
+
+task :default => :test
+
+require 'rubygems/tasks'
+Gem::Tasks.new
+
+require 'rake/testtask'
+Rake::TestTask.new do |test|
+  test.libs << 'test'
+  test.pattern = 'test/**/test_*.rb'
+  test.verbose = true
+end
+
+require 'yard'
+YARD::Rake::YardocTask.new  
+task :doc => :yard

data/app/detect.rb

@@ -0,0 +1,166 @@
+require "optparse"
+require "optparse/pathname"
+require "epub/parser"
+require "epub/cfi"
+require "epub/searcher"
+require "rouge"
+require "rouge/lexers/fluentd"
+require "colored"
+require "pirka/library"
+require_relative "subcommand"
+
+module Pirka
+  class App
+    class Detect
+      PROGRAM_NAME = "detect"
+      DESCRIPTION = "Detects source code from EPUB file and generate library file"
+      ARGS = %w[EPUB_FILE]
+
+      include Subcommand
+
+      def initialize(config)
+        super
+
+        @library_path = nil
+        @interactive = false
+
+        @available_lexers = Rouge::Lexer.all.sort_by(&:tag).each_with_object({}).with_index {|(lexer, lexers), index|
+          lexers[(index + 1).to_s] = lexer
+        }
+        initial = nil
+        @lexers_display = @available_lexers.collect {|(index, lexer)|
+          init = lexer.title[0].upcase
+          if initial == init
+            option = ""
+          else
+            option = "\n"
+            initial = init
+          end
+          option << "#{index})".bold << " " << lexer.title
+          option << "(#{lexer.aliases.join(', ')})" unless lexer.aliases.empty?
+          option
+        }.join("  ")
+        @commands = ["s) skip", "q) quit", "c) show code", "o) show options"].join("  ")
+        @commands = {
+          "s" => "skip",
+          "q" => "quit",
+          "c" => "show code",
+          "o" => "show options"
+        }.collect {|(key, command)|
+          "#{key})".bold << " " << command
+        }.join("  ")
+      end
+
+      def run(argv)
+        parse_options! argv
+
+        epub_path = argv.shift
+        raise ArgumentError, 'Specify EPUB file' unless epub_path
+
+        begin
+          # @todo Make this optional
+          require 'epub/maker/ocf/physical_container/zipruby'
+          EPUB::OCF::PhysicalContainer.adapter = :Zipruby
+        rescue LoadError
+        end
+        epub = EPUB::Parser.parse(epub_path)
+        $stderr.puts "Detecting code from \"#{epub.title}\""
+
+        codelist = {}
+        library = Library.new
+        library.metadata["Release Identifier"] = epub.release_identifier
+        library.metadata["title"] = epub.title
+        catch do |quit|
+          EPUB::Searcher.search_element(epub, css: 'code').each do |result|
+            item = result[:itemref].item
+            if @interactive
+              catch do |skip|
+                show_item item
+                show_code result[:element]
+                show_options
+                show_commands
+                i = ask
+
+                while true
+                  case i
+                  when "s"
+                    throw skip
+                  when "q"
+                    throw quit
+                  when "c"
+                    show_item item
+                    show_code(result[:element])
+                    show_options
+                    show_commands
+                    i = ask
+                  when "o"
+                    show_options
+                    show_commands
+                    i = ask
+                  else
+                    lexer = @available_lexers[i]
+                    unless lexer
+                      i = ask
+                      next
+                    end
+                    library.codelist[result[:location]] = {"language" => lexer.tag}
+                    break
+                  end
+                end
+              end
+            else
+              library.codelist[result[:location]] = ({
+                "language" => nil,
+                "item" => result[:itemref].item.entry_name,
+                "code" => result[:element].content
+              })
+            end
+          end
+
+          path = library.save(@library_path)
+          $stderr.puts "Library file was saved to:"
+          $stdout.puts path
+        end
+      end
+
+      private
+
+      def parse_options!(argv)
+        super do |opt|
+          opt.separator ""
+          opt.on "-i", "--interactive" do
+            @interactive = true
+          end
+          opt.on "-o", "--output=FILE", "File to save library data", Pathname do |path|
+            @library_path = path
+          end
+        end
+      end
+
+      def show_item(item)
+        $stderr.puts
+        $stderr.puts item.entry_name
+      end
+
+      def show_code(code)
+        $stderr.puts
+        $stderr.puts code.content
+        $stderr.puts
+      end
+
+      def show_options
+        $stderr.puts @lexers_display
+      end
+
+      def show_commands
+        $stderr.puts
+        $stderr.puts @commands
+      end
+
+      def ask
+        $stderr.print "Which language?  "
+        $stdin.gets.chomp
+      end
+    end
+  end
+end

data/app/highlight.rb

@@ -0,0 +1,161 @@
+require "pathname"
+require "optparse"
+require "optparse/pathname"
+require "epub/parser"
+require "epub/maker"
+require "rouge"
+require "rouge/lexers/fluentd"
+require "pirka/library"
+require_relative "subcommand"
+
+module Pirka
+  class App
+    class Highlight
+      PROGRAM_NAME = "highlight"
+      DESCRIPTION = "Highlights source code in EPUB file"
+      ARGS = %w[EPUB_FILE]
+
+      include Subcommand
+
+      DUMMY_ORIGIN = Addressable::URI.parse("file:///")
+      CSS_PATH = "pirka/style.css" # @todo Avoid conflict with existing item by other than Pirka
+      CSS_CLASS_NAME = "pirka"
+      SCOPE = "code.#{CSS_CLASS_NAME}"
+      THEME = "github"
+
+      def initialize(config)
+        super
+        @library_path = nil
+      end
+
+      # @todo Handle multiple renditions
+      def run(argv)
+        parse_options! argv
+
+        epub_path = argv.shift
+        raise ArgumentError, 'Specify EPUB file' unless epub_path
+
+        begin
+          # @todo Make this optional
+          require 'epub/maker/ocf/physical_container/zipruby'
+          EPUB::OCF::PhysicalContainer.adapter = :Zipruby
+        rescue LoadError
+        end
+        epub = EPUB::Parser.parse(epub_path)
+        library = find_library(epub.unique_identifier, epub.modified)
+        raise RuntimeError, "Cannot find code list #{Library.filename(epub.release_identifier)} for #{epub.release_identifier}(#{epub_path}) in any directory of #{Library.directories.join(", ")}" unless library
+
+        css_item = add_css_file(epub)
+        need_save = highlight_contents(epub, css_item, library)
+        need_save << css_item
+        need_save.uniq!
+        update_modified_date(epub, Time.now)
+
+        save_file epub, need_save
+      end
+
+      # @param [EPUB::Book, EPUB::Book::Features] epub
+      # @return [EPUB::Publication::Package::Manifest::Item] item indicating added CSS file
+      def add_css_file(epub)
+        rootfile_path = DUMMY_ORIGIN + epub.ocf.container.rootfile.full_path
+        style = Rouge::Theme.find(THEME).new(scope: SCOPE).render
+
+        epub.package.manifest.make_item {|item|
+          item.href = (DUMMY_ORIGIN + CSS_PATH).route_from(rootfile_path)
+          # IMPROVEMENT: Want to call item.entry_name = css_path
+          item.media_type = 'text/css'
+          item.id = CSS_PATH.gsub('/', '-') # @todo Avoid conflict with existing items
+          item.content = style
+        }
+      end
+
+      # @todo Do the best when file for release identifier is not find but for unique identifier found
+      def find_library(unique_identifier, modified)
+        @library_path ? Library.load_file(@library_path) :
+          Library.find_by_release_identifier("#{unique_identifier}@#{modified}")
+      end
+
+      # @todo Consider descendant elements of code
+      def highlight_contents(epub, css_item, library)
+        need_save = []
+
+        formatter = Rouge::Formatters::HTML.new
+
+        library.each.reverse_each do |(cfi, data)|
+          lang = data["language"]
+          unless lang
+            warn "Language for #{cfi} is not detected"
+            next
+          end
+          itemref, elem, _ = EPUB::Searcher.search_by_cfi(epub, cfi)
+          item = itemref.item
+          doc = elem.document
+          lexer = Rouge::Lexer.find(lang) || Rouge::Lexer.guess(source: elem.content)
+          unless lexer
+            warn "Cannot find lexer for #{lang}"
+            next
+          end
+          elem.inner_html = formatter.format(lexer.lex(elem.content)) # @todo Consider the case `elem` has descendants
+
+          classes = (elem["class"] || "").split(/\s+/)
+          unless classes.include? CSS_CLASS_NAME
+            classes << CSS_CLASS_NAME
+            elem["class"] = classes.join(" ")
+          end
+
+          link = doc.at('#pirka') # @todo Avoid conflict with existing link
+          unless link
+            item_entry_name = DUMMY_ORIGIN + item.entry_name
+            entry_name = DUMMY_ORIGIN + css_item.entry_name
+            href = entry_name.route_from(item_entry_name)
+            link = Nokogiri::XML::Node.new('link', doc)
+            link['href'] = href
+            link['type'] = 'text/css'
+            link['rel'] = 'stylesheet'
+            link['id'] = 'pirka'
+            head = (doc/'head').first
+            head << link
+          end
+          item.content = doc.to_xml
+          need_save << item
+        end
+
+        need_save
+      end
+
+      def update_modified_date(epub, time = Time.now)
+        modified = epub.modified
+        unless modified
+          modified = EPUB::Publication::Package::Metadata::Meta.new
+          modified.property = 'dcterms:modified'
+          epub.package.metadata.metas << modified
+        end
+        modified.content = time.utc.iso8601
+
+        modified
+      end
+
+      def save_file(epub, need_save)
+        need_save.each do |item|
+          item.save
+          epub.package.manifest << item
+        end
+        epub.package.edit
+      end
+
+      private
+
+      # @todo theme
+      # @todo CSS file path
+      # @todo scope
+      def parse_options!(argv)
+        super do |opt|
+          opt.separator ""
+          opt.on "-l", "--library=FILE", "library file", Pathname do |path|
+            @library_path = path
+          end
+        end
+      end
+    end
+  end
+end

data/app/lib.rb

@@ -0,0 +1,52 @@
+require "pirka/library"
+
+module Pirka
+  class App
+    class Lib
+      PROGRAM_NAME = "lib"
+      DESCRIPTION = "Show library infomation"
+
+      include Subcommand
+
+      def run(argv)
+        # show all
+        # show remote repos
+        # show data dirs
+        # show data home
+        # show books
+        # show book metadata
+
+        no_dir_file_length = Library::SUBDIR_LENGTH + Library::EXT.length
+        Library.directories.each do |dir|
+          next unless dir.directory?
+
+          dir.each_child do |child|
+            if child.to_path.length < no_dir_file_length && child.extname != Library::EXT
+              show_info child
+              next
+            end
+            next unless child.directory?
+            next unless child.basename.to_path.length == Library::SUBDIR_LENGTH
+            child.each_child do |lib|
+              next unless lib.extname == Library::EXT
+              next unless lib.file?
+              show_info lib
+            end
+          end
+        end
+      end
+
+      def show_info(path)
+        $stdout.puts Library.load_file(path).metadata.to_yaml
+        $stdout.puts "library: #{path}"
+      end
+
+      private
+
+      def parse_options!(argv)
+        super do |opt|
+        end
+      end
+    end
+  end
+end

data/app/subcommand.rb

@@ -0,0 +1,35 @@
+module Pirka
+  class App
+    module Subcommand
+      class << self
+        def included(base)
+          APPS[base::PROGRAM_NAME] = base
+        end
+      end
+
+      # @param [Config]
+      def initialize(config)
+        @config = config
+      end
+
+      private
+
+      # @todo Consider the case the subcommand has no option
+      def parse_options!(argv)
+        parser = OptionParser.new {|opt|
+          usage = "Usage: #{opt.program_name} [options] #{self.class::PROGRAM_NAME}"
+          usage << " " << self.class::ARGS.join(" ") if self::class.const_defined?(:ARGS)
+
+          opt.program_name = "#{opt.program_name} [global options] #{self.class::PROGRAM_NAME}"
+          opt.banner = <<EOB
+#{self::class::DESCRIPTION}
+
+#{usage}
+EOB
+          yield opt if block_given?
+        }
+        parser.order! argv
+      end
+    end
+  end
+end