aipp 0.1.3 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6ce24174fda71fe6649f2d1f02eab03a2d9de0a59da6dbb581d581fe2dc2df4c
-  data.tar.gz: 7643f1ec9c399e244ddbc13ef9a3a5df31eaf63e78c197a55c55838a0536d8ad
+  metadata.gz: 279a7722a7ee0070603236b477cdd96bb1babfea88e8e70cce7088069683e783
+  data.tar.gz: cb9453b4aca266eafc8aa7ae8d949d402c42f748d2193796b160fc17c9280068
 SHA512:
-  metadata.gz: d4029172c77e70998f89c10eb19c6bb827d042fd7a9c9190538bb422e319c0387c85534de28e8d0357b0c676f99e7a222479419bbc34d00cc34ca3c918636987
-  data.tar.gz: 3a294be41d8eb5067dc286b60ed304db554607694e094b7fa92a114ce9bac663261ba4246d45d4da133b753689535f2a708d09dd47dc06204dd5f86c5dd982f6
+  metadata.gz: 4102f1a2c77ea39f2b3200c61d49c7dd194768944688f42d4cb0740707d431b2dc290e35a81215de0c646284a59b8ed9fee4dff15a6cb65616e70e3cf979868a
+  data.tar.gz: c8087248c414c41e415c1361370f5ec84116e6a11d6aa23336eceb353f9af13f72de7b8459b7d4504a44ff4bb9206c042734e18da15a02010bd8f8074ff830d5

data/.gitignore

@@ -3,4 +3,6 @@ Gemfile.lock
 pkg/*
 *.gem
 .bundle
+.yardoc
 *.aixm
+*.ofmx

data/.ruby-version

@@ -1 +1 @@
-ruby-2.5.0
+ruby-2.5.3

data/.yardopts

@@ -0,0 +1,3 @@
+--no-private
+lib/**/*.rb -
+README.md CHANGELOG.md LICENSE.txt

data/CHANGELOG.md

@@ -1,22 +1,37 @@
+## 0.2.0
+
+#### Changes
+* Complete rewrite of the framework in order to allow cross-AIP parsing made necessary due to recent changes in LF AIP.
+
+#### Additions
+* LF/ENR-2.1
+* Handling of errors and warnings optimized for parser development
+
+#### Removals
+* LF/AD-1.5
+
 ## 0.1.3
 
-* Additons:
-  LF/AD-1.5
-* Minor changes:
-  * Summary at end of run
+#### Changes
+* Summary at end of run
+
+#### Additions
+* LF/AD-1.5
+* Source file line number evaluation
 
 ## 0.1.2
 
-* Additions:
-  * LF/ENR-4.3
+#### Additions:
+* LF/ENR-4.3
 
 ## 0.1.1
 
-* Additions:
-  * Helper modules
-  * LF/ENR-4.1
+#### Additions:
+* LF/ENR-4.1
+* Helper modules
 
 ## 0.1.0
 
-* Initial implementation of aip2aixm executable and infrastructure
+#### Initial Implementation
+* Framework and aip2aixm executable
 * LF/ENR-5.1

data/README.md

@@ -8,9 +8,7 @@
 
 Parser for Aeronautical Information Publication (AIP) available online.
 
-This gem incluces an executable to download and parse aeronautical data, then
-export is as [AIXM](https://github.com/svoop/aixm) which can be consumed by
-[Open Flightmaps](https://openflightmaps.org).
+This gem incluces two executables to download and parse aeronautical data, then export is as [AIXM](https://github.com/svoop/aixm) or [OFMX](https://github.com/openflightmaps/ofmx/wiki).
 
 * [Homepage](https://github.com/svoop/aipp)
 * Author: [Sven Schwyn - Bitcetera](http://www.bitcetera.com)
@@ -27,38 +25,126 @@ gem aipp
 
 ```
 aip2aixm --help
+aip2ofmx --help
 ```
 
-## Parsers
+## Storage
 
-Parsers are defined as modules and named <tt>lib/aipp/parser/{FIR}/{AIP}.rb</tt>. For
-them to plug in, you have to define the following public methods:
+AIPP uses a storage directory for configuration, caching and in order to keep the results of previous runs. The default location is `~/.aipp`, however, you can pass a different directory with the `--storage` argument.
 
-* `url`<br>Must return the download URL of the AIP HTML as a string.
-* `convert!`<br>Takes `html` ([Nokogiri document](https://github.com/sparklemotion/nokogiri)) to parse and populate `aixm` ([AIXM document](https://github.com/svoop/aixm))
+## Regions
 
-You should read and honor the following attributes passed in from `aip2aixm`
-arguments:
+The reference implementation is region "LF" (France).
 
-* `@fir`
-* `@aip`
-* `@airac`
-* `@limit`
+To implement a region, you have to create a new directory <tt>lib/aipp/regions/{REGION}</tt> and place the following files there:
 
-You should `fail` on fatal and `warn` on non-fatal problems. If `$DEBUG` is
-+true+ (e.g. by use of the `-D` option), a Pry session will open if you use
-`warn` as follows:
+### helper.rb
+
+Create the file <tt>helper.rb</tt> which defines the module `AIPP::LF::Helper` and usually contains the URL builder method `url_for` used by all AIP parsers:
+
+```ruby
+module AIPP
+  module LF
+    module Helper
+
+      def url_for(aip_file)
+        # build and return the download URL for the aip file
+      end
+
+    end
+  end
+end
+```
+
+### AIP Parsers
+
+Say, you want to parse ENR-4.3, you have to create the file <tt>ENR-4.3.rb</tt> which defines the class `AIPP::LF::ENR43` as follows:
+
+```ruby
+module AIPP
+  module LF
+    class ENR43 < AIP
+
+      DEPENDS = %w(ENR-2.1 ENR-2.2)   # declare dependencies to other AIPs
+
+      def parse
+        html = load_html
+        # read from "html" (Nokogiri::HTML::Document) and write to "aixm"
+      end
+
+    end
+  end
+end
+```
+
+Some AIP may be split over several files which require a little more code to load the individual HTML source files:
+
+```ruby
+module AIPP
+  module LF
+    class AD2 < AIP
+
+      def parse
+        %i(one two three).each do |part|
+          html = load_html(aip_file: "#{aip}.#{part}")
+          # read from "html" (Nokogiri::HTML::Document) and write to "aixm"
+        end
+      end
+
+    end
+  end
+end
+```
+
+Inside the `parse` method, you have access to the following objects:
+
+* `aixm` – target: instance of `AIXM::Document` (see [AIXM Rubygem](https://github.com/svoop/aixm))
+* `options` – arguments read from <tt>aip2aixm</tt> or <tt>aip2ofmx</tt> respectively
+* `config` – configuration read from <tt>config.yml</tt>
+* `cache` – virgin `OStruct` instance to make objects available across AIPs
+
+Furthermore, you have access to any method defined in <tt>helper.rb</tt> and you can overwrite any of them if need be (most notably `url_for`).
+
+### Source File Line Numbers
+
+In order to reference the source of an AIXM/OFMX feature, it's necessary to know the line number where a particular node occurs in the HTML source file. You can ask any HTML element as follows:
+
+```ruby
+tr.line
+```
+
+:warning: Make sure you have build Nokogumbo `--with-libxml2`. Otherwise, all elements will report line number `0` and therefore render OFMX documents invalid. See the [Nokogumbo README](https://github.com/rubys/nokogumbo/blob/master/README.md#flavors-of-nokogumbo) for more on this.
+
+### Errors
+
+You should `fail` on fatal problems. The `-E` command line argument will open a Pry session when such an error occurs. Issue errors as usual:
 
 ```ruby
-warn("my message", binding)
+fail "my message"
 ```
 
-## Helpers
+### Warnings
+
+You should `warn` on non-fatal problems. The `-W ID` command line argument will open a Pry session when then warning with the given ID occurs. To issue a warning:
+
+```ruby
+warn("my message", context: binding)   # open Pry with binding context
+warn("my message", context: error)     # open Pry with error context
+```
+
+### Informational Messages
+
+You may `info` any other useful information:
+
+```ruby
+info("my message")                  # show info only in verbose mode (-V)
+info("my message", force: true)     # always show info
+info("my message", color: :green)   # show info with this color
+```
 
-Any modules in <tt>lib/aipp/parser/{FIR}/helpers</tt> are required automatically and
-can be included and used in the parsers.
+## AIRAC Date Calculations
 
-## AIRAC date calculations
+The `AIPP::AIRAC` class is used to calculate AIRAC cycles:
 
 ```ruby
 airac = AIPP::AIRAC.new(Date.parse('2017-12-24'))
@@ -70,9 +156,13 @@ airac.next_id     # => 1801
 
 ## References
 
-* AIP authorities
-  * [LF: SIA](https://www.sia.aviation-civile.gouv.fr)
-* [Open Flightmaps](https://openflightmaps.org)
+* LF - France
+  * [SIA – AIP publisher](https://www.sia.aviation-civile.gouv.fr)
+  * [OpenData – public data files](https://www.data.gouv.fr)
+  * [Protected Planet – protected area data files](https://www.protectedplanet.net)
+* [Geo Maps – programmatically generated GeoJSON maps](https://github.com/simonepri/geo-maps)
+* [Open Flightmaps – open-source aeronautical maps](https://openflightmaps.org)
+* [AIXM Rubygem – AIXM/OFMX generator for Ruby](https://github.com/svoop/aixm)
 
 ## Development
 
@@ -88,8 +178,7 @@ Please submit issues on:
 
 https://github.com/svoop/aipp/issues
 
-To contribute code, fork the project on Github, add your code and submit a
-pull request:
+To contribute code, fork the project on Github, add your code and submit a pull request:
 
 https://help.github.com/articles/fork-a-repo
 

data/aipp.gemspec

@@ -30,8 +30,10 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'guard'
   spec.add_development_dependency 'guard-minitest'
 
-  spec.add_runtime_dependency 'aixm', '~> 0', '>= 0.2.3'
+  spec.add_runtime_dependency 'aixm', '~> 0', '>= 0.3.2'
   spec.add_runtime_dependency 'nokogiri', '~> 1'
-  spec.add_runtime_dependency 'nokogumbo', '~> 1'
+  spec.add_runtime_dependency 'nokogumbo', '~> 2'
+  spec.add_runtime_dependency 'colorize', '~> 0'
   spec.add_runtime_dependency 'pry', '~> 0'
+  spec.add_runtime_dependency 'pry-rescue', '~> 1'
 end

data/exe/aip2aixm

@@ -1,8 +1,6 @@
 #!/usr/bin/env ruby
 
 require 'bundler/inline'
-require 'optparse'
-require 'yaml'
 
 gemfile do
   source 'https://rubygems.org'
@@ -10,51 +8,4 @@ gemfile do
   gem 'aipp', '~> 0'
 end
 
-class Executable
-  attr_reader :fir, :aip, :airac, :ofm, :limit, :verbose
-
-  def initialize
-    @ogn = false
-    @airac = AIPP::AIRAC.new(Date.today).date
-    @limit = Float::INFINITY
-    @verbose = false
-    OptionParser.new do |o|
-      o.banner = <<~END
-        Download online AIP and convert it to AIXM for VFR.
-        Usage: #{File.basename($0)} [options]
-      END
-      o.on('-A', '--about', 'author and license information') { puts 'Written by Sven Schwyn (bitcetera.com) and distributed under MIT license.'; exit }
-      o.on('-l', '--list', "list available FIR/AIP and exit") { puts AIPP::Loader.list.to_yaml; exit }
-      o.on('-f', '--fir STRING', String, 'FIR (flight information region, e.g. "LF")') { |v| @fir = v }
-      o.on('-a', '--aip STRING', String, 'AIP (aeronautical information publication, e.g. "ENR-5.1")') { |v| @aip = v }
-      o.on('-d', '--airac DATE', String, 'AIRAC date (e.g. "2018-01-04", default: current)') { |v| @airac = v }
-      o.on('-o', '--[no-]ofm', 'Use OFM extensions (default: false)') { |v| @ofm = v }
-      o.on('-L', '--limit INTEGER', Integer, 'Stop conversion after this many features') { |v| @limit = v }
-      o.on('-D', '--[no-]debug', 'Enable debug mode (default: false)') { |v| $DEBUG = v }
-      o.on('-v', '--[no-]verbose', 'Verbose error reporting (default: false)') { |v| @verbose = v }
-    end.parse!
-    fail(ArgumentError, "FIR must be supplied") unless @fir
-    fail(ArgumentError, "AIP must be supplied") unless @aip
-    fail(ArgumentError, "AIRAC date must be supplied") unless @airac
-  end
-
-  def run
-    extensions = [(:ofm if ofm)].compact
-    filename = [fir, aip, airac].join('_') + '.aixm'
-    loader = AIPP::Loader.new(fir: fir, aip: aip, airac: airac, limit: limit)
-    File.write(Pathname.new(Dir.pwd).join(filename), loader.aixm.to_aixm(*extensions))
-    puts "#{loader.aixm.features.count} feature(s) written to #{filename}"
-  end
-end
-
-begin
-  executable = Executable.new
-  executable.run
-rescue => exception
-  if executable&.verbose
-    raise exception
-  else
-    puts "#{File.basename($0)}: #{exception.message}"
-    exit 1
-  end
-end
+AIPP::Executable.new(schema: File.basename($0)[4..-1].to_sym).run

data/exe/aip2ofmx

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+
+require 'bundler/inline'
+
+gemfile do
+  source 'https://rubygems.org'
+  ruby '>= 2.5'
+  gem 'aipp', '~> 0'
+end
+
+AIPP::Executable.new(schema: File.basename($0)[4..-1].to_sym).run

data/lib/aipp.rb

@@ -1,12 +1,29 @@
-require 'tmpdir'
+require 'forwardable'
+require 'colorize'
+require 'pry'
+require 'pry-rescue'
+require 'optparse'
+require 'yaml'
 require 'pathname'
 require 'open-uri'
+require 'securerandom'
+require 'tsort'
+require 'ostruct'
 require 'date'
 require 'nokogiri'
 require 'nokogumbo'
 require 'aixm'
 
 require_relative 'aipp/version'
-require_relative 'aipp/airac'
 require_relative 'aipp/refinements'
-require_relative 'aipp/loader'
+require_relative 'aipp/t_hash'
+require_relative 'aipp/executable'
+require_relative 'aipp/progress'
+require_relative 'aipp/airac'
+require_relative 'aipp/aip'
+require_relative 'aipp/parser'
+
+# Disable "did you mean?" suggestions
+module DidYouMean::Correctable
+  remove_method :to_s
+end

data/lib/aipp/aip.rb

@@ -0,0 +1,63 @@
+module AIPP
+
+  # @abstract
+  class AIP
+    using AIPP::Refinements
+    include AIPP::Progress
+    extend Forwardable
+
+    DEPENDS = []
+
+    # @return [String] AIP name (e.g. "ENR-2.1")
+    attr_reader :aip
+
+    # @return [AIXM::Document] target document
+    attr_reader :aixm
+
+    # @!method aixm
+    #   @return (see AIPP::Parser#aixm)
+    # @!method config
+    #   @return (see AIPP::Parser#config)
+    # @!method options
+    #   @return (see AIPP::Parser#options)
+    # @!method cache
+    #   @return (see AIPP::Parser#cache)
+    def_delegators :@parser, :aixm, :config, :options, :cache
+
+    def initialize(aip:, parser:)
+      @aip, @parser = aip, parser
+      self.class.include [:AIPP, options[:region], :Helper].constantize
+    end
+
+    # Load an AIP source file
+    #
+    # Depending on whether a local copy of the file exists, either:
+    # * download from URL to local storage and read from local storage
+    # * read from local storage
+    #
+    # An URL builder method +url_for(aip_file)+ must be defined either in
+    # +helper.rb+ or in the AIP parser definition (e.g. +ENR-2.1.rb+).
+    #
+    # @param aip_file [String] e.g. "ENR-2.1" or "AD-2.LFMV" (default: +aip+)
+    # @return [Nokogiri::HTML5] HTML document
+    def load_html(aip_file: nil)
+      aip_file ||= aip
+      unless (storage_path = storage_path(aip_file)).exist?
+        info("Downloading #{storage_file}", force: true)
+        storage_path.mkpath
+        IO.copy_stream(open(url_for(aip_file)), storage_path)
+      end
+      Nokogiri::HTML5(storage_path)
+    end
+
+    private
+
+    def storage_path(aip_file=nil)
+      options[:storage].
+        join(options[:airac].date.xmlschema).
+        join(("#{aip_file}.html" if aip_file).to_s)
+    end
+
+  end
+
+end

data/lib/aipp/airac.rb

@@ -1,43 +1,51 @@
 module AIPP
 
-  ##
-  # Calculate the AIRAC date and AIRAC ID for the given +any_date+
+  # AIRAC cycle date calculations
+  #
+  # @example
+  #   airac = AIPP::AIRAC.new('2018-01-01')
+  #   airac.date        # => #<Date: 2017-12-07 ((2458095j,0s,0n),+0s,2299161j)>
+  #   airac.id          # => 1713
+  #   airac.next_date   # => #<Date: 2018-01-04 ((2458123j,0s,0n),+0s,2299161j)>
+  #   airac.next_id     # => 1801
   class AIRAC
-    ##
     # First AIRAC date following the last cycle length modification
     ROOT_DATE = Date.parse('2015-06-25').freeze
 
-    ##
     # Length of one AIRAC cycle
     DAYS_PER_CYCLE = 28
 
-    attr_reader :date, :id
+    # @return [Date] AIRAC effective on date
+    attr_reader :date
 
+    # @return [Integer] AIRAC cycle ID
+    attr_reader :id
+
+    # @param any_date [Date] any date within the AIRAC cycle (default: today)
     def initialize(any_date = nil)
-      any_date ||= Date.today
-      fail(ArgumentError, "argument must be of class Date") unless any_date.is_a? Date
+      any_date = any_date ? Date.parse(any_date.to_s) : Date.today
       fail(ArgumentError, "cannot calculate dates before #{ROOT_DATE}") if any_date < ROOT_DATE
       @date = date_for(any_date)
       @id = id_for(@date)
     end
 
+    # @return [Date] next AIRAC effective on date
     def next_date
       date + DAYS_PER_CYCLE
     end
 
+    # @return [Integer] next AIRAC cycle ID
     def next_id
       id_for next_date
     end
 
     private
 
-    ##
     # Find the AIRAC date for +any_date+
     def date_for(any_date)
       ROOT_DATE + (any_date - ROOT_DATE).to_i / DAYS_PER_CYCLE * DAYS_PER_CYCLE
     end
 
-    ##
     # Find the AIRAC ID for the AIRAC +date+
     def id_for(date)
       (date.year % 100) * 100 + ((date.yday - 1) / 28) + 1