alf 0.9.3 → 0.10.0

data/README.md

@@ -1,4 +1,4 @@
-# Alf - Relational Algebra at your fingertips (version 0.9.3)
+# Alf - Relational Algebra at your fingertips (version 0.10.0)
 
 ## Description
 
@@ -12,19 +12,24 @@ _relations_... Let's stop the segregation ;-)
 
 ### Install
 
-    % [sudo] gem install alf
+    % [sudo] gem install alf [fastercsv, ...]
     % alf --help
 
 ### Bundler & Require 
 
     # API is not considered stable enough for now, please use
-    gem "alf", "= 0.9.2"
+    gem "alf", "= 0.10.0"
+    
+    # The following should not break your code, but is a bit less safe, 
+    # until 1.0.0 has been reached
+    gem "alf", "~> 0.10.0"
 
 ### Links
     
-* {http://rubydoc.info/github/blambeau/alf/master/frames} (read this file there!)
-* {http://github.com/blambeau/alf} (source code)
-* {http://revision-zero.org} (author's blog)
+* http://blambeau.github.com/alf
+* http://rubydoc.info/gems/alf
+* http://github.com/blambeau/alf
+* http://rubygems.org/gems/alf
 
 ### Quick overview
 
@@ -65,9 +70,9 @@ of a truly relational algebra approach. Objectives behind Alf are manifold:
 
 * Alf is also an educational tool, that I've written to draw people attention
   about the ill-known relational theory (and ill-represented by SQL). The tool
-  is largely inspired from TUTORIAL D, the tutorial language of Chris Date and 
+  is largely inspired from **Tutorial D**, the tutorial language of Chris Date and 
   Hugh Darwen in their books, more specifically in 
-  {http://www.thethirdmanifesto.com/ The Third Manifesto (TTM)}. 
+  {http://www.thethirdmanifesto.com/ *The Third Manifesto* (TTM)}. 
   However, Alf only provides an overview of the relational _algebra_ defined 
   there (Alf is neither a relational _database_, nor a relational _language_). 
   I hope that people (especially talented developers) will be sufficiently 
@@ -76,7 +81,7 @@ of a truly relational algebra approach. Objectives behind Alf are manifold:
   the following query for the kind of things that you'll never ever have in SQL 
   (see also 'alf help quota', 'alf help wrap', 'alf help group', ...):
   
-        % alf --text summarize supplies --by=sid -- total "sum(:qty)" -- which "group(:pid)"
+        % alf --text summarize supplies -- sid -- total "sum{ qty }" which "collect{ pid }"
   
 * Last, but not least, Alf is an attempt to help me test some research ideas and 
   communicate about them with people that already know (all or part) of the TTM 
@@ -87,7 +92,7 @@ of a truly relational algebra approach. Objectives behind Alf are manifold:
   'research work in progress', and used with care because not necessarily in 
   conformance with the TTM.
   
-        % alf --text quota supplies --by=sid --order=qty -- pos "count()"  
+        % alf --text quota supplies -- sid -- qty -- pos "count()"  
 
 ## Overview of relational theory
 
@@ -233,7 +238,7 @@ you want! The same query, in shell:
     
 ## What is Alf exactly?   
 
-The Third Manifesto defines a series of prescriptions, proscriptions and very 
+*The Third Manifesto* defines a series of prescriptions, proscriptions and very 
 strong suggestions for designing a truly relational _language_, called a _D_, 
 as an alternative to SQL for managing relational databases. This is far behind
 my objective with Alf, as it does not touch at database issues at all (persistence,
@@ -267,7 +272,7 @@ section about versioning policy at the end of this file).
 If you take a look at .alf example files, you'll find functional ruby expressions 
 like the following (called Lispy expressions):
 
-    % cat examples/minus.alf
+    % cat examples/operators/minus.alf
 
     # Give all suppliers, except those living in Paris
     (minus :suppliers, 
@@ -280,7 +285,7 @@ like the following (called Lispy expressions):
 You can simply execute such expressions with the alf command line itself (the 
 three following invocations return the same result):
  
-    % alf examples/minus.alf | alf show
+    % alf examples/operators/minus.alf | alf show
     % alf show minus
     % alf -e "(restrict :suppliers, lambda{ city != 'Paris' })" | alf show
 
@@ -373,7 +378,7 @@ operators works on relations and return relations), you can use a sub expression
               :supplies), 
         :cities),
       [:country],
-      :which => Agg::group(:pid),
+      :which => Agg::collect(:pid),
       :total => Agg::sum{ qty })
 
 Of course, complex queries quickly become unreadable that way. But you can always
@@ -384,30 +389,19 @@ split complex tasks in more simple ones:
     supplying      = (join with_countries, :supplies)
     (summarize supplying,
                [:country],
-               :which => Agg::group(:pid),
+               :which => Agg::collect(:pid),
                :total => Agg::sum{ qty })
 
 And here is the result !
 
-    +----------+----------+--------+
-    | :country | :which   | :total |
-    +----------+----------+--------+
-    | England  | +------+ |   2200 |
-    |          | | :pid | |        |
-    |          | +------+ |        |
-    |          | | P1   | |        |
-    |          | | P2   | |        |
-    |          | | P3   | |        |
-    |          | | P4   | |        |
-    |          | | P5   | |        |
-    |          | | P6   | |        |
-    |          | +------+ |        |
-    | France   | +------+ |    200 |
-    |          | | :pid | |        |
-    |          | +------+ |        |
-    |          | | P2   | |        |
-    |          | +------+ |        |
-    +----------+----------+--------+
+    +------+--------+--------------------------+
+    | :sid | :total | :which                   |
+    +------+--------+--------------------------+
+    | S1   |   1300 | [P1, P2, P3, P4, P5, P6] |
+    | S2   |    700 | [P1, P2]                 |
+    | S3   |    200 | [P2]                     |
+    | S4   |    900 | [P2, P4, P5]             |
+    +------+--------+--------------------------+
 
 ### Reference API
 
@@ -522,11 +516,11 @@ For example, try the following:
 ### Recognized data streams/files (.rash files)
     
 For educational purposes, 'suppliers' and 'cities' inputs are magically resolved 
-as denoting the files examples/suppliers.rash and examples/cities.rash, 
-respectively. You'll find other data files: parts.rash, supplies.rash that are 
-resolved magically as well and with which you can play. For non-educational 
-purposes, operands may always be explicit files, or you can force the folder in
-which datasource files have to be found:
+as denoting the files examples/operators/suppliers.rash and 
+examples/operators/cities.rash, respectively. You'll find other data files: 
+parts.rash, supplies.rash that are resolved magically as well and with which you 
+can play. For non-educational purposes, operands may always be explicit files, 
+or you can force the folder in which datasource files have to be found:
 
     # The following invocations are equivalent
     % alf restrict /tmp/foo.rash -- "..." 
@@ -541,13 +535,13 @@ operand on standard input if not specific as command argument. For example, the
 invocation below is equivalent to the one given above.
 
     # display suppliers that live in Paris
-    % cat examples/suppliers.rash | alf restrict -- "city == 'Paris'"  
+    % cat examples/operators/suppliers.rash | alf restrict -- "city == 'Paris'"  
 
 Similarly, when only one operand is present in invocations of binary operators, 
 they read their left operand from standard input. Therefore, the join given in
 previous section can also be written as follows:
 
-    % cat examples/suppliers.rash | alf join cities
+    % cat examples/operators/suppliers.rash | alf join cities
 
 The relational algebra is _closed_ under its operators, which means that these 
 operators take relations as operands and return a relation. Therefore operator 
@@ -595,7 +589,7 @@ a .yaml file, as follows:
 You'll also find .alf files in the examples folder, that contain more complex
 examples in the Ruby functional syntax (see section below). 
 
-    % cat examples/group.alf
+    % cat examples/operators/group.alf
     #!/usr/bin/env alf
     (group :supplies, [:pid, :qty], :supplying)
 
@@ -779,7 +773,7 @@ following template for contributions in lib/alf/renderer
   third manifesto paper} itself? 
 - Also have a look at {http://www.dcs.warwick.ac.uk/~hugh/TTM/Projects.html other 
   implementation projects}, especially {http://dbappbuilder.sourceforge.net/Rel.php Rel}
-  which provides an implementation of the TUTORIAL D language.
+  which provides an implementation of the **Tutorial D** language.
 - {https://github.com/dkubb/veritas Dan Kubb's Veritas} project is worth considering 
   also in the Ruby community. While very similar to Alf in providing a pure ruby
   algebra implementation, Veritas mostly provides a framework for manipulating

data/TODO.md

@@ -4,7 +4,7 @@
         (rename :suppliers, [:name, :city], :suffix => "_sup")
         (rename :suppliers, [:name, :city], lambda{|name| name.upcase}) 
 
-* WRAP: provide a multi-wraping ability?
+* WRAP: provide a multi-wrapping ability?
 
         (wrap (wrap :supplies, [:a, :b], :x), [:x, :c], :y)
          => (wrap :supplies, :x => [:a, :b], :y => [:x, :c])

data/alf.gemspec

@@ -30,7 +30,7 @@ Gem::Specification.new do |s|
   s.description = "Alf brings the relational algebra both in Shell and in Ruby. In Shell, because \nmanipulating any relation-like data source should be as straightforward as a \none-liner. In Ruby, because I've never understood why programming languages \nprovide data structures like arrays, hashes, sets, trees and graphs but not \n_relations_... Let's stop the segregation ;-)"
   
   # The URL of this gem home page (optional)
-  s.homepage = "http://rubydoc.info/github/blambeau/alf/master/frames"
+  s.homepage = "http://blambeau.github.com/alf"
 
   # Gem publication date (required but auto)
   #
@@ -124,15 +124,18 @@ Gem::Specification.new do |s|
   # for each development dependency. These gems are required for developers
   #
   s.add_development_dependency("rake", "~> 0.9.2")
-  s.add_development_dependency("bundler", "~> 1.0")
   s.add_development_dependency("rspec", "~> 2.6.0")
+  s.add_development_dependency("wlang", "~> 0.10.2")
   s.add_development_dependency("rcov", "~> 0.9.9")
   s.add_development_dependency("yard", "~> 0.7.2")
-  s.add_development_dependency("bluecloth", "~> 2.0.9")
-  s.add_development_dependency("wlang", "~> 0.10.1")
-  s.add_development_dependency("noe", "~> 1.3.0")
-  s.add_dependency("quickl", "~> 0.2.2")
-  s.add_dependency("myrrha", "~> 1.0.0")
+  s.add_development_dependency("bluecloth", "~> 2.1.0")
+  s.add_development_dependency("redcarpet", "~> 1.17.2")
+  s.add_development_dependency("fastercsv", "~> 1.5.4")
+  s.add_development_dependency("request-log-analyzer", "~> 1.11.0")
+  s.add_development_dependency("sequel", "~> 3.25.0")
+  s.add_development_dependency("sqlite3", "~> 1.3.0")
+  s.add_dependency("quickl", "~> 0.4.0")
+  s.add_dependency("myrrha", "~> 1.2.0")
 
   # The version of ruby required by this gem
   #

data/alf.noespec

@@ -1,16 +1,22 @@
 template-info:
   name: "ruby"
-  version: 1.3.0
+  version: 1.5.0
   manifest:
+    tasks/spec_test.rake:
+      ignore: true
+    spec/test___lower__.rb:
+      ignore: true
     tasks/unit_test.rake: 
       safe-override: false
+    tasks/debug_mail.txt:
+      safe-override: false
 variables:
   lower: 
     alf
   upper:
     Alf
   version:
-    0.9.3
+    0.10.0
   summary: |-
     Relational Algebra at your fingertips
   description: |-
@@ -22,17 +28,22 @@ variables:
   authors: 
     - {name: Bernard Lambeau, email: blambeau@gmail.com}
   links:
-    - http://rubydoc.info/github/blambeau/alf/master/frames
+    - http://blambeau.github.com/alf
     - http://github.com/blambeau/alf
     - http://rubygems.org/gems/alf
+    - http://rubydoc.info/gems/alf
   dependencies: 
-    - {name: rake,         version: "~> 0.9.2",  groups: [development]}
-    - {name: bundler,      version: "~> 1.0",    groups: [development]}
-    - {name: rspec,        version: "~> 2.6.0",  groups: [development]}
-    - {name: rcov,         version: "~> 0.9.9",  groups: [development]}
-    - {name: yard,         version: "~> 0.7.2",  groups: [development]}
-    - {name: bluecloth,    version: "~> 2.0.9",  groups: [development]}
-    - {name: wlang,        version: "~> 0.10.1", groups: [development]}
-    - {name: noe,          version: "~> 1.3.0",  groups: [development]}
-    - {name: quickl,       version: "~> 0.2.2",  groups: [runtime]}
-    - {name: myrrha,       version: "~> 1.0.0",  groups: [runtime]}    
+    - {name: quickl,       version: "~> 0.4.0",  groups: [runtime      ]}
+    - {name: myrrha,       version: "~> 1.2.0",  groups: [runtime      ]}
+    - {name: rake,         version: "~> 0.9.2",  groups: [test, release]}
+    - {name: rspec,        version: "~> 2.6.0",  groups: [test, release]}
+    - {name: wlang,        version: "~> 0.10.2", groups: [release      ]}
+    - {name: rcov,         version: "~> 0.9.9",  groups: [doc          ]}
+    - {name: yard,         version: "~> 0.7.2",  groups: [doc          ]}
+    - {name: bluecloth,    version: "~> 2.1.0",  groups: [doc          ]}
+    - {name: redcarpet,    version: "~> 1.17.2", groups: [doc          ]}
+    - {name: fastercsv,    version: "~> 1.5.4",  groups: [extra        ]}
+    - {name: request-log-analyzer, version: "~> 1.11.0", groups: [extra]}
+    - {name: sequel,       version: "~> 3.25.0", groups: [extra        ]}
+    - {name: sqlite3,      version: "~> 1.3.0",  groups: [extra        ]}
+

data/bin/alf

@@ -18,10 +18,10 @@ module AlfLauncher
   def self.start(argv)
     load
     if ENV["ALF_OPTS"]
-      argv = Alf::Tools::parse_commandline_args(ENV["ALF_OPTS"]) + argv
+      argv = Quickl.parse_commandline_args(ENV["ALF_OPTS"]) + argv
     end
     Alf::Command::Main.run(argv, __FILE__)
   end
 
 end # module AlfLaucher
-AlfLauncher.start(ARGV)
+AlfLauncher.start(ARGV)

data/doc/commands/exec.md

@@ -0,0 +1,16 @@
+
+Executes an .alf file on current environment
+
+SYNOPSIS
+
+    alf #(command_name) [FILE]
+
+OPTIONS
+
+    #(summarized_options)
+
+DESCRIPTION
+
+This command executes the .alf file passed as first argument (or what comes
+on standard input) as a alf query to be executed on the current environment.
+

data/doc/commands/help.md

@@ -0,0 +1,11 @@
+
+Show help about a specific command
+
+SYNOPSIS
+
+    alf #(command_name) COMMAND
+
+OPTIONS
+
+    #(summarized_options)
+

data/doc/commands/main.md

@@ -0,0 +1,33 @@
+
+alf - Relational algebra at your fingertips
+
+SYNOPSIS
+
+    alf [--version] [--help] 
+    alf -e '(lispy command)'
+    alf [FILE.alf]
+    alf [alf opts] OPERATOR [operator opts] ARGS ...
+    alf help OPERATOR
+
+OPTIONS
+
+    #(summarized_options)
+
+RELATIONAL OPERATORS
+
+    #(summarized_subcommands relational_operators)
+
+EXPERIMENTAL RELATIONAL OPERATORS
+
+    #(summarized_subcommands experimental_operators)
+
+NON-RELATIONAL OPERATORS
+
+    #(summarized_subcommands non_relational_operators)
+
+OTHER NON-RELATIONAL COMMANDS
+
+    #(summarized_subcommands other_non_relational_commands)
+
+See 'alf help COMMAND' for details about a specific command.
+

data/doc/commands/show.md

@@ -0,0 +1,19 @@
+
+Output input tuples through a specific renderer (text, yaml, ...)
+
+SYNOPSIS
+
+    alf #(command_name) DATASET
+
+OPTIONS
+
+    #(summarized_options)
+
+DESCRIPTION
+
+When a dataset name is specified as commandline arg, request the 
+environment to provide this dataset and prints it. Otherwise, take what 
+comes on standard input.
+
+Note that this command is not an operator and should not be piped anymore.
+

data/doc/operators/non_relational/autonum.md

@@ -0,0 +1,23 @@
+
+Extend its operand with an unique autonumber attribute
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This non-relational operator guarantees uniqueness of output tuples by adding 
+an attribute called AS whose value is an auto-numbered Integer. 
+
+If the presence of duplicates was the only "non-relational" aspect of the input, 
+the result is a valid relation for which AS is a candidate key.
+
+EXAMPLE
+
+    # Autonumber suppliers with default attribute name
+    !(alf autonum suppliers)
+
+    # Autonumber suppliers with a `unique_id` attribute
+    !(alf autonum suppliers -- unique_id)
+

data/doc/operators/non_relational/clip.md

@@ -0,0 +1,31 @@
+
+Clip input tuples to a subset of attributes
+
+SYNOPSIS
+
+    #(signature)
+
+OPTIONS
+
+    #(summarized_options)
+
+DESCRIPTION
+
+This operator clips tuples on attributes whose names are specified in 
+ATTRIBUTES. This is similar to the relational PROJECT operator, expect 
+that CLIP does not remove duplicates afterwards.
+
+Clipping may therefore lead to bags of tuples instead of sets. The result
+is therefore **not** a valid relation unless a candidate key is preserved.
+
+With the allbut option, the operator keeps attributes in ATTRIBUTES, instead 
+of projecting them away. 
+
+EXAMPLE
+
+    # Clip suppliers on `name` and `city`
+    !(alf clip suppliers -- name city)
+
+    # Clip suppliers on all other attributes
+    !(alf clip suppliers --allbut -- name city)
+

data/doc/operators/non_relational/coerce.md

@@ -0,0 +1,15 @@
+
+Force attribute coercion according to a heading
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This operator coerces attributes of the input tuples according to HEADING.
+
+EXAMPLE
+
+  alf coerce parts -- weight Float color Color
+

data/doc/operators/non_relational/compact.md

@@ -0,0 +1,20 @@
+
+Remove tuple duplicates
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This non-relational operator removes duplicates from its input operand.
+In other words, it converts a bag of tuples to a set of tuples in a 
+brute-force manner.
+
+If the presence of duplicates was the only "non-relational" aspect of 
+the operand, the result is a valid relation.
+
+EXAMPLE
+
+  alf compact suppliers
+

data/doc/operators/non_relational/defaults.md

@@ -0,0 +1,32 @@
+
+Force default values on missing/nil attributes
+
+SYNOPSIS
+
+    #(signature)
+
+OPTIONS
+
+    #(summarized_options)
+
+DESCRIPTION
+
+This non-relational operator rewrites input tuples to ensure that all values 
+for attribute names specified in DEFAULTS are present and not nil. Missing or 
+nil attributes are replaced by the specified default value.
+
+A value specified in DEFAULTS may be any tuple expression. This allows computing 
+the default value as an expression on the current tuple.
+
+With the strict option, the operator projects resulting tuples on attributes for 
+which a default value has been specified. Using the strict mode guarantees that 
+the heading of all tuples is the same, and that no nil value ever remains. 
+
+Note that this operator never removes duplicates. Even in strict mode the result 
+might be an invalid relation. 
+
+EXAMPLE
+
+  alf defaults suppliers -- country "'Belgium'"
+  alf defaults --strict suppliers -- country "'Belgium'"
+

data/doc/operators/non_relational/generator.md

@@ -0,0 +1,20 @@
+
+Generate a relation with an auto-numbered attribute
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This non-relational operator generates a relation of one attribute called AS, 
+whose value is an auto-number ranging from 1 to SIZE, inclusively.
+
+EXAMPLE
+
+    # Default behavior: AS is `num` and SIZE is 10
+    !(alf generator)
+
+    # That you can override
+    !(alf generator -- 5 -- id)
+

data/doc/operators/non_relational/sort.md

@@ -0,0 +1,24 @@
+
+Sort input tuples according to an order relation
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This non-relational operator sorts input tuples according to ORDERING. 
+
+This is, of course, a non relational operator as relations are unordered 
+sets. It is provided for displaying purposes and normalization of 
+non-relational inputs.
+
+EXAMPLE
+
+  # Sort suppliers by name
+  !(alf sort suppliers -- name asc)
+
+  # Sort suppliers by city in descending order, then on name on ascending 
+  # order
+  !(alf sort suppliers -- city desc name asc)
+

data/doc/operators/relational/extend.md

@@ -0,0 +1,18 @@
+
+Relational extension (additional, computed attributes)
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This operator extends its operand with new attributes whose value is the result 
+of evaluating tuple expressions specified in EXT. The latter are specified as 
+(name, expression) pairs.
+
+EXAMPLE
+
+    # Compute a few attributes on suppliers by extension
+    !(alf extend supplies -- big "qty > 100"  price "qty * 12.2")
+

data/doc/operators/relational/group.md

@@ -0,0 +1,27 @@
+
+Relational grouping (relation-valued attributes)
+
+SYNOPSIS
+
+    #(signature)
+
+OPTIONS
+
+    #(summarized_options)
+
+DESCRIPTION
+
+This operator groups attributes in ATTRIBUTES as a new, relation-valued 
+attribute named AS.
+
+With the allbut option, it groups all attributes not specified in ATTRIBUTES
+instead.
+
+EXAMPLE
+
+    # Group pid and qty as a relation-valued attribute names supplying
+    !(alf group supplies -- pid qty -- supplying)
+
+    # Group all but pid ...
+    !(alf group supplies --allbut -- pid -- supplying)
+

data/doc/operators/relational/intersect.md

@@ -0,0 +1,13 @@
+
+Relational intersection (aka a logical and)
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This operator computes the relational intersection of its operands. Such
+intersection is simply the set of tuples that appear both in LEFT and in 
+RIGHT.
+

data/doc/operators/relational/join.md

@@ -0,0 +1,27 @@
+
+Relational join (and cartesian product)
+
+SYNOPSIS
+
+    #(signature)
+
+DESCRIPTION
+
+This operator computes the (natural) join of its operands. 
+
+Natural join means that, unlike what is commonly used in SQL, the join is 
+performed on common attribute names. You can use the `rename` operator if this 
+behavior does not fit your needs.
+
+When operands have no attribute in common, this operator naturally "degenerates" 
+to a cartesian product.
+
+EXAMPLE
+
+    # Computes natural join of suppliers and supplies (on sid, the only 
+    # attribute they have in common)
+    !(alf join suppliers supplies)
+
+    # The following example demontrates the cartesian product with a generated 
+    # relation
+    !(alf generator -- 3 -- num | alf join cities)