ftpd 0.2.2 → 0.3.1

data/Changelog.md

@@ -1,3 +1,30 @@
+### 0.3.1
+
+API changes
+
+The file system interface for directory listing was completely
+rewritten.  It no longer shells out to ls, which removes potential
+command injection security holes, and improves prospects for
+portability.
+
+* Removed {Ftpd::DiskFileSystem::Ls}
+* Removed {Ftpd::DiskFileSystem::NameList}.  NLIST now uses the
+  functions in {Ftpd::DiskFileSystem::List}.
+* Removed {Ftpd::DiskFileSystem::List#list}.  The formatting of
+  directory output is now done by ftpd, not by the file system driver.
+* Added {Ftpd::DiskFileSystem::List#file_info}, used by LIST.
+* Added {Ftpd::DiskFileSystem::List#dir}, used by LIST and NLST.
+
+Bug fixes
+
+* LIST and NLST support globs again.
+* STOU (store unique) works in Ruby 1.8.7
+
+Enhancements
+
+* The output of the "LIST" command can be customized (see
+  {Ftpd::FtpServer#list_formatter})
+
 ### 0.2.2
 
 Bug fixes
@@ -8,9 +35,10 @@ Bug fixes
   PASS
 * Open PASV mode data connection on same local IP as control connection.
   This is required by RFC 1123.
-* Disabled globbing in LIST (for now) due to code injection
-  vulnerability.  This patch also disables globbing in NLST, but NLST
-  probably shouldn't do globbing.
+* Disabled globbing in LIST (for now) due to a command (shell)
+  injection vulnerability.  This patch also disables globbing in NLST,
+  but NLST probably shouldn't do globbing.  Thanks to Larry Cashdollar
+  for the report.
 
 Enhancements
 

data/Gemfile

@@ -9,5 +9,6 @@ group :development do
   gem 'rake'
   gem 'redcarpet'
   gem 'rspec'
+  gem 'timecop'
   gem 'yard'
 end

data/Gemfile.lock

@@ -36,6 +36,7 @@ GEM
     rspec-expectations (2.13.0)
       diff-lcs (>= 1.1.3, < 2.0)
     rspec-mocks (2.13.0)
+    timecop (0.5.9.2)
     yard (0.8.5.2)
 
 PLATFORMS
@@ -49,4 +50,5 @@ DEPENDENCIES
   rake
   redcarpet
   rspec
+  timecop
   yard

data/README.md

@@ -92,10 +92,6 @@ Ftpd is not yet RFC compliant.  It does most of RFC969, and enough TLS
 to get by.  {file:doc/rfc.md Here} is a list of RFCs, indicating how
 much of each Ftpd complies with.
 
-The DiskFileSystem class only works in Linux.  This is because it
-shells out to the "ls" command.  This affects the example, which uses
-the DiskFileSystem.
-
 To bind the server to an external interface, the interface must be set
 to the public IP of that interface (e.g. "1.2.3.4"), not to "0.0.0.0".
 That's because the interface IP is used both for binding server ports,
@@ -103,10 +99,6 @@ _and_ for advertising to the client which IP to connect to.  Binding
 to 0.0.0.0 will work fine, but when the client tries to connect to
 0.0.0.0, it won't get to the server.
 
-LIST doesn't accept globs.  It has other problems (it accepts
-arbitrary ls arguments!) and needs to be rewritten to not shell out to
-"ls".
-
 ## RUBY COMPATABILITY
 
 The tests pass with these Rubies:

data/VERSION

@@ -1 +1 @@
-0.2.2
+0.3.1

data/doc/references.md

@@ -1,5 +1,7 @@
 # REFERENCES
 
+## RFCs
+
 _This list of references comes from the README of the em-ftpd gem,
 which is licensed under the same MIT license as this gem, and is
 Copyright (c) 2008 James Healy_
@@ -30,3 +32,8 @@ including the ye old RFC114 from 1971, "A File Transfer Protocol"
 There is a {http://secureftp-test.com public test server} which is
 very handy for checking out clients, and seeing how at least one
 server behaves.
+
+## LIST output format
+
+* {http://www.gnu.org/software/coreutils/manual/html_node/What-information-is-listed.html#What-information-is-listed GNU docs for ls}
+* {http://cr.yp.to/ftp/list/eplf.html Easily Parsed LIST format (EPLF)}

data/examples/example.rb

@@ -10,6 +10,7 @@ require 'optparse'
 module Example
   class Arguments
 
+    attr_reader :eplf
     attr_reader :interface
     attr_reader :port
     attr_reader :tls
@@ -39,6 +40,9 @@ module Example
               'Select TLS support (off, explicit, implicit)') do |t|
           @tls = t
         end
+        op.on('--eplf', 'LIST uses EPLF format') do |t|
+          @eplf = t
+        end
       end
     end
 
@@ -94,6 +98,9 @@ module Example
       @server.port = @args.port
       @server.tls = @args.tls
       @server.certfile_path = insecure_certfile_path
+      if @args.eplf
+        @server.list_formatter = Ftpd::ListFormat::Eplf
+      end
       @server.start
       display_connection_info
       create_connection_script

data/features/example/eplf.feature

@@ -0,0 +1,14 @@
+Feature: Example
+
+  As a programmer
+  I want to enable EPLF list format
+  So that I can test this library with an EPLF client
+
+  Background:
+    Given the example has argument "--eplf"
+    And the example server is started
+
+  Scenario: List directory
+    Given a successful login
+    When the client successfully lists the directory
+    Then the list should be in EPLF format

data/features/example/step_definitions/example_server.rb

@@ -1,3 +1,11 @@
+def example_args
+  @example_args ||= []
+end
+
+Given /^the example has argument "(.*?)"$/ do |arg|
+  example_args << arg
+end
+
 Given /^the example server is started$/ do
-  @server = ExampleServer.new
+  @server = ExampleServer.new(example_args)
 end

data/features/ftp_server/list.feature

@@ -41,8 +41,15 @@ Feature: List
     Then the file list should be in long form
     And the file list should contain "foo"
 
+  Scenario: After CWD
+    Given a successful login
+    And the server has file "subdir/foo"
+    And the client successfully cd's to "subdir"
+    When the client successfully lists the directory
+    Then the file list should be in long form
+    And the file list should contain "foo"
+
   Scenario: Glob
-    Given PENDING "Disabled (for now) due to code injection vulnerability"
     Given a successful login
     And the server has file "foo"
     And the server has file "bar"

data/features/ftp_server/name_list.feature

@@ -62,7 +62,7 @@ Feature: Name List
     Then the server returns a not logged in error
 
   Scenario: List not enabled
-    Given the test server is started without name_list
+    Given the test server is started without list
     And a successful login
     When the client name-lists the directory
     Then the server returns an unimplemented command error

data/features/step_definitions/list.rb

@@ -18,6 +18,12 @@ class FileList
     !long_form?
   end
 
+  def eplf_format?
+    @lines.all? do |line|
+      line =~ /^\+.*\t.*$/
+    end
+  end
+
   def empty?
     @lines.empty?
   end
@@ -61,3 +67,7 @@ end
 Then /^the file list should be empty$/ do
   @list.should be_empty
 end
+
+Then /^the list should be in EPLF format$/ do
+  @list.should be_eplf_format
+end

data/features/support/example_server.rb

@@ -9,10 +9,11 @@ class ExampleServer
   include FileUtils
   include TestServerFiles
 
-  def initialize
+  def initialize(args = nil)
     command = [
       File.expand_path('../../examples/example.rb',
-                       File.dirname(__FILE__))
+                       File.dirname(__FILE__)),
+      args,
     ].join(' ')
     @io = IO.popen(command, 'r+')
     @output = read_output

data/features/support/test_server.rb

@@ -16,7 +16,6 @@ class TestServer
     attr_accessor :delete
     attr_accessor :list
     attr_accessor :mkdir
-    attr_accessor :name_list
     attr_accessor :read
     attr_accessor :rename
     attr_accessor :rmdir
@@ -27,7 +26,6 @@ class TestServer
       @delete = true
       @list = true
       @mkdir = true
-      @name_list = true
       @read = true
       @rename = true
       @rmdir = true
@@ -43,7 +41,6 @@ class TestServer
                                :delete => @delete,
                                :list => @list,
                                :mkdir => @mkdir,
-                               :name_list => @name_list,
                                :read => @read,
                                :rename => @rename,
                                :rmdir => @rmdir,
@@ -147,10 +144,6 @@ class TestServer
           include Ftpd::DiskFileSystem::List
         end
 
-        if opts[:name_list]
-          include Ftpd::DiskFileSystem::NameList
-        end
-
         if opts[:mkdir]
           include Ftpd::DiskFileSystem::Mkdir
         end
@@ -218,7 +211,6 @@ class TestServer
   def_delegator :@driver, :'delete='
   def_delegator :@driver, :'list='
   def_delegator :@driver, :'mkdir='
-  def_delegator :@driver, :'name_list='
   def_delegator :@driver, :'rmdir='
   def_delegator :@driver, :'read='
   def_delegator :@driver, :'rename='

data/ftpd.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = "ftpd"
-  s.version = "0.2.2"
+  s.version = "0.3.1"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Wayne Conrad"]
-  s.date = "2013-03-02"
+  s.date = "2013-03-04"
   s.description = "ftpd is a pure Ruby FTP server library.  It supports implicit and explicit TLS, passive and active mode, and most of the commands specified in RFC 969.  It an be used as part of a test fixture or embedded in a program."
   s.email = "wconrad@yagni.com"
   s.extra_rdoc_files = [
@@ -28,6 +28,7 @@ Gem::Specification.new do |s|
     "doc/rfc-compliance.md",
     "examples/example.rb",
     "examples/hello_world.rb",
+    "features/example/eplf.feature",
     "features/example/example.feature",
     "features/example/step_definitions/example_server.rb",
     "features/ftp_server/allo.feature",
@@ -110,9 +111,12 @@ Gem::Specification.new do |s|
     "lib/ftpd/error.rb",
     "lib/ftpd/exception_translator.rb",
     "lib/ftpd/exceptions.rb",
+    "lib/ftpd/file_info.rb",
     "lib/ftpd/file_system_error_translator.rb",
     "lib/ftpd/ftp_server.rb",
     "lib/ftpd/insecure_certificate.rb",
+    "lib/ftpd/list_format/eplf.rb",
+    "lib/ftpd/list_format/ls.rb",
     "lib/ftpd/server.rb",
     "lib/ftpd/session.rb",
     "lib/ftpd/temp_dir.rb",
@@ -127,7 +131,10 @@ Gem::Specification.new do |s|
     "spec/command_sequence_checker_spec.rb",
     "spec/disk_file_system_spec.rb",
     "spec/exception_translator_spec.rb",
+    "spec/file_info_spec.rb",
     "spec/file_system_error_translator_spec.rb",
+    "spec/list_format/eplf_spec.rb",
+    "spec/list_format/ls_spec.rb",
     "spec/spec_helper.rb",
     "spec/translate_exceptions_spec.rb"
   ]
@@ -148,6 +155,7 @@ Gem::Specification.new do |s|
       s.add_development_dependency(%q<rake>, [">= 0"])
       s.add_development_dependency(%q<redcarpet>, [">= 0"])
       s.add_development_dependency(%q<rspec>, [">= 0"])
+      s.add_development_dependency(%q<timecop>, [">= 0"])
       s.add_development_dependency(%q<yard>, [">= 0"])
     else
       s.add_dependency(%q<memoizer>, ["~> 1.0.1"])
@@ -157,6 +165,7 @@ Gem::Specification.new do |s|
       s.add_dependency(%q<rake>, [">= 0"])
       s.add_dependency(%q<redcarpet>, [">= 0"])
       s.add_dependency(%q<rspec>, [">= 0"])
+      s.add_dependency(%q<timecop>, [">= 0"])
       s.add_dependency(%q<yard>, [">= 0"])
     end
   else
@@ -167,6 +176,7 @@ Gem::Specification.new do |s|
     s.add_dependency(%q<rake>, [">= 0"])
     s.add_dependency(%q<redcarpet>, [">= 0"])
     s.add_dependency(%q<rspec>, [">= 0"])
+    s.add_dependency(%q<timecop>, [">= 0"])
     s.add_dependency(%q<yard>, [">= 0"])
   end
 end

data/lib/ftpd.rb

@@ -7,10 +7,15 @@ require 'socket'
 require 'tmpdir'
 
 module Ftpd
+  module ListFormat
+    autoload :Eplf,                    'ftpd/list_format/eplf'
+    autoload :Ls,                      'ftpd/list_format/ls'
+  end
   autoload :CommandSequenceChecker,    'ftpd/command_sequence_checker'
   autoload :DiskFileSystem,            'ftpd/disk_file_system'
   autoload :Error,                     'ftpd/error'
   autoload :ExceptionTranslator,       'ftpd/exception_translator'
+  autoload :FileInfo,                  'ftpd/file_info'
   autoload :FileSystemErrorTranslator, 'ftpd/file_system_error_translator'
   autoload :FileSystemMethodMissing,   'ftpd/file_system_method_missing'
   autoload :FtpServer,                 'ftpd/ftp_server'

data/lib/ftpd/command_sequence_checker.rb

@@ -1,7 +1,8 @@
 # Some commands are supposed to occur in sequence.  For example, USER
 # must be immediately followed by PASS.  This class keeps track of
-# when a specific command is expected, and raises a "bad sequence"
-# error when that command is not next.
+# when a specific command either must arrive or must not arrive, and
+# raises a "bad sequence" error when commands arrive in the wrong
+# sequence.
 
 module Ftpd
   class CommandSequenceChecker

data/lib/ftpd/disk_file_system.rb

@@ -200,38 +200,6 @@ module Ftpd
 
   end
 
-  class DiskFileSystem
-
-    # Ls interface used by List and NameList
-
-    module Ls
-
-      include Shellwords
-
-      def ls(ftp_path, option)
-        path = expand_ftp_path(ftp_path)
-        dirname = File.dirname(path)
-        filename = File.basename(path)
-        command = [
-          'ls',
-          option,
-          filename,
-        ].compact
-        if File.exists?(dirname)
-          list = Dir.chdir(dirname) do
-            `#{shelljoin(command)} 2>&1`
-          end
-        else
-          list = ''
-        end
-        list = "" if $? != 0
-        list = list.gsub(/^total \d+\n/, '')
-      end
-
-    end
-
-  end
-
   class DiskFileSystem
 
     # DiskFileSystem mixin providing directory listing
@@ -240,65 +208,86 @@ module Ftpd
 
       include TranslateExceptions
 
-      # Get a file list, long form.  This returns a long-form
-      # directory listing.  The FTP standard does not specify the
-      # format of the listing, but many systems emit a *nix style
-      # directory listing:
+      # Get information about a single file or directory.
       #
-      #     -rw-r--r-- 1 wayne wayne 4 Feb 18 18:36 a
-      #     -rw-r--r-- 1 wayne wayne 8 Feb 18 18:36 b
+      # Should follow symlinks (per
+      # {http://cr.yp.to/ftp/list/eplf.html}, "lstat() is not a good
+      # idea for FTP directory listings").
+      #
+      # @return [FileInfo]
+      #
+      # Called for:
+      # * LIST
+      #
+      # If missing, then these commands are not supported.
+
+      def file_info(path)
+        stat = File.stat(expand_ftp_path(path))
+        FileInfo.new(:ftype => stat.ftype,
+                     :group => gid_name(stat.gid),
+                     :identifier => identifier(stat),
+                     :mode => stat.mode,
+                     :mtime => stat.mtime,
+                     :nlink => stat.nlink,
+                     :owner => uid_name(stat.uid),
+                     :path => path,
+                     :size => stat.size)
+      end
+      translate_exceptions :file_info
+
+      # Expand a path that may contain globs into a list of paths of
+      # matching files and directories.
       #
-      # some emit a Windows style listing.  Some emit EPLF (Easily
-      # Parsed List Format):
+      # * If the path matches no files, returns an empty list.
       #
-      #     +i8388621.48594,m825718503,r,s280, djb.html
-      #     +i8388621.50690,m824255907,/, 514
-      #     +i8388621.48598,m824253270,r,s612, 514.html
+      # * If the path has no glob and matches a directory, then the
+      #   list contains only that directory.
       #
-      # EPLF is a draft internet standard for the output of LIST:
+      # * If the patch has a glob, it may return multiple entries.
       #
-      #     http://cr.yp.to/ftp/list/eplf.html
+      # The paths returned are fully qualified, relative to the root
+      # of the virtual file system.
+      # 
+      # For example, suppose these files exist on the physical file
+      # system:
       #
-      # Some FTP clients know how to parse EPLF; those clients will
-      # display the EPLF in a more user-friendly format.  Clients that
-      # don't recognize EPLF will display it raw.  The advantages of
-      # EPLF are that it's easier for clients to parse, and the client
-      # can display the LIST output in any format it likes.
+      #   /var/lib/ftp/foo/foo
+      #   /var/lib/ftp/foo/subdir/bar
+      #   /var/lib/ftp/foo/subdir/baz
       #
-      # This class emits a *nix style listing.  It does so by shelling
-      # to the "ls" command, so it won't run on Windows at all.
+      # and that the directory /var/lib/ftp is the root of the virtual
+      # file system.  Then:
+      #
+      #   dir('foo')         # => ['/foo']
+      #   dir('subdir')      # => ['/subdir']
+      #   dir('subdir/*')    # => ['/subdir/bar', '/subdir/baz']
+      #   dir('*')           # => ['/foo', '/subdir']
       #
       # Called for:
       # * LIST
+      # * NLST
       #
       # If missing, then these commands are not supported.
 
-      def list(ftp_path)
-        ls(ftp_path, '-l')
+      def dir(path)
+        Dir[expand_ftp_path(path)].map do |path|
+          path.sub(/^#{@data_dir}/, '')
+        end
       end
+      translate_exceptions :dir
 
-    end
-  end
-
-  class DiskFileSystem
-
-    # DiskFileSystem mixin providing directory name listing
-
-    module NameList
+      private
 
-      include Ls
+      def uid_name(uid)
+        Etc.getpwuid(uid).name
+      end
 
-      # Get a file list, short form.
-      #
-      # This returns one filename per line, and nothing else
-      #
-      # Called for:
-      # * NLST
-      #
-      # If missing, then these commands are not supported.
+      def gid_name(gid)
+        Etc.getgrgid(gid).name
+      end
 
-      def name_list(ftp_path)
-        ls(ftp_path, '-1')
+      def identifier(stat)
+        [stat.dev, stat.ino].join('.')
       end
 
     end
@@ -367,7 +356,6 @@ module Ftpd
     include DiskFileSystem::Delete
     include DiskFileSystem::List
     include DiskFileSystem::Mkdir
-    include DiskFileSystem::NameList
     include DiskFileSystem::Read
     include DiskFileSystem::Rename
     include DiskFileSystem::Rmdir