fernyb_davclient 0.0.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/LICENSE ADDED
@@ -0,0 +1,53 @@
1
+ DavClient (davclient and the dav utility) is copyrighted free software by Thomas Flemming
2
+ <thomas dot flemming at usit dot uio do no> and contributors. You can redistribute it
3
+ and/or modify it under either the terms of the GPL2 or the conditions below:
4
+
5
+ 1. You may make and give away verbatim copies of the source form of the
6
+ software without restriction, provided that you duplicate all of the
7
+ original copyright notices and associated disclaimers.
8
+
9
+ 2. You may modify your copy of the software in any way, provided that
10
+ you do at least ONE of the following:
11
+
12
+ a) place your modifications in the Public Domain or otherwise make them
13
+ Freely Available, such as by posting said modifications to Usenet or an
14
+ equivalent medium, or by allowing the author to include your
15
+ modifications in the software.
16
+
17
+ b) use the modified software only within your corporation or
18
+ organization.
19
+
20
+ c) rename any non-standard executables so the names do not conflict with
21
+ standard executables, which must also be provided.
22
+
23
+ d) make other distribution arrangements with the author.
24
+
25
+ 3. You may distribute the software in object code or executable
26
+ form, provided that you do at least ONE of the following:
27
+
28
+ a) distribute the executables and library files of the software,
29
+ together with instructions (in the manual page or equivalent) on where
30
+ to get the original distribution.
31
+
32
+ b) accompany the distribution with the machine-readable source of the
33
+ software.
34
+
35
+ c) give non-standard executables non-standard names, with
36
+ instructions on where to get the original software distribution.
37
+
38
+ d) make other distribution arrangements with the author.
39
+
40
+ 4. You may modify and include the part of the software into any other
41
+ software (possibly commercial). But some files in the distribution
42
+ are not written by the author, so that they are not under this terms.
43
+
44
+ 5. The scripts and library files supplied as input to or produced as
45
+ output from the software do not automatically fall under the
46
+ copyright of the software, but belong to whomever generated them,
47
+ and may be sold commercially, and may be aggregated with this
48
+ software.
49
+
50
+ 6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
51
+ IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
52
+ WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
53
+ PURPOSE.
@@ -0,0 +1,38 @@
1
+ bin/dav
2
+ examples/clean_files.rb
3
+ examples/meta_tags_in_use.rb
4
+ examples/remove_ds_store.rb
5
+ examples/scrape_site.rb
6
+ examples/simple_find.rb
7
+ fernyb_davclient.gemspec
8
+ lib/davclient/curl_commands.rb
9
+ lib/davclient/dav-ls.rb
10
+ lib/davclient/dav-propfind.rb
11
+ lib/davclient/dav-put.rb
12
+ lib/davclient/davcli.rb
13
+ lib/davclient/hpricot_extensions.rb
14
+ lib/davclient/simple.rb
15
+ lib/davclient/termutil.rb
16
+ lib/davclient/util.rb
17
+ lib/davclient.rb
18
+ LICENSE
19
+ Manifest
20
+ Rakefile
21
+ README.rdoc
22
+ tests/dav.rb
23
+ tests/tc_dav-cat.rb
24
+ tests/tc_dav-cd.rb
25
+ tests/tc_dav-cp.rb
26
+ tests/tc_dav-delete.rb
27
+ tests/tc_dav-get.rb
28
+ tests/tc_dav-ls.rb
29
+ tests/tc_dav-mkcol.rb
30
+ tests/tc_dav-mv.rb
31
+ tests/tc_dav-propfind.rb
32
+ tests/tc_dav-put.rb
33
+ tests/tc_util.rb
34
+ tests/tc_webdav_basic.rb
35
+ tests/tc_webdav_publish.rb
36
+ tests/test_helper.rb
37
+ tests/ts_davclient.rb
38
+ todo
@@ -0,0 +1,134 @@
1
+ = DavClient
2
+
3
+ * http://davclient.rubyforge.org
4
+ * http://rubyforge.org/mailman/listinfo/davclient-general
5
+
6
+ == DESCRIPTION:
7
+
8
+ A scriptable WebDAV command line client for Ruby for managing content on
9
+ webservers that support the WebDAV extensions.
10
+
11
+ This is still very much work in progress. Especially the command line interface
12
+ is changed a lot recently. The ruby library however is starting to settle.
13
+
14
+
15
+ == Requirements
16
+
17
+ The command line utility curl installed and available in your unix path.
18
+
19
+ == LIRBRARY SYNOPSIS:
20
+
21
+ require 'rubygems'
22
+ require 'davclient'
23
+
24
+ # Print url of all files in webdav folder recursively
25
+ # with basic tree walking
26
+
27
+ url = 'http://test.webdav.org/dav/'
28
+ WebDAV.find(url, :recursive => true) do |item|
29
+ puts item.href
30
+ end
31
+
32
+ == COMMAND LINE UTILITIES:
33
+
34
+ DavClient includes the command line utility 'dav'. It is inspired by git and should be familiar to unix users.
35
+ The command 'dav cd url' sets current working url, 'dav ls' list files and 'dav pwd'
36
+ prints current working url. , users can access files, folders and their properties on webdav servers.
37
+
38
+ The only authentication method supported at the moment, is by reading usernames and passwords from
39
+ a file named ~/.netrc. If username is missing, the 'dav' command will print out detailed instructions
40
+ on how to add username etc. to the '~/.netrc' file.
41
+
42
+ == COMMAND LINE SYNOPSIS:
43
+
44
+ bash $ dav cd http://test.webdav.org/dav/
45
+ http://test.webdav.org/dav/
46
+ bash $ dav ls
47
+ images/
48
+ index.html
49
+ bash $ dav pwd
50
+ http://test.webdav.org/dav/
51
+ bash $ get ./index.html
52
+ bash $ dav --help
53
+ usage: dav COMMAND [ARGS]
54
+
55
+ Available dav commands:
56
+ ls List files on webdav server
57
+ pwd Print current working url
58
+ cd Change current working url
59
+ cp Copy resource
60
+ mv Move resource
61
+ rm Remove resource
62
+ cat Print content of resource
63
+ mkdir Create remote collection (directory) (mkcol alias)
64
+ get Download resource
65
+ put Upload local file
66
+ propfind Print webdav properties for url
67
+ mkcol Make collection
68
+ options Display webservers WebDAV options
69
+ edit Edit contents of remote file with editor
70
+
71
+ == USE WITH IRB:
72
+
73
+ IRB can also be used as an interactive dav console:
74
+
75
+ bash $ irb -rubygems -rdavclient/simple
76
+ >> cd "https://webdav.org/projects/"
77
+ => "https://webdav.org/projects/"
78
+ >> pwd
79
+ => "https://webdav.org/projects/"
80
+ >> content = get "index.html"
81
+ => "index.html"
82
+ >> ls
83
+ index.html
84
+ >> cd ".."
85
+ => "https://webdav.org/"
86
+
87
+
88
+ == INSTALL:
89
+
90
+ If you can type the command 'curl' in your terminal window, everything is ok.
91
+
92
+ DavClient uses the command line utility cURL to interact with servers.
93
+ Curl comes preinstalled on Mac OS X. It can also be downloaded from
94
+ http://curl.haxx.se/ . If you are using debian or ubuntu, it
95
+ can be installed with apt-get:
96
+
97
+ sudo apt-get install curl
98
+
99
+ Then install DavClient:
100
+
101
+ [sudo] gem install davclient
102
+
103
+ or
104
+
105
+ git clone git://github.com/thomasfl/davclient.git
106
+ cd davclient
107
+ sudo rake install
108
+
109
+
110
+ == Background:
111
+
112
+ There has been posted a few examples on the web of how to make a WebDAV client.
113
+ The problem is that they all seem to support only one type of username and password
114
+ authentication. DavClient instead uses the command line tool 'curl' to do all the
115
+ authentication and networking. To avoid handling authentication all togheter, curl
116
+ are told to look up all usernames and passwords are in a file named ~/.netrc.
117
+
118
+ The command line utility 'dav' is non-interactive and inspired by git, making it more suitable for
119
+ use in scripts. If you can script in Ruby, the examples folder includes sample
120
+ scripts using the davclient Ruby library.
121
+
122
+ == Licence
123
+
124
+ DavClient (davclient and the dav utility) is copyrighted free software by Thomas Flemming
125
+ <thomas dot flemming at usit.uio.no> and contributors. You can redistribute it
126
+ and/or modify it under either the terms of the GPL2 or the conditions below:
127
+
128
+ See LICENCE file for details.
129
+
130
+ == Comments
131
+
132
+ Feel free to contact me at thomas dot flemming at usit.uio.no.
133
+
134
+
@@ -0,0 +1,95 @@
1
+ # Rakefile for DavClient -*- ruby -*-
2
+ # Usage:
3
+ #
4
+ # rake package Create gem
5
+ # rake rdoc Create doc
6
+ # rake upload-docs Upload docs to rubyforge
7
+ #
8
+ # rake install Create and install gem
9
+ # rake manifest Create manifest
10
+ # rake release Release to rubyforge
11
+
12
+ require 'rake/rdoctask'
13
+ require 'rake/gempackagetask'
14
+
15
+ GEM_VERSION = "0.0.9"
16
+
17
+ spec = Gem::Specification.new do |s|
18
+ s.name = "fernyb_davclient"
19
+ s.version = GEM_VERSION
20
+ s.author = "Thomas Flemming"
21
+ s.email = "thomasfl@usit.uio.no"
22
+ s.homepage = "http://folk.uio.no/thomasfl"
23
+ s.platform = Gem::Platform::RUBY
24
+ s.summary = "Command line WebDAV client and Ruby library."
25
+ s.description = "WebDAV command line client written in Ruby for managing " +
26
+ "content on webservers that support the WebDAV extensions."
27
+ s.requirements << "cURL command line tool available from http://curl.haxx.se/"
28
+ s.requirements << "Servername, username and password must be supplied in ~/.netrc file."
29
+ s.files = ["lib/davclient.rb","lib/davclient/hpricot_extensions.rb",
30
+ "lib/davclient/curl_commands.rb", "bin/dav", "lib/davclient/davcli.rb",
31
+ "lib/davclient/dav-put.rb", "lib/davclient/dav-ls.rb",
32
+ "lib/davclient/dav-propfind.rb","lib/davclient/util.rb","lib/davclient/termutil.rb"]
33
+ s.executables = ["dav"]
34
+ s.require_path = "lib"
35
+ s.rubyforge_project = "fernyb_davclient"
36
+ # s.test_files = FileList["{test}/**/*test.rb"].to_a
37
+ s.has_rdoc = true
38
+ s.extra_rdoc_files = ["README.rdoc"]
39
+ s.add_dependency("hpricot", ">= 0.6")
40
+ s.add_dependency("termios", ">= 0.9") # For password prompt
41
+ s.add_dependency("ZenTest", ">= 3.5") # For tests
42
+ end
43
+
44
+ Rake::GemPackageTask.new(spec) do |pkg|
45
+ pkg.need_zip = false
46
+ pkg.need_tar = false
47
+ end
48
+
49
+
50
+ html_dir = 'doc/html'
51
+ library = 'Ruby DavClient'
52
+ Rake::RDocTask.new('rdoc') do |t|
53
+ t.rdoc_files.include('README.rdoc',
54
+ 'lib/davclient.rb',
55
+ 'lib/davclient/hpricot_extensions.rb',
56
+ 'lib/davclient/simple.rb')
57
+ t.main = 'README.rdoc'
58
+ t.title = "#{library} API documentation"
59
+ t.rdoc_dir = html_dir
60
+ end
61
+
62
+ rubyforge_user = 'thomasfl'
63
+ rubyforge_project = 'fernyb_davclient'
64
+ rubyforge_path = "/var/www/gforge-projects/#{rubyforge_project}/"
65
+ desc 'Upload documentation to RubyForge.'
66
+ task 'upload-docs' => ['rdoc'] do
67
+ sh "scp -r #{html_dir}/* " +
68
+ "#{rubyforge_user}@rubyforge.org:#{rubyforge_path}"
69
+ end
70
+
71
+
72
+ website_dir = 'site'
73
+ desc 'Update project website to RubyForge.'
74
+ task 'upload-site' do
75
+ sh "scp -r #{website_dir}/* " +
76
+ "#{rubyforge_user}@rubyforge.org:/var/www/gforge-projects/project/"
77
+ end
78
+
79
+ desc 'Update API docs and project website to RubyForge.'
80
+ task 'publish' => ['upload-docs', 'upload-site']
81
+
82
+
83
+
84
+ require 'echoe'
85
+
86
+ Echoe.new('fernyb_davclient', GEM_VERSION) do |p|
87
+ p.description = "Command line WebDAV client and Ruby library."
88
+ p.url = "http://davclient.rubyforge.org/davclient"
89
+ p.author = "Thomas Flemming"
90
+ p.email = "thomasfl@usit.uio.no"
91
+ p.ignore_pattern = ["svn_user.yml", "svn_project.rake"]
92
+ p.project = "fernyb_davclient"
93
+ end
94
+
95
+ Dir["#{File.dirname(__FILE__)}/tasks/*.rake"].sort.each { |ext| load ext }
data/bin/dav ADDED
@@ -0,0 +1,6 @@
1
+ #!/usr/bin/env ruby
2
+ require 'rubygems'
3
+ require 'davclient'
4
+ require 'davclient/davcli'
5
+
6
+ DavCLI.dav(ARGV)
@@ -0,0 +1,131 @@
1
+ # -*- coding: utf-8 -*-
2
+ $LOAD_PATH << File.expand_path( File.dirname(__FILE__) + '/../lib' )
3
+ require 'rubygems'
4
+ require 'davclient/simple'
5
+
6
+ # Logg to file. Close file to be sure nothing bad happens.
7
+ def log_update(url,msg)
8
+ path = File.expand_path( File.dirname(__FILE__) ) + "/cleaned_files.log"
9
+ file = File.open(path, "a")
10
+ file.puts url
11
+ file.puts msg
12
+ file.puts "----"
13
+ file.close
14
+ end
15
+
16
+ # Stealth mode editing
17
+ # Removes traces after editing resources by resetting properties to
18
+ # their original value.
19
+ def stealth_mode(item, &block)
20
+ original_item = item.clone
21
+ # puts "DEBUG: Leser props." + item.getlastmodified
22
+ yield item
23
+ # puts "DEBUG: Skriver props."
24
+
25
+ # Det har ingen effekt å sette disse:
26
+ # <v:propertiesModifiedBy xmlns:v="vrtx">#{original_item.propertiesModifiedBy}</v:propertiesModifiedBy>
27
+ # <d:getlastmodified>#{original_item.getlastmodified}</d:getlastmodified>
28
+ # <v:propertiesLastModified xmlns:v="vrtx">#{original_item.propertiesLastModified}</v:propertiesLastModified>
29
+
30
+ props = ""
31
+ if(original_item.contentLastModified != nil and original_item.contentLastModified != "")
32
+ props += "<v:contentLastModified xmlns:v=\"vrtx\">#{original_item.contentLastModified}</v:contentLastModified>"
33
+ end
34
+ if(original_item.modifiedBy != nil and original_item.modifiedBy != "")
35
+ props += "<v:modifiedBy xmlns:v=\"vrtx\">#{original_item.modifiedBy}</v:modifiedBy>"
36
+ end
37
+ if(original_item.contentModifiedBy != nil and original_item.contentModifiedBy != "")
38
+ props += "<v:contentModifiedBy xmlns:v=\"vrtx\">#{original_item.contentModifiedBy}</v:contentModifiedBy>"
39
+ end
40
+ if(props != "")
41
+ begin
42
+ item.proppatch(props)
43
+ rescue RuntimeError
44
+ puts "Kunne ikke patche " + item.href
45
+ log_update(item.href, "Kunne ikke patche properties")
46
+ end
47
+ end
48
+ end
49
+
50
+
51
+ def clean_files_from_file(baseurl, filename)
52
+ checked_files = 1
53
+ lines = ""
54
+ file = File.open(filename, 'r')
55
+ file.each do |line|
56
+ url = baseurl + line.gsub(/\(/,"\\(").gsub(/\)/,"\\)").gsub(/ /,"\\ ")
57
+ puts checked_files.to_s + " Sjekker: " + url
58
+ begin
59
+ content = WebDAV.get(url)
60
+ rescue RuntimeError => details
61
+ puts "Error: " + details
62
+ log_update(url, "Error: " + details )
63
+ end
64
+ if(content =~ /404 - Not Found/)
65
+ puts "Warning: 404 not found"
66
+ log_update(url, "Warning: 404 not found")
67
+ end
68
+
69
+ checked_files += 1
70
+ if(content != nil)
71
+ meta = content.grep(/<meta\s*name[^>]*>/im).join("")
72
+ if(meta != "" )
73
+ puts "Cleaning: " + url
74
+ puts meta
75
+ clean_file(url, content)
76
+ log_update(url, meta)
77
+ end
78
+ end
79
+ # exit if(checked_files > 10)
80
+ end
81
+ end
82
+
83
+ def clean_file(url, content)
84
+ begin
85
+ props = WebDAV.propfind(url, :xml => true)
86
+ doc = Hpricot(props)
87
+ item = doc.search("//d:response").first
88
+ stealth_mode(item) do |item|
89
+ item.content = item.content.gsub(/<meta.*\s+name\s*=[^>]*>[\s\n\r]*/im, "")
90
+ end
91
+ rescue RuntimeError => details
92
+ puts "Error: " + details
93
+ log_update(url, "Error: " + details )
94
+ end
95
+ end
96
+
97
+ def find_and_clean_files(url)
98
+ checked_files = 0
99
+ find(url, :recursive => true) do |item|
100
+ checked_files += 1
101
+ puts "count:" + checked_files.to_s + " " + item.href
102
+ if(item.resourceType =~ /html/ and item.href =~ /\..?html$/)
103
+ content = item.content
104
+
105
+ # uspesifiserte dokumenttyper har resourceType "html" eller "xhtml10trans"
106
+ if(item.content != nil)
107
+ meta = item.content.grep(/<meta\s*name[^>]*>/im).join("")
108
+
109
+ if(meta != "" )
110
+ log_update(item.href, meta)
111
+ puts item.href + ":"
112
+ puts meta
113
+ puts
114
+ stealth_mode(item) do |item|
115
+ # item.content = item.content.gsub(/<meta.*\s+name\s*=[^>]*>[\s\n\r]*/im, "")
116
+ item.content = item.content.gsub(/<meta\s+name\s*=[^>]*>/im, "")
117
+ item.content = item.content.gsub(/<meta\s+content\s*=[^>]*>/im, "")
118
+ end
119
+
120
+ end
121
+
122
+ end
123
+ end
124
+ end
125
+ end
126
+
127
+ if($0 == __FILE__)
128
+ baseurl = ARGV.shift
129
+ filename = ARGV.shift or abort "usage: #{$0} baseurl filename"
130
+ clean_files_from_file( baseurl, filename)
131
+ end