iudex-http-test 1.1.0-java
Sign up to get free protection for your applications and to get access to all the features.
- data/.gemtest +0 -0
- data/History.rdoc +2 -0
- data/Manifest.txt +16 -0
- data/README.rdoc +25 -0
- data/Rakefile +36 -0
- data/bin/iudex-http-test-fg +50 -0
- data/lib/iudex-http-test.rb +19 -0
- data/lib/iudex-http-test/base.rb +23 -0
- data/lib/iudex-http-test/broken_server.rb +50 -0
- data/lib/iudex-http-test/helper.rb +78 -0
- data/lib/iudex-http-test/server.rb +40 -0
- data/lib/iudex-http-test/test_app.rb +194 -0
- data/public/atom.xml +1230 -0
- data/test/setup.rb +28 -0
- data/test/test_broken_server.rb +62 -0
- data/test/test_server.rb +53 -0
- data/test/test_test_app.rb +104 -0
- metadata +144 -0
data/public/atom.xml
ADDED
|
@@ -0,0 +1,1230 @@
|
|
|
1
|
+
<?xml version="1.0" encoding="utf-8"?>
|
|
2
|
+
<feed xmlns="http://www.w3.org/2005/Atom">
|
|
3
|
+
|
|
4
|
+
<title>void * blog</title>
|
|
5
|
+
<link href="http://gravitext.com/atom.xml" rel="self"/>
|
|
6
|
+
<link href="http://gravitext.com/"/>
|
|
7
|
+
<updated>2011-02-26T13:59:09-08:00</updated>
|
|
8
|
+
<id>http://gravitext.com/</id>
|
|
9
|
+
<author>
|
|
10
|
+
<name>David Kellum</name>
|
|
11
|
+
<email>dek-blog@gravitext.com</email>
|
|
12
|
+
</author>
|
|
13
|
+
|
|
14
|
+
<entry>
|
|
15
|
+
<title>Iyyov</title>
|
|
16
|
+
<link href="http://gravitext.com/2010/03/13/Iyyov.html"/>
|
|
17
|
+
<updated>2010-03-13T00:00:00-08:00</updated>
|
|
18
|
+
<id>id:/2010/03/13/Iyyov</id>
|
|
19
|
+
<content type="html"><p>Toward the goal of fully self-contained java/ruby service gems and ruby <a href='/2008/11/09/scripted-server-setup.html'>scripted setup</a>, some issues have lingered. With the recent releases of <a href='http://hashdot.sourceforge.net/'>Hashdot</a> 1.4 and the new <a href='http://github.com/dekellum/iyyov#readme'>Iyyov</a> monitor, a complete solution is now available. Here is what it looks like:</p>
|
|
20
|
+
|
|
21
|
+
<ul>
|
|
22
|
+
<li>
|
|
23
|
+
<p>Package and deploy your service as a versioned rubygem.</p>
|
|
24
|
+
</li>
|
|
25
|
+
|
|
26
|
+
<li>
|
|
27
|
+
<p>Include a version-specific service init script written in ruby and annotated will all necessary JVM/jruby settings (i.e -server -Xmx2g, etc.)</p>
|
|
28
|
+
</li>
|
|
29
|
+
|
|
30
|
+
<li>
|
|
31
|
+
<p><a href='http://hashdot.sourceforge.net/'>Hashdot</a> will reliably bootstrap your service as a fully UNIX compliant fork&#8217;d, setsid, and IO redirected daemon with an instance exclusive PID file and recognizable process name.</p>
|
|
32
|
+
</li>
|
|
33
|
+
|
|
34
|
+
<li>
|
|
35
|
+
<p><a href='http://github.com/dekellum/iyyov#readme'>Iyyov</a> will start your service on system boot, keep it running, and automatically restart on config changes or gem upgrades.</p>
|
|
36
|
+
</li>
|
|
37
|
+
|
|
38
|
+
<li>
|
|
39
|
+
<p>No external bash scripts, user environment settings, or other cruft required.</p>
|
|
40
|
+
</li>
|
|
41
|
+
</ul>
|
|
42
|
+
|
|
43
|
+
<h2 id='packaging_your_service_gem'>Packaging your Service Gem</h2>
|
|
44
|
+
|
|
45
|
+
<p>MRI rubygem authors would expect to place a daemon startup script in the gem&#8217;s bin directory. On install, rubygems will create a wrapper script for the original that arranges for gem initializing, makes the script available to the user PATH, and provides a convenient version selection mechanism. This is a great mechanism for short lived command-line interaction with gems, but has one very troubling shortcoming for a JRuby service writer:</p>
|
|
46
|
+
|
|
47
|
+
<blockquote>
|
|
48
|
+
<p>There is no obvious mechanism to reliably control JVM or jruby settings on a daemon or version specific basis and package these settings within a gem.</p>
|
|
49
|
+
</blockquote>
|
|
50
|
+
|
|
51
|
+
<p>Jruby defaults to the -client JVM, 500Mb of heap, and no specific GC tunings. This is reasonable default behavior, but most any production-grade service will want and need a mechanism to override these defaults.</p>
|
|
52
|
+
|
|
53
|
+
<p>To achieve this we need to package a non-wrapped executable script. Here is a sample from <a href='http://github.com/dekellum/hashdot-test-daemon'>hashdot-test-daemon</a>:</p>
|
|
54
|
+
<div class='highlight'><pre><code class='ruby'><span class='c1'>#!/usr/bin/env jruby</span>
|
|
55
|
+
<span class='c1'>#-*- ruby -*-</span>
|
|
56
|
+
<span class='c1'>#. hashdot.profile += daemon</span>
|
|
57
|
+
<span class='c1'>#. hashdot.pid_file = ./hashdot-test-daemon.pid</span>
|
|
58
|
+
<span class='c1'>#. hashdot.io_redirect.file = ./hashdot-test-daemon.log</span>
|
|
59
|
+
<span class='c1'>#. hashdot.vm.options += -Xmx64m</span>
|
|
60
|
+
<span class='c1'>#. hashdot.vm.options += -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled</span>
|
|
61
|
+
|
|
62
|
+
<span class='nb'>require</span> <span class='s1'>&#39;rubygems&#39;</span>
|
|
63
|
+
<span class='n'>gem</span><span class='p'>(</span> <span class='s2'>&quot;hashdot-test-daemon&quot;</span><span class='p'>,</span> <span class='s2'>&quot;= 1.1.0&quot;</span> <span class='p'>)</span>
|
|
64
|
+
|
|
65
|
+
<span class='nb'>require</span> <span class='s1'>&#39;rjack-logback&#39;</span>
|
|
66
|
+
<span class='no'>RJack</span><span class='o'>::</span><span class='no'>Logback</span><span class='o'>.</span><span class='n'>config_console</span><span class='p'>(</span> <span class='ss'>:full</span> <span class='o'>=&gt;</span> <span class='kp'>true</span><span class='p'>,</span> <span class='ss'>:thread</span> <span class='o'>=&gt;</span> <span class='kp'>true</span> <span class='p'>)</span>
|
|
67
|
+
|
|
68
|
+
<span class='nb'>require</span> <span class='s1'>&#39;hashdot-test-daemon&#39;</span>
|
|
69
|
+
<span class='no'>Hashdot</span><span class='o'>::</span><span class='no'>Daemon</span><span class='o'>::</span><span class='no'>Runner</span><span class='o'>.</span><span class='n'>new</span><span class='o'>.</span><span class='n'>run</span>
|
|
70
|
+
</code></pre>
|
|
71
|
+
</div>
|
|
72
|
+
<p>Note that if a system level <a href='http://hashdot.sourceforge.net/'>Hashdot</a> dependency gives you concern, you could achieve an approximately similar and perhaps sufficient result with a more elaborate shell script and/or other daemonizing options less complete for JRuby.</p>
|
|
73
|
+
|
|
74
|
+
<p>In any case, we want to be able to launch the correct service version by referencing the full path to the script in the local gem installation. For example if the gems are installed to /opt/jruby/gems, then the following should launch the 1.1.0 version of service in the background with any 1.1.0 specific settings:</p>
|
|
75
|
+
<div class='highlight'><pre><code class='bash'>/opt/jruby/gems/gems/hashdot-test-daemon-1.1.0-java/init/hashdot-test-daemon
|
|
76
|
+
</code></pre>
|
|
77
|
+
</div>
|
|
78
|
+
<h2 id='service_runtime_layout'>Service runtime layout</h2>
|
|
79
|
+
|
|
80
|
+
<p>Given java&#8217;s native threads its unlikely that it will be necessary to run a large number of separate daemon instances. However there is still plenty of reasons to want to run different services on the same host or different configurations or versions of the same service. Toward this end I use a separate directory for each service, and relative path references to instance specific configuration, log, and PID files, i.e:</p>
|
|
81
|
+
<div class='highlight'><pre><code class='bash'>/opt/var/hashdot-test-deamon-1
|
|
82
|
+
├── hashdot-test-daemon.log
|
|
83
|
+
├── hashdot-test-daemon.log.1.gz
|
|
84
|
+
├── hashdot-test-daemon.pid
|
|
85
|
+
└── config.rb
|
|
86
|
+
/opt/var/service-test-daemon-2
|
|
87
|
+
└── ...
|
|
88
|
+
</code></pre>
|
|
89
|
+
</div>
|
|
90
|
+
<p>A fully expanded shell based launch of the service might look like:</p>
|
|
91
|
+
<div class='highlight'><pre><code class='bash'><span class='nb'>cd</span> /opt/var/hashdot-test-deamon-1 <span class='o'>&amp;&amp;</span> <span class='se'>\</span>
|
|
92
|
+
<span class='nb'>exec</span> /opt/jruby/gems/gems/hashdot-test-daemon-1.1.0-java/init/hashdot-test-daemon -c config.rb
|
|
93
|
+
</code></pre>
|
|
94
|
+
</div>
|
|
95
|
+
<p>And in fact, one approach to boot-time startup or restart would be to encode something like this in an init.d script, inittab, upstart, or crontab. The last three can achieve automatic restart, provided that the service supports reliable instance-specific mutual exclusion (via PID file, etc.) Hashdot supports this reliably.</p>
|
|
96
|
+
|
|
97
|
+
<h2 id='iyyov'>Iyyov</h2>
|
|
98
|
+
|
|
99
|
+
<p>Once I arrived at the above solution for system layout and packaging service startup details in the gem, I went looking for a process monitor solution that would handle restarts and I could customize to keep the setup simple. I first looked at <a href='http://mmonit.com/monit/'>Monit</a>, and while I was able to get it to work with these daemons in basic form, configuration was &#8220;involved&#8221;, and customization impractical.</p>
|
|
100
|
+
|
|
101
|
+
<p>Next I tried <a href='http://god.rubyforge.org/'>God</a>. While God&#8217;s integration with kernel events makes it all powerful, and its ruby based configuration is awesome, it is also a bit of a liability to install. Furthermore, God doesn&#8217;t run on JRuby, due to a combination of heavy native non-FFI extension, and reliance on low level ruby API&#8217;s not yet completely supported. In short, I found <a href='http://god.rubyforge.org/'>God</a> to be a bit <em>more</em> than I needed. So instead I began work on and created <a href='http://github.com/dekellum/iyyov#readme'>Iyyov</a>&#8217;s configuration in <a href='http://god.rubyforge.org/'>God</a>&#8217;s image. See the <a href='http://github.com/dekellum/iyyov#readme'>Iyyov</a> README for a more detailed feature set and examples. The hashdot-test-daemon in the above example can be configured and launched like so:</p>
|
|
102
|
+
<div class='highlight'><pre><code class='ruby'><span class='no'>Iyyov</span><span class='o'>.</span><span class='n'>context</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>c</span><span class='o'>|</span>
|
|
103
|
+
<span class='n'>c</span><span class='o'>.</span><span class='n'>define_daemon</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>d</span><span class='o'>|</span>
|
|
104
|
+
<span class='n'>d</span><span class='o'>.</span><span class='n'>name</span> <span class='o'>=</span> <span class='s2'>&quot;hashdot-test-daemon&quot;</span>
|
|
105
|
+
<span class='n'>d</span><span class='o'>.</span><span class='n'>version</span> <span class='o'>=</span> <span class='s2'>&quot;~&gt; 1.0&quot;</span>
|
|
106
|
+
<span class='n'>d</span><span class='o'>.</span><span class='n'>log_rotate</span>
|
|
107
|
+
<span class='k'>end</span>
|
|
108
|
+
<span class='k'>end</span>
|
|
109
|
+
</code></pre>
|
|
110
|
+
</div>
|
|
111
|
+
<p>Which highlights several Iyyov features:</p>
|
|
112
|
+
|
|
113
|
+
<ul>
|
|
114
|
+
<li>
|
|
115
|
+
<p>By default, Iyyov uses the rubygems API to find the highest matching gem version and init script with the same name. Thus in the common case, configuration is absolutely DRY.</p>
|
|
116
|
+
</li>
|
|
117
|
+
|
|
118
|
+
<li>
|
|
119
|
+
<p>Built in log rotation/compression that meshes well with Hashdot&#8217;s IO redirection and SIGHUP log reopening support. The end result is a complete logging solution that also redirects the often neglected JVM level output in the event of a JVM crash, uncaught exception, or &#8216;kill -QUIT&#8217; arranged stack dump. The external Linux logrotate utility could also be used here.</p>
|
|
120
|
+
</li>
|
|
121
|
+
</ul>
|
|
122
|
+
|
|
123
|
+
<p>Iyyov is also a practical cron replacement supporting fixed or periodic time scheduling of supporting tasks. For example:</p>
|
|
124
|
+
<div class='highlight'><pre><code class='ruby'><span class='no'>Iyyov</span><span class='o'>.</span><span class='n'>context</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>c</span><span class='o'>|</span>
|
|
125
|
+
|
|
126
|
+
<span class='c1'># Backup service state at 1am local time each weekday</span>
|
|
127
|
+
<span class='c1'># Run :async, in a &quot;background&quot; thread, since this might take a while.</span>
|
|
128
|
+
<span class='n'>c</span><span class='o'>.</span><span class='n'>schedule_at</span><span class='p'>(</span> <span class='ss'>:name</span> <span class='o'>=&gt;</span> <span class='s2'>&quot;backup&quot;</span><span class='p'>,</span> <span class='ss'>:mode</span> <span class='o'>=&gt;</span> <span class='ss'>:async</span><span class='p'>,</span>
|
|
129
|
+
<span class='ss'>:fixed_times</span> <span class='o'>=&gt;</span> <span class='o'>[</span> <span class='s1'>&#39;1:00&#39;</span> <span class='o'>]</span><span class='p'>,</span> <span class='ss'>:fixed_days</span> <span class='o'>=&gt;</span> <span class='p'>(</span><span class='mi'>1</span><span class='o'>.</span><span class='n'>.</span><span class='mi'>5</span><span class='p'>)</span> <span class='p'>)</span> <span class='k'>do</span>
|
|
130
|
+
<span class='nb'>system</span><span class='p'>(</span> <span class='s2'>&quot;rsync ...&quot;</span> <span class='p'>)</span>
|
|
131
|
+
<span class='k'>end</span>
|
|
132
|
+
|
|
133
|
+
<span class='k'>end</span>
|
|
134
|
+
</code></pre>
|
|
135
|
+
</div>
|
|
136
|
+
<p>As a final example, since the configuration is fully scriptable ruby, we might select which service to run by hostname in a single maintained configuration file:</p>
|
|
137
|
+
<div class='highlight'><pre><code class='ruby'><span class='no'>Iyyov</span><span class='o'>.</span><span class='n'>context</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>c</span><span class='o'>|</span>
|
|
138
|
+
|
|
139
|
+
<span class='k'>case</span> <span class='sb'>`hostname -s`</span><span class='o'>.</span><span class='n'>strip</span>
|
|
140
|
+
|
|
141
|
+
<span class='k'>when</span><span class='sr'> /^server-[012]$/</span>
|
|
142
|
+
<span class='n'>c</span><span class='o'>.</span><span class='n'>define_daemon</span> <span class='p'>{</span> <span class='o'>|</span><span class='n'>d</span><span class='o'>|</span> <span class='n'>d</span><span class='o'>.</span><span class='n'>name</span> <span class='o'>=</span> <span class='s2'>&quot;widget-factory&quot;</span> <span class='p'>}</span>
|
|
143
|
+
|
|
144
|
+
<span class='k'>when</span> <span class='s1'>&#39;server-3&#39;</span>
|
|
145
|
+
<span class='n'>c</span><span class='o'>.</span><span class='n'>define_daemon</span> <span class='p'>{</span> <span class='o'>|</span><span class='n'>d</span><span class='o'>|</span> <span class='n'>d</span><span class='o'>.</span><span class='n'>name</span> <span class='o'>=</span> <span class='s2'>&quot;assembly-line&quot;</span> <span class='p'>}</span>
|
|
146
|
+
|
|
147
|
+
<span class='k'>end</span>
|
|
148
|
+
<span class='k'>end</span>
|
|
149
|
+
</code></pre>
|
|
150
|
+
</div>
|
|
151
|
+
<h2 id='gem_maintenance'>Gem maintenance</h2>
|
|
152
|
+
|
|
153
|
+
<p>Because the above described per-gem init script isn&#8217;t wrapped by rubygems, the corresponding gem version should be hard coded within to guarantee the resolution of this and other gem dependencies. This presents a maintenance problem which can be solved through various means. My solution is the <a href='http://rjack.rubyforge.org/tarpit'>Tarpit</a> 1.2 test_line_match feature that may be installed in the gem project&#8217;s Rakefile, i.e:</p>
|
|
154
|
+
<div class='highlight'><pre><code class='ruby'><span class='n'>task</span> <span class='ss'>:check_init_version</span> <span class='k'>do</span>
|
|
155
|
+
<span class='n'>t</span><span class='o'>.</span><span class='n'>test_line_match</span><span class='p'>(</span> <span class='s1'>&#39;init/hashdot-test-daemon&#39;</span><span class='p'>,</span><span class='sr' />
|
|
156
|
+
<span class='sr'> /^gem.+hashdot-test-daemon/</span><span class='p'>,</span> <span class='o'>/=</span> <span class='c1'>#{t.version}/ )</span>
|
|
157
|
+
<span class='k'>end</span>
|
|
158
|
+
</code></pre>
|
|
159
|
+
</div>
|
|
160
|
+
<p>A simple error message is generated when attempting to build the gem without this version being set properly. My recent gems use this feature extensively and I think its less net maintenance then the other option I considered: generating such files with ERB templates.</p></content>
|
|
161
|
+
</entry>
|
|
162
|
+
|
|
163
|
+
<entry>
|
|
164
|
+
<title>RJack Updates via Gemcutter</title>
|
|
165
|
+
<link href="http://gravitext.com/2009/12/19/rjack-updates-via-gemcutter.html"/>
|
|
166
|
+
<updated>2009-12-19T00:00:00-08:00</updated>
|
|
167
|
+
<id>id:/2009/12/19/rjack-updates-via-gemcutter</id>
|
|
168
|
+
<content type="html"><p>Another batch of <a href='http://rjack.rubyforge.org/'>RJack</a> updates has been accumulating, but was blocked behind figuring out what has changed with <a href='http://gems.rubyforge.org/'>Gemcutter</a> and gem publishing to the canonical repo. Gemcutter appears to be good forward progress in modernizing gem distribution. While the <a href='http://update.gemcutter.org/'>blog</a> posts and FAQ provide clues, it would help more if there was some canonical status page (not a twitter timeline) that provides a clear summary of the current state of accounts, publishing routes, and gem repo behavior. Somewhere I thought I read that mirroring of the old rubyforge project gem file hosting has stopped, and a look at <a href='http://rubyforge.org'>rubyforge.org</a> shows the last published gem recorded there was on 2009-12-10. So without risking an old style rubyforge release (Hoe&#8217;s :release task) with unknown results, I decided it was time to start using &#8220;gem push&#8221;.</p>
|
|
169
|
+
|
|
170
|
+
<p>The previous introduction of <a href='http://rjack.rubyforge.org/tarpit/History_rdoc.html'>rjack-tarpit</a> made the changes for rjack much simpler. I first released a new rjack-tarpit-1.1.0 which provides new &#8220;push&#8221; and &#8220;install&#8221; tasks as shown below:</p>
|
|
171
|
+
<div class='highlight'><pre><code class='ruby'><span class='c1'># Define gem push and install tasks</span>
|
|
172
|
+
<span class='k'>def</span> <span class='nf'>define_gem_tasks</span>
|
|
173
|
+
<span class='n'>desc</span> <span class='s2'>&quot;gem push (gemcutter)&quot;</span>
|
|
174
|
+
<span class='n'>task</span> <span class='ss'>:push</span> <span class='o'>=&gt;</span> <span class='o'>[</span> <span class='ss'>:gem</span> <span class='o'>]</span> <span class='k'>do</span>
|
|
175
|
+
<span class='nb'>require</span> <span class='s1'>&#39;rubygems&#39;</span>
|
|
176
|
+
<span class='n'>cm</span> <span class='o'>=</span> <span class='no'>Gem</span><span class='o'>::</span><span class='no'>CommandManager</span><span class='o'>::</span><span class='n'>instance</span>
|
|
177
|
+
<span class='n'>cm</span><span class='o'>.</span><span class='n'>run</span><span class='p'>(</span> <span class='o'>[</span> <span class='s1'>&#39;push&#39;</span><span class='p'>,</span> <span class='s1'>&#39;-V&#39;</span><span class='p'>,</span> <span class='s2'>&quot;pkg/</span><span class='si'>#{</span><span class='nb'>name</span><span class='si'>}</span><span class='s2'>-</span><span class='si'>#{</span><span class='n'>version</span><span class='si'>}</span><span class='s2'>.gem&quot;</span> <span class='o'>]</span> <span class='p'>)</span>
|
|
178
|
+
<span class='k'>end</span>
|
|
179
|
+
|
|
180
|
+
<span class='n'>desc</span> <span class='s2'>&quot;gem install (default install dir)&quot;</span>
|
|
181
|
+
<span class='n'>task</span> <span class='ss'>:install</span> <span class='o'>=&gt;</span> <span class='o'>[</span> <span class='ss'>:gem</span> <span class='o'>]</span> <span class='k'>do</span>
|
|
182
|
+
<span class='nb'>require</span> <span class='s1'>&#39;rubygems&#39;</span>
|
|
183
|
+
<span class='n'>cm</span> <span class='o'>=</span> <span class='no'>Gem</span><span class='o'>::</span><span class='no'>CommandManager</span><span class='o'>::</span><span class='n'>instance</span>
|
|
184
|
+
<span class='n'>cm</span><span class='o'>.</span><span class='n'>run</span><span class='p'>(</span> <span class='o'>[</span> <span class='s1'>&#39;install&#39;</span><span class='p'>,</span> <span class='s1'>&#39;--no-ri&#39;</span><span class='p'>,</span> <span class='s1'>&#39;-V&#39;</span><span class='p'>,</span> <span class='s2'>&quot;pkg/</span><span class='si'>#{</span><span class='nb'>name</span><span class='si'>}</span><span class='s2'>-</span><span class='si'>#{</span><span class='n'>version</span><span class='si'>}</span><span class='s2'>.gem&quot;</span> <span class='o'>]</span> <span class='p'>)</span>
|
|
185
|
+
<span class='k'>end</span>
|
|
186
|
+
<span class='k'>end</span>
|
|
187
|
+
</code></pre>
|
|
188
|
+
</div>
|
|
189
|
+
<p>(I hope CommandManager::run is a &#8220;supported&#8221; approach to running gem commands in process. This has a speed advantage under jruby by avoiding a separate interpreter start-up.)</p>
|
|
190
|
+
|
|
191
|
+
<p>Using the new gemcutter based :push task I was able to release all of these accumulated updates:</p>
|
|
192
|
+
|
|
193
|
+
<ul>
|
|
194
|
+
<li><a href='http://rjack.rubyforge.org/slf4j/History_rdoc.html'>rjack-slf4j-1-5.10.0</a></li>
|
|
195
|
+
|
|
196
|
+
<li><a href='http://rjack.rubyforge.org/logback/History_rdoc.html'>rjack-logback-0.9.18.0</a></li>
|
|
197
|
+
|
|
198
|
+
<li><a href='http://rjack.rubyforge.org/httpclient-4/History_rdoc.html'>rjack-httpclient-4-4.0.1.0</a></li>
|
|
199
|
+
|
|
200
|
+
<li><a href='http://rjack.rubyforge.org/jetty/History_rdoc.html'>rjack-jetty-6.1.22.0</a></li>
|
|
201
|
+
|
|
202
|
+
<li><a href='http://rjack.rubyforge.org/jetty-jsp/History_rdoc.html'>rjack-jetty-jsp-6.1.22.2.1.0</a></li>
|
|
203
|
+
|
|
204
|
+
<li><a href='http://rjack.rubyforge.org/commons-dbutils/History_rdoc.html'>rjack-commons-dbutils-1.3.0</a></li>
|
|
205
|
+
|
|
206
|
+
<li><a href='http://rjack.rubyforge.org/commons-pool/History_rdoc.html'>rjack-commons-pool-1.5.4.0</a></li>
|
|
207
|
+
</ul>
|
|
208
|
+
|
|
209
|
+
<p>This eliminates one Hoe dependency, and with tarpit encapsulating these concerns, this change feels like a step in the direction of removing the Hoe dependency (and the quirky Manifest handling) all together. I like my implementation better than the recently released <a href='http://seattlerb.rubyforge.org/hoe/Hoe/Gemcutter.html#M000012'>Hoe 2.4 Gemcutter plugin</a>.</p></content>
|
|
210
|
+
</entry>
|
|
211
|
+
|
|
212
|
+
<entry>
|
|
213
|
+
<title>Jekyll</title>
|
|
214
|
+
<link href="http://gravitext.com/2009/11/04/jekyll.html"/>
|
|
215
|
+
<updated>2009-11-04T00:00:00-08:00</updated>
|
|
216
|
+
<id>id:/2009/11/04/jekyll</id>
|
|
217
|
+
<content type="html"><p>A new post for a newly revamped <a href='/#vblog'>void * blog</a> and a new personal home page constructed with <a href='http://github.com/mojombo/jekyll'>Jekyll</a>, a ruby based static site generator. In relation to my needs, Jekyll offers:</p>
|
|
218
|
+
|
|
219
|
+
<ul>
|
|
220
|
+
<li>
|
|
221
|
+
<p>A back-to-basics approach of static site generation from &#8220;source&#8221; as HTML templates, CSS, and markdown, all easily maintained by me in git and emacs.</p>
|
|
222
|
+
</li>
|
|
223
|
+
|
|
224
|
+
<li>
|
|
225
|
+
<p>Supports blog-style dated posts with excerpts and template driven atom feed generation.</p>
|
|
226
|
+
</li>
|
|
227
|
+
|
|
228
|
+
<li>
|
|
229
|
+
<p>Usable as a simple but complete and powerful CMS, not just blog posts. I want this for both maintaining a home page for myself, but also for developing technical site docs (usage, design, reference, etc.) for various projects.</p>
|
|
230
|
+
</li>
|
|
231
|
+
|
|
232
|
+
<li>
|
|
233
|
+
<p>Supports inline code sample syntax highlighting, something that was lacking in the previous blog.</p>
|
|
234
|
+
</li>
|
|
235
|
+
</ul>
|
|
236
|
+
|
|
237
|
+
<p>For now, the generated site remains on my home server and is served via Nginx. But I could easily push it elsewhere, for example, S3; as it requires no dynamic server component.</p>
|
|
238
|
+
|
|
239
|
+
<p>A missing feature vs. my previous blog is comment support, which has traditionally required a dynamic server, means of storage, spam detection, and many other complications. But java script embeddable comment <a href='http://www.knowliz.com/2009/06/disqus-intensedebate-js-kit-analysis.html'>services</a> like DISQUS are now winning conversions even from mainstream blog platforms based on superior features and identity integration. Its on my list to integrate one of these. Its both amusing and convenient to have Web 2.0 handling the last mile, and allowing me to get back to basics with a <em>Web 0.1</em> static site generator.</p>
|
|
240
|
+
|
|
241
|
+
<p>I&#8217;ll end the post with an example, the liquid template used to produce this blog&#8217;s atom feed:</p>
|
|
242
|
+
<!-- ZWSP U+200B added bellow to avoid interpreting nested liquid tags --><div class='highlight'><pre><code class='xml'><span class='cp'>&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;</span>
|
|
243
|
+
<span class='nt'>&lt;feed</span> <span class='na'>xmlns=</span><span class='s'>&quot;http://www.w3.org/2005/Atom&quot;</span><span class='nt'>&gt;</span>
|
|
244
|
+
|
|
245
|
+
<span class='nt'>&lt;title&gt;</span>void * blog<span class='nt'>&lt;/title&gt;</span>
|
|
246
|
+
<span class='nt'>&lt;link</span> <span class='na'>href=</span><span class='s'>&quot;http://gravitext.com/atom.xml&quot;</span> <span class='na'>rel=</span><span class='s'>&quot;self&quot;</span><span class='nt'>/&gt;</span>
|
|
247
|
+
<span class='nt'>&lt;link</span> <span class='na'>href=</span><span class='s'>&quot;http://gravitext.com/&quot;</span><span class='nt'>/&gt;</span>
|
|
248
|
+
<span class='nt'>&lt;updated&gt;</span>{{ site.time | date_to_xmlschema }}<span class='nt'>&lt;/updated&gt;</span>
|
|
249
|
+
<span class='nt'>&lt;id&gt;</span>http://gravitext.com/<span class='nt'>&lt;/id&gt;</span>
|
|
250
|
+
<span class='nt'>&lt;author&gt;</span>
|
|
251
|
+
<span class='nt'>&lt;name&gt;</span>David Kellum<span class='nt'>&lt;/name&gt;</span>
|
|
252
|
+
<span class='nt'>&lt;email&gt;</span>dek-blog@gravitext.com<span class='nt'>&lt;/email&gt;</span>
|
|
253
|
+
<span class='nt'>&lt;/author&gt;</span>
|
|
254
|
+
|
|
255
|
+
{% for post in site.posts %}
|
|
256
|
+
<span class='nt'>&lt;entry&gt;</span>
|
|
257
|
+
<span class='nt'>&lt;title&gt;</span>{{ post.title }}<span class='nt'>&lt;/title&gt;</span>
|
|
258
|
+
<span class='nt'>&lt;link</span> <span class='na'>href=</span><span class='s'>&quot;http://gravitext.com{{ post.url }}&quot;</span><span class='nt'>/&gt;</span>
|
|
259
|
+
<span class='nt'>&lt;updated&gt;</span>{{ post.date | date_to_xmlschema }}<span class='nt'>&lt;/updated&gt;</span>
|
|
260
|
+
<span class='nt'>&lt;id&gt;</span>id:{{ post.id }}<span class='nt'>&lt;/id&gt;</span>
|
|
261
|
+
<span class='nt'>&lt;content</span> <span class='na'>type=</span><span class='s'>&quot;html&quot;</span><span class='nt'>&gt;</span>{{ post.content | xml_escape }}<span class='nt'>&lt;/content&gt;</span>
|
|
262
|
+
<span class='nt'>&lt;/entry&gt;</span>
|
|
263
|
+
{% endfor %}
|
|
264
|
+
|
|
265
|
+
<span class='nt'>&lt;/feed&gt;</span>
|
|
266
|
+
</code></pre>
|
|
267
|
+
</div></content>
|
|
268
|
+
</entry>
|
|
269
|
+
|
|
270
|
+
<entry>
|
|
271
|
+
<title>RJack Updates</title>
|
|
272
|
+
<link href="http://gravitext.com/2009/10/24/rjack-updates.html"/>
|
|
273
|
+
<updated>2009-10-24T00:00:00-07:00</updated>
|
|
274
|
+
<id>id:/2009/10/24/rjack-updates</id>
|
|
275
|
+
<content type="html"><p>This week&#8217;s addition to the growing set of rjack gems is <a href='http://rjack.rubyforge.org/rome/'>rjack-rome-1.0.0</a> based on the ROME RSS/Atom feed reader/generator.</p>
|
|
276
|
+
|
|
277
|
+
<p>A new SVG intra-gem dependency diagram, built using the Graphviz Dot tool, was also added to the rjack project home page. The gems are linked to the RDoc:</p>
|
|
278
|
+
<div style='text-align: center;'>
|
|
279
|
+
<object data='http://rjack.rubyforge.org/rjack.svg' type='image/svg+xml' height='267' width='711'>
|
|
280
|
+
<img src='http://rjack.rubyforge.org/rjack.png' height='267' width='711' />
|
|
281
|
+
</object>
|
|
282
|
+
</div></content>
|
|
283
|
+
</entry>
|
|
284
|
+
|
|
285
|
+
<entry>
|
|
286
|
+
<title>RJack Updates</title>
|
|
287
|
+
<link href="http://gravitext.com/2009/10/18/rjack-updates.html"/>
|
|
288
|
+
<updated>2009-10-18T00:00:00-07:00</updated>
|
|
289
|
+
<id>id:/2009/10/18/rjack-updates</id>
|
|
290
|
+
<content type="html"><p>The <a href='http://rjack.rubyforge.org/'>RJack</a> project home page has been further improved. New and updated gems are detailed below.</p>
|
|
291
|
+
|
|
292
|
+
<h2 id='new'>New</h2>
|
|
293
|
+
|
|
294
|
+
<p>The following new gems have been released after consultations with the upstream project owners:</p>
|
|
295
|
+
|
|
296
|
+
<ul>
|
|
297
|
+
<li><a href='http://rjack.rubyforge.org/jdom/'>rjack-jdom-1.1.0.0</a> (JDOM XML processor)</li>
|
|
298
|
+
|
|
299
|
+
<li><a href='http://rjack.rubyforge.org/jets3t/'>rjack-jets3t-0.7.1.0</a> (Amazon S3/CloudFront client)</li>
|
|
300
|
+
</ul>
|
|
301
|
+
|
|
302
|
+
<p>The rjack-jets3t gem in particular is demonstrating returns on the gem based dependency management approach of rjack. This new gems depends on rjack-httpclient-3, rjack-commons-codec, and rjack-slf4j.</p>
|
|
303
|
+
|
|
304
|
+
<p>With these additions, rjack is up to 31 third-party jars packaged in 13 gems. With the recently released rjack-tarpit making it much easier to setup and maintain these gems, I am much more inclined to push all Java third-party jars into rjack gems rather than packing them directly in the consuming projects. This makes the consuming projects easier to maintain and distribute, and avoids version conflicts.</p>
|
|
305
|
+
|
|
306
|
+
<p>Next up for inclusion is an rjack-rome packaging of the ROME feed parser (which will also dependent on rjack-jdom.)</p>
|
|
307
|
+
|
|
308
|
+
<h2 id='updated'>Updated</h2>
|
|
309
|
+
|
|
310
|
+
<p>Additional gem updates since the last RJack post include:</p>
|
|
311
|
+
|
|
312
|
+
<ul>
|
|
313
|
+
<li><a href='http://rjack.rubyforge.org/tarpit/History_rdoc.html'>rjack-tarpit-1.0.1</a></li>
|
|
314
|
+
|
|
315
|
+
<li><a href='http://rjack.rubyforge.org/jetty/History_rdoc.html'>rjack-jetty-6.1.21.0</a></li>
|
|
316
|
+
|
|
317
|
+
<li><a href='http://rjack.rubyforge.org/jetty-jsp/History_rdoc.html'>rjack-jetty-6.1.21.2.1.0</a></li>
|
|
318
|
+
</ul></content>
|
|
319
|
+
</entry>
|
|
320
|
+
|
|
321
|
+
<entry>
|
|
322
|
+
<title>TarPit</title>
|
|
323
|
+
<link href="http://gravitext.com/2009/10/11/tarpit.html"/>
|
|
324
|
+
<updated>2009-10-11T00:00:00-07:00</updated>
|
|
325
|
+
<id>id:/2009/10/11/tarpit</id>
|
|
326
|
+
<content type="html"><p>Admission: I was a bit rash to have declared <a href='/2008/12/07/maven-must-die.html'>Jihad on Maven</a> back in December. (Is it ever wise to write &#8220;Jihad&#8221; on your blog?) Now 10 months later I&#8217;m still using Maven to build java and manage cross-java dependencies at build time. In the interest of peace and reconciliation let me say that Maven has really been performing amiably in its now reduced responsibility of invoking javac with a proper classpath, and packaging jars from the results. While this stalemate still seems less than ideal, I think I might continue to be able to get along with Maven, provided I don&#8217;t need to write any &#8220;plugins&#8221; for it, or wrestle with it for any less well trodden tasks.</p>
|
|
327
|
+
|
|
328
|
+
<p>Besides the war mongering, the original post does manage to outline the goals and associated set of issues around building gems for JRuby with java jars included. I&#8217;ve had good success with this strategy for releasing componentized systems to both dedicated colo and Amazon EC2 production environments in the last year. Enough so that investing some more time in build automation has become warranted. This comes in the form of a new Rake helper gem called <a href='http://rjack.rubyforge.org/tarpit'>TarPit</a>, released as part of the <a href='http://rjack.rubyforge.org/'>RJack</a> project. What was a very wet and copied around jars-in-gem rakefile recipe, can now be expressed in a short and declarative style. For example, the rjack-logback <a href='http://github.com/dekellum/rjack/blob/master/logback/Rakefile'>Rakefile</a>:</p>
|
|
329
|
+
<div class='highlight'><pre><code class='ruby'><span class='nb'>require</span> <span class='s1'>&#39;rjack-tarpit&#39;</span>
|
|
330
|
+
|
|
331
|
+
<span class='n'>t</span> <span class='o'>=</span> <span class='no'>RJack</span><span class='o'>::</span><span class='no'>TarPit</span><span class='o'>.</span><span class='n'>new</span><span class='p'>(</span> <span class='s1'>&#39;rjack-logback&#39;</span><span class='p'>,</span> <span class='no'>RJack</span><span class='o'>::</span><span class='no'>Logback</span><span class='o'>::</span><span class='no'>VERSION</span> <span class='p'>)</span>
|
|
332
|
+
|
|
333
|
+
<span class='n'>t</span><span class='o'>.</span><span class='n'>specify</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>h</span><span class='o'>|</span>
|
|
334
|
+
<span class='n'>h</span><span class='o'>.</span><span class='n'>developer</span><span class='p'>(</span> <span class='s2'>&quot;David Kellum&quot;</span><span class='p'>,</span> <span class='s2'>&quot;dek-oss@gravitext.com&quot;</span> <span class='p'>)</span>
|
|
335
|
+
<span class='n'>h</span><span class='o'>.</span><span class='n'>extra_deps</span> <span class='o'>&lt;&lt;</span> <span class='o'>[</span> <span class='s1'>&#39;rjack-slf4j&#39;</span><span class='p'>,</span> <span class='s1'>&#39;~&gt; 1.5.8&#39;</span> <span class='o'>]</span>
|
|
336
|
+
<span class='n'>h</span><span class='o'>.</span><span class='n'>rubyforge_name</span> <span class='o'>=</span> <span class='s2'>&quot;rjack&quot;</span>
|
|
337
|
+
<span class='n'>h</span><span class='o'>.</span><span class='n'>remote_rdoc_dir</span> <span class='o'>=</span> <span class='s2'>&quot;logback&quot;</span>
|
|
338
|
+
<span class='k'>end</span>
|
|
339
|
+
|
|
340
|
+
<span class='n'>t</span><span class='o'>.</span><span class='n'>jars</span> <span class='o'>=</span> <span class='sx'>%w{ core classic access }</span><span class='o'>.</span><span class='n'>map</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>n</span><span class='o'>|</span>
|
|
341
|
+
<span class='s2'>&quot;logback-</span><span class='si'>#{</span><span class='n'>n</span><span class='si'>}</span><span class='s2'>-</span><span class='si'>#{</span> <span class='no'>RJack</span><span class='o'>::</span><span class='no'>Logback</span><span class='o'>::</span><span class='no'>LOGBACK_VERSION</span> <span class='si'>}</span><span class='s2'>.jar&quot;</span>
|
|
342
|
+
<span class='k'>end</span>
|
|
343
|
+
|
|
344
|
+
<span class='n'>file</span> <span class='s1'>&#39;Manifest.txt&#39;</span> <span class='o'>=&gt;</span> <span class='o'>[</span> <span class='s2'>&quot;lib/</span><span class='si'>#{</span><span class='n'>t</span><span class='o'>.</span><span class='n'>name</span><span class='si'>}</span><span class='s2'>/base.rb&quot;</span> <span class='o'>]</span>
|
|
345
|
+
|
|
346
|
+
<span class='n'>t</span><span class='o'>.</span><span class='n'>assembly_version</span> <span class='o'>=</span> <span class='mi'>1</span><span class='o'>.</span><span class='mi'>0</span>
|
|
347
|
+
|
|
348
|
+
<span class='n'>t</span><span class='o'>.</span><span class='n'>define_tasks</span>
|
|
349
|
+
</code></pre>
|
|
350
|
+
</div>
|
|
351
|
+
<h2 id='how_it_all_works'>How it all works</h2>
|
|
352
|
+
|
|
353
|
+
<ul>
|
|
354
|
+
<li>
|
|
355
|
+
<p>The t.specify() call wraps Hoe.spec() (<a href='http://seattlerb.rubyforge.org/hoe/'>Hoe</a> 2.x) where |h| is the hoe object. All of the normal Hoe.spec options are still available, but TarPit importantly gains control of when in the Rake setup process Hoe.spec() is actually called (hint: last in define_tasks, once any needed Manifest.txt update has already been performed.)</p>
|
|
356
|
+
</li>
|
|
357
|
+
|
|
358
|
+
<li>
|
|
359
|
+
<p>There are some strategy options on the initial TarPit.new call, including: :jars_from_assembly (where only Maven knows the versioned jars to be included) and :no_assembly (when you are only including an internal jar built from source). In the :jars_from_assembly case, given that Hoe still insists on a Manifest.txt complete on disk at start of Rake setup, you still need to manually run &#8220;jrake manifest&#8221; as needed. If instead you can specify the jars, for example, as above from a ruby version number, then Manifest.txt will be built as needed and before any dependent hoe tasks.</p>
|
|
360
|
+
</li>
|
|
361
|
+
|
|
362
|
+
<li>
|
|
363
|
+
<p>In either case TarPit will setup Rake to run &#8220;mvn package&#8221; as a dependency whenever any java source files (src/**), pom.xml, or assembly.xml have been updated. The Maven produced jars, including &#8220;assembled&#8221; external jars are symlinked from the normal Maven target location to your lib/gemname/ directory.</p>
|
|
364
|
+
</li>
|
|
365
|
+
</ul></content>
|
|
366
|
+
</entry>
|
|
367
|
+
|
|
368
|
+
<entry>
|
|
369
|
+
<title>RJack Expanded</title>
|
|
370
|
+
<link href="http://gravitext.com/2009/10/10/rjack-expanded.html"/>
|
|
371
|
+
<updated>2009-10-10T00:00:00-07:00</updated>
|
|
372
|
+
<id>id:/2009/10/10/rjack-expanded</id>
|
|
373
|
+
<content type="html"><p>The <a href='http://rjack.rubyforge.org/'>RJack</a> project has expanded its mission in providing common open-source java components as gems for use with JRuby.</p>
|
|
374
|
+
|
|
375
|
+
<h2 id='name_changes'>Name Changes</h2>
|
|
376
|
+
|
|
377
|
+
<p>When I started RJack with the slf4j and logback gems, I figured I was pretty safe in using these names verbatim, since the gems were a very minimal and direct packaging of the same named upstream projects. Same with the jetty inclusion. Over time however I found potential inclusions colliding with existing ruby gems, for example: hc-httpclient was so named because httpclient, not surprisingly, was not available. With the later additions a certain <em>rjack style</em> also started to jell:</p>
|
|
378
|
+
|
|
379
|
+
<ul>
|
|
380
|
+
<li>
|
|
381
|
+
<p>SLF4J is the logging interface: every included rjack gem will use it to resolve java logging dependencies where found, including via legacy interfaces like commons logging, etc.</p>
|
|
382
|
+
</li>
|
|
383
|
+
|
|
384
|
+
<li>
|
|
385
|
+
<p>RJack will strive to expose every aspect of component setup, which might be referred to as &#8220;configuration&#8221; in the upstream java project, as easy to use and extend ruby classes, instead of less flexible XML, YAML, or the like.</p>
|
|
386
|
+
</li>
|
|
387
|
+
|
|
388
|
+
<li>
|
|
389
|
+
<p>When appropriate and useful, RJack gems will include ruby adapters to make these components more amendable to direct use from ruby code.</p>
|
|
390
|
+
</li>
|
|
391
|
+
</ul>
|
|
392
|
+
|
|
393
|
+
<p>Given these stylistic choices, and in order to better consolidate the effort and prepare for further expansion, I&#8217;ve decided at this point to release new versions of the existing gems under a &#8220;rjack-&#8221; gem prefix name and ::RJack ruby module. The following table shows old and new names and the current release version. New rjack-gems are also shown.</p>
|
|
394
|
+
<table><thead><tr><th>Old Name</th><th>New Name</th><th>Updated Version</th></tr></thead><tbody><tr><td style='text-align: center;'>slf4j</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/slf4j/History_rdoc.html'>rjack-slf4j</a></td><td style='text-align: center;'>1.5.8.1</td>
|
|
395
|
+
</tr><tr><td style='text-align: center;'>logback</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/logback/History_rdoc.html'>rjack-logback</a></td><td style='text-align: center;'>0.9.17.1</td>
|
|
396
|
+
</tr><tr><td style='text-align: center;'>hc-httpclient</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/httpclient-3/History_rdoc.html'>rjack-httpclient-3</a></td><td style='text-align: center;'>3.1.3</td>
|
|
397
|
+
</tr><tr><td style='text-align: center;'>-</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/httpclient-4/History_rdoc.html'>rjack-httpclient-4</a></td><td style='text-align: center;'>4.0.0</td>
|
|
398
|
+
</tr><tr><td style='text-align: center;'>jetty</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/jetty/History_rdoc.html'>rjack-jetty</a></td><td style='text-align: center;'>6.1.20.1</td>
|
|
399
|
+
</tr><tr><td style='text-align: center;'>jetty-jsp</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/jetty-jsp/History_rdoc.html'>rjack-jetty-jsp</a></td><td style='text-align: center;'>6.1.20.2.1.1</td>
|
|
400
|
+
</tr><tr><td style='text-align: center;'>-</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/commons-codec/History_rdoc.html'>rjack-commons-codec</a></td><td style='text-align: center;'>1.4.0</td>
|
|
401
|
+
</tr><tr><td style='text-align: center;'>-</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/commons-dbcp/History_rdoc.html'>rjack-commons-dbcp</a></td><td style='text-align: center;'>1.2.2.0</td>
|
|
402
|
+
</tr><tr><td style='text-align: center;'>-</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/commons-dbutils/History_rdoc.html'>rjack-commons-dbutils</a></td><td style='text-align: center;'>1.2.0</td>
|
|
403
|
+
</tr><tr><td style='text-align: center;'>-</td><td style='text-align: center;'><a href='http://rjack.rubyforge.org/commons-pool/History_rdoc.html'>rjack-commons-pool</a></td><td style='text-align: center;'>1.5.3.0</td>
|
|
404
|
+
</tr></tbody></table>
|
|
405
|
+
<p>The rjack-httpclient-[3,4] gems were a special consideration. I maintain continuity between the two versions, and the ruby setup interfaces are compatible, but require&#8217;ing both versions in the same process is permissible (like the associated Java code). This supports incremental transition to 4.x, a reality given lack of backward compatibility in the java interface and lack of significant pressure to migrate from 3.x.</p>
|
|
406
|
+
|
|
407
|
+
<p>In my other projects that use these gems, I&#8217;ve found converting to the new names a pretty easy grep and replace operation. Since they are also some cross dependencies between these gems, I recommend transition en masse. The gems with the old names will remain in the RubyForge gem repo, and significant bug fixes can be back ported if needed.</p>
|
|
408
|
+
|
|
409
|
+
<h2 id='new_features'>New Features</h2>
|
|
410
|
+
|
|
411
|
+
<p>Finally, there are some other new features included:</p>
|
|
412
|
+
|
|
413
|
+
<ul>
|
|
414
|
+
<li>
|
|
415
|
+
<p>The SLF4J gem now has better support for both ruby and java exceptions logged from ruby, and in mapping Ruby Module::Class names into java/SLF4J Logger dot notation.</p>
|
|
416
|
+
</li>
|
|
417
|
+
|
|
418
|
+
<li>
|
|
419
|
+
<p><em>Mapped Diagnostic Content</em> support in SLF4J and Logback</p>
|
|
420
|
+
</li>
|
|
421
|
+
|
|
422
|
+
<li>
|
|
423
|
+
<p>A new reusable jar-in-gem build tool, <a href='http://rjack.rubyforge.org/tarpit/'>rjack-tarpit</a> the subject of a future post.</p>
|
|
424
|
+
</li>
|
|
425
|
+
</ul></content>
|
|
426
|
+
</entry>
|
|
427
|
+
|
|
428
|
+
<entry>
|
|
429
|
+
<title>Insidious Whitespace</title>
|
|
430
|
+
<link href="http://gravitext.com/2009/06/28/insidious-whitespace.html"/>
|
|
431
|
+
<updated>2009-06-28T00:00:00-07:00</updated>
|
|
432
|
+
<id>id:/2009/06/28/insidious-whitespace</id>
|
|
433
|
+
<content type="html"><p>Over the past few months I&#8217;ve converted just about every active project of mine from Subversion to Git. This now includes the Evri development projects, as well as my open source projects, and even home linux server code (i.e. the ruby script that runs this blog.)</p>
|
|
434
|
+
|
|
435
|
+
<h2 id='background'>Background</h2>
|
|
436
|
+
|
|
437
|
+
<p>I was originally drawn to git based on its merits in supporting free forking of open source projects. Publishing several of my personal open source projects to github has cemented my appreciation. That said, as has been discussed broadly, the learning curve of git is quite steep. With this kind of power must come some complexity. Its relatively simple to push your sole project contributions to a github repo, but the first time I offered commits and a pull request off a fork of <a href='http://github.com/dekellum/activerecord-jdbc-adapter/'>activerecord-jdbc-adapter</a> I was a bit embarrassed to find that I really didn&#8217;t yet know what I was doing. That was a good lesson for me, perhaps learned the only practical way, and since then I&#8217;ve put a little extra git education time into just about every project. Milestones crossed:</p>
|
|
438
|
+
|
|
439
|
+
<ul>
|
|
440
|
+
<li>I&#8217;m no longer terrified by the unknown that was &#8220;git rebase&#8221;</li>
|
|
441
|
+
|
|
442
|
+
<li>I&#8217;m only mildly leery of &#8220;git svn.&#8221;</li>
|
|
443
|
+
</ul>
|
|
444
|
+
|
|
445
|
+
<p>Yes I will make my history look the way it should: nice independent linear patches, forward-only merges when possible, etc. And I think this is slowly starting to pay dividends in my development process. I&#8217;ve re-enabled some development tactics that I had previously decided, over the years of using SVN, CVS, and those before it, just weren&#8217;t worth the trouble. And I&#8217;ve learned some new tactics.</p>
|
|
446
|
+
|
|
447
|
+
<p>My mode of operation with Subversion had been to avoid branches and merging at all costs:</p>
|
|
448
|
+
|
|
449
|
+
<ul>
|
|
450
|
+
<li>Don&#8217;t check that in until I tag the next release!</li>
|
|
451
|
+
|
|
452
|
+
<li>Lets split this code base up so we can work better in parallel!</li>
|
|
453
|
+
</ul>
|
|
454
|
+
|
|
455
|
+
<p>While there remains merit in the last approach (appropriate modular decomposition), it shouldn&#8217;t be driven by limitations of a tool, and besides I never figured out exactly where to put those &#8220;trunk&#8221; directories.</p>
|
|
456
|
+
|
|
457
|
+
<h2 id='making_the_tools_work'>Making the tools work</h2>
|
|
458
|
+
|
|
459
|
+
<p>What parts of my development tool set need to change with git?</p>
|
|
460
|
+
|
|
461
|
+
<p>Maven might have been an issue (isn&#8217;t it always), but my recent conversion to java/ruby gems packaging has the convenient side effect of liberating me from the maven release plugin (though I&#8217;m told there is workable maven/git integration). I use rake for that now, and a simple &#8220;rake tag&#8221; task covers my git interaction.</p>
|
|
462
|
+
|
|
463
|
+
<p>I&#8217;m still inclined to use Eclipse, while its not ideal, as a java code editor, primarily for the refactoring support that java seems to need, as a junit test launcher, and occasional debugger. The egit plugin for eclipse works well for basic operations, but the command line is still my main interface to git.</p>
|
|
464
|
+
|
|
465
|
+
<p>But with merging in play, I really want my commits to be recognizable as clean atomic patch sets. An important aspect of this is not cluttering up those patches with arbitrary white noise in the form of randomly changing whitespace. With emacs, there is whitespace-mode and surrounding commands. Problem solved for all files accept java, but Eclipse is just not that sophisticated as a text editor.</p>
|
|
466
|
+
|
|
467
|
+
<h2 id='cleanws_script'>cleanws script</h2>
|
|
468
|
+
|
|
469
|
+
<p>So I worked up a new tool. Its a simple text file whitespace filter, but I&#8217;ve also added some nice git integration and color coding to make it reasonably convenient. Ruby BTW is quite nice for this kind of task. Before doing a &#8220;git add&#8221;, I run &#8220;cleanws -g&#8221; to ensure good canonical whitespace in new/updated files. See the following example:</p>
|
|
470
|
+
<div class='image'>
|
|
471
|
+
<img src='/images/cleanws_sample_3.png' height='352' width='555' />
|
|
472
|
+
</div>
|
|
473
|
+
<p>The second call fixes the file and (-w)rites it to disk. Any detected evil tab characters are left in place to be manually resolved.</p>
|
|
474
|
+
|
|
475
|
+
<p>Here is the <a href='/share/cleanws'>full script</a>. Perhaps it will find its way into a gem and github it there is interest or if I continue to expand on it. Enjoy, and please keep it clean out there.</p></content>
|
|
476
|
+
</entry>
|
|
477
|
+
|
|
478
|
+
<entry>
|
|
479
|
+
<title>Hashdot 1.3 Released</title>
|
|
480
|
+
<link href="http://gravitext.com/2009/03/09/hashdot-1.3.html"/>
|
|
481
|
+
<updated>2009-03-09T00:00:00-07:00</updated>
|
|
482
|
+
<id>id:/2009/03/09/hashdot-1.3</id>
|
|
483
|
+
<content type="html"><p>Hashdot 1.3 has been released on SourceForge.</p>
|
|
484
|
+
|
|
485
|
+
<h2 id='13_200939'>1.3 (2009-3-9)</h2>
|
|
486
|
+
|
|
487
|
+
<ul>
|
|
488
|
+
<li>Added hashdot.user.home property to current real user home directory.</li>
|
|
489
|
+
|
|
490
|
+
<li>Added hashdot.header.comment property in support of languages not using &#8217;#&#8217; as a line comment. The groovy.hdp and rhino.hdp profiles now set this property to &#8220;//&#8221;. Associated examples demonstrate alternative header formats.</li>
|
|
491
|
+
|
|
492
|
+
<li>Added hashdot.script.dir. This and hashdot.script properties are now always absolute paths. Clarified use of java.class.path.</li>
|
|
493
|
+
|
|
494
|
+
<li>Added clojure (clj) profile and example. [Phil Hagelberg]</li>
|
|
495
|
+
|
|
496
|
+
<li>Refined examples, jruby is only default INSTALL_SYMLINK.</li>
|
|
497
|
+
</ul></content>
|
|
498
|
+
</entry>
|
|
499
|
+
|
|
500
|
+
<entry>
|
|
501
|
+
<title>Hashdot 1.2 Released</title>
|
|
502
|
+
<link href="http://gravitext.com/2008/12/22/hashdot-1.2-released.html"/>
|
|
503
|
+
<updated>2008-12-22T00:00:00-08:00</updated>
|
|
504
|
+
<id>id:/2008/12/22/hashdot-1.2-released</id>
|
|
505
|
+
<content type="html"><p>Hashdot 1.2 has been released on SourceForge. Special thanks to Matt Sanford for contributing the initial Mac OS X port and helping me through some additional OS X idiosyncrasies. Details in the change log below:</p>
|
|
506
|
+
|
|
507
|
+
<h2 id='12_20081222'>1.2 (2008-12-22)</h2>
|
|
508
|
+
|
|
509
|
+
<ul>
|
|
510
|
+
<li>Added HUP signal handler to reopen STDOUT/STDERR when both daemonize and io_redirect are used</li>
|
|
511
|
+
|
|
512
|
+
<li>Added support for hashdot.chdir</li>
|
|
513
|
+
|
|
514
|
+
<li>Ported to Mac OS X. See default.hdp for differences in profile configuration. Note on Mac, the final test/test_cmdline.rb currently fails due to lack of a suitable equivalent to the Linux prctl() for process rename. However, this is not strictly required for useful operation.</li>
|
|
515
|
+
|
|
516
|
+
<li>Simplified default.hdp and shortlived.hdp using new hashdot.vm.lib delayed expansion property.</li>
|
|
517
|
+
|
|
518
|
+
<li>Added explicit int casts to avoid x64 warnings</li>
|
|
519
|
+
|
|
520
|
+
<li>Improved INSTALL notes.</li>
|
|
521
|
+
</ul></content>
|
|
522
|
+
</entry>
|
|
523
|
+
|
|
524
|
+
<entry>
|
|
525
|
+
<title>Maven Must Die</title>
|
|
526
|
+
<link href="http://gravitext.com/2008/12/07/maven-must-die.html"/>
|
|
527
|
+
<updated>2008-12-07T00:00:00-08:00</updated>
|
|
528
|
+
<id>id:/2008/12/07/maven-must-die</id>
|
|
529
|
+
<content type="html"><p>The RubyGems packaging system has several advantages for java-hosted components. It goes places that Maven won&#8217;t go, like all the way into production and managing non-trivial executables from the command line or inittab. In my <a href='/2008/11/06/rjack-updates.html'>first post on rjack</a> I make some further case for this approach and describe some considerations in packaging ruby/java components as gems. In this post I&#8217;ll describe some of the complexities with building such gems, and workarounds to common problems.</p>
|
|
530
|
+
|
|
531
|
+
<h2 id='gems'>Gems with jars</h2>
|
|
532
|
+
|
|
533
|
+
<p>The rjack project is best understood with an install of jruby, optimally hashdot, and for example, the jetty gem:</p>
|
|
534
|
+
<pre>
|
|
535
|
+
% jgem install jetty
|
|
536
|
+
% jetty-service
|
|
537
|
+
</pre>
|
|
538
|
+
<p>This will afford the opportunity to look at how the gem is built, as the full source including the build mechanism is included. The top two levels of the gem directory structure are shown below:</p>
|
|
539
|
+
<pre>
|
|
540
|
+
/opt/jruby/gems/gems/jetty-6.1.14.1/
|
|
541
|
+
|-- History.txt
|
|
542
|
+
|-- Manifest.txt
|
|
543
|
+
|-- README.txt
|
|
544
|
+
|-- Rakefile
|
|
545
|
+
|-- assembly.xml
|
|
546
|
+
|-- bin/
|
|
547
|
+
| `-- jetty-service*
|
|
548
|
+
|-- lib/
|
|
549
|
+
| |-- jetty/
|
|
550
|
+
| `-- jetty.rb
|
|
551
|
+
|-- pom.xml
|
|
552
|
+
|-- src/
|
|
553
|
+
| `-- main/
|
|
554
|
+
|-- test/
|
|
555
|
+
| `-- test_jetty.rb
|
|
556
|
+
`-- webapps/
|
|
557
|
+
|-- test/
|
|
558
|
+
`-- test.war
|
|
559
|
+
</pre>
|
|
560
|
+
<p>Note the presence of both a Rakefile, and the Maven pom.xml and assembly.xml. Other projects that produce gems including jars manage the jar files as if they were source, but this becomes a maintenance liability in any of the following scenarios:</p>
|
|
561
|
+
|
|
562
|
+
<ul>
|
|
563
|
+
<li>
|
|
564
|
+
<p>Included jar(s) are undergoing frequent updates and the gem should be kept current.</p>
|
|
565
|
+
</li>
|
|
566
|
+
|
|
567
|
+
<li>
|
|
568
|
+
<p>Included jar(s) have additional jar dependencies themselves undergoing updates, and these jars must also be included.</p>
|
|
569
|
+
</li>
|
|
570
|
+
|
|
571
|
+
<li>
|
|
572
|
+
<p>The gem includes original java source code that must be built, packaged as a jar and included in with the gem.</p>
|
|
573
|
+
</li>
|
|
574
|
+
|
|
575
|
+
<li>
|
|
576
|
+
<p>External components, perhaps themselves packaged as gems, have dependencies on the packaged java. Thus the internally built java must also be published for resolving external java dependencies.</p>
|
|
577
|
+
</li>
|
|
578
|
+
</ul>
|
|
579
|
+
|
|
580
|
+
<h2 id='hoe_rake_maven_the_unholy_union'>Hoe, Rake, Maven: The unholy union</h2>
|
|
581
|
+
|
|
582
|
+
<p>The jetty gem Rakefile (see below) attempts the rather perilous integration of Maven into Rake, with <a href='http://seattlerb.rubyforge.org/hoe/'>Hoe</a> for gem building and rubyforge publishing support. Here you will find subtle and not-so-subtle workarounds for a variety of issues encountered, including:</p>
|
|
583
|
+
|
|
584
|
+
<ol>
|
|
585
|
+
<li>
|
|
586
|
+
<p>In standard gem build tradition, the gem version is actually obtained by loading ruby source. This has the advantage of the version being specified in one (fewer) place. However, there is an unfortunate incompatibility with JRuby and Rake both currently defining Object::import (JRUBY-3171). There is a simple workaround: Include an alternative ruby include that just defines version constants without any java imports. In this case its lib/jetty/base.rb, which defines VERSION and JETTY_VERSION in the Jetty module.</p>
|
|
587
|
+
</li>
|
|
588
|
+
|
|
589
|
+
<li>
|
|
590
|
+
<p>Hoe requires a Manifest.txt file on disk a priori, apparently to protect us from making mistakes in what is included in the gem. The problem with this is that jar names to be included may frequently change, and only maven knows the full dependency details. The workaround is to generate the Manifest.txt file with a :manifest task, but this task must be run and completed in its own process before any Hoe tasks utilize it as part of task definition. In practice we need to remember to update the manifest when jar&#8217;s change, less than ideal.</p>
|
|
591
|
+
</li>
|
|
592
|
+
|
|
593
|
+
<li>
|
|
594
|
+
<p>In the jetty gem, the full list of JARS (l28) are mapped from the jetty/base.rb included VERSIONs. In other gems the assembly brings in arbitrary dependencies of mixed versions, it becomes necessary to Dir.glob( &#8220;..assembly/*.jar&#8221; ) for these and the :manifest task acquires a &#8220;mvn package&#8221; task dependency.</p>
|
|
595
|
+
</li>
|
|
596
|
+
|
|
597
|
+
<li>
|
|
598
|
+
<p>We know to build the assembly via &#8220;mvn package&#8221; if it doesn&#8217;t yet exist, or if the pom.xml or assembly.xml is newer. The assembly is made a dependency of the :gem and :test tasks (l84). However, for gems packing java source, the dependencies are extended to all java source files as well.</p>
|
|
599
|
+
</li>
|
|
600
|
+
|
|
601
|
+
<li>
|
|
602
|
+
<p>Hoe wants to use a graphics tool called &#8220;dot&#8221; for rdoc generation. I don&#8217;t. Disabled via a crufty ENV[&#8216;NOTDOT&#8217;] setter.</p>
|
|
603
|
+
</li>
|
|
604
|
+
</ol>
|
|
605
|
+
|
|
606
|
+
<p>These might appear to constitute a rant, but I&#8217;m well aware of the fact that all of this software is free. I&#8217;m just fishing for a better solution in a relatively new problem space.</p>
|
|
607
|
+
|
|
608
|
+
<h2 id='gemjar_dependencies'>Gem/jar dependencies</h2>
|
|
609
|
+
|
|
610
|
+
<p>For a more complicated example, consider a dependency graph between two gems both containing java source:</p>
|
|
611
|
+
|
|
612
|
+
<p><img src='/images/gem_jar_deps.png' alt='gem jar dependencies' /></p>
|
|
613
|
+
|
|
614
|
+
<p>There is no dependency loops and everything works surprisingly well once gem and maven releases have been made to the respective repositories. However its complicated, manual, and error prone when implementing new versions of both gems in parallel, and needing to share changes with other developers. We must abandon Maven&#8217;s <code>mvn release:prepare</code> functionality for tagging a release version and incrementing to the next snapshot. Instead the following steps are typical:</p>
|
|
615
|
+
|
|
616
|
+
<ol>
|
|
617
|
+
<li>For gem-a, <code>jrake clean</code> to remove older jar versions</li>
|
|
618
|
+
|
|
619
|
+
<li>Update versions numbers in the pom.xml, lib/gem-a/base.rb, and a new version to History.txt</li>
|
|
620
|
+
|
|
621
|
+
<li><code>jrake manifest</code> to update Manifest.text with new jars.</li>
|
|
622
|
+
|
|
623
|
+
<li><code>jrake test gem</code> to build, test and package the gem.</li>
|
|
624
|
+
|
|
625
|
+
<li><code>mvn install</code> to make a.jar available for gem-b java build.</li>
|
|
626
|
+
|
|
627
|
+
<li><code>jgem install pkg/gem-a-VERSION.gem</code> to make gem-a available for gem-b</li>
|
|
628
|
+
|
|
629
|
+
<li>For gem-b, repeat steps 1-4, etc.</li>
|
|
630
|
+
</ol>
|
|
631
|
+
|
|
632
|
+
<p>Also consider that much of the above will need to be repeated if gem-a undergoes changes that gem-b is dependent on. One saving feature is that from a Maven perspective, gem-a and gem-b still look like &#8220;normal&#8221; java jar projects, if gem-a and gem-b java changes can be made in parallel with typical maven means, including use of parent pom declaring gem-a and gem-b as modules. But in general, its clear that a better integrated tool, offering full build automation for gems with jars is needed. An evaluation of alternatives will be saved for a subsequent post.</p>
|
|
633
|
+
|
|
634
|
+
<h2 id='rakefile_for_jetty_gem'>Rakefile for jetty gem</h2>
|
|
635
|
+
<div class='highlight'><pre><code class='ruby'><span class='nb'>require</span> <span class='s1'>&#39;rubygems&#39;</span>
|
|
636
|
+
|
|
637
|
+
<span class='no'>ENV</span><span class='o'>[</span><span class='s1'>&#39;NODOT&#39;</span><span class='o'>]</span> <span class='o'>=</span> <span class='s2'>&quot;no thank you&quot;</span>
|
|
638
|
+
<span class='nb'>require</span> <span class='s1'>&#39;hoe&#39;</span>
|
|
639
|
+
|
|
640
|
+
<span class='vg'>$LOAD_PATH</span> <span class='o'>&lt;&lt;</span> <span class='s1'>&#39;./lib&#39;</span>
|
|
641
|
+
<span class='nb'>require</span> <span class='s1'>&#39;jetty/base&#39;</span>
|
|
642
|
+
|
|
643
|
+
<span class='no'>JARS</span> <span class='o'>=</span> <span class='sx'>%w{ jetty jetty-util jetty-rewrite-handler }</span><span class='o'>.</span><span class='n'>map</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>n</span><span class='o'>|</span>
|
|
644
|
+
<span class='s2'>&quot;</span><span class='si'>#{</span><span class='n'>n</span><span class='si'>}</span><span class='s2'>-</span><span class='si'>#{</span> <span class='no'>Jetty</span><span class='o'>::</span><span class='no'>JETTY_VERSION</span> <span class='si'>}</span><span class='s2'>.jar&quot;</span>
|
|
645
|
+
<span class='k'>end</span>
|
|
646
|
+
<span class='no'>JARS</span> <span class='o'>&lt;&lt;</span> <span class='s2'>&quot;servlet-api-</span><span class='si'>#{</span> <span class='no'>Jetty</span><span class='o'>::</span><span class='no'>SERVLET_API_VERSION</span> <span class='si'>}</span><span class='s2'>-</span><span class='si'>#{</span> <span class='no'>Jetty</span><span class='o'>::</span><span class='no'>JETTY_VERSION</span> <span class='si'>}</span><span class='s2'>.jar&quot;</span>
|
|
647
|
+
<span class='no'>JARS</span> <span class='o'>&lt;&lt;</span> <span class='s1'>&#39;gravitext-testservlets-1.0.jar&#39;</span>
|
|
648
|
+
<span class='no'>JAR_FILES</span> <span class='o'>=</span> <span class='no'>JARS</span><span class='o'>.</span><span class='n'>map</span> <span class='p'>{</span> <span class='o'>|</span><span class='n'>jar</span><span class='o'>|</span> <span class='s2'>&quot;lib/jetty/</span><span class='si'>#{</span><span class='n'>jar</span><span class='si'>}</span><span class='s2'>&quot;</span> <span class='p'>}</span>
|
|
649
|
+
|
|
650
|
+
<span class='n'>desc</span> <span class='s2'>&quot;Update the Manifest with actual jars&quot;</span>
|
|
651
|
+
<span class='n'>task</span> <span class='ss'>:manifest</span> <span class='k'>do</span>
|
|
652
|
+
<span class='n'>out</span> <span class='o'>=</span> <span class='no'>File</span><span class='o'>.</span><span class='n'>new</span><span class='p'>(</span> <span class='s1'>&#39;Manifest.txt&#39;</span><span class='p'>,</span> <span class='s1'>&#39;w&#39;</span> <span class='p'>)</span>
|
|
653
|
+
<span class='k'>begin</span>
|
|
654
|
+
<span class='n'>out</span><span class='o'>.</span><span class='n'>write</span> <span class='o'>&lt;&lt;</span><span class='no'>END</span>
|
|
655
|
+
<span class='sh'>History.txt</span>
|
|
656
|
+
<span class='sh'>Manifest.txt</span>
|
|
657
|
+
<span class='sh'>README.txt</span>
|
|
658
|
+
<span class='sh'>Rakefile</span>
|
|
659
|
+
<span class='sh'>pom.xml</span>
|
|
660
|
+
<span class='sh'>assembly.xml</span>
|
|
661
|
+
<span class='sh'>bin/jetty-service</span>
|
|
662
|
+
<span class='sh'>lib/jetty.rb</span>
|
|
663
|
+
<span class='sh'>lib/jetty/base.rb</span>
|
|
664
|
+
<span class='sh'>lib/jetty/rewrite.rb</span>
|
|
665
|
+
<span class='sh'>lib/jetty/test-servlets.rb</span>
|
|
666
|
+
<span class='sh'>src/main/java/com/gravitext/testservlets/PerfTestServlet.java</span>
|
|
667
|
+
<span class='sh'>src/main/java/com/gravitext/testservlets/SnoopServlet.java</span>
|
|
668
|
+
<span class='sh'>test/test_jetty.rb</span>
|
|
669
|
+
<span class='sh'>test/test.txt</span>
|
|
670
|
+
<span class='sh'>webapps/test.war</span>
|
|
671
|
+
<span class='sh'>webapps/test/WEB-INF/web.xml</span>
|
|
672
|
+
<span class='sh'>webapps/test/index.html</span>
|
|
673
|
+
<span class='no'>END</span>
|
|
674
|
+
<span class='n'>out</span><span class='o'>.</span><span class='n'>puts</span> <span class='no'>JAR_FILES</span>
|
|
675
|
+
<span class='k'>ensure</span>
|
|
676
|
+
<span class='n'>out</span><span class='o'>.</span><span class='n'>close</span>
|
|
677
|
+
<span class='k'>end</span>
|
|
678
|
+
<span class='k'>end</span>
|
|
679
|
+
|
|
680
|
+
<span class='no'>ASSEMBLY</span> <span class='o'>=</span> <span class='s2'>&quot;target/gravitext-testservlets-1.0-bin.dir&quot;</span>
|
|
681
|
+
|
|
682
|
+
<span class='n'>file</span> <span class='s1'>&#39;webapps/test.war&#39;</span> <span class='o'>=&gt;</span> <span class='o'>[</span> <span class='s1'>&#39;webapps/test/index.html&#39;</span><span class='p'>,</span>
|
|
683
|
+
<span class='s1'>&#39;webapps/test/WEB-INF/web.xml&#39;</span> <span class='o'>]</span> <span class='k'>do</span>
|
|
684
|
+
<span class='n'>sh</span><span class='p'>(</span> <span class='s1'>&#39;jar cvf webapps/test.war &#39;</span> <span class='o'>+</span>
|
|
685
|
+
<span class='s1'>&#39;-C webapps/test index.html -C webapps/test WEB-INF/web.xml&#39;</span> <span class='p'>)</span>
|
|
686
|
+
<span class='k'>end</span>
|
|
687
|
+
|
|
688
|
+
<span class='n'>file</span> <span class='no'>ASSEMBLY</span> <span class='o'>=&gt;</span> <span class='o'>[</span> <span class='s1'>&#39;pom.xml&#39;</span><span class='p'>,</span> <span class='s1'>&#39;assembly.xml&#39;</span> <span class='o'>]</span> <span class='k'>do</span>
|
|
689
|
+
<span class='n'>sh</span><span class='p'>(</span> <span class='s1'>&#39;mvn package&#39;</span> <span class='p'>)</span>
|
|
690
|
+
<span class='k'>end</span>
|
|
691
|
+
|
|
692
|
+
<span class='no'>JARS</span><span class='o'>.</span><span class='n'>each</span> <span class='k'>do</span> <span class='o'>|</span><span class='n'>jar</span><span class='o'>|</span>
|
|
693
|
+
<span class='n'>file</span> <span class='s2'>&quot;lib/jetty/</span><span class='si'>#{</span><span class='n'>jar</span><span class='si'>}</span><span class='s2'>&quot;</span> <span class='o'>=&gt;</span> <span class='o'>[</span> <span class='no'>ASSEMBLY</span> <span class='o'>]</span> <span class='k'>do</span>
|
|
694
|
+
<span class='n'>cp_r</span><span class='p'>(</span> <span class='no'>File</span><span class='o'>.</span><span class='n'>join</span><span class='p'>(</span> <span class='no'>ASSEMBLY</span><span class='p'>,</span> <span class='n'>jar</span> <span class='p'>),</span> <span class='s1'>&#39;lib/jetty&#39;</span> <span class='p'>)</span>
|
|
695
|
+
<span class='k'>end</span>
|
|
696
|
+
<span class='k'>end</span>
|
|
697
|
+
|
|
698
|
+
<span class='o'>[</span> <span class='ss'>:gem</span><span class='p'>,</span> <span class='ss'>:test</span> <span class='o'>].</span><span class='n'>each</span> <span class='p'>{</span> <span class='o'>|</span><span class='n'>t</span><span class='o'>|</span> <span class='n'>task</span> <span class='n'>t</span> <span class='o'>=&gt;</span> <span class='no'>JAR_FILES</span> <span class='o'>+</span> <span class='o'>[</span> <span class='s1'>&#39;webapps/test.war&#39;</span> <span class='o'>]</span> <span class='p'>}</span>
|
|
699
|
+
|
|
700
|
+
<span class='n'>task</span> <span class='ss'>:mvn_clean</span> <span class='k'>do</span>
|
|
701
|
+
<span class='n'>rm_f</span><span class='p'>(</span> <span class='no'>JAR_FILES</span> <span class='o'>+</span> <span class='o'>[</span> <span class='s1'>&#39;webapps/test.war&#39;</span> <span class='o'>]</span> <span class='p'>)</span>
|
|
702
|
+
<span class='n'>sh</span><span class='p'>(</span> <span class='s1'>&#39;mvn clean&#39;</span> <span class='p'>)</span>
|
|
703
|
+
<span class='k'>end</span>
|
|
704
|
+
<span class='n'>task</span> <span class='ss'>:clean</span> <span class='o'>=&gt;</span> <span class='ss'>:mvn_clean</span>
|
|
705
|
+
|
|
706
|
+
<span class='n'>task</span> <span class='ss'>:tag</span> <span class='k'>do</span>
|
|
707
|
+
<span class='n'>tag</span> <span class='o'>=</span> <span class='s2'>&quot;jetty-</span><span class='si'>#{</span><span class='no'>Jetty</span><span class='o'>::</span><span class='no'>VERSION</span><span class='si'>}</span><span class='s2'>&quot;</span>
|
|
708
|
+
<span class='n'>svn_base</span> <span class='o'>=</span> <span class='s1'>&#39;svn://localhost/subversion.repo/src/gems&#39;</span>
|
|
709
|
+
<span class='n'>tag_url</span> <span class='o'>=</span> <span class='s2'>&quot;</span><span class='si'>#{</span><span class='n'>svn_base</span><span class='si'>}</span><span class='s2'>/tags/</span><span class='si'>#{</span><span class='n'>tag</span><span class='si'>}</span><span class='s2'>&quot;</span>
|
|
710
|
+
|
|
711
|
+
<span class='n'>dname</span> <span class='o'>=</span> <span class='no'>File</span><span class='o'>.</span><span class='n'>dirname</span><span class='p'>(</span> <span class='bp'>__FILE__</span> <span class='p'>)</span>
|
|
712
|
+
<span class='n'>dname</span> <span class='o'>=</span> <span class='s1'>&#39;.&#39;</span> <span class='k'>if</span> <span class='no'>Dir</span><span class='o'>.</span><span class='n'>getwd</span> <span class='o'>==</span> <span class='n'>dname</span>
|
|
713
|
+
<span class='n'>stat</span> <span class='o'>=</span> <span class='sb'>`svn status </span><span class='si'>#{</span><span class='n'>dname</span><span class='si'>}</span><span class='sb'>`</span>
|
|
714
|
+
<span class='n'>stat</span><span class='o'>.</span><span class='n'>strip!</span> <span class='k'>if</span> <span class='n'>stat</span>
|
|
715
|
+
<span class='k'>if</span> <span class='p'>(</span> <span class='n'>stat</span> <span class='o'>&amp;&amp;</span> <span class='n'>stat</span><span class='o'>.</span><span class='n'>length</span> <span class='o'>&gt;</span> <span class='mi'>0</span> <span class='p'>)</span>
|
|
716
|
+
<span class='vg'>$stderr</span><span class='o'>.</span><span class='n'>puts</span><span class='p'>(</span> <span class='s2'>&quot;Resolve the following before tagging (svn status):&quot;</span> <span class='p'>)</span>
|
|
717
|
+
<span class='vg'>$stderr</span><span class='o'>.</span><span class='n'>puts</span><span class='p'>(</span> <span class='n'>stat</span> <span class='p'>)</span>
|
|
718
|
+
<span class='k'>else</span>
|
|
719
|
+
<span class='n'>sh</span><span class='p'>(</span> <span class='s2'>&quot;svn cp -m &#39;tag [</span><span class='si'>#{</span><span class='n'>tag</span><span class='si'>}</span><span class='s2'>]&#39; </span><span class='si'>#{</span><span class='n'>dname</span><span class='si'>}</span><span class='s2'> </span><span class='si'>#{</span><span class='n'>tag_url</span><span class='si'>}</span><span class='s2'>&quot;</span> <span class='p'>)</span>
|
|
720
|
+
<span class='k'>end</span>
|
|
721
|
+
<span class='k'>end</span>
|
|
722
|
+
|
|
723
|
+
<span class='n'>hoe</span> <span class='o'>=</span> <span class='no'>Hoe</span><span class='o'>.</span><span class='n'>new</span><span class='p'>(</span> <span class='s2'>&quot;jetty&quot;</span><span class='p'>,</span> <span class='no'>Jetty</span><span class='o'>::</span><span class='no'>VERSION</span> <span class='p'>)</span> <span class='k'>do</span> <span class='o'>|</span><span class='nb'>p</span><span class='o'>|</span>
|
|
724
|
+
<span class='nb'>p</span><span class='o'>.</span><span class='n'>developer</span><span class='p'>(</span> <span class='s2'>&quot;David Kellum&quot;</span><span class='p'>,</span> <span class='s2'>&quot;dek-ruby@gravitext.com&quot;</span> <span class='p'>)</span>
|
|
725
|
+
<span class='nb'>p</span><span class='o'>.</span><span class='n'>rubyforge_name</span> <span class='o'>=</span> <span class='s2'>&quot;rjack&quot;</span>
|
|
726
|
+
<span class='nb'>p</span><span class='o'>.</span><span class='n'>rdoc_pattern</span> <span class='o'>=</span> <span class='sr'>/^(lib.*\.(rb|txt))|[^\/]*\.txt$/</span>
|
|
727
|
+
<span class='k'>end</span>
|
|
728
|
+
</code></pre>
|
|
729
|
+
</div></content>
|
|
730
|
+
</entry>
|
|
731
|
+
|
|
732
|
+
<entry>
|
|
733
|
+
<title>RJack Updates</title>
|
|
734
|
+
<link href="http://gravitext.com/2008/12/06/rjack-updates.html"/>
|
|
735
|
+
<updated>2008-12-06T00:00:00-08:00</updated>
|
|
736
|
+
<id>id:/2008/12/06/rjack-updates</id>
|
|
737
|
+
<content type="html"><p>The <a href='http://rjack.rubyforge.org/'>RJack</a> project home page has been beautified.</p>
|
|
738
|
+
|
|
739
|
+
<p>The gem updates include:</p>
|
|
740
|
+
|
|
741
|
+
<ul>
|
|
742
|
+
<li>jetty and jetty-jsp 6.1.14</li>
|
|
743
|
+
|
|
744
|
+
<li>slf4j 1.5.6</li>
|
|
745
|
+
|
|
746
|
+
<li>logback 0.9.13</li>
|
|
747
|
+
</ul>
|
|
748
|
+
|
|
749
|
+
<p>Also, as part of this update I tested compatibility with newly released JRuby 1.1.6-RC1. Assuming the RC status is meaningfully applied and temporary, all should be in good for the 1.1.6 general release.</p>
|
|
750
|
+
|
|
751
|
+
<p>The void * blog is now running these gems.</p></content>
|
|
752
|
+
</entry>
|
|
753
|
+
|
|
754
|
+
<entry>
|
|
755
|
+
<title>Scripted Server Setup</title>
|
|
756
|
+
<link href="http://gravitext.com/2008/11/09/scripted-server-setup.html"/>
|
|
757
|
+
<updated>2008-11-09T00:00:00-08:00</updated>
|
|
758
|
+
<id>id:/2008/11/09/scripted-server-setup</id>
|
|
759
|
+
<content type="html"><p>I&#8217;ve made several posts on Hashdot and the RJack gems, but have yet to post on the <em>why?</em> or <em>what?</em> of it. This post should fill that gap by way of a non-trivial example.</p>
|
|
760
|
+
|
|
761
|
+
<p>At <a href='http://www.evri.com/'>Evri</a> I&#8217;m responsible for content acquisition and text extraction. We&#8217;re parsing volumes of news and general web content, and separating the <em>kernels</em> of semantically rich text from the <em>chaff</em> i.e. site navigational boxes, advertisements, etc. Normally this content is processed &#8220;offline&#8221;, but I had reason recently to package it as a HTTP-XML service for transforming web content in real time. As the content processing system already existed as a mixture of java (for the heavy lifting) and ruby, I went looking for a solution for service construction that would be simple, flexible enough to reuse the existing processing code and setup, and well performing.</p>
|
|
762
|
+
|
|
763
|
+
<p>As the XML production (output) code was already in java, and because I wanted the performance advantages of the JVM&#8217;s native threads, I decided to write two java servlets: one to simply test obtaining the web content by URL as a proxy (using thread-safe and connection pooling commons-httpclient), and the second extending the first to actually transform the content HTML into an internal XML representation. So now I have two servlets that need to be setup in an HTTP server, I want them to share a common HTTPClient setup, and the transforming servlet also needs a reference to a custom setup content processor. How was I going to wire all this together?</p>
|
|
764
|
+
|
|
765
|
+
<h2 id='standards_frameworks_and_xml_configuration'>Standards, Frameworks, and XML configuration</h2>
|
|
766
|
+
|
|
767
|
+
<p>Classical java would have me create a Web Application Archive (WAR) which contains the compiled Servlets plus a web.xml Deployment Descripter providing the mapping of servlets to URIs. Since the container will be constructing the servlets, access to the HTTPClient and processor must be arranged by non-direct means, presumably through a ServletContext which allows Object-type attributes. Back in the day. with earlier Servlet specs, this would have been done with some form of initialization servlet. Now its possible to register a context listener which does the same. I could write all the java classes and XML myself, or depend on something like the Spring Framework&#8217;s <a href='http://static.springframework.org/spring/docs/2.5.x/reference/beans.html#context-create'>ContextLoaderListener</a>. In either case it will be a challenge to keep the ruby based content processor setup without resorting to an additionally complex integration, with java setting up a jruby interpreter (via BSF or otherwise). Note also that none of this complexity helps to address such matters as how to get this all installed in a production environment, how the service will be started, monitored, etc. The two servlets are themselves pretty simple. How might we shed some of the surrounding baggage?</p>
|
|
768
|
+
|
|
769
|
+
<h2 id='uninversion_of_control'>Un-Inversion of Control</h2>
|
|
770
|
+
|
|
771
|
+
<p>Enter the beautifully designed <a href='http://www.mortbay.org/jetty/'>Jetty Web Server</a> which offers full compliance with all of the committee Servlet/JSP standards, but also offers direct programmatic setup under the guise of <a href='http://docs.codehaus.org/display/JETTY/Embedding+Jetty'>embedding</a>. It was quick work to prototype the same embedded HTTP server setup in jruby. I later took up authoring a generic jetty gem packaging and setup façade as a side project.</p>
|
|
772
|
+
|
|
773
|
+
<p>As I&#8217;ve setup a lot of classic java web applications in the past, reviewing the script for this post gave me an eery feeling that somehow there must be a few more XML configuration files or bash scripts somewhere that were needed to make this all work. That&#8217;s not the case here. Lets quickly review the prerequisites:</p>
|
|
774
|
+
|
|
775
|
+
<ol>
|
|
776
|
+
<li>Java JDK or JRE</li>
|
|
777
|
+
|
|
778
|
+
<li><a href='/2008/10/27/jruby-setup-with-hashdot.html'>Hashdot installed</a> with JRuby in /opt.</li>
|
|
779
|
+
|
|
780
|
+
<li>RJack <a href='http://rjack.rubyforge.org/logback/'>logback</a> and <a href='http://rjack.rubyforge.org/jetty/'>jetty</a> gems: <pre>
|
|
781
|
+
% jgem install logback jetty
|
|
782
|
+
</pre></li>
|
|
783
|
+
|
|
784
|
+
<li>A commons-httpclient gem which has not yet been published.</li>
|
|
785
|
+
|
|
786
|
+
<li>The Evri proprietary content_processor and proxy_transformer ruby Modules which, sorry to say, I won&#8217;t be sharing.</li>
|
|
787
|
+
</ol>
|
|
788
|
+
|
|
789
|
+
<p>(Note: I&#8217;ve also changed a few of the setup details of these to avoid giving too many details.)</p>
|
|
790
|
+
|
|
791
|
+
<h2 id='dependency_injection'>Dependency Injection</h2>
|
|
792
|
+
|
|
793
|
+
<p>Fancy terminology aye? The HTTP Client and content processor are simply created as local variables <code>http</code> (l28) and <code>processor</code> (l33) in the script, and passed directly on construction of the ProxyServlet and TransformServlet (l46-7). Isn&#8217;t that refreshingly direct? The <code>Jetty::ServerFactory</code> provided in the jetty gem is used to <code>create</code> a the complete HTTP server, with a healthy dose of sensible defaults. The <code>set_context_servlets</code> call (l35) uses a ruby hash for mapping paths of the root <code>'/'</code> context to these servlets. There is no Web Application, because there is no need for one. (However the jetty gem supports full webapps as well when needed.)</p>
|
|
794
|
+
|
|
795
|
+
<h2 id='lifecycle_events'>Lifecycle Events</h2>
|
|
796
|
+
|
|
797
|
+
<p>More fancy terminology. We have some shutdown work we&#8217;d like to do after the server port is closed. Once Jetty is started via <code>server.start</code> (l51), the <code>server.join</code> call (l52) will block until the server exits, for example via SIGTERM. We then have an opportunity to cleanup, for example, gracefully closing the HTTP client and any kept-alive connections (l55). Other example might include writing out some final reporting information.</p>
|
|
798
|
+
|
|
799
|
+
<h2 id='code_or_configuration'>Code or Configuration?</h2>
|
|
800
|
+
|
|
801
|
+
<p>The original Jetty <a href='http://docs.codehaus.org/display/JETTY/Embedding+Jetty'>embedding</a> examples are written in Java and thus need to be compiled. Any settings such as the HTTP port number are thus <em>hard coded</em> in the jetty examples. The distinction is a good bit less clear in the case of this ruby script. The script itself is almost entirely in a declarative style, save perhaps for the final server start, join and cleanup (l51-55). It also uses some local variables, but this isn&#8217;t unlike named object references in Jetty or Spring XML configuration. It would certainly be easy enough to break a final set of configuration elements out into a separate XML configuration file (or YAML, or even java proprieties), but what is the gain? Is it not easy enough to change any necessary settings in this script directly? Is there not a clear advantage to having all aspects of the service setup in a single unified file, rather than spread out into multiple XML configuration files and the typically required bash wrapper script?</p>
|
|
802
|
+
|
|
803
|
+
<h2 id='where_is_the_ruby_at_runtime'>Where is the Ruby at Runtime?</h2>
|
|
804
|
+
|
|
805
|
+
<p>While ruby is used for setup, wiring, dependency injection, or whatever you&#8217;d like to call it, ruby is interestingly absent as an explicit reference in any of the Java code, where there is not a a single JRuby or BSF dependency reference. Furthermore, the <code>ProxyServlet</code> never utilizes a single line of ruby code in its runtime execution. We&#8217;ve effectively made good use of ruby to setup this servlet but take on absolutely zero ruby performance penalties during runtime.</p>
|
|
806
|
+
|
|
807
|
+
<p>The content processor by comparison, as used by the <code>TransformServlet</code> uses a pipeline or chain-of-responsibility pattern with some links in the chain implemented in ruby. The ruby links are non-performance intensive processing logic steps, and don&#8217;t measurably detract from the performance. Note that if they did become a performance bottleneck, they could be rewritten in pure-java without needing to modify or recompile the <code>TransformServlet</code>. Thus by putting ruby in control of service boot-strapping, ruby can easily be injected into the service and mixed with java components where advantageous and without any complexities incurred on the java side. Some additional example code for this kind of ruby injection can be found in the <a href='http://rjack.rubyforge.org/jetty/RJack/Jetty/ServerFactory.html'>Jetty::ServerFactory</a> rdoc, where for example, I demonstrate implementing a Servlet in ruby and passing it to Jetty.</p>
|
|
808
|
+
|
|
809
|
+
<h2 id='logging_daemonizing_and_other_oft_ignored_subtleties'>Logging, Daemonizing, and other oft ignored Subtleties</h2>
|
|
810
|
+
|
|
811
|
+
<p>The Hashdot launcher and hashdot properties are used to arrange for the service to daemonize, redirect STDOUT/STDERR to a log file, and set a Java heap size. This will make it simple to add the service to inittab or other UNIX process monitor. The logback gem is loaded (l10,14) which loads the slf4j gem as a dependency. Jetty in turn detects the presence of SLF4J and will use it for logging. SLF4J via Logback output to STDOUT at INFO level is arranged pragmatically (l22-5), thus avoiding a separate logback.xml. Below is an example of log output using jetty-service script provided with the jetty gem itself (and with the logback gem available).</p>
|
|
812
|
+
<pre>
|
|
813
|
+
1 INFO org.mortbay.log - Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog
|
|
814
|
+
58 INFO org.mortbay.log - jetty-6.1.12
|
|
815
|
+
280 INFO org.mortbay.log - NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
|
|
816
|
+
388 INFO org.mortbay.log - Started SelectChannelConnector@0.0.0.0:38225
|
|
817
|
+
388 INFO jetty-service - Listening on port: 38225
|
|
818
|
+
</pre>
|
|
819
|
+
<p>Its important to redirect STDOUT and STDERR because any fatal errors like uncaught runtime exceptions or java crash dumps will end up here. Its also convenient when debugging a problem to be able to <code>pkill -HUP proxy-transform-service</code> and get all of the thread stack dumps into the log. Finally, its quite common for an application which brings in many open-source components to have cases where these components write to STDOUT or STDERR under <em>unusual</em> circumstances. An approach that redirects STDOUT/STDERR to <code>/dev/null</code> or an obscure alternative log file will commonly result in this information being lost. Thus I find a certain old school elegance in coalescing all the logging output to STDOUT, using Hashdot to redirect this to a file at the system (not just java) level, and using an external tool like logrotate to provide log rotation.</p>
|
|
820
|
+
|
|
821
|
+
<h2 id='proxytransformservice_script'>proxy-transform-service script</h2>
|
|
822
|
+
<div class='highlight'><pre><code class='ruby'><span class='c1'>#!/opt/bin/jruby</span>
|
|
823
|
+
<span class='c1'>#. hashdot.profile += daemon</span>
|
|
824
|
+
<span class='c1'>#. hashdot.vm.options += -Xmx256m</span>
|
|
825
|
+
<span class='c1'>#. hashdot.io_redirect.file = /var/log/proxy-transformer.log</span>
|
|
826
|
+
|
|
827
|
+
<span class='nb'>require</span> <span class='s1'>&#39;rubygems&#39;</span>
|
|
828
|
+
|
|
829
|
+
<span class='n'>gem</span> <span class='s1'>&#39;logback&#39;</span><span class='p'>,</span> <span class='s1'>&#39;~&gt; 0.9&#39;</span>
|
|
830
|
+
<span class='n'>gem</span> <span class='s1'>&#39;jetty&#39;</span><span class='p'>,</span> <span class='s1'>&#39;~&gt; 6.1&#39;</span>
|
|
831
|
+
<span class='n'>gem</span> <span class='s1'>&#39;commons-httpclient&#39;</span><span class='p'>,</span> <span class='s1'>&#39;~&gt; 3.1&#39;</span>
|
|
832
|
+
|
|
833
|
+
<span class='nb'>require</span> <span class='s1'>&#39;logback&#39;</span>
|
|
834
|
+
<span class='nb'>require</span> <span class='s1'>&#39;jetty&#39;</span>
|
|
835
|
+
<span class='nb'>require</span> <span class='s1'>&#39;commons-httpclient&#39;</span>
|
|
836
|
+
|
|
837
|
+
<span class='nb'>require</span> <span class='s1'>&#39;content_processor&#39;</span>
|
|
838
|
+
<span class='nb'>require</span> <span class='s1'>&#39;proxy_transformer&#39;</span>
|
|
839
|
+
|
|
840
|
+
<span class='c1'># Logback</span>
|
|
841
|
+
<span class='no'>Logback</span><span class='o'>.</span><span class='n'>configure</span> <span class='k'>do</span>
|
|
842
|
+
<span class='no'>Logback</span><span class='o'>.</span><span class='n'>root</span><span class='o'>.</span><span class='n'>add_appender</span><span class='p'>(</span> <span class='no'>Logback</span><span class='o'>::</span><span class='no'>ConsoleAppender</span><span class='o'>.</span><span class='n'>new</span> <span class='p'>)</span>
|
|
843
|
+
<span class='no'>Logback</span><span class='o'>.</span><span class='n'>root</span><span class='o'>.</span><span class='n'>level</span> <span class='o'>=</span> <span class='no'>Logback</span><span class='o'>::</span><span class='no'>INFO</span>
|
|
844
|
+
<span class='k'>end</span>
|
|
845
|
+
|
|
846
|
+
<span class='c1'># HTTP Client</span>
|
|
847
|
+
<span class='n'>http</span> <span class='o'>=</span> <span class='no'>CommonsHttpClient</span><span class='o'>::</span><span class='no'>Facade</span><span class='o'>.</span><span class='n'>new</span>
|
|
848
|
+
<span class='n'>http</span><span class='o'>.</span><span class='n'>max_total_connections</span> <span class='o'>=</span> <span class='mi'>100</span>
|
|
849
|
+
<span class='n'>http</span><span class='o'>.</span><span class='n'>connection_timeout</span> <span class='o'>=</span> <span class='mi'>1500</span> <span class='c1'>#ms</span>
|
|
850
|
+
|
|
851
|
+
<span class='c1'># Content Processor</span>
|
|
852
|
+
<span class='n'>processor</span> <span class='o'>=</span> <span class='no'>ContentProcessor</span><span class='o'>.</span><span class='n'>new</span> <span class='k'>do</span> <span class='o'>|</span><span class='nb'>p</span><span class='o'>|</span>
|
|
853
|
+
<span class='nb'>p</span><span class='o'>.</span><span class='n'>centroid_weight</span> <span class='o'>=</span> <span class='mi'>0</span><span class='o'>.</span><span class='mi'>3</span>
|
|
854
|
+
<span class='nb'>p</span><span class='o'>.</span><span class='n'>filter_threads</span> <span class='o'>=</span> <span class='mi'>1</span>
|
|
855
|
+
<span class='k'>end</span>
|
|
856
|
+
|
|
857
|
+
<span class='c1'># HTTP Server</span>
|
|
858
|
+
<span class='n'>factory</span> <span class='o'>=</span> <span class='no'>Jetty</span><span class='o'>::</span><span class='no'>ServerFactory</span><span class='o'>.</span><span class='n'>new</span>
|
|
859
|
+
<span class='n'>factory</span><span class='o'>.</span><span class='n'>max_threads</span> <span class='o'>=</span> <span class='mi'>50</span>
|
|
860
|
+
<span class='n'>factory</span><span class='o'>.</span><span class='n'>port</span> <span class='o'>=</span> <span class='mi'>8080</span>
|
|
861
|
+
<span class='n'>factory</span><span class='o'>.</span><span class='n'>max_idle_time_ms</span> <span class='o'>=</span> <span class='mi'>10000</span>
|
|
862
|
+
|
|
863
|
+
<span class='kp'>include</span> <span class='no'>ProxyTransformer</span>
|
|
864
|
+
<span class='n'>factory</span><span class='o'>.</span><span class='n'>set_context_servlets</span><span class='p'>(</span> <span class='s1'>&#39;/&#39;</span><span class='p'>,</span>
|
|
865
|
+
<span class='p'>{</span> <span class='s1'>&#39;/proxy&#39;</span> <span class='o'>=&gt;</span> <span class='no'>ProxyServlet</span><span class='o'>.</span><span class='n'>new</span><span class='p'>(</span> <span class='n'>http</span><span class='o'>.</span><span class='n'>client</span> <span class='p'>),</span>
|
|
866
|
+
<span class='s1'>&#39;/transform&#39;</span> <span class='o'>=&gt;</span> <span class='no'>TransformServlet</span><span class='o'>.</span><span class='n'>new</span><span class='p'>(</span> <span class='n'>http</span><span class='o'>.</span><span class='n'>client</span><span class='p'>,</span> <span class='n'>processor</span> <span class='p'>)</span> <span class='p'>}</span>
|
|
867
|
+
|
|
868
|
+
<span class='n'>server</span> <span class='o'>=</span> <span class='n'>factory</span><span class='o'>.</span><span class='n'>create</span>
|
|
869
|
+
|
|
870
|
+
<span class='n'>server</span><span class='o'>.</span><span class='n'>start</span>
|
|
871
|
+
<span class='n'>server</span><span class='o'>.</span><span class='n'>join</span>
|
|
872
|
+
|
|
873
|
+
<span class='c1'># Shutdown, cleanup</span>
|
|
874
|
+
<span class='n'>http</span><span class='o'>.</span><span class='n'>shutdown</span>
|
|
875
|
+
</code></pre>
|
|
876
|
+
</div></content>
|
|
877
|
+
</entry>
|
|
878
|
+
|
|
879
|
+
<entry>
|
|
880
|
+
<title>RJack Updates</title>
|
|
881
|
+
<link href="http://gravitext.com/2008/11/09/rjack-updates.html"/>
|
|
882
|
+
<updated>2008-11-09T00:00:00-08:00</updated>
|
|
883
|
+
<id>id:/2008/11/09/rjack-updates</id>
|
|
884
|
+
<content type="html"><p>Two gem updates released to the <a href='http://rjack.rubyforge.org/'>RJack</a> project on RubyForge:</p>
|
|
885
|
+
|
|
886
|
+
<ul>
|
|
887
|
+
<li><a href='http://rjack.rubyforge.org/jetty/'>jetty</a></li>
|
|
888
|
+
|
|
889
|
+
<li><a href='http://rjack.rubyforge.org/jetty-jsp/'>jetty-jsp</a></li>
|
|
890
|
+
</ul>
|
|
891
|
+
|
|
892
|
+
<p>based on the latest <a href='http://www.mortbay.org/jetty/'>Jetty Web Server</a> 6.1.12.</p></content>
|
|
893
|
+
</entry>
|
|
894
|
+
|
|
895
|
+
<entry>
|
|
896
|
+
<title>Hashdot 1.3 Dev</title>
|
|
897
|
+
<link href="http://gravitext.com/2008/11/07/hashdot-1.2-dev.html"/>
|
|
898
|
+
<updated>2008-11-07T00:00:00-08:00</updated>
|
|
899
|
+
<id>id:/2008/11/07/hashdot-1.2-dev</id>
|
|
900
|
+
<content type="html"><p>A quick status update on Hashdot:</p>
|
|
901
|
+
|
|
902
|
+
<p>To better facilitate merging changes with a colleague who is investigating the Mac OS X port, I&#8217;ve enabled the SVN repository for Hashdot on SourceForge, imported the 1.1 source, and then committed a set of changes for Hashdot 1.2 I had previously made locally. The changes are listed below as a preview for interested parties:</p>
|
|
903
|
+
<pre>
|
|
904
|
+
* Added change_log starting with version 1.2
|
|
905
|
+
|
|
906
|
+
svn merge -r 464:HEAD
|
|
907
|
+
|
|
908
|
+
* Added HUP signal handler to reopen STDOUT/STDERR for logrotate
|
|
909
|
+
compatibility.
|
|
910
|
+
* Fixed hashbang.
|
|
911
|
+
* Added explicit int casts to avoid x64 warnings
|
|
912
|
+
* VERSION -> 1.2
|
|
913
|
+
* Added support for hashdot.chdir, incl. test and doc.
|
|
914
|
+
</pre></content>
|
|
915
|
+
</entry>
|
|
916
|
+
|
|
917
|
+
<entry>
|
|
918
|
+
<title>RJack Updates</title>
|
|
919
|
+
<link href="http://gravitext.com/2008/11/06/rjack-updates.html"/>
|
|
920
|
+
<updated>2008-11-06T00:00:00-08:00</updated>
|
|
921
|
+
<id>id:/2008/11/06/rjack-updates</id>
|
|
922
|
+
<content type="html"><p>On Monday, I managed to complete the release of two new gems and two gem updates, all part of the <a href='http://rjack.rubyforge.org/'>RJack</a> project hosted on RubyForge. As this is my first post on RJack, here is a an introduction without quoting the linked project summary, and some details on jar in gem packaging.</p>
|
|
923
|
+
|
|
924
|
+
<p>RJack provides gem packaging of various broadly used Java components for use with JRuby. Included in this release set are upstream updates to the <a href='http://rjack.rubyforge.org/slf4j/'>slf4j</a> and <a href='http://rjack.rubyforge.org/logback/'>logback</a> logging system, and the new gems <a href='http://rjack.rubyforge.org/jetty/'>jetty</a> and <a href='http://rjack.rubyforge.org/jetty-jsp/'>jetty-jsp</a>, which package the Jetty Web Server. A guiding design principal is that each of these gems may be used simply and solely to deliver java package dependencies to a consuming ruby/java application. However, three of the four gems in the current set also provide either façades for setup or adapters for direct use from ruby code. For these inclusions, care is given to avoid incurring any additional external dependencies.</p>
|
|
925
|
+
|
|
926
|
+
<h2 id='why_gems'>Why gems?</h2>
|
|
927
|
+
|
|
928
|
+
<p>The <a href='http://www.rubygems.org/'>RubyGems</a> package management system offers several advantages for working with server application components in both development and production environments, including a straight forward command line interface, multiple remote and local gem repositories, versioned dependency management, support for multiple versions of the same component, and support for installing command-line accessable executables. While in common use for pure and C-extended ruby components, JRuby provides an opportunity to similarly utilize gems for java-extended ruby or even near-pure java components. The nearest alternative from the java comunity would be Maven and its derivatives, but these are far more appropriate for handling java dependencies at compile time than for production deployment.</p>
|
|
929
|
+
|
|
930
|
+
<p>Once the logical decision to deploy gems is made, the first order of business is determining how to divide the various components and their java jar dependencies into gems for maximum benefit including reuse, reliability, and ease/safety of upgrade with independent release cycles. Java components are ultimately loaded by searching through an order list of jars or directories, the &#8220;classpath&#8221;. If multiple versions of the same java package are found on the classpath, the first instance wins. Problems arise if two utilized components include incompatible versions of the same package. It is therefore advantageous to break common java dependencies out into individual gems and utilize the version management offered by the rubygems system.</p>
|
|
931
|
+
|
|
932
|
+
<p>As a prime example, the SLF4J system is predicated on the application developer selecting a final logging destination by installing the appropriate logging adapter in the classpath. If however various gems with some java dependencies each in turn include their own versions of these or other logging system jars, then this logging output flexibility is lost. As SLF4J is now the best available logging interface for Java, I&#8217;ve released an independent slf4j gem, for reuse and in the hope of encouraging other java gem packagers to use the slf4j gem to resolve any java logging dependencies, in lieu of embedding alternative logging components in their gems.</p>
|
|
933
|
+
|
|
934
|
+
<h2 id='gems_package_projects'>Gems package projects</h2>
|
|
935
|
+
|
|
936
|
+
<p>Since Maven manages dependencies at the jar level, a one-to-one mapping would suggest a gem per jar. While some discussion has taken place on automating direct jar to gem packing, a significant offering has yet to be published. I&#8217;m also not convinced this is the right mapping. Consider that in ruby, the <code>require</code> method is included in normal program flow, and its common to use multiple <code>require</code> calls to selectively bring in different parts of a single gem. JRuby extends <code>require</code> to support adding a named jar file to the classpath of its custom class loader. Its therefore quite reasonable for gems to map directly to a java project which provides more than one jar. A clear example of this is the RJack slf4j gem. See the <a href='http://rjack.rubyforge.org/slf4j/RJack/SLF4J.html'>SLF4J module</a> documentation for <code>require</code> mappings to eight additional input/output adapter jars beyond the <code>slf4j-api-version.jar</code> loaded with the base <code>require &#39;slf4j&#39;</code>. There will be other cases where a single jar is best packaged as a gem, but its clearly not necessary to do so when a project releases multiple jar&#8217;s in version lock-step.</p>
|
|
937
|
+
|
|
938
|
+
<p>A counter argument for finer grain gems is the case where, as with the jetty-jsp gem, large jars are included and some of these are commonly not needed. The jetty gem contains the core server components and servlet-api (4 gems) at 0.7M where as the jetty-jsp gem adds JSP support (4 more gems) for an additional 5.2M. There are also cases where external project jar dependencies are added to a gem. The argument for this would be that it is unlikely that a particular dependency will be used in another context. Of course this could be a mistake, but its one that can be fixed later with a patch release, breaking the dependency out into its own gem.</p>
|
|
939
|
+
|
|
940
|
+
<h2 id='other_considerations'>Other considerations</h2>
|
|
941
|
+
|
|
942
|
+
<p>Finally its worth noting that these gems defer to the upstream java project version numbers, adding a single gem package release number to the end of the upstream version. This puts emphasis on version compatibility in upstream project upgrades and will encourage timing any significant ruby layer changes with upstream major releases.</p>
|
|
943
|
+
|
|
944
|
+
<p>Along similar lines, each of these gems is released under identical license terms to the upstream project, thereby simplifying license considerations for consuming applications.</p></content>
|
|
945
|
+
</entry>
|
|
946
|
+
|
|
947
|
+
<entry>
|
|
948
|
+
<title>JRuby setup with Hashdot</title>
|
|
949
|
+
<link href="http://gravitext.com/2008/10/27/jruby-setup-with-hashdot.html"/>
|
|
950
|
+
<updated>2008-10-27T00:00:00-07:00</updated>
|
|
951
|
+
<id>id:/2008/10/27/jruby-setup-with-hashdot</id>
|
|
952
|
+
<content type="html"><p>In <a href='/2008/10/22/introducing-hashdot.html'>Introducing Hashdot</a> I laid out various problems with using the standard &#8216;java&#8217; launcher and the &#8216;jruby&#8217; wrapper bash script provided with the JRuby distribution. In this post, I&#8217;m going to describe in detail an alternative installation layout and setup of JRuby using Hashdot to best advantage. Some goals for this setup:</p>
|
|
953
|
+
|
|
954
|
+
<ol>
|
|
955
|
+
<li>
|
|
956
|
+
<p>Any and all invocations of &#8216;jruby&#8217; should be launched via hashdot to take advantage of central configuration, better process report, and hashdot script headers where appropriate.</p>
|
|
957
|
+
</li>
|
|
958
|
+
|
|
959
|
+
<li>
|
|
960
|
+
<p>No user environment dependencies. We want JRuby to function properly for any user on the system, without requiring each user to maintain environment variables, etc. in their own profile.</p>
|
|
961
|
+
</li>
|
|
962
|
+
|
|
963
|
+
<li>
|
|
964
|
+
<p>A separate gem repository for JRuby, independent of the versioned JRuby distribution. This will enable JRuby upgrades without needing to rebuild or copy to a new gem repository.</p>
|
|
965
|
+
</li>
|
|
966
|
+
|
|
967
|
+
<li>
|
|
968
|
+
<p>MRI-ruby and JRuby need to coexist. This means &#8216;ruby&#8217; launches MRI and &#8216;jruby&#8217; launches JRuby. Such utilities as &#8216;ri&#8217;, &#8216;gem&#8217;, and &#8216;rake&#8217; should invoke MRI ruby, while we also want convenient access to the same via JRuby as &#8216;jri&#8217;, &#8216;jgem&#8217;, &#8216;jrake&#8217;, etc.</p>
|
|
969
|
+
</li>
|
|
970
|
+
|
|
971
|
+
<li>
|
|
972
|
+
<p>The current latest &#8216;jruby&#8217; interpreter, &#8216;jgem&#8217;, &#8216;jri&#8217;, &#8216;jrake&#8217; and any bin scripts installed in the jruby gem repository should be available on a system standard PATH.</p>
|
|
973
|
+
</li>
|
|
974
|
+
</ol>
|
|
975
|
+
|
|
976
|
+
<h2 id='layout_in_opt'>Layout in /opt</h2>
|
|
977
|
+
|
|
978
|
+
<p>The &#8216;/opt&#8217; directory appears most appropriate for this kind of system wide but custom setup. Alternatively &#8216;/usr/local&#8217; could be used. The following structure is recommended and will be described in detail below.</p>
|
|
979
|
+
<pre>
|
|
980
|
+
/opt
|
|
981
|
+
|-- bin
|
|
982
|
+
| |-- hashdot
|
|
983
|
+
| |-- jgem
|
|
984
|
+
| |-- jirb -> ../dist/jruby/bin/jirb
|
|
985
|
+
| |-- jrake -> ../jruby/gems/bin/rake
|
|
986
|
+
| |-- jri -> ../dist/jruby/bin/ri
|
|
987
|
+
| |-- jruby -> hashdot
|
|
988
|
+
| `-- jruby-shortlived -> hashdot
|
|
989
|
+
|-- dist
|
|
990
|
+
| |-- jdk_sun_1.6.0_05_x32
|
|
991
|
+
| |-- jdk_sun_1.6.0_07_x32
|
|
992
|
+
| |-- jruby -> jruby-1.1.4
|
|
993
|
+
| |-- jruby-1.1.3
|
|
994
|
+
| `-- jruby-1.1.4
|
|
995
|
+
|-- hashdot
|
|
996
|
+
| `-- profiles
|
|
997
|
+
`-- jruby
|
|
998
|
+
`-- gems
|
|
999
|
+
</pre>
|
|
1000
|
+
<h2 id='create_the_opt_skeleton'>Create the /opt skeleton</h2>
|
|
1001
|
+
|
|
1002
|
+
<p>As root:</p>
|
|
1003
|
+
<pre>
|
|
1004
|
+
% mkdir /opt /opt/bin /opt/dist /opt/jruby
|
|
1005
|
+
</pre>
|
|
1006
|
+
<h2 id='configure_make_install_hashdot'>Configure, make, install Hashdot</h2>
|
|
1007
|
+
|
|
1008
|
+
<p>Configuring hashdot currently involves editing the Makefile including with the source distribution. Set the following Makefile variables:</p>
|
|
1009
|
+
<pre>
|
|
1010
|
+
INSTALL_BIN=/opt/bin
|
|
1011
|
+
PROFILE_DIR=/opt/hashdot/profiles
|
|
1012
|
+
</pre>
|
|
1013
|
+
<p>Hashdot may then be built or rebuilt via &#8216;make clean all&#8217;. You might want to then customize the various profiles before running &#8216;make install&#8217; as root. You can also customize them in /opt/hashdot/profiles after install.</p>
|
|
1014
|
+
|
|
1015
|
+
<h2 id='install_distributions'>Install distributions</h2>
|
|
1016
|
+
|
|
1017
|
+
<p>As root, unpack the Java JDK (or JRE) and JRuby under /opt/dist. Note I prefer to use a more descriptive name including the full java version number. Since we also reference some jruby distro provided bin scripts, symlink the latest jruby-1.1.4 to jruby. As shown, its possible for multiple versions of java/jruby to coexist for testing purposes.</p>
|
|
1018
|
+
|
|
1019
|
+
<h2 id='create_optjrubygems_repository'>Create /opt/jruby/gems repository</h2>
|
|
1020
|
+
|
|
1021
|
+
<p>To create a starting gems repository, as root:</p>
|
|
1022
|
+
<pre>
|
|
1023
|
+
% cp -a /opt/dist/jruby-1.1.4/lib/ruby/gems /opt/dist/jruby/gems
|
|
1024
|
+
</pre>
|
|
1025
|
+
<p>To make sure this repository is used for all jruby invocations, GEM_HOME and GEM_PATH environment variables are set via Hashdot in the jruby.hdp profile (see below.) This has an important advantage over setting in a user profile: The MRI ruby gem home/path will not be effected and can be kept separate.</p>
|
|
1026
|
+
|
|
1027
|
+
<h2 id='create_optbin_symlinks'>Create /opt/bin symlinks</h2>
|
|
1028
|
+
|
|
1029
|
+
<p>The above list directory tree for /opt/bin should be self explanatory. We create &#8216;jirb&#8217; and &#8216;jri&#8217; symlinks to dist/jruby/bin in order to make these easy to use. Note however that &#8216;jruby -S ri&#8217; will also still work for these. Similarly we create jrake as a symlink to gems/bin/rake, since in a development environment we will likely want MRI and jruby versions both available. Finally, we choose to customize &#8216;jgem&#8217; by copying and changing the hashbang to:</p>
|
|
1030
|
+
<pre>
|
|
1031
|
+
#!/opt/bin/jruby-shortlived
|
|
1032
|
+
</pre>
|
|
1033
|
+
<p>Which gives a bit of a performance advantage in common usage. Note that with the given hashdot profiles, the server vm is default, which is almost always the best choice for any long lived application.</p>
|
|
1034
|
+
|
|
1035
|
+
<h2 id='system_path'>System PATH</h2>
|
|
1036
|
+
|
|
1037
|
+
<p>Adjusting the standard system PATH is Linux (or UNIX) distribution dependent and might involve modifying /etc/profile or an equivalent. In any case, /opt/bin should be found early in the path (since we only include &#8216;j*&#8217; commands here), and /opt/jruby/gems/bin should be added late in the path to get additional jruby gem bin scripts available, but without overriding any MRI ruby gem equivalents. For example:</p>
|
|
1038
|
+
<pre>
|
|
1039
|
+
/opt/bin:/usr/local/bin:/bin:/usr/sbin:/usr/bin:/opt/jruby/gems/bin
|
|
1040
|
+
</pre>
|
|
1041
|
+
<p>In the case where you need both the MRI and jruby gem bin script versions to coexist in the PATH, create a &#8216;j*&#8217; symlink in /opt/bin for the jruby version.</p>
|
|
1042
|
+
|
|
1043
|
+
<h2 id='hashdot_profiles'>Hashdot profiles</h2>
|
|
1044
|
+
|
|
1045
|
+
<p>The customized profiles I use on my 32bit development environment are given below.</p>
|
|
1046
|
+
|
|
1047
|
+
<p>default.hdp:</p>
|
|
1048
|
+
<div class='highlight'><pre><code class='bash'><span class='c'># HashDot default launch profile</span>
|
|
1049
|
+
<span class='c'># This profile is always read before any other profiles or hashdot</span>
|
|
1050
|
+
<span class='c'># properties</span>
|
|
1051
|
+
|
|
1052
|
+
<span class='c'>## Setup the default JVM</span>
|
|
1053
|
+
|
|
1054
|
+
<span class='c'># JVM install directory</span>
|
|
1055
|
+
hashdot.vm.home <span class='o'>=</span> /opt/dist/jdk_sun_1.6.0_07_x32
|
|
1056
|
+
|
|
1057
|
+
<span class='c'># Use &quot;amd64&quot; instead of i386 for linux 64bit platforms</span>
|
|
1058
|
+
hashdot.vm.arch <span class='o'>=</span> i386
|
|
1059
|
+
|
|
1060
|
+
<span class='c'># Use server by default (see alternative in shortlived.hdp)</span>
|
|
1061
|
+
hashdot.vm.mode <span class='o'>=</span> server
|
|
1062
|
+
|
|
1063
|
+
hashdot.vm.libbase <span class='o'>=</span> <span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.home</span><span class='k'>}</span>/jre/lib/<span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.arch</span><span class='k'>}</span>
|
|
1064
|
+
|
|
1065
|
+
<span class='c'># Setup libpath to find libjvm.so and dependencies.</span>
|
|
1066
|
+
hashdot.vm.libpath <span class='o'>=</span> <span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.libbase</span><span class='k'>}</span>
|
|
1067
|
+
hashdot.vm.libpath +<span class='o'>=</span> <span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.libbase</span><span class='k'>}</span>/<span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.mode</span><span class='k'>}</span>
|
|
1068
|
+
|
|
1069
|
+
<span class='c'># TZ often needed for Java</span>
|
|
1070
|
+
hashdot.env.TZ <span class='o'>=</span> America/Los_Angeles
|
|
1071
|
+
</code></pre>
|
|
1072
|
+
</div>
|
|
1073
|
+
<p>jruby.hdp:</p>
|
|
1074
|
+
<div class='highlight'><pre><code class='bash'><span class='c'># HashDot launch profile for JRuby (http://jruby.codehaus.org/)</span>
|
|
1075
|
+
|
|
1076
|
+
jruby.home <span class='o'>=</span> /opt/dist/jruby-1.1.4
|
|
1077
|
+
jruby.lib <span class='o'>=</span> <span class='k'>${</span><span class='nv'>jruby</span><span class='p'>.home</span><span class='k'>}</span>/lib
|
|
1078
|
+
jruby.script <span class='o'>=</span> jruby
|
|
1079
|
+
jruby.shell <span class='o'>=</span> /bin/sh
|
|
1080
|
+
|
|
1081
|
+
<span class='c'># Identical defaults to jruby launcher</span>
|
|
1082
|
+
<span class='c'># These can still be overridden by the individual scripts</span>
|
|
1083
|
+
hashdot.vm.options +<span class='o'>=</span> -Xmx500m -Xss1024k
|
|
1084
|
+
|
|
1085
|
+
<span class='c'># Only jruby.jar is required for typical usage (scripts can require</span>
|
|
1086
|
+
<span class='c'># bsf.jar or JIP profiler if desired).</span>
|
|
1087
|
+
hashdot.vm.options +<span class='o'>=</span> -Xbootclasspath/a:<span class='k'>${</span><span class='nv'>jruby</span><span class='p'>.lib</span><span class='k'>}</span>/jruby.jar
|
|
1088
|
+
|
|
1089
|
+
hashdot.main <span class='o'>=</span> org.jruby.Main
|
|
1090
|
+
|
|
1091
|
+
<span class='c'># Arguments following these flags are _not_ a script to scan for</span>
|
|
1092
|
+
<span class='c'># hashdot headers.</span>
|
|
1093
|
+
hashdot.parse_flags.value_args <span class='o'>=</span> -F -I -r
|
|
1094
|
+
|
|
1095
|
+
<span class='c'># Give up looking for a script header with any of these</span>
|
|
1096
|
+
hashdot.parse_flags.terminal <span class='o'>=</span> -C -e -S
|
|
1097
|
+
|
|
1098
|
+
<span class='c'># GEM PATH/HOME</span>
|
|
1099
|
+
hashdot.env.GEM_HOME <span class='o'>=</span> /opt/jruby/gems
|
|
1100
|
+
hashdot.env.GEM_PATH <span class='o'>=</span> <span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.env.GEM_HOME</span><span class='k'>}</span>
|
|
1101
|
+
</code></pre>
|
|
1102
|
+
</div>
|
|
1103
|
+
<p>shortlived.hdp:</p>
|
|
1104
|
+
<div class='highlight'><pre><code class='bash'><span class='c'># HashDot profile for short lived processes.</span>
|
|
1105
|
+
<span class='c'># Overrides default.hdp to use the client VM.</span>
|
|
1106
|
+
|
|
1107
|
+
<span class='c'># Note: for current amd64 java distros, this entire profile should be</span>
|
|
1108
|
+
<span class='c'># commented out to disable it, since these only include a server VM</span>
|
|
1109
|
+
|
|
1110
|
+
hashdot.vm.mode <span class='o'>=</span> client
|
|
1111
|
+
|
|
1112
|
+
hashdot.vm.libpath <span class='o'>=</span> <span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.libbase</span><span class='k'>}</span>
|
|
1113
|
+
hashdot.vm.libpath +<span class='o'>=</span> <span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.libbase</span><span class='k'>}</span>/<span class='k'>${</span><span class='nv'>hashdot</span><span class='p'>.vm.mode</span><span class='k'>}</span>
|
|
1114
|
+
</code></pre>
|
|
1115
|
+
</div>
|
|
1116
|
+
<p>jruby-shortlived.hdp:</p>
|
|
1117
|
+
<div class='highlight'><pre><code class='bash'><span class='c'># HashDot profile for shortlived jruby scripts.</span>
|
|
1118
|
+
|
|
1119
|
+
<span class='c'># Extends shortlived (client VM) and (jruby) profiles with further</span>
|
|
1120
|
+
<span class='c'># startup time tweaks.</span>
|
|
1121
|
+
hashdot.profile <span class='o'>=</span> shortlived jruby
|
|
1122
|
+
|
|
1123
|
+
<span class='c'># JMX setup slows startup and wont be used</span>
|
|
1124
|
+
jruby.management.enabled<span class='o'>=</span><span class='nb'>false</span>
|
|
1125
|
+
</code></pre>
|
|
1126
|
+
</div></content>
|
|
1127
|
+
</entry>
|
|
1128
|
+
|
|
1129
|
+
<entry>
|
|
1130
|
+
<title>Introducing Hashdot</title>
|
|
1131
|
+
<link href="http://gravitext.com/2008/10/22/introducing-hashdot.html"/>
|
|
1132
|
+
<updated>2008-10-22T00:00:00-07:00</updated>
|
|
1133
|
+
<id>id:/2008/10/22/introducing-hashdot</id>
|
|
1134
|
+
<content type="html"><p>From the summary of <a href='http://hashdot.sourceforge.net/'>Hashdot</a> as published on SourceForge:</p>
|
|
1135
|
+
|
|
1136
|
+
<blockquote>
|
|
1137
|
+
<p>Hashdot elevates Java-platform script interpreters to first class status on Unix-like operating systems. It provides a script aware replacement to the stock &#8216;java&#8217; launcher, and thus avoids numerous issues in using the &#8216;java&#8217; launcher to bootstrap a script interpreter. All relevant interpreter and JVM options (i.e: Java heap size) may be specified directly in a script header and/or via system profiles, without resorting to environment variables, command line arguments, and the additional wrapper shell scripts needed to maintain them.</p>
|
|
1138
|
+
</blockquote>
|
|
1139
|
+
|
|
1140
|
+
<p>There are some interesting aspects to this work. Firstly, Hashdot really needed to be written in C for tight UNIX integration (fork(), dlopen(), prctl(), setsid(), etc.) Though I had developed in C and C++ for many years, it&#8217;s been a while, and I was surprised at how cumbersome I now found it.</p>
|
|
1141
|
+
|
|
1142
|
+
<p>Its a bit of of an odd predicament to be forced to write C, essentially as <em>glue code</em> between UNIX, java, and a Java-platform script interpreter. When I took on my first large scale java project in the late 90&#8217;s (Java 1.1, Netscape Search federation and UI) I reserved the right to do any heavy lifting that was required in C++. Surprisingly, it didn&#8217;t come to that then, and only rarely has in the last 10 years. I <em>have</em> spent some time optimizing inner loops in a style of java that looks all too close to C. So here I am writing actual C again, but as glue code rather than for performance reasons, and simply to gain some modest comforts while writing ruby and java for the JVM!</p>
|
|
1143
|
+
|
|
1144
|
+
<p>Its doubly odd to be writing C glue code in order to make decidedly incremental and cosmetic improvements to a UNIX, Java, and JRuby stack. What if however, the aggregate total of current stack integration warts, annoyances, and complexities might otherwise retard broad acceptance of that stack by any of the UNIX, Java, or Ruby camps? Maybe then it would be worth diving in.</p>
|
|
1145
|
+
|
|
1146
|
+
<h2 id='the_stock_java_launcher'>The Stock &#8216;java&#8217; Launcher</h2>
|
|
1147
|
+
|
|
1148
|
+
<p>Understanding the <em>why</em> of Hashdot must begin with a problem statement, and the problems begin with the stock &#8216;java&#8217; launcher command line:</p>
|
|
1149
|
+
<pre>
|
|
1150
|
+
% java -h
|
|
1151
|
+
Usage: java [-options] class [args...]
|
|
1152
|
+
(to execute a class)
|
|
1153
|
+
or java [-options] -jar jarfile [args...]
|
|
1154
|
+
(to execute a jar file)
|
|
1155
|
+
|
|
1156
|
+
where options include:
|
|
1157
|
+
-client to select the "client" VM
|
|
1158
|
+
-server to select the "server" VM
|
|
1159
|
+
-cp &lt;class search path of directories and zip/jar files>
|
|
1160
|
+
-classpath &lt;class search path of directories and zip/jar files>
|
|
1161
|
+
A : separated list of directories, JAR archives,
|
|
1162
|
+
and ZIP archives to search for class files.
|
|
1163
|
+
-D&lt;name>=&lt;value>
|
|
1164
|
+
set a system property
|
|
1165
|
+
...
|
|
1166
|
+
</pre>
|
|
1167
|
+
<p>Like many before and after, the Java programming language started life as a toy. It must have seemed reasonable at that point, for a user starting a java program via a class&#8217;s Main method to effectively link its dependencies by passing -cp on the command line. Anyone using Java in any sort of performance intensive application would know that also setting an appropriate maximum heap size (-Xmx) and, for long lived services, using the -server VM is essentual. All told the following example &#8220;command line&#8221; is typical for a full production java service:</p>
|
|
1168
|
+
<pre>
|
|
1169
|
+
java -server -Xss512k -Xms32m -Xmx256m -XX:NewSize=16m -Xnoclassgc
|
|
1170
|
+
-Dsun.net.inetaddr.ttl=30 -Dsun.net.inetaddr.negative.ttl=0
|
|
1171
|
+
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
|
|
1172
|
+
-Djava.util.logging.config.file=/home/david/run/\
|
|
1173
|
+
apache-tomcat-5.5.20/conf/logging.properties
|
|
1174
|
+
-Djava.endorsed.dirs=/home/david/run/apache-tomcat-5.5.20/common/endorsed
|
|
1175
|
+
-classpath :/home/david/run/apache-tomcat-5.5.20/bin/bootstrap.jar\
|
|
1176
|
+
:/home/david/run/apache-tomcat-5.5.20/bin/commons-logging-api.jar
|
|
1177
|
+
-Dcatalina.base=/home/david/run/apache-tomcat-5.5.20
|
|
1178
|
+
-Dcatalina.home=/home/david/run/apache-tomcat-5.5.20
|
|
1179
|
+
-Djava.io.tmpdir=/home/david/run/apache-tomcat-5.5.20/temp
|
|
1180
|
+
org.apache.catalina.startup.Bootstrap start
|
|
1181
|
+
</pre>
|
|
1182
|
+
<p>Since its rather impractical to type this on the command line, the obvious solution is to write a shell script wrapper that exec&#8217;s it. Ironically, by not offering a better facility for process boot strapping, and in the name of portability, Sun pushed developers into writing highly non-portable shell scripts. For a developer starting his career in Java, this was an awkward introduction to (ba)sh scripts. For a complete server such as Tomcat used in the above example, I count 1276 lines of shell script (sh + bat) and a maintenance job in its own right.</p>
|
|
1183
|
+
|
|
1184
|
+
<h2 id='enter_jruby'>Enter JRuby</h2>
|
|
1185
|
+
|
|
1186
|
+
<p>Scripting languages like ruby, perl, or python have all done a much better job than java in supporting cross-platform portability while also not hamstringing OS, file system, or console operablility. This was but one aspect that drew me to jruby. Would it not be possible to deploy entire, potentially complex, java-based services as easily installed gems and directly executable ruby scripts, rather than the typical amalgam of tarballs, XML config files, and wrapper bash scripts?</p>
|
|
1187
|
+
|
|
1188
|
+
<p>I believe the answer is a resounding &#8220;yes!&#8221; and that it will be well worth the effort, but there is some work to do to really get there. Lets consider the following very simple theoretical service implemented in java with a jruby based launch script:</p>
|
|
1189
|
+
<div class='highlight'><pre><code class='ruby'><span class='c1'>#!/opt/dist/jruby-1.1.4/bin/jruby</span>
|
|
1190
|
+
|
|
1191
|
+
<span class='nb'>require</span> <span class='s1'>&#39;myservice.jar&#39;</span>
|
|
1192
|
+
<span class='n'>import</span> <span class='s1'>&#39;com.foobar.MyService&#39;</span>
|
|
1193
|
+
|
|
1194
|
+
<span class='n'>service</span> <span class='o'>=</span> <span class='no'>MyService</span><span class='o'>.</span><span class='n'>new</span>
|
|
1195
|
+
<span class='n'>service</span><span class='o'>.</span><span class='n'>run</span>
|
|
1196
|
+
</code></pre>
|
|
1197
|
+
</div>
|
|
1198
|
+
<p>Note the use of a UNIX-standard &#8216;#!&#8217; <em>hashbang</em> (or <em>shebang</em>) to reference the interpreter for the script. The UNIX shell or system exec calls will detect the hashbang and invoke the named interpreter executable, passing the script file as an argument. While this works fine with any native script interpreter (ruby, perl, python, bash, etc.) <em>it won&#8217;t work out of the box with jruby or any other java-platform script interpreters not resorting to native code.</em> The reason is that the referenced &#8216;jruby&#8217; launcher is itself a bash script which needs to exec a java command line with the required details. On Linux at least, multiple scripts/intpreters can not be chained in this way. The work around presented for jruby is:</p>
|
|
1199
|
+
<div class='highlight'><pre><code class='bash'><span class='c'>#!/usr/bin/env jruby</span>
|
|
1200
|
+
</code></pre>
|
|
1201
|
+
</div>
|
|
1202
|
+
<p>&#8230;which then requires something like:</p>
|
|
1203
|
+
<div class='highlight'><pre><code class='bash'><span class='nv'>PATH</span><span class='o'>=</span>/opt/dist/jruby-1.1.4/bin:<span class='nv'>$PATH</span>
|
|
1204
|
+
</code></pre>
|
|
1205
|
+
</div>
|
|
1206
|
+
<p>&#8230;to be set in a user or system global profile. Besides raising some traditional security concerns, the use of the env workaround has notable side effects. First, it introduces a PATH dependency that must be managed external to the script. Next consider wanting to set various jruby intpreter and java options made available as command line flags to jruby, for example enabling full compilation and the java max heap size:</p>
|
|
1207
|
+
<div class='highlight'><pre><code class='bash'><span class='c'>#!/usr/bin/env jruby -X+C -J-Xmx256m</span>
|
|
1208
|
+
</code></pre>
|
|
1209
|
+
</div>
|
|
1210
|
+
<p>But <em>this also doesn&#8217;t work</em>, at least on current versions of Linux. When used as a hashbang, the <code>jruby -X+C -J-Xmx256m</code> is interpreted as the single parameter, and (fortunately) isn&#8217;t found in the path. Similarly the jruby script launcher supports JRUBY_OPTS as an alternative means to set these options:</p>
|
|
1211
|
+
<div class='highlight'><pre><code class='bash'><span class='c'>#!/usr/bin/env JRUBY_OPTS=&#39;-X+C -J-Xmx256m&#39; jruby</span>
|
|
1212
|
+
</code></pre>
|
|
1213
|
+
</div>
|
|
1214
|
+
<p>But again, on current versions of Linux, <em>this approach fails even more spectacularly: it causes an infinite loop! Brilliant!</em></p>
|
|
1215
|
+
|
|
1216
|
+
<p>You could tune all of the java and jruby option flags by directly modifying the jruby launcher script in the jruby distribution, but then you are stuck with one set of settings per host and jruby install. Poor at best. The only practical option (short of native code) is to write a wrapper script to launch any ruby script we want to set options for:</p>
|
|
1217
|
+
<div class='highlight'><pre><code class='bash'><span class='c'>#!/bin/bash</span>
|
|
1218
|
+
|
|
1219
|
+
<span class='c'># Hard code myservice.rb location or have additional fun determining</span>
|
|
1220
|
+
<span class='c'># relative location via $0</span>
|
|
1221
|
+
<span class='nb'>exec</span> /opt/dist/jruby-1.1.4/bin/jruby -X+C -J-Xmx256m -J-server <span class='se'>\</span>
|
|
1222
|
+
/home/david/run/myservice.rb
|
|
1223
|
+
</code></pre>
|
|
1224
|
+
</div>
|
|
1225
|
+
<p>Isn&#8217;t this fun? We are now back where we started with the java launcher: needing to write wrapper scripts around any significant production application. Besides being a nuisance and hiding important details in a seperate wrapper script file, this approach is particularly at odds with the bin script support offered by RubyGems. Without a native launcher like Hashdot, does this lack of exec integration not deminish jruby by comparison, not with java, but with MRI (native) ruby?</p>
|
|
1226
|
+
|
|
1227
|
+
<p>In my next post on Hashdot, I&#8217;ll demonstrate how Hashdot solves these and other historic blemishes and limitations of java and java-platform script interpreters like JRuby.</p></content>
|
|
1228
|
+
</entry>
|
|
1229
|
+
|
|
1230
|
+
</feed>
|