mdarray 0.5.3-java → 0.5.4-java
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +325 -70
- data/doc/20130625 MDArray Internals.docx +0 -0
- data/lib/colt/colt.rb +2 -0
- data/lib/colt/matrix/colt_matrix.rb +365 -0
- data/lib/colt/matrix/matrix2D_floating_algebra.rb +325 -0
- data/lib/colt/matrix/matrix_hierarchy.rb +258 -0
- data/lib/mdarray.rb +54 -3
- data/lib/mdarray/access.rb +16 -0
- data/lib/mdarray/counter.rb +16 -0
- data/lib/mdarray/creation.rb +13 -1
- data/lib/mdarray/lazy_mdarray.rb +6 -2
- data/lib/mdarray/lazy_operators.rb +8 -0
- data/lib/mdarray/printing.rb +18 -3
- data/lib/mdarray/section.rb +101 -0
- data/lib/netcdf/attribute.rb +154 -0
- data/lib/netcdf/cdm_node.rb +71 -0
- data/lib/netcdf/dimension.rb +146 -0
- data/lib/netcdf/file.rb +253 -0
- data/lib/netcdf/file_writer.rb +474 -0
- data/lib/netcdf/group.rb +205 -0
- data/lib/netcdf/netcdf.rb +151 -0
- data/lib/netcdf/variable.rb +520 -0
- data/test/colt/test_complete.rb +1 -2
- data/test/colt/test_double_matrix2d.rb +186 -0
- data/test/colt/test_float_matrix2d.rb +171 -0
- data/test/colt/test_math.rb +21 -0
- data/test/colt/test_matrix.rb +172 -0
- data/test/complete.rb +9 -1
- data/test/env.rb +11 -1
- data/test/mdarray/test_complete.rb +2 -0
- data/test/mdarray/test_creation.rb +19 -28
- data/test/mdarray/test_non_numeric.rb +97 -0
- data/test/mdarray/test_sections.rb +94 -0
- data/test/mdarray/test_views.rb +23 -1
- data/test/netcdf/netcdfwriter.rb +197 -0
- data/test/netcdf/test_complete.rb +27 -0
- data/test/netcdf/test_netcdf.rb +331 -0
- data/test/netcdf/test_redefine.rb +120 -0
- data/vendor/incanter.jar +0 -0
- data/vendor/{netcdfAll-4.3.16.jar → netcdfAll-4.3.18.jar} +0 -0
- data/version.rb +1 -1
- metadata +44 -26
- data/vendor/parallelcolt-0.10.0.jar +0 -0
data/README.md
CHANGED
@@ -1,89 +1,344 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
Announcement
|
2
|
+
============
|
3
|
+
|
4
|
+
MDArray version 0.5.4 has Just been released. MDArray is a multi dimensional array implemented
|
5
|
+
for JRuby inspired by NumPy (www.numpy.org) and Masahiro Tanaka´s Narray (narray.rubyforge.org).
|
6
|
+
MDArray stands on the shoulders of Java-NetCDF and Parallel Colt. At this point MDArray has
|
7
|
+
libraries for mathematical, trigonometric and descriptive statistics methods.
|
3
8
|
|
4
|
-
MDArray is a multi dimensional array implemented for JRuby inspired by NumPy (www.numpy.org)
|
5
|
-
and Narray (narray.rubyforge.org) by Masahiro Tanaka. MDArray stands on the shoulders of
|
6
|
-
Java-NetCDF and Parallel Colt.
|
7
|
-
|
8
9
|
NetCDF-Java Library is a Java interface to NetCDF files, as well as to many other types of
|
9
|
-
scientific data formats. It is developed and distributed by Unidata
|
10
|
+
scientific data formats. It is developed and distributed by Unidata
|
11
|
+
(http://www.unidata.ucar.edu).
|
12
|
+
|
13
|
+
Parallel Colt (http://grepcode.com/snapshot/repo1.maven.org/maven2/net.sourceforge.parallelcolt/
|
14
|
+
parallelcolt/0.10.0/) is a multithreaded version of Colt (http://acs.lbl.gov/software/colt/).
|
15
|
+
Colt provides a set of Open Source Libraries for High Performance Scientific and Technical
|
16
|
+
Computing in Java. Scientific and technical computing is characterized by demanding problem
|
17
|
+
sizes and a need for high performance at reasonably small memory footprint.
|
18
|
+
|
19
|
+
For more information and (some) documentation please go to: https://github.com/rbotafogo/mdarray/wiki
|
20
|
+
|
21
|
+
What´s new:
|
22
|
+
===========
|
23
|
+
|
24
|
+
NetCDF-3 File Support
|
25
|
+
---------------------
|
26
|
+
|
27
|
+
From Wikipedia, the free encyclopedia:
|
28
|
+
|
29
|
+
"NetCDF (Network Common Data Form) is a set of software libraries and self-describing,
|
30
|
+
machine-independent data formats that support the creation, access, and sharing of array-oriented
|
31
|
+
scientific data. The project homepage is hosted by the Unidata program at the University
|
32
|
+
Corporation for Atmospheric Research (UCAR). They are also the chief source of netCDF software,
|
33
|
+
standards development, updates, etc. The format is an open standard. NetCDF Classic and 64-bit
|
34
|
+
Offset Format are an international standard of the Open Geospatial Consortium.
|
10
35
|
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
and technical computing is characterized by demanding problem sizes and a need for high
|
15
|
-
performance at reasonably small memory footprint.
|
36
|
+
The project is actively supported by UCAR. Version 4.0 (released in 2008) allows the use of the
|
37
|
+
HDF5 data file format. Version 4.1 (2010) adds support for C and Fortran client access to
|
38
|
+
specified subsets of remote data via OPeNDAP.
|
16
39
|
|
17
|
-
|
18
|
-
|
40
|
+
The format was originally based on the conceptual model of the Common Data Format developed by
|
41
|
+
NASA, but has since diverged and is not compatible with it."
|
19
42
|
|
20
|
-
MDArray
|
43
|
+
This version of MDArray implements NetCDF-3 file support only. NetCDF-4 is not yet supported. At
|
44
|
+
the end of this announcement we show the MDArray implementation of the NetCDF-3 writing from the
|
45
|
+
tutorial at: http://www.unidata.ucar.edu/software/netcdf-java/tutorial/NetcdfWriting.html
|
21
46
|
|
22
|
-
|
23
|
-
and
|
47
|
+
|
48
|
+
MDArray and SciRuby:
|
49
|
+
====================
|
50
|
+
|
51
|
+
MDArray subscribes fully to the SciRuby Manifesto (http://sciruby.com/).
|
52
|
+
|
53
|
+
“Ruby has for some time had no equivalent to the beautifully constructed NumPy, SciPy, and
|
54
|
+
matplotlib libraries for Python.
|
24
55
|
|
25
56
|
We believe that the time for a Ruby science and visualization package has come. Sometimes
|
26
57
|
when a solution of sugar and water becomes super-saturated, from it precipitates a pure,
|
27
|
-
delicious, and diabetes-inducing crystal of sweetness, induced by no more than the tap
|
28
|
-
|
29
|
-
|
30
|
-
Main properties
|
31
|
-
===============
|
32
|
-
|
33
|
-
+ Homogeneous multidimensional array, a table of elements (usually numbers), all of the
|
34
|
-
same type, indexed by a tuple of positive integers;
|
35
|
-
+ Easy calculation for large numerical multi dimensional arrays;
|
36
|
-
+ Basic types are: boolean, byte, short, int, long, float, double, string, structure;
|
37
|
-
+ Based on JRuby, which allows importing Java libraries;
|
38
|
-
+ Operator: +,-,*,/,%,**, >, >=, etc.
|
39
|
-
+ Functions: abs, ceil, floor, truncate, is_zero, square, cube, fourth;
|
40
|
-
+ Binary Operators: &, |, ^, ~ (binary_ones_complement), <<, >>;
|
41
|
-
+ Ruby Math functions: acos, acosh, asin, asinh, atan, atan2, atanh, cbrt, cos, erf, exp,
|
42
|
-
gamma, hypot, ldexp, log, log10, log2, sin, sinh, sqrt, tan, tanh, neg;
|
43
|
-
+ Boolean operations on boolean arrays: and, or, not;
|
44
|
-
+ Fast descriptive statistics from Parallel Colt (complete list found bellow);
|
45
|
-
+ Easy manipulation of arrays: reshape, reduce dimension, permute, section, slice, etc.
|
46
|
-
+ Reading of two dimensional arrays from CSV files (mainly for debugging and simple
|
47
|
-
testing purposes);
|
48
|
-
+ StatList: a list that can grow/shrink and that can compute Parallel Colt descriptive
|
49
|
-
statistics.
|
50
|
-
|
51
|
-
Descriptive statistics methods
|
52
|
-
==============================
|
53
|
-
|
54
|
-
auto_correlation, correlation, covariance, durbin_watson, frequencies, geometric_mean,
|
55
|
-
harmonic_mean, kurtosis, lag1, max, mean, mean_deviation, median, min, moment, moment3,
|
56
|
-
moment4, pooled_mean, pooled_variance, product, quantile, quantile_inverse,
|
57
|
-
rank_interpolated, rms, sample_covariance, sample_kurtosis,
|
58
|
-
sample_kurtosis_standard_error, sample_skew, sample_skew_standard_error,
|
59
|
-
sample_standard_deviation, sample_variance, sample_weighted_variance, skew, split,
|
60
|
-
standard_deviation, standard_error, sum, sum_of_inversions, sum_of_logarithms,
|
61
|
-
sum_of_powers, sum_of_power_deviations, sum_of_squares, sum_of_squared_deviations,
|
62
|
-
trimmed_mean, variance, weighted_mean, weighted_rms, weighted_sums, winsorized_mean.
|
63
|
-
|
64
|
-
Installation and download
|
65
|
-
=========================
|
58
|
+
delicious, and diabetes-inducing crystal of sweetness, induced by no more than the tap of a
|
59
|
+
finger. So is occurring now, we believe, with numeric and visualization libraries for Ruby.”
|
66
60
|
|
67
|
-
|
68
|
-
|
61
|
+
MDArray main properties are:
|
62
|
+
============================
|
69
63
|
|
70
|
-
|
71
|
-
|
64
|
+
+ Homogeneous multidimensional array, a table of elements (usually numbers), all of the
|
65
|
+
same type, indexed by a tuple of positive integers;
|
66
|
+
+ Easy calculation for large numerical multi dimensional arrays;
|
67
|
+
+ Basic types are: boolean, byte, short, int, long, float, double, string, structure;
|
68
|
+
+ Based on JRuby, which allows importing Java libraries;
|
69
|
+
+ Operator: +,-,*,/,%,**, >, >=, etc.;
|
70
|
+
+ Functions: abs, ceil, floor, truncate, is_zero, square, cube, fourth;
|
71
|
+
+ Binary Operators: &, |, ^, ~ (binary_ones_complement), <<, >>;
|
72
|
+
+ Ruby Math functions: acos, acosh, asin, asinh, atan, atan2, atanh, cbrt, cos, erf, exp,
|
73
|
+
gamma, hypot, ldexp, log, log10, log2, sin, sinh, sqrt, tan, tanh, neg;
|
74
|
+
+ Boolean operations on boolean arrays: and, or, not;
|
75
|
+
+ Fast descriptive statistics from Parallel Colt (complete list found bellow);
|
76
|
+
+ Easy manipulation of arrays: reshape, reduce dimension, permute, section, slice, etc.;
|
77
|
+
+ Support for reading and writing NetCDF-3 files;
|
78
|
+
+ Reading of two dimensional arrays from CSV files (mainly for debugging and simple testing
|
79
|
+
purposes);
|
80
|
+
+ StatList: a list that can grow/shrink and that can compute Parallel Colt descriptive
|
81
|
+
statistics;
|
82
|
+
+ Experimental lazy evaluation (still slower than eager evaluation).
|
83
|
+
|
84
|
+
Descriptive statistics methods imported from Parallel Colt:
|
85
|
+
===========================================================
|
86
|
+
|
87
|
+
+ auto_correlation, correlation, covariance, durbin_watson, frequencies, geometric_mean,
|
88
|
+
+ harmonic_mean, kurtosis, lag1, max, mean, mean_deviation, median, min, moment, moment3,
|
89
|
+
+ moment4, pooled_mean, pooled_variance, product, quantile, quantile_inverse,
|
90
|
+
+ rank_interpolated, rms, sample_covariance, sample_kurtosis, sample_kurtosis_standard_error,
|
91
|
+
+ sample_skew, sample_skew_standard_error, sample_standard_deviation, sample_variance,
|
92
|
+
+ sample_weighted_variance, skew, split, standard_deviation, standard_error, sum,
|
93
|
+
+ sum_of_inversions, sum_of_logarithms, sum_of_powers, sum_of_power_deviations,
|
94
|
+
+ sum_of_squares, sum_of_squared_deviations, trimmed_mean, variance, weighted_mean,
|
95
|
+
+ weighted_rms, weighted_sums, winsorized_mean.
|
72
96
|
|
73
|
-
|
97
|
+
Double and Float methods from Parallel Colt:
|
98
|
+
============================================
|
74
99
|
|
75
|
-
|
76
|
-
|
100
|
+
+ acos, asin, atan, atan2, ceil, cos, exp, floor, greater, IEEEremainder, inv, less, lg,
|
101
|
+
+ log, log2, rint, sin, sqrt, tan.
|
102
|
+
|
103
|
+
Double, Float, Long and Int methods from Parallel Colt:
|
104
|
+
=======================================================
|
105
|
+
|
106
|
+
+ abs, compare, div, divNeg, equals, isEqual (is_equal), isGreater (is_greater),
|
107
|
+
+ isles (is_less), max, min, minus, mod, mult, multNeg (mult_neg), multSquare (mult_square),
|
108
|
+
+ neg, plus (add), plusAbs (plus_abs), pow (power), sign, square.
|
109
|
+
|
110
|
+
Long and Int methods from Parallel Colt
|
111
|
+
=======================================
|
112
|
+
|
113
|
+
+ and, dec, factorial, inc, not, or, shiftLeft (shift_left), shiftRightSigned
|
114
|
+
(shift_right_signed), shiftRightUnsigned (shift_right_unsigned), xor.
|
115
|
+
|
116
|
+
MDArray installation and download:
|
117
|
+
==================================
|
118
|
+
|
119
|
+
+ Install Jruby
|
120
|
+
+ jruby –S gem install mdarray
|
121
|
+
|
122
|
+
MDArray Homepages:
|
123
|
+
==================
|
77
124
|
|
78
125
|
+ http://rubygems.org/gems/mdarray
|
79
126
|
+ https://github.com/rbotafogo/mdarray/wiki
|
80
127
|
|
128
|
+
Contributors:
|
129
|
+
=============
|
130
|
+
Contributors are welcome.
|
131
|
+
|
132
|
+
MDArray History:
|
133
|
+
================
|
134
|
+
|
135
|
+
+ 07/08/2013: Version 0.5.4 - Support for reading and writing NetCDF-3 files
|
136
|
+
+ 24/06/2013: Version 0.5.3 – Over 90% Performance improvements for methods imported
|
137
|
+
from Parallel Colt and over 40% performance improvements for all other methods
|
138
|
+
(implemented in Ruby);
|
139
|
+
+ 16/05/2013: Version 0.5.0 - All loops transferred to Java with over 50% performance
|
140
|
+
improvements. Descriptive statistics from Parallel Colt;
|
141
|
+
+ 19/04/2013: Version 0.4.3 - Fixes a simple, but fatal bug in 0.4.2. No new features;
|
142
|
+
+ 17/04/2013: Version 0.4.2 - Adds simple statistics and boolean operators;
|
143
|
+
+ 05/04/2013: Version 0.4.0 – Initial release.
|
144
|
+
|
145
|
+
NetCDF-3 Writing with MDArray API
|
146
|
+
=================================
|
147
|
+
|
148
|
+
require 'mdarray'
|
149
|
+
|
150
|
+
class NetCDF
|
151
|
+
|
152
|
+
attr_reader :dir, :filename, :max_strlen
|
153
|
+
|
154
|
+
#---------------------------------------------------------------------------------------
|
155
|
+
#
|
156
|
+
#---------------------------------------------------------------------------------------
|
157
|
+
|
158
|
+
def initialize
|
159
|
+
@dir = "~/tmp"
|
160
|
+
@filename1 = "testWriter"
|
161
|
+
@filename2 = "testWriteRecord2"
|
162
|
+
@max_strlen = 80
|
163
|
+
end
|
164
|
+
|
165
|
+
#---------------------------------------------------------------------------------------
|
166
|
+
# Define the NetCDF-3 file
|
167
|
+
#---------------------------------------------------------------------------------------
|
168
|
+
|
169
|
+
def define_file
|
170
|
+
|
171
|
+
# We pass the directory, filename, filetype and optionaly the outside_scope.
|
172
|
+
#
|
173
|
+
# I'm implementing in cygwin, so the need for method cygpath that converts the
|
174
|
+
# directory name to a Windows name. In another environment, just pass the directory
|
175
|
+
# name.
|
176
|
+
#
|
177
|
+
# Inside a block we have another scope, so the block cannot access any variables, etc.
|
178
|
+
# from the ouside scope. If we pass the outside scope, in this case we are passing self,
|
179
|
+
# we can access variables in the outside scope by using @outside_scope.<variable>.
|
180
|
+
NetCDF.define(cygpath(@dir), @filename1, "netcdf3", self) do
|
181
|
+
|
182
|
+
# add dimensions
|
183
|
+
dimension "lat", 64
|
184
|
+
dimension "lon", 128
|
185
|
+
|
186
|
+
# add variables and attributes
|
187
|
+
# add Variable double temperature(lat, lon)
|
188
|
+
variable "temperature", "double", [@dim_lat, @dim_lon]
|
189
|
+
variable_att @var_temperature, "units", "K"
|
190
|
+
variable_att @var_temperature, "scale", [1, 2, 3]
|
191
|
+
|
192
|
+
# add a string-value variable: char svar(80)
|
193
|
+
# note that this is created as a scalar variable although in NetCDF-3 there is no
|
194
|
+
# string type and the string has to be represented as a char type.
|
195
|
+
variable "svar", "string", [], {:max_strlen => @outside_scope.max_strlen}
|
196
|
+
|
197
|
+
# add a 2D string-valued variable: char names(names, 80)
|
198
|
+
dimension "names", 3
|
199
|
+
variable "names", "string", [@dim_names], {:max_strlen => @outside_scope.max_strlen}
|
200
|
+
|
201
|
+
# add a scalar variable
|
202
|
+
variable "scalar", "double", []
|
203
|
+
|
204
|
+
# add global attributes
|
205
|
+
global_att "yo", "face"
|
206
|
+
global_att "versionD", 1.2, "double"
|
207
|
+
global_att "versionF", 1.2, "float"
|
208
|
+
global_att "versionI", 1, "int"
|
209
|
+
global_att "versionS", 2, "short"
|
210
|
+
global_att "versionB", 3, "byte"
|
211
|
+
|
212
|
+
end
|
213
|
+
|
214
|
+
end
|
215
|
+
|
216
|
+
#---------------------------------------------------------------------------------------
|
217
|
+
# write data on the above define file
|
218
|
+
#---------------------------------------------------------------------------------------
|
219
|
+
|
220
|
+
def write_file
|
221
|
+
|
222
|
+
NetCDF.write(cygpath(@dir), @filename1, self) do
|
223
|
+
|
224
|
+
temperature = find_variable("temperature")
|
225
|
+
shape = temperature.shape
|
226
|
+
data = MDArray.fromfunction("double", shape) do |i, j|
|
227
|
+
i * 1_000_000 + j * 1_000
|
228
|
+
end
|
229
|
+
write(temperature, data)
|
230
|
+
|
231
|
+
svar = find_variable("svar")
|
232
|
+
write_string(svar, "Two pairs of ladies stockings!")
|
233
|
+
|
234
|
+
names = find_variable("names")
|
235
|
+
# careful here with the shape of a string variable. A string variable has one
|
236
|
+
# more dimension than it should as there is no string type in NetCDF-3. As such,
|
237
|
+
# if we look as names' shape it has 2 dimensions, be we need to create a one
|
238
|
+
# dimension string array.
|
239
|
+
data = MDArray.string([3], ["No pairs of ladies stockings!",
|
240
|
+
"One pair of ladies stockings!",
|
241
|
+
"Two pairs of ladies stockings!"])
|
242
|
+
write_string(names, data)
|
243
|
+
|
244
|
+
# write scalar data
|
245
|
+
scalar = find_variable("scalar")
|
246
|
+
write(scalar, 222.333 )
|
247
|
+
|
248
|
+
end
|
249
|
+
|
250
|
+
end
|
251
|
+
|
252
|
+
#---------------------------------------------------------------------------------------
|
253
|
+
# Define a file for writing one record at a time
|
254
|
+
#---------------------------------------------------------------------------------------
|
255
|
+
|
256
|
+
def define_one_at_time
|
257
|
+
|
258
|
+
NetCDF.define(cygpath(@dir), @filename2, "netcdf3", self) do
|
259
|
+
|
260
|
+
dimension "lat", 3
|
261
|
+
dimension "lon", 4
|
262
|
+
# zero sized dimension is an unlimited dimension
|
263
|
+
dimension "time", 0
|
264
|
+
|
265
|
+
variable "lat", "float", [@dim_lat]
|
266
|
+
variable_att @var_lat, "units", "degree_north"
|
267
|
+
|
268
|
+
variable "lon", "float", [@dim_lon]
|
269
|
+
variable_att @var_lon, "units", "degree_east"
|
270
|
+
|
271
|
+
variable "rh", "int", [@dim_time, @dim_lat, @dim_lon]
|
272
|
+
variable_att @var_rh, "long_name", "relative humidity"
|
273
|
+
variable_att @var_rh, "units", "percent"
|
274
|
+
|
275
|
+
variable "T", "double", [@dim_time, @dim_lat, @dim_lon]
|
276
|
+
variable_att @var_t, "long_name", "surface temperature"
|
277
|
+
variable_att @var_t, "units", "degC"
|
278
|
+
|
279
|
+
variable "time", "int", [@dim_time]
|
280
|
+
variable_att @var_time, "units", "hours since 1990-01-01"
|
281
|
+
|
282
|
+
end
|
283
|
+
|
284
|
+
end
|
285
|
+
|
286
|
+
#---------------------------------------------------------------------------------------
|
287
|
+
# Define a file for writing one record at a time
|
288
|
+
#---------------------------------------------------------------------------------------
|
289
|
+
|
290
|
+
def write_one_at_time
|
291
|
+
|
292
|
+
NetCDF.write(cygpath(@dir), @filename2, self) do
|
293
|
+
|
294
|
+
lat = find_variable("lat")
|
295
|
+
lon = find_variable("lon")
|
296
|
+
write(lat, MDArray.float([3], [41, 40, 39]))
|
297
|
+
write(lon, MDArray.float([4], [-109, -107, -105, -103]))
|
298
|
+
|
299
|
+
# get variables from file
|
300
|
+
rh = find_variable("rh")
|
301
|
+
time = find_variable("time")
|
302
|
+
t = find_variable("T")
|
303
|
+
|
304
|
+
# there is no method find_dimension for NetcdfFileWriter, so we need to get the
|
305
|
+
# dimension from a variable.
|
306
|
+
rh_shape = rh.shape
|
307
|
+
dim_lat = rh_shape[1]
|
308
|
+
dim_lon = rh_shape[2]
|
309
|
+
|
310
|
+
(0...10).each do |time_idx|
|
311
|
+
|
312
|
+
# fill rh_data array
|
313
|
+
rh_data = MDArray.fromfunction("int", [dim_lat, dim_lon]) do |lat, lon|
|
314
|
+
time_idx * lat * lon
|
315
|
+
end
|
316
|
+
# reshape rh_data so that it has the same shape as rh variable
|
317
|
+
# Method reshape! reshapes the array in-place without data copying.
|
318
|
+
rh_data.reshape!([1, dim_lat, dim_lon])
|
319
|
+
|
320
|
+
# fill temp_data array
|
321
|
+
temp_data = MDArray.fromfunction("double", [dim_lat, dim_lon]) do |lat, lon|
|
322
|
+
time_idx * lat * lon / 3.14159
|
323
|
+
end
|
324
|
+
# reshape temp_data array so that it has the same shape as temp variable.
|
325
|
+
temp_data.reshape!([1, dim_lat, dim_lon])
|
326
|
+
|
327
|
+
# write the variables
|
328
|
+
write(time, MDArray.int([1], [time_idx * 12]), [time_idx])
|
329
|
+
write(rh, rh_data, [time_idx, 0, 0])
|
330
|
+
write(t, temp_data, [time_idx, 0, 0])
|
331
|
+
|
332
|
+
end # End time_idx loop
|
333
|
+
|
334
|
+
end
|
335
|
+
|
336
|
+
end
|
81
337
|
|
82
|
-
|
83
|
-
=======
|
338
|
+
end
|
84
339
|
|
85
|
-
|
86
|
-
|
87
|
-
|
88
|
-
|
89
|
-
|
340
|
+
netcdf = NetCDF.new
|
341
|
+
netcdf.define_file
|
342
|
+
netcdf.write_file
|
343
|
+
netcdf.define_one_at_time
|
344
|
+
netcdf.write_one_at_time
|
Binary file
|
data/lib/colt/colt.rb
CHANGED
@@ -0,0 +1,365 @@
|
|
1
|
+
# -*- coding: utf-8 -*-
|
2
|
+
|
3
|
+
##########################################################################################
|
4
|
+
# Copyright © 2013 Rodrigo Botafogo. All Rights Reserved. Permission to use, copy, modify,
|
5
|
+
# and distribute this software and its documentation for educational, research, and
|
6
|
+
# not-for-profit purposes, without fee and without a signed licensing agreement, is hereby
|
7
|
+
# granted, provided that the above copyright notice, this paragraph and the following two
|
8
|
+
# paragraphs appear in all copies, modifications, and distributions. Contact Rodrigo
|
9
|
+
# Botafogo - rodrigo.a.botafogo@gmail.com for commercial licensing opportunities.
|
10
|
+
#
|
11
|
+
# IN NO EVENT SHALL RODRIGO BOTAFOGO BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL,
|
12
|
+
# INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF
|
13
|
+
# THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF RODRIGO BOTAFOGO HAS BEEN ADVISED OF THE
|
14
|
+
# POSSIBILITY OF SUCH DAMAGE.
|
15
|
+
#
|
16
|
+
# RODRIGO BOTAFOGO SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
|
17
|
+
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE
|
18
|
+
# SOFTWARE AND ACCOMPANYING DOCUMENTATION, IF ANY, PROVIDED HEREUNDER IS PROVIDED "AS IS".
|
19
|
+
# RODRIGO BOTAFOGO HAS NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS,
|
20
|
+
# OR MODIFICATIONS.
|
21
|
+
##########################################################################################
|
22
|
+
|
23
|
+
require 'java'
|
24
|
+
|
25
|
+
class MDMatrix
|
26
|
+
include_package "cern.colt.matrix.tdouble.impl"
|
27
|
+
include_package "cern.colt.matrix.tdouble.algo"
|
28
|
+
|
29
|
+
include_package "cern.colt.matrix.tfloat.impl"
|
30
|
+
include_package "cern.colt.matrix.tfloat.algo"
|
31
|
+
|
32
|
+
include_package "cern.colt.matrix.tlong.impl"
|
33
|
+
include_package "cern.colt.matrix.tlong.algo"
|
34
|
+
|
35
|
+
include_package "cern.colt.matrix.tint.impl"
|
36
|
+
include_package "cern.colt.matrix.tint.algo"
|
37
|
+
|
38
|
+
|
39
|
+
attr_reader :colt_matrix
|
40
|
+
attr_reader :mdarray
|
41
|
+
attr_reader :rank
|
42
|
+
|
43
|
+
#------------------------------------------------------------------------------------
|
44
|
+
#
|
45
|
+
#------------------------------------------------------------------------------------
|
46
|
+
|
47
|
+
def initialize(mdarray, colt_matrix)
|
48
|
+
@mdarray = mdarray
|
49
|
+
@colt_matrix = colt_matrix
|
50
|
+
@rank = @mdarray.rank
|
51
|
+
@algebra = nil
|
52
|
+
end
|
53
|
+
|
54
|
+
#------------------------------------------------------------------------------------
|
55
|
+
#
|
56
|
+
#------------------------------------------------------------------------------------
|
57
|
+
|
58
|
+
def self.build(type, shape, storage = nil)
|
59
|
+
if (shape.size > 3)
|
60
|
+
raise "Cannot create MDMatrix of size greater than 3"
|
61
|
+
end
|
62
|
+
self.from_mdarray(MDArray.build(type, shape, storage))
|
63
|
+
end
|
64
|
+
|
65
|
+
#------------------------------------------------------------------------------------
|
66
|
+
#
|
67
|
+
#------------------------------------------------------------------------------------
|
68
|
+
|
69
|
+
def self.double(shape, storage = nil)
|
70
|
+
self.build("double", shape, storage)
|
71
|
+
end
|
72
|
+
|
73
|
+
#------------------------------------------------------------------------------------
|
74
|
+
#
|
75
|
+
#------------------------------------------------------------------------------------
|
76
|
+
|
77
|
+
def self.float(shape, storage = nil)
|
78
|
+
self.build("float", shape, storage)
|
79
|
+
end
|
80
|
+
|
81
|
+
#------------------------------------------------------------------------------------
|
82
|
+
#
|
83
|
+
#------------------------------------------------------------------------------------
|
84
|
+
|
85
|
+
def self.long(shape, storage = nil)
|
86
|
+
self.build("long", shape, storage)
|
87
|
+
end
|
88
|
+
|
89
|
+
#------------------------------------------------------------------------------------
|
90
|
+
#
|
91
|
+
#------------------------------------------------------------------------------------
|
92
|
+
|
93
|
+
def self.int(shape, storage = nil)
|
94
|
+
self.build("int", shape, storage)
|
95
|
+
end
|
96
|
+
|
97
|
+
#------------------------------------------------------------------------------------
|
98
|
+
# Creates a MDMatrix from an MDArray.
|
99
|
+
# (int rows, int columns, double[] elements, int rowZero, int columnZero,
|
100
|
+
# int rowStride, int columnStride, boolean isView)
|
101
|
+
#------------------------------------------------------------------------------------
|
102
|
+
|
103
|
+
def self.from_mdarray(mdarray)
|
104
|
+
|
105
|
+
case mdarray.rank
|
106
|
+
|
107
|
+
when 1
|
108
|
+
dense1D(mdarray)
|
109
|
+
when 2
|
110
|
+
dense2D(mdarray)
|
111
|
+
when 3
|
112
|
+
dense3D(mdarray)
|
113
|
+
else
|
114
|
+
raise "Cannot create MDMatrix of rank greater than 3"
|
115
|
+
end
|
116
|
+
|
117
|
+
end
|
118
|
+
|
119
|
+
#------------------------------------------------------------------------------------
|
120
|
+
# Creates a new MDMatrix from a given colt_matrix
|
121
|
+
#------------------------------------------------------------------------------------
|
122
|
+
|
123
|
+
def self.from_colt_matrix(colt_matrix)
|
124
|
+
|
125
|
+
if (colt_matrix.is_a? DenseDoubleMatrix3D)
|
126
|
+
mdarray = MDArray.from_jstorage("double",
|
127
|
+
[colt_matrix.slices, colt_matrix.rows,
|
128
|
+
colt_matrix.columns], colt_matrix.elements)
|
129
|
+
return DoubleMDMatrix3D.from_mdarray(mdarray)
|
130
|
+
elsif (colt_matrix.is_a? DenseFloatMatrix3D)
|
131
|
+
mdarray = MDArray.from_jstorage("float",
|
132
|
+
[colt_matrix.slices, colt_matrix.rows,
|
133
|
+
colt_matrix.columns], colt_matrix.elements)
|
134
|
+
return FloatMDMatrix3D.from_mdarray(mdarray)
|
135
|
+
elsif (colt_matrix.is_a? DenseLongMatrix3D)
|
136
|
+
mdarray = MDArray.from_jstorage("long",
|
137
|
+
[colt_matrix.slices, colt_matrix.rows,
|
138
|
+
colt_matrix.columns], colt_matrix.elements)
|
139
|
+
return LongMDMatrix3D.from_mdarray(mdarray)
|
140
|
+
elsif (colt_matrix.is_a? DenseIntMatrix3D)
|
141
|
+
mdarray = MDArray.from_jstorage("int",
|
142
|
+
[colt_matrix.slices, colt_matrix.rows,
|
143
|
+
colt_matrix.columns], colt_matrix.elements)
|
144
|
+
return IntMDMatrix3D.from_mdarray(mdarray)
|
145
|
+
elsif (colt_matrix.is_a? DenseDoubleMatrix2D)
|
146
|
+
mdarray = MDArray.from_jstorage("double", [colt_matrix.rows, colt_matrix.columns],
|
147
|
+
colt_matrix.elements)
|
148
|
+
return DoubleMDMatrix2D.from_mdarray(mdarray)
|
149
|
+
elsif (colt_matrix.is_a? DenseFloatMatrix2D)
|
150
|
+
mdarray = MDArray.from_jstorage("float", [colt_matrix.rows, colt_matrix.columns],
|
151
|
+
colt_matrix.elements)
|
152
|
+
return FloatMDMatrix2D.from_mdarray(mdarray)
|
153
|
+
elsif (colt_matrix.is_a? DenseLongMatrix2D)
|
154
|
+
mdarray = MDArray.from_jstorage("long", [colt_matrix.rows, colt_matrix.columns],
|
155
|
+
colt_matrix.elements)
|
156
|
+
return LongMDMatrix2D.from_mdarray(mdarray)
|
157
|
+
elsif (colt_matrix.is_a? DenseIntMatrix2D)
|
158
|
+
mdarray = MDArray.from_jstorage("int", [colt_matrix.rows, colt_matrix.columns],
|
159
|
+
colt_matrix.elements)
|
160
|
+
return IntMDMatrix2D.from_mdarray(mdarray)
|
161
|
+
elsif (colt_matrix.is_a? DenseDoubleMatrix1D)
|
162
|
+
mdarray = MDArray.from_jstorage("double", [colt_matrix.size], colt_matrix.elements)
|
163
|
+
return DoubleMDMatrix1D.from_mdarray(mdarray)
|
164
|
+
elsif (colt_matrix.is_a? DenseFloatMatrix1D)
|
165
|
+
mdarray = MDArray.from_jstorage("float", [colt_matrix.size], colt_matrix.elements)
|
166
|
+
return FloatMDMatrix1D.from_mdarray(mdarray)
|
167
|
+
elsif (colt_matrix.is_a? DenseLongMatrix1D)
|
168
|
+
mdarray = MDArray.from_jstorage("long", [colt_matrix.size], colt_matrix.elements)
|
169
|
+
return LongMDMatrix1D.from_mdarray(mdarray)
|
170
|
+
elsif (colt_matrix.is_a? DenseIntMatrix1D)
|
171
|
+
mdarray = MDArray.from_jstorage("int", [colt_matrix.size], colt_matrix.elements)
|
172
|
+
return IntMDMatrix1D.from_mdarray(mdarray)
|
173
|
+
end
|
174
|
+
|
175
|
+
end
|
176
|
+
|
177
|
+
#------------------------------------------------------------------------------------
|
178
|
+
#
|
179
|
+
#------------------------------------------------------------------------------------
|
180
|
+
|
181
|
+
def normalize!
|
182
|
+
@colt_matrix.normalize
|
183
|
+
end
|
184
|
+
|
185
|
+
#------------------------------------------------------------------------------------
|
186
|
+
#
|
187
|
+
#------------------------------------------------------------------------------------
|
188
|
+
|
189
|
+
def sum
|
190
|
+
@colt_matrix.zSum
|
191
|
+
end
|
192
|
+
|
193
|
+
#------------------------------------------------------------------------------------
|
194
|
+
#
|
195
|
+
#------------------------------------------------------------------------------------
|
196
|
+
|
197
|
+
def print
|
198
|
+
|
199
|
+
case mdarray.type
|
200
|
+
|
201
|
+
when "double"
|
202
|
+
formatter = DoubleFormatter.new
|
203
|
+
when "float"
|
204
|
+
formatter = FloatFormatter.new
|
205
|
+
when "long"
|
206
|
+
formatter = LongFormatter.new
|
207
|
+
when "int"
|
208
|
+
formatter = IntFormatter.new
|
209
|
+
|
210
|
+
end
|
211
|
+
|
212
|
+
printf(formatter.toString(@colt_matrix))
|
213
|
+
|
214
|
+
end
|
215
|
+
|
216
|
+
#------------------------------------------------------------------------------------
|
217
|
+
#
|
218
|
+
#------------------------------------------------------------------------------------
|
219
|
+
|
220
|
+
private
|
221
|
+
|
222
|
+
#------------------------------------------------------------------------------------
|
223
|
+
#
|
224
|
+
#------------------------------------------------------------------------------------
|
225
|
+
|
226
|
+
def self.dense1D(mdarray)
|
227
|
+
|
228
|
+
storage = mdarray.nc_array.getStorage()
|
229
|
+
index = mdarray.nc_array.getIndex()
|
230
|
+
size = index.size
|
231
|
+
|
232
|
+
klass = index.getClass
|
233
|
+
field = klass.getDeclaredField("stride0")
|
234
|
+
field.setAccessible true
|
235
|
+
stride0 = field.get(index)
|
236
|
+
# p stride0
|
237
|
+
|
238
|
+
klass = klass.getSuperclass()
|
239
|
+
field = klass.getDeclaredField("offset")
|
240
|
+
field.setAccessible true
|
241
|
+
offset = field.get(index)
|
242
|
+
# p offset
|
243
|
+
|
244
|
+
case mdarray.type
|
245
|
+
when "double"
|
246
|
+
colt_matrix = DenseDoubleMatrix1D.new(size, storage, offset, stride0, false)
|
247
|
+
DoubleMDMatrix1D.new(mdarray, colt_matrix)
|
248
|
+
when "float"
|
249
|
+
colt_matrix = DenseFloatMatrix1D.new(size, storage, offset, stride0, false)
|
250
|
+
FloatMDMatrix1D.new(mdarray, colt_matrix)
|
251
|
+
when "long"
|
252
|
+
colt_matrix = DenseLongMatrix1D.new(size, storage, offset, stride0, false)
|
253
|
+
LongMDMatrix1D.new(mdarray, colt_matrix)
|
254
|
+
when "int"
|
255
|
+
colt_matrix = DenseIntMatrix1D.new(size, storage, offset, stride0, false)
|
256
|
+
IntMDMatrix1D.new(mdarray, colt_matrix)
|
257
|
+
end
|
258
|
+
|
259
|
+
end
|
260
|
+
|
261
|
+
#------------------------------------------------------------------------------------
|
262
|
+
#
|
263
|
+
#------------------------------------------------------------------------------------
|
264
|
+
|
265
|
+
def self.dense2D(mdarray)
|
266
|
+
|
267
|
+
storage = mdarray.nc_array.getStorage()
|
268
|
+
index = mdarray.nc_array.getIndex()
|
269
|
+
shape = index.getShape()
|
270
|
+
|
271
|
+
klass = index.getClass
|
272
|
+
field = klass.getDeclaredField("stride0")
|
273
|
+
field.setAccessible true
|
274
|
+
stride0 = field.get(index)
|
275
|
+
# p stride0
|
276
|
+
|
277
|
+
field = klass.getDeclaredField("stride1")
|
278
|
+
field.setAccessible true
|
279
|
+
stride1 = field.get(index)
|
280
|
+
# p stride1
|
281
|
+
|
282
|
+
klass = klass.getSuperclass()
|
283
|
+
field = klass.getDeclaredField("offset")
|
284
|
+
field.setAccessible true
|
285
|
+
offset = field.get(index)
|
286
|
+
# p offset
|
287
|
+
|
288
|
+
case mdarray.type
|
289
|
+
when "double"
|
290
|
+
colt_matrix = DenseDoubleMatrix2D.new(shape[0], shape[1], storage, offset, 0,
|
291
|
+
stride0, stride1, false)
|
292
|
+
DoubleMDMatrix2D.new(mdarray, colt_matrix)
|
293
|
+
when "float"
|
294
|
+
colt_matrix = DenseFloatMatrix2D.new(shape[0], shape[1], storage, offset, 0,
|
295
|
+
stride0, stride1, false)
|
296
|
+
FloatMDMatrix2D.new(mdarray, colt_matrix)
|
297
|
+
when "long"
|
298
|
+
colt_matrix = DenseLongMatrix2D.new(shape[0], shape[1], storage, offset, 0,
|
299
|
+
stride0, stride1, false)
|
300
|
+
LongMDMatrix2D.new(mdarray, colt_matrix)
|
301
|
+
when "int"
|
302
|
+
colt_matrix = DenseIntMatrix2D.new(shape[0], shape[1], storage, offset, 0,
|
303
|
+
stride0, stride1, false)
|
304
|
+
IntMDMatrix2D.new(mdarray, colt_matrix)
|
305
|
+
end
|
306
|
+
|
307
|
+
end
|
308
|
+
|
309
|
+
#------------------------------------------------------------------------------------
|
310
|
+
#
|
311
|
+
#------------------------------------------------------------------------------------
|
312
|
+
|
313
|
+
def self.dense3D(mdarray)
|
314
|
+
|
315
|
+
storage = mdarray.nc_array.getStorage()
|
316
|
+
index = mdarray.nc_array.getIndex()
|
317
|
+
shape = index.getShape()
|
318
|
+
|
319
|
+
klass = index.getClass
|
320
|
+
field = klass.getDeclaredField("stride0")
|
321
|
+
field.setAccessible true
|
322
|
+
stride0 = field.get(index)
|
323
|
+
# p stride0
|
324
|
+
|
325
|
+
field = klass.getDeclaredField("stride1")
|
326
|
+
field.setAccessible true
|
327
|
+
stride1 = field.get(index)
|
328
|
+
# p stride1
|
329
|
+
|
330
|
+
field = klass.getDeclaredField("stride2")
|
331
|
+
field.setAccessible true
|
332
|
+
stride2 = field.get(index)
|
333
|
+
# p stride2
|
334
|
+
|
335
|
+
klass = klass.getSuperclass()
|
336
|
+
field = klass.getDeclaredField("offset")
|
337
|
+
field.setAccessible true
|
338
|
+
offset = field.get(index)
|
339
|
+
# p offset
|
340
|
+
|
341
|
+
case mdarray.type
|
342
|
+
when "double"
|
343
|
+
colt_matrix = DenseDoubleMatrix3D.new(shape[0], shape[1], shape[2], storage,
|
344
|
+
offset, 0, 0, stride0, stride1, stride2, false)
|
345
|
+
DoubleMDMatrix3D.new(mdarray, colt_matrix)
|
346
|
+
when "float"
|
347
|
+
colt_matrix = DenseFloatMatrix3D.new(shape[0], shape[1], shape[2], storage,
|
348
|
+
offset, 0, 0, stride0, stride1, stride2, false)
|
349
|
+
FloatMDMatrix3D.new(mdarray, colt_matrix)
|
350
|
+
when "long"
|
351
|
+
colt_matrix = DenseLongMatrix3D.new(shape[0], shape[1], shape[2], storage,
|
352
|
+
offset, 0, 0, stride0, stride1, stride2, false)
|
353
|
+
LongMDMatrix3D.new(mdarray, colt_matrix)
|
354
|
+
when "int"
|
355
|
+
colt_matrix = DenseIntMatrix3D.new(shape[0], shape[1], shape[2], storage,
|
356
|
+
offset, 0, 0, stride0, stride1, stride2, false)
|
357
|
+
IntMDMatrix3D.new(mdarray, colt_matrix)
|
358
|
+
end
|
359
|
+
|
360
|
+
end
|
361
|
+
|
362
|
+
end # MDMatrix
|
363
|
+
|
364
|
+
require_relative 'matrix_hierarchy'
|
365
|
+
require_relative 'matrix2D_floating_algebra'
|