ndav 0.0.7 → 0.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.yardopts +0 -1
- data/README.md +9 -5
- data/Rakefile +1 -1
- data/ext/ndav.c +48 -73
- data/lib/ndav/converter.rb +2 -2
- data/lib/ndav/flags.rb +7 -1
- data/lib/ndav/memory_viewable.rb +40 -20
- data/lib/ndav.rb +67 -61
- data/ndav.gemspec +1 -2
- data/test/helper.rb +0 -2
- data/test/test_flags.rb +5 -11
- data/test/test_ndav.rb +21 -24
- data/test/test_ractor.rb +5 -0
- metadata +2 -18
- data/lib/ndav/ffi.rb +0 -97
- data/test/test_ffi.rb +0 -117
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 95b06d29aad658c09480cbe88ad3030c77aca3bad0e98ea20d02f29168de76cc
|
|
4
|
+
data.tar.gz: 6fce2648d0ff6a4ed82741f9357194c1d5de8b2c81183d1d16de9804cb19adcb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: edd30f7e86ef25155769684c2432383ae9f8fd4b788eaddd25067ebd99fd9f210f5d88a48018dd76cc8b5baab378a404d61f0610e98401aff0e10c2f6b891101
|
|
7
|
+
data.tar.gz: 917020fcfde03cfab43b4d47b631fba0e94d4c94a0b72f0b72080f933d50123874c50dcf682e391fa4dd86a61a8b770df94ec2af96c26cbbcf80baeb07b7e1cd
|
data/.yardopts
CHANGED
data/README.md
CHANGED
|
@@ -112,7 +112,7 @@ On the other hand, it also exports MemoryView. You can pass NDAV arrays *directl
|
|
|
112
112
|
samples = waveform.to_ndav
|
|
113
113
|
whisper.full(params, samples)
|
|
114
114
|
|
|
115
|
-
|
|
115
|
+
Additionally, classes that `include` `NDAV::MemoryViewable`, including `Numo::NArray`, `OnnxRuntime::OrtValue` and `Torch::Tensor`, export MemoryViews:
|
|
116
116
|
|
|
117
117
|
memory_view = Fiddle::MemoryView.new(numo)
|
|
118
118
|
memory_view = Fiddle::MemoryView.new(ort_value)
|
|
@@ -124,6 +124,10 @@ Notice that NDAV is just a memory view and libraries share a memory address. If
|
|
|
124
124
|
|
|
125
125
|
Additionally, you potentially encounter odd data corruption or segmentation fault. These might be bugs in bridge libraries such as [ndav-numo-narray][]. As a user, you don't need to worry about this kind of memory management, but it's worth knowing such situations may occur.
|
|
126
126
|
|
|
127
|
+
### Notice On Export Flags ###
|
|
128
|
+
|
|
129
|
+
When the first argument to `NDAV.new` or `.new_override` is a MemoryView producer, NDAV assumes it exports the same metadata - such as `byte_size`, `format`, and `shape` - regardless of any flags. If you find a MemoryView producer which exports different metadata dependent on the flags, please file an issue.
|
|
130
|
+
|
|
127
131
|
INSTALLATION
|
|
128
132
|
------------
|
|
129
133
|
|
|
@@ -140,11 +144,11 @@ BRIDGES
|
|
|
140
144
|
|
|
141
145
|
There are some bridges using NDAV:
|
|
142
146
|
|
|
143
|
-
* {file:ndav/ffi ndav/ffi}: [`FFI::MemoryPointer`][FFI Pointer], [`FFI::Pointer`][FFI Pointer] <-> `NDAV`
|
|
144
147
|
* [ndav-numo-narray][]: [`Numo::NArray`][Numo::NArray] <-> `NDAV`
|
|
145
148
|
* [ndav-ort_value][]: [`OnnxRuntime::OrtValue`][ONNX Runtime Ruby] <-> `NDAV`
|
|
146
149
|
* [ndav-torch-tensor][]: [`Torch::Tensor`][Torch.rb] <-> `NDAV`
|
|
147
150
|
* [onnxruntime-torch-tensor][]: [`Torch::Tensor`][Torch.rb] <-> [`OnnxRuntime::OrtValue`][ONNX Runtime Ruby] via `NDAV`
|
|
151
|
+
* [ndav-magick-image][]: [`Magick::Image`][RMagick] <-> `NDAV`
|
|
148
152
|
|
|
149
153
|
APPLICATIONS AND LIBRARIES WHICH USE NDAV
|
|
150
154
|
-----------------------------------------
|
|
@@ -159,7 +163,7 @@ Refer to existing bridge implementations listed above to create your bridge.
|
|
|
159
163
|
|
|
160
164
|
The points are:
|
|
161
165
|
|
|
162
|
-
* Implement `FromNDAV#from_ndav`, `MemoryViewable#ndav_descriptor`, optionally `ToNDAV#to_ndav`, {NDAV.register register} them, and `NDAV.from_your_data` and `NDAV#to_your_data` are automatically derived and the array
|
|
166
|
+
* Implement `FromNDAV#from_ndav`, `MemoryViewable#ndav_descriptor`, optionally `ToNDAV#to_ndav`, {NDAV.register register} them, and `NDAV.from_your_data` and `NDAV#to_your_data` are automatically derived and the array becomes able to export a MemoryView
|
|
163
167
|
* When initializing NDAV object from your object, use `lifetime` keyword argument for {NDAV#initialize} effectively to prevent Ruby from GCing your object, which would lead to a dangling pointer
|
|
164
168
|
* When initializing your object from NDAV object, keep NDAV object alive to prevent Ruby from GCing NDAV object, which would lead to a dangling pointer, [ndav-numo-narray][], for instance, embeds the NDAV object in an instance variable
|
|
165
169
|
|
|
@@ -183,13 +187,13 @@ BSD-2-Clause license. See {file:LICENSE.txt} file.
|
|
|
183
187
|
[ONNX Runtime Ruby]: https://github.com/ankane/onnxruntime-ruby
|
|
184
188
|
[Red Arrow]: https://github.com/apache/arrow/tree/main/ruby
|
|
185
189
|
[Red Arrow Numo::NArray]: https://github.com/red-data-tools/red-arrow-numo-narray
|
|
186
|
-
[ndav/ffi]: https://gitlab.com/KitaitiMakoto/ndav/-/blob/main/lib/ndav/ffi.rb
|
|
187
|
-
[FFI Pointer]: https://github.com/ffi/ffi/wiki/Pointers
|
|
188
190
|
[whispercpp]: https://github.com/ggml-org/whisper.cpp/tree/master/bindings/ruby
|
|
189
191
|
[ndav-numo-narray]: https://gitlab.com/KitaitiMakoto/ndav-numo-narray
|
|
190
192
|
[ndav-ort_value]: https://gitlab.com/KitaitiMakoto/ndav-ort_value
|
|
191
193
|
[ndav-torch-tensor]: https://gitlab.com/KitaitiMakoto/ndav-torch-tensor
|
|
192
194
|
[onnxruntime-torch-tensor]: https://gitlab.com/KitaitiMakoto/onnxruntime-torch-tensor
|
|
195
|
+
[ndav-magick-image]: https://gitlab.com/KitaitiMakoto/ndav-magick-image
|
|
196
|
+
[RMagick]: https://rmagick.github.io/
|
|
193
197
|
[GTCRN]: https://gitlab.com/KitaitiMakoto/gtcrn
|
|
194
198
|
[TorchAudio]: https://github.com/ankane/torchaudio-ruby
|
|
195
199
|
[Itak]: https://gitlab.com/KitaitiMakoto/itak
|
data/Rakefile
CHANGED
|
@@ -8,7 +8,7 @@ DL_BUILD_PATH = File.join("ext", DL_NAME)
|
|
|
8
8
|
DL_PATH = File.join("lib", DL_NAME)
|
|
9
9
|
|
|
10
10
|
SRC = FileList["ext/**.{h,c,rb}"]
|
|
11
|
-
CLEAN.include
|
|
11
|
+
CLEAN.include FileList["ext/**.{o,so,bundle,dll}"]
|
|
12
12
|
|
|
13
13
|
task default: :test
|
|
14
14
|
|
data/ext/ndav.c
CHANGED
|
@@ -1,7 +1,16 @@
|
|
|
1
1
|
#include <ruby.h>
|
|
2
2
|
#include <ruby/memory_view.h>
|
|
3
3
|
|
|
4
|
+
#ifndef NUM2PTR
|
|
5
|
+
#if SIZEOF_LONG == SIZEOF_VOIDP
|
|
6
|
+
#define NUM2PTR(x) ((void *)(NUM2LONG(x)))
|
|
7
|
+
#else
|
|
8
|
+
#define NUM2PTR(x) ((void *)(NUM2ULL(x)))
|
|
9
|
+
#endif // SIZEOF_LONG == SIZEOF_VOIDP
|
|
10
|
+
#endif // NUM2PTR
|
|
11
|
+
|
|
4
12
|
static ID id_ndav_validated_descriptor;
|
|
13
|
+
static VALUE sym_data;
|
|
5
14
|
static VALUE sym_addr;
|
|
6
15
|
static VALUE sym_shape;
|
|
7
16
|
static VALUE sym_strides;
|
|
@@ -14,9 +23,10 @@ static VALUE sym_item_size;
|
|
|
14
23
|
static ID id_new;
|
|
15
24
|
static VALUE cItemComponent;
|
|
16
25
|
|
|
17
|
-
|
|
18
|
-
VALUE
|
|
19
|
-
|
|
26
|
+
struct private_data {
|
|
27
|
+
VALUE data;
|
|
28
|
+
ssize_t buf[];
|
|
29
|
+
};
|
|
20
30
|
|
|
21
31
|
static bool
|
|
22
32
|
fill_size_array(VALUE src, ssize_t *dest, ssize_t size)
|
|
@@ -31,23 +41,6 @@ fill_size_array(VALUE src, ssize_t *dest, ssize_t size)
|
|
|
31
41
|
return true;
|
|
32
42
|
}
|
|
33
43
|
|
|
34
|
-
static void
|
|
35
|
-
release_memview_common(rb_memory_view_t *view)
|
|
36
|
-
{
|
|
37
|
-
if (view->shape) {
|
|
38
|
-
free((void *)view->shape);
|
|
39
|
-
view->shape = NULL;
|
|
40
|
-
}
|
|
41
|
-
if (view->strides) {
|
|
42
|
-
free((void *)view->strides);
|
|
43
|
-
view->strides = NULL;
|
|
44
|
-
}
|
|
45
|
-
if (view->sub_offsets) {
|
|
46
|
-
free((void *)view->sub_offsets);
|
|
47
|
-
view->sub_offsets = NULL;
|
|
48
|
-
}
|
|
49
|
-
}
|
|
50
|
-
|
|
51
44
|
static bool
|
|
52
45
|
ndav_memory_viewable_get_memory_view(const VALUE ndav, rb_memory_view_t *view, int flags)
|
|
53
46
|
{
|
|
@@ -69,67 +62,49 @@ ndav_memory_viewable_get_memory_view(const VALUE ndav, rb_memory_view_t *view, i
|
|
|
69
62
|
}
|
|
70
63
|
tmp.readonly = readonly;
|
|
71
64
|
tmp.obj = ndav;
|
|
72
|
-
|
|
65
|
+
VALUE data = rb_hash_aref(desc, sym_data);
|
|
73
66
|
VALUE item_size = rb_hash_aref(desc, sym_item_size);
|
|
74
67
|
tmp.item_size = NUM2SSIZET(item_size);
|
|
75
68
|
tmp.byte_size = byte_size;
|
|
76
69
|
VALUE ndim = rb_hash_aref(desc, sym_ndim);
|
|
77
70
|
tmp.ndim = NUM2SSIZET(ndim);
|
|
78
71
|
VALUE format = rb_hash_aref(desc, sym_format);
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
// TODO: Commonalize
|
|
72
|
+
char *view_format = StringValueCStr(format);
|
|
82
73
|
VALUE shape = rb_hash_aref(desc, sym_shape);
|
|
83
|
-
ssize_t *view_shape = (ssize_t *)malloc(sizeof(ssize_t) * tmp.ndim);
|
|
84
|
-
if (!view_shape) {
|
|
85
|
-
release_memview_common(&tmp);
|
|
86
|
-
return false;
|
|
87
|
-
}
|
|
88
|
-
if (!fill_size_array(shape, view_shape, tmp.ndim)) {
|
|
89
|
-
free(view_shape);
|
|
90
|
-
release_memview_common(&tmp);
|
|
91
|
-
return false;
|
|
92
|
-
}
|
|
93
|
-
tmp.shape = view_shape;
|
|
94
|
-
|
|
95
74
|
VALUE strides = rb_hash_aref(desc, sym_strides);
|
|
96
|
-
ssize_t *view_strides = (ssize_t *)malloc(sizeof(ssize_t) * tmp.ndim);
|
|
97
|
-
if (!view_strides) {
|
|
98
|
-
release_memview_common(&tmp);
|
|
99
|
-
return false;
|
|
100
|
-
}
|
|
101
|
-
if (!fill_size_array(strides, view_strides, tmp.ndim)) {
|
|
102
|
-
free(view_strides);
|
|
103
|
-
release_memview_common(&tmp);
|
|
104
|
-
return false;
|
|
105
|
-
}
|
|
106
|
-
tmp.strides = view_strides;
|
|
107
|
-
|
|
108
75
|
VALUE sub_offsets = rb_hash_aref(desc, sym_sub_offsets);
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
76
|
+
|
|
77
|
+
size_t num_ary = NIL_P(sub_offsets) ? 2 : 3;
|
|
78
|
+
long format_len = RSTRING_LEN(format);
|
|
79
|
+
struct private_data *private_data = malloc(sizeof(struct private_data) + sizeof(ssize_t) * tmp.ndim * num_ary + sizeof(char) * (format_len + 1));
|
|
80
|
+
if (!private_data) {
|
|
113
81
|
return false;
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
82
|
+
}
|
|
83
|
+
ssize_t pdt_idx;
|
|
84
|
+
if (!fill_size_array(shape, &private_data->buf[0], tmp.ndim)) {
|
|
85
|
+
free(private_data);
|
|
118
86
|
return false;
|
|
119
|
-
}
|
|
120
|
-
tmp.sub_offsets = view_sub_offsets;
|
|
121
87
|
}
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
rb_warn("failed to alloc private_data");
|
|
127
|
-
return false;
|
|
88
|
+
tmp.shape = &private_data->buf[0];
|
|
89
|
+
if (!fill_size_array(strides, &private_data->buf[tmp.ndim], tmp.ndim)) {
|
|
90
|
+
free(private_data);
|
|
91
|
+
return false;
|
|
128
92
|
}
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
93
|
+
tmp.strides = &private_data->buf[tmp.ndim];
|
|
94
|
+
if (!NIL_P(sub_offsets)) {
|
|
95
|
+
pdt_idx = tmp.ndim * 2;
|
|
96
|
+
if (!fill_size_array(sub_offsets, &private_data->buf[pdt_idx], tmp.ndim)) {
|
|
97
|
+
free(private_data);
|
|
98
|
+
return false;
|
|
99
|
+
}
|
|
100
|
+
tmp.sub_offsets = &private_data->buf[pdt_idx];
|
|
101
|
+
}
|
|
102
|
+
pdt_idx = tmp.ndim * num_ary;
|
|
103
|
+
memcpy((void *)&private_data->buf[pdt_idx], view_format, format_len + 1);
|
|
104
|
+
tmp.format = (char *)&private_data->buf[pdt_idx];
|
|
105
|
+
private_data->data = data;
|
|
106
|
+
rb_gc_register_address(&private_data->data);
|
|
107
|
+
tmp.private_data = (void *)private_data;
|
|
133
108
|
|
|
134
109
|
*view = tmp;
|
|
135
110
|
|
|
@@ -139,13 +114,12 @@ ndav_memory_viewable_get_memory_view(const VALUE ndav, rb_memory_view_t *view, i
|
|
|
139
114
|
static bool
|
|
140
115
|
ndav_memory_viewable_release_memory_view(const VALUE ndav, rb_memory_view_t *view)
|
|
141
116
|
{
|
|
142
|
-
release_memview_common(view);
|
|
143
117
|
if (view->private_data) {
|
|
144
|
-
|
|
145
|
-
rb_gc_unregister_address(&private_data->
|
|
146
|
-
free(private_data);
|
|
147
|
-
view->private_data = NULL;
|
|
118
|
+
struct private_data *private_data = view->private_data;
|
|
119
|
+
rb_gc_unregister_address(&private_data->data);
|
|
148
120
|
}
|
|
121
|
+
free(view->private_data);
|
|
122
|
+
view->private_data = NULL;
|
|
149
123
|
|
|
150
124
|
return true;
|
|
151
125
|
}
|
|
@@ -246,6 +220,7 @@ void
|
|
|
246
220
|
Init_ndav(void)
|
|
247
221
|
{
|
|
248
222
|
id_ndav_validated_descriptor = rb_intern("ndav_validated_descriptor");
|
|
223
|
+
sym_data = ID2SYM(rb_intern("data"));
|
|
249
224
|
sym_addr = ID2SYM(rb_intern("addr"));
|
|
250
225
|
sym_shape = ID2SYM(rb_intern("shape"));
|
|
251
226
|
sym_strides = ID2SYM(rb_intern("strides"));
|
data/lib/ndav/converter.rb
CHANGED
|
@@ -24,7 +24,7 @@ class NDAV
|
|
|
24
24
|
unless mdl.const_defined?(:ToNDAV)
|
|
25
25
|
mdl.const_set(:ToNDAV, Module.new {
|
|
26
26
|
def to_ndav(lifetime: self, **)
|
|
27
|
-
::NDAV.
|
|
27
|
+
::NDAV.new_override(self, lifetime:, **)
|
|
28
28
|
end
|
|
29
29
|
})
|
|
30
30
|
end
|
|
@@ -54,7 +54,7 @@ class NDAV
|
|
|
54
54
|
elsif array.respond_to?(:to_ndav)
|
|
55
55
|
array.to_ndav(*, **)
|
|
56
56
|
else
|
|
57
|
-
::NDAV.
|
|
57
|
+
::NDAV.new_override(array, *, **)
|
|
58
58
|
end
|
|
59
59
|
end
|
|
60
60
|
end
|
data/lib/ndav/flags.rb
CHANGED
|
@@ -13,11 +13,13 @@ class NDAV
|
|
|
13
13
|
|
|
14
14
|
module_function
|
|
15
15
|
|
|
16
|
-
def
|
|
16
|
+
def decode_for_request(flags)
|
|
17
17
|
FLAGS.each_with_object({}) {|(name, value), state|
|
|
18
18
|
state[name] = case name
|
|
19
19
|
when :simple
|
|
20
20
|
flags == value
|
|
21
|
+
when :row_major, :column_major
|
|
22
|
+
flags & FLAGS[:any_contiguous] == FLAGS[name]
|
|
21
23
|
when :any_contiguous
|
|
22
24
|
(flags & FLAGS[:row_major]) == FLAGS[:row_major] ||
|
|
23
25
|
(flags & FLAGS[:column_major]) == FLAGS[:column_major]
|
|
@@ -26,5 +28,9 @@ class NDAV
|
|
|
26
28
|
end
|
|
27
29
|
}
|
|
28
30
|
end
|
|
31
|
+
|
|
32
|
+
def combine(*flags)
|
|
33
|
+
flags.reduce(FLAGS[:simple]) {|encoded, flag| encoded | FLAGS[flag]}
|
|
34
|
+
end
|
|
29
35
|
end
|
|
30
36
|
end
|
data/lib/ndav/memory_viewable.rb
CHANGED
|
@@ -5,6 +5,7 @@ class NDAV
|
|
|
5
5
|
# +#ndav_descriptor+ 's keyword arguments are flags of MemoryView including
|
|
6
6
|
# +simple+, +writable+, +format+, +multi_dimensional+, +strides+, +row_major+, +column_major+, +any_contiguous+, +indirect+.
|
|
7
7
|
# For the meaning of them, see MemoryView document.
|
|
8
|
+
# +any_contiguous+ ' meaning is ambiguous but assumed as +row_major+ **OR** +column_major+ is requested here.
|
|
8
9
|
#
|
|
9
10
|
# It must return a Hash whose keys includes:
|
|
10
11
|
#
|
|
@@ -52,66 +53,85 @@ class NDAV
|
|
|
52
53
|
|
|
53
54
|
private
|
|
54
55
|
|
|
56
|
+
# TODO: Distinguish validation error and other errors
|
|
55
57
|
def ndav_validated_descriptor(flags = ::NDAV::Flags::FLAGS[:simple])
|
|
56
|
-
flags = ::NDAV::Flags.
|
|
58
|
+
flags = ::NDAV::Flags.decode_for_request(flags)
|
|
57
59
|
desc = ndav_descriptor(**flags).to_h
|
|
58
60
|
desc => {data:, shape:, strides:, format:, byte_size:, readonly?: readonly}
|
|
59
61
|
sub_offsets = desc[:sub_offsets]
|
|
60
62
|
|
|
61
63
|
[:data, :shape, :strides, :format, :byte_size, :readonly?].each do |key|
|
|
62
64
|
if desc[key].nil?
|
|
63
|
-
|
|
64
|
-
return false
|
|
65
|
+
raise ":#{key} is nil"
|
|
65
66
|
end
|
|
66
67
|
end
|
|
67
68
|
[:shape, :strides].each do |key|
|
|
68
69
|
unless desc[key].instance_of? Array
|
|
69
|
-
|
|
70
|
-
return false
|
|
70
|
+
raise "#{key} not Array"
|
|
71
71
|
end
|
|
72
72
|
end
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
73
|
+
ndim = shape.length
|
|
74
|
+
raise "wrong length of strides" unless strides.length == ndim
|
|
75
|
+
if sub_offsets
|
|
76
|
+
raise "sub_offsets not Array" unless sub_offsets.instance_of?(Array)
|
|
77
|
+
raise "wrong length of sub_offsets" unless sub_offsets.length == ndim
|
|
76
78
|
end
|
|
79
|
+
format = format.to_s
|
|
77
80
|
item_size = ::NDAV::ITEM_SIZES[format]
|
|
78
|
-
unless shape.reduce(item_size, :*) == byte_size
|
|
79
|
-
warn "item size from format and shape not match byte_size"
|
|
80
|
-
return false
|
|
81
|
-
end
|
|
82
81
|
|
|
83
82
|
if flags[:writable] && readonly
|
|
84
|
-
|
|
85
|
-
return false
|
|
83
|
+
raise "not writable"
|
|
86
84
|
end
|
|
87
85
|
|
|
88
86
|
# TODO: Research the meaning of MULTI_DIMENSIONAL and validate it
|
|
89
87
|
|
|
90
88
|
if flags[:indirect] && !sub_offsets
|
|
91
|
-
|
|
92
|
-
return false
|
|
89
|
+
raise "indirect requested but sub_offsets is NULL"
|
|
93
90
|
end
|
|
94
91
|
|
|
95
92
|
row_major_contiguous = ::NDAV.row_major_contiguous?(format:, shape:, strides:)
|
|
96
93
|
column_major_contiguous = ::NDAV.column_major_contiguous?(format:, shape:, strides:)
|
|
97
94
|
if flags[:row_major] && !row_major_contiguous
|
|
98
|
-
|
|
95
|
+
raise "row-major contiguous requested but it isn't"
|
|
99
96
|
end
|
|
100
97
|
if flags[:column_major] && !column_major_contiguous
|
|
101
|
-
|
|
98
|
+
raise "column-major contiguous requested but it isn't"
|
|
102
99
|
end
|
|
103
100
|
if flags[:any_contiguous] && !row_major_contiguous && !column_major_contiguous
|
|
104
|
-
|
|
101
|
+
raise "contiguous requested but it isn't"
|
|
102
|
+
end
|
|
103
|
+
if row_major_contiguous || column_major_contiguous
|
|
104
|
+
unless shape.reduce(item_size, :*) == byte_size
|
|
105
|
+
raise "item size from format and shape not match byte_size"
|
|
106
|
+
end
|
|
107
|
+
else
|
|
108
|
+
max_offset = 0
|
|
109
|
+
min_offset = 0
|
|
110
|
+
shape.zip(strides) do |s, stride|
|
|
111
|
+
next if s == 0
|
|
112
|
+
|
|
113
|
+
extent = (s - 1) * stride
|
|
114
|
+
max_offset += [extent, 0].max
|
|
115
|
+
min_offset += [extent, 0].min
|
|
116
|
+
end
|
|
117
|
+
if min_offset < 0
|
|
118
|
+
raise "min offset must be positive"
|
|
119
|
+
end
|
|
120
|
+
if max_offset + item_size > byte_size
|
|
121
|
+
raise "max offset too large"
|
|
122
|
+
end
|
|
105
123
|
end
|
|
106
124
|
|
|
107
125
|
desc.merge(
|
|
108
126
|
addr: ::Fiddle::Pointer.to_ptr(data).to_i,
|
|
109
|
-
ndim
|
|
127
|
+
ndim:,
|
|
128
|
+
format:,
|
|
110
129
|
item_size:,
|
|
111
130
|
sub_offsets:
|
|
112
131
|
)
|
|
113
132
|
rescue => err
|
|
114
133
|
warn err
|
|
134
|
+
warn err.backtrace.join($/) if $VERBOSE
|
|
115
135
|
return false
|
|
116
136
|
end
|
|
117
137
|
end
|
data/lib/ndav.rb
CHANGED
|
@@ -17,7 +17,7 @@ class NDAV
|
|
|
17
17
|
8 => "q"
|
|
18
18
|
}
|
|
19
19
|
|
|
20
|
-
attr_reader :ndim, :shape, :format, :item_size, :byte_size, :strides, :sub_offsets, :
|
|
20
|
+
attr_reader :ndim, :shape, :format, :item_size, :byte_size, :strides, :sub_offsets, :lifetime
|
|
21
21
|
|
|
22
22
|
class << self
|
|
23
23
|
# * Doesn't modify byte_size
|
|
@@ -104,58 +104,58 @@ class NDAV
|
|
|
104
104
|
end
|
|
105
105
|
end
|
|
106
106
|
|
|
107
|
-
def initialize(obj,
|
|
108
|
-
|
|
109
|
-
# For instance, Fiddle::Pointer's MemoryView is readonly when writable flag is not passed and is not readonly when writable flag is passed.
|
|
110
|
-
# How should we solve this mismatching? Delaying to determine the flags?
|
|
111
|
-
# Considers when it becomes actual problem.
|
|
112
|
-
@fmv = obj.kind_of?(::Fiddle::MemoryView) ? obj : ::Fiddle::MemoryView.new(obj)
|
|
113
|
-
# Fiddle::MemoryView#to_s is zero-copy
|
|
114
|
-
# Fiddle::MemoryView#to_s embeds obj into the String, so we can prevent GC from collecting it
|
|
115
|
-
@to_s = @fmv.to_s
|
|
107
|
+
def initialize(obj, format: nil, item_size: nil, shape: nil, strides: nil, sub_offsets: nil, lifetime: nil)
|
|
108
|
+
@obj = obj
|
|
116
109
|
@lifetime = lifetime
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
110
|
+
is_fmv = @obj.kind_of? Fiddle::MemoryView
|
|
111
|
+
fmv = is_fmv ? @obj : Fiddle::MemoryView.new(@obj)
|
|
112
|
+
begin
|
|
113
|
+
@byte_size = fmv.byte_size
|
|
114
|
+
|
|
115
|
+
case [format, item_size]
|
|
116
|
+
in [nil, nil]
|
|
117
|
+
@format = fmv.format || "C"
|
|
118
|
+
@item_size = fmv.item_size
|
|
119
|
+
in [_, nil]
|
|
120
|
+
@format = format
|
|
121
|
+
@item_size = ITEM_SIZES[@format]
|
|
122
|
+
in [nil, _]
|
|
123
|
+
@item_size = item_size
|
|
124
|
+
@format = DEFAULT_FORMATS[@item_size] || fmv.format
|
|
125
|
+
else
|
|
126
|
+
@format = format
|
|
127
|
+
@item_size = item_size
|
|
128
|
+
end
|
|
133
129
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
130
|
+
case [shape, @byte_size]
|
|
131
|
+
in [nil, nil]
|
|
132
|
+
@shape = fmv.shape
|
|
133
|
+
@byte_size = @shape.reduce(@item_size, :*)
|
|
134
|
+
in [_, nil]
|
|
135
|
+
@shape = shape
|
|
136
|
+
@byte_size = @shape.reduce(@item_size, :*)
|
|
137
|
+
in [nil, _]
|
|
138
|
+
@shape = fmv.shape || [@byte_size / @item_size]
|
|
139
|
+
else
|
|
140
|
+
@shape = shape
|
|
141
|
+
end
|
|
146
142
|
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
143
|
+
@ndim = @shape.size
|
|
144
|
+
@strides = strides || fmv.strides || self.class.default_strides(shape: @shape, item_size: @item_size)
|
|
145
|
+
@sub_offsets = sub_offsets || fmv.sub_offsets
|
|
146
|
+
@row_major_contiguous = self.class.row_major_contiguous?(format: @format, shape: @shape, strides: @strides)
|
|
147
|
+
@column_major_contiguous = self.class.column_major_contiguous?(format: @format, shape: @shape, strides: @strides)
|
|
148
|
+
ensure
|
|
149
|
+
fmv.release unless is_fmv
|
|
150
|
+
end
|
|
153
151
|
|
|
154
152
|
validate
|
|
155
153
|
end
|
|
156
154
|
|
|
157
|
-
def
|
|
158
|
-
|
|
155
|
+
def to_s
|
|
156
|
+
# Fiddle::MemoryView#to_s is zero-copy
|
|
157
|
+
# Fiddle::MemoryView#to_s embeds obj into the String, so we can prevent GC from collecting it
|
|
158
|
+
(@obj.kind_of?(Fiddle::MemoryView) ? @obj : Fiddle::MemoryView.new(@obj)).to_s
|
|
159
159
|
end
|
|
160
160
|
|
|
161
161
|
def item_desc
|
|
@@ -171,27 +171,33 @@ class NDAV
|
|
|
171
171
|
end
|
|
172
172
|
|
|
173
173
|
def to_ptr
|
|
174
|
-
::Fiddle::Pointer[
|
|
174
|
+
::Fiddle::Pointer[to_s]
|
|
175
175
|
end
|
|
176
176
|
alias to_fiddle_pointer to_ptr
|
|
177
177
|
|
|
178
|
-
def release
|
|
179
|
-
@fmv.release
|
|
180
|
-
@fmv = @ndim = @readonly = @shape = @format = @item_size = @byte_size = @strides = @sub_offsets = @lifetime = @to_s = @row_major_contiguous = @column_major_contiguous = nil
|
|
181
|
-
end
|
|
182
|
-
|
|
183
178
|
private
|
|
184
179
|
|
|
185
|
-
def ndav_descriptor(**)
|
|
186
|
-
|
|
187
|
-
|
|
180
|
+
def ndav_descriptor(writable:, **)
|
|
181
|
+
fmv = if @obj.kind_of? Fiddle::MemoryView
|
|
182
|
+
raise "not writable" if writable && @obj.readonly?
|
|
183
|
+
@obj
|
|
184
|
+
else
|
|
185
|
+
if Fiddle::VERSION <= "1.1.8"
|
|
186
|
+
fmv = Fiddle::MemoryView.new(@obj)
|
|
187
|
+
raise "not writable" if writable && fmv.readonly?
|
|
188
|
+
fmv
|
|
189
|
+
else
|
|
190
|
+
flgs = Flags.combine(writable ? :writable : :simple)
|
|
191
|
+
Fiddle::MemoryView.new(@obj, flgs)
|
|
192
|
+
end
|
|
193
|
+
end
|
|
188
194
|
{
|
|
189
|
-
data:
|
|
195
|
+
data: fmv.to_s,
|
|
190
196
|
shape: @shape,
|
|
191
197
|
strides: @strides,
|
|
192
198
|
format: @format,
|
|
193
199
|
byte_size: @byte_size,
|
|
194
|
-
readonly?:
|
|
200
|
+
readonly?: fmv.readonly?,
|
|
195
201
|
sub_offsets: @sub_offsets
|
|
196
202
|
}
|
|
197
203
|
end
|
|
@@ -206,14 +212,14 @@ class NDAV
|
|
|
206
212
|
r = @byte_size % @item_size
|
|
207
213
|
raise ArgumentError, "byte_size must be n-times of item_size" unless r.zero?
|
|
208
214
|
|
|
209
|
-
raise ArgumentError, "shape length
|
|
215
|
+
raise ArgumentError, "shape length doesn't match ndim" unless @shape.length == @ndim
|
|
210
216
|
if @row_major_contiguous || @column_major_contiguous
|
|
211
217
|
unless @byte_size == @shape.reduce(@item_size, :*)
|
|
212
|
-
raise ArgumentError, "shape not match byte_size"
|
|
218
|
+
raise ArgumentError, "shape not match byte_size. byte_size: #{@byte_size}, item_size: #{@item_size}, shape: #{@shape}"
|
|
213
219
|
end
|
|
214
220
|
end
|
|
215
221
|
|
|
216
|
-
raise ArgumentError, "strides length
|
|
222
|
+
raise ArgumentError, "strides length doesn't match ndim" unless @strides.length == @ndim
|
|
217
223
|
if @row_major_contiguous
|
|
218
224
|
stride = @item_size
|
|
219
225
|
@strides.reverse_each.with_index do |s, i|
|
|
@@ -234,7 +240,7 @@ class NDAV
|
|
|
234
240
|
# NDAV uses Fiddle::MemoryView#to_s and it doesn't allow negative min offset
|
|
235
241
|
# Some MemoryView might allow negative min offset
|
|
236
242
|
# Remove this check if such the MemoryView producer is found
|
|
237
|
-
raise ArgumentError, "min offset must be
|
|
243
|
+
raise ArgumentError, "min offset must be non-negative"
|
|
238
244
|
end
|
|
239
245
|
raise ArgumentError, "max offset too large" if max_offset + @item_size > @byte_size
|
|
240
246
|
end
|
data/ndav.gemspec
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Gem::Specification.new do |s|
|
|
2
2
|
s.name = "ndav"
|
|
3
|
-
s.version = "0.0.
|
|
3
|
+
s.version = "0.0.9"
|
|
4
4
|
s.summary = "N-Dimensional Array View"
|
|
5
5
|
s.authors = ["Kitaiti Makoto"]
|
|
6
6
|
s.licenses = ["BSD-2-Clause"]
|
|
@@ -20,6 +20,5 @@ Gem::Specification.new do |s|
|
|
|
20
20
|
s.add_development_dependency "rubygems-tasks"
|
|
21
21
|
s.add_development_dependency "rubygems-requirements-system"
|
|
22
22
|
s.add_development_dependency "red-arrow"
|
|
23
|
-
s.add_development_dependency "ffi"
|
|
24
23
|
s.add_development_dependency "yard"
|
|
25
24
|
end
|
data/test/helper.rb
CHANGED
|
@@ -4,7 +4,6 @@ require "test/unit/notify"
|
|
|
4
4
|
require "fiddle"
|
|
5
5
|
require "arrow"
|
|
6
6
|
require "ndav"
|
|
7
|
-
require "ndav/ffi"
|
|
8
7
|
|
|
9
8
|
class TestBase < Test::Unit::TestCase
|
|
10
9
|
private
|
|
@@ -16,7 +15,6 @@ class TestBase < Test::Unit::TestCase
|
|
|
16
15
|
assert_equal [2], array.strides
|
|
17
16
|
assert_equal 2, array.item_size
|
|
18
17
|
assert_equal 6, array.byte_size
|
|
19
|
-
assert_true array.readonly?
|
|
20
18
|
assert_equal [1, 2, 3].pack("s*"), array.to_s
|
|
21
19
|
end
|
|
22
20
|
end
|
data/test/test_flags.rb
CHANGED
|
@@ -4,25 +4,19 @@ class TestFlags < TestBase
|
|
|
4
4
|
include NDAV::Flags
|
|
5
5
|
|
|
6
6
|
def test_any_contiguous
|
|
7
|
-
decoded =
|
|
8
|
-
|
|
9
|
-
|
|
7
|
+
decoded = decode_for_request(combine(:any_contiguous))
|
|
8
|
+
assert_false decoded[:row_major]
|
|
9
|
+
assert_false decoded[:column_major]
|
|
10
10
|
assert_true decoded[:any_contiguous]
|
|
11
11
|
|
|
12
|
-
decoded =
|
|
12
|
+
decoded = decode_for_request(combine(:row_major))
|
|
13
13
|
assert_true decoded[:row_major]
|
|
14
14
|
assert_false decoded[:column_major]
|
|
15
15
|
assert_true decoded[:any_contiguous]
|
|
16
16
|
|
|
17
|
-
decoded =
|
|
17
|
+
decoded = decode_for_request(combine(:column_major))
|
|
18
18
|
assert_false decoded[:row_major]
|
|
19
19
|
assert_true decoded[:column_major]
|
|
20
20
|
assert_true decoded[:any_contiguous]
|
|
21
21
|
end
|
|
22
|
-
|
|
23
|
-
private
|
|
24
|
-
|
|
25
|
-
def combine_flags(*flags)
|
|
26
|
-
flags.reduce(0) {|composed, name| composed | FLAGS[name]}
|
|
27
|
-
end
|
|
28
22
|
end
|
data/test/test_ndav.rb
CHANGED
|
@@ -29,6 +29,27 @@ class TestNDAV < TestBase
|
|
|
29
29
|
assert_int_array ndav
|
|
30
30
|
end
|
|
31
31
|
|
|
32
|
+
def test_flags
|
|
33
|
+
omit if Fiddle::VERSION <= "1.1.8"
|
|
34
|
+
|
|
35
|
+
flags = NDAV::Flags.combine(:writable)
|
|
36
|
+
ptr = Fiddle::Pointer[[1, 2, 3].pack("s*")]
|
|
37
|
+
ndav = NDAV.new(ptr)
|
|
38
|
+
|
|
39
|
+
Fiddle::MemoryView.export ndav do |memview|
|
|
40
|
+
assert_true memview.readonly?
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
Fiddle::MemoryView.export ndav, flags do |memview|
|
|
44
|
+
assert_false memview.readonly?
|
|
45
|
+
|
|
46
|
+
p = Fiddle::Pointer[memview.to_s]
|
|
47
|
+
p[0] = 4
|
|
48
|
+
assert_equal [4, 2, 3], memview.to_s.unpack("s*")
|
|
49
|
+
end
|
|
50
|
+
assert_equal [4, 2, 3], ndav.to_s.unpack("s*")
|
|
51
|
+
end
|
|
52
|
+
|
|
32
53
|
def test_string
|
|
33
54
|
str = [1, 2, 3].pack("s*")
|
|
34
55
|
ndav = NDAV.from_string(str, format: "s")
|
|
@@ -57,18 +78,6 @@ class TestNDAV < TestBase
|
|
|
57
78
|
assert_true ndav.column_major_contiguous?
|
|
58
79
|
end
|
|
59
80
|
|
|
60
|
-
def test_release
|
|
61
|
-
ndav = NDAV.new(@array)
|
|
62
|
-
|
|
63
|
-
ndav.release
|
|
64
|
-
[:ndim, :shape, :format, :item_size, :byte_size, :strides, :sub_offsets, :lifetime, :to_s].each do |attr|
|
|
65
|
-
assert_nil ndav.send(attr)
|
|
66
|
-
end
|
|
67
|
-
assert_nil ndav.readonly?
|
|
68
|
-
assert_nil ndav.row_major_contiguous?
|
|
69
|
-
assert_nil ndav.column_major_contiguous?
|
|
70
|
-
end
|
|
71
|
-
|
|
72
81
|
def test_parse_item_format
|
|
73
82
|
assert_equal [
|
|
74
83
|
NDAV::ItemComponent.new("f", false, IO::Buffer::HOST_ENDIAN == IO::Buffer::LITTLE_ENDIAN, 0, 4, 1)
|
|
@@ -190,17 +199,5 @@ class TestNDAV < TestBase
|
|
|
190
199
|
assert_equal 6, memview.byte_size
|
|
191
200
|
end
|
|
192
201
|
end
|
|
193
|
-
|
|
194
|
-
def test_overwrite_readonly?
|
|
195
|
-
assert_raise ArgumentError do
|
|
196
|
-
NDAV.new(@array, readonly: false)
|
|
197
|
-
end
|
|
198
|
-
|
|
199
|
-
ndav = NDAV.new_override(@array, readonly: false)
|
|
200
|
-
Fiddle::MemoryView.export ndav do |memview|
|
|
201
|
-
assert_false memview.readonly?
|
|
202
|
-
assert_equal 6, memview.byte_size
|
|
203
|
-
end
|
|
204
|
-
end
|
|
205
202
|
end
|
|
206
203
|
end
|
data/test/test_ractor.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ndav
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.0.
|
|
4
|
+
version: 0.0.9
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Kitaiti Makoto
|
|
@@ -135,20 +135,6 @@ dependencies:
|
|
|
135
135
|
- - ">="
|
|
136
136
|
- !ruby/object:Gem::Version
|
|
137
137
|
version: '0'
|
|
138
|
-
- !ruby/object:Gem::Dependency
|
|
139
|
-
name: ffi
|
|
140
|
-
requirement: !ruby/object:Gem::Requirement
|
|
141
|
-
requirements:
|
|
142
|
-
- - ">="
|
|
143
|
-
- !ruby/object:Gem::Version
|
|
144
|
-
version: '0'
|
|
145
|
-
type: :development
|
|
146
|
-
prerelease: false
|
|
147
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
148
|
-
requirements:
|
|
149
|
-
- - ">="
|
|
150
|
-
- !ruby/object:Gem::Version
|
|
151
|
-
version: '0'
|
|
152
138
|
- !ruby/object:Gem::Dependency
|
|
153
139
|
name: yard
|
|
154
140
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -179,7 +165,6 @@ files:
|
|
|
179
165
|
- ext/ndav.c
|
|
180
166
|
- lib/ndav.rb
|
|
181
167
|
- lib/ndav/converter.rb
|
|
182
|
-
- lib/ndav/ffi.rb
|
|
183
168
|
- lib/ndav/flags.rb
|
|
184
169
|
- lib/ndav/import.rb
|
|
185
170
|
- lib/ndav/item_component.rb
|
|
@@ -188,7 +173,6 @@ files:
|
|
|
188
173
|
- test/helper.rb
|
|
189
174
|
- test/test_converter.rb
|
|
190
175
|
- test/test_export.rb
|
|
191
|
-
- test/test_ffi.rb
|
|
192
176
|
- test/test_flags.rb
|
|
193
177
|
- test/test_ndav.rb
|
|
194
178
|
- test/test_package.rb
|
|
@@ -212,7 +196,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
212
196
|
- !ruby/object:Gem::Version
|
|
213
197
|
version: '0'
|
|
214
198
|
requirements: []
|
|
215
|
-
rubygems_version: 4.0.
|
|
199
|
+
rubygems_version: 4.0.20
|
|
216
200
|
specification_version: 4
|
|
217
201
|
summary: N-Dimensional Array View
|
|
218
202
|
test_files: []
|
data/lib/ndav/ffi.rb
DELETED
|
@@ -1,97 +0,0 @@
|
|
|
1
|
-
require "ffi"
|
|
2
|
-
|
|
3
|
-
class NDAV
|
|
4
|
-
module FFI
|
|
5
|
-
TYPE_SIZE_TO_FORMAT = {
|
|
6
|
-
1 => "c",
|
|
7
|
-
2 => "s",
|
|
8
|
-
4 => "l",
|
|
9
|
-
8 => "q"
|
|
10
|
-
}
|
|
11
|
-
|
|
12
|
-
module MemoryPointer
|
|
13
|
-
module FromNDAV
|
|
14
|
-
# NOTE: Copies data
|
|
15
|
-
def from_ndav(array)
|
|
16
|
-
from_string(array.to_s)
|
|
17
|
-
end
|
|
18
|
-
end
|
|
19
|
-
|
|
20
|
-
module ToNDAV
|
|
21
|
-
def to_ndav(format: TYPE_SIZE_TO_FORMAT[type_size], lifetime: self, **)
|
|
22
|
-
shape = [size / type_size]
|
|
23
|
-
byte_size = shape.reduce(::NDAV::ITEM_SIZES[format], :*)
|
|
24
|
-
ptr = ::Fiddle::Pointer.new(address, byte_size)
|
|
25
|
-
::NDAV.new(ptr, shape:, format:, lifetime:, **)
|
|
26
|
-
end
|
|
27
|
-
end
|
|
28
|
-
|
|
29
|
-
module MemoryViewable
|
|
30
|
-
def ndav_descriptor(**)
|
|
31
|
-
shape = [size / type_size]
|
|
32
|
-
format = TYPE_SIZE_TO_FORMAT[type_size]
|
|
33
|
-
item_size = ::NDAV::ITEM_SIZES[format]
|
|
34
|
-
byte_size = shape.reduce(item_size, :*)
|
|
35
|
-
ptr = ::Fiddle::Pointer.new(address, byte_size)
|
|
36
|
-
|
|
37
|
-
{
|
|
38
|
-
data: Fiddle::MemoryView.new(ptr).to_s,
|
|
39
|
-
shape:,
|
|
40
|
-
strides: ::NDAV.default_strides(shape:, item_size:),
|
|
41
|
-
format:,
|
|
42
|
-
byte_size:,
|
|
43
|
-
readonly?: true
|
|
44
|
-
}
|
|
45
|
-
end
|
|
46
|
-
end
|
|
47
|
-
|
|
48
|
-
module Converter
|
|
49
|
-
def FFIMemoryPointer(array, *, **)
|
|
50
|
-
if ::FFI::MemoryPointer === array
|
|
51
|
-
array
|
|
52
|
-
elsif array.respond_to? :to_ffi_memory_pointer
|
|
53
|
-
array.to_ffi_memory_pointer(*, **)
|
|
54
|
-
elsif array.respond_to? :to_ndav
|
|
55
|
-
array.to_ndav(*, **).to_ffi_memory_pointer
|
|
56
|
-
else
|
|
57
|
-
::NDAV.new(array, *, **).to_ffi_memory_pointer
|
|
58
|
-
end
|
|
59
|
-
end
|
|
60
|
-
end
|
|
61
|
-
end
|
|
62
|
-
|
|
63
|
-
module Pointer
|
|
64
|
-
module FromNDAV
|
|
65
|
-
def from_ndav(array)
|
|
66
|
-
new(array.item_size, array.to_ptr.to_i)
|
|
67
|
-
end
|
|
68
|
-
end
|
|
69
|
-
|
|
70
|
-
module ToNDAV
|
|
71
|
-
def to_ndav(shape:, format: TYPE_SIZE_TO_FORMAT[type_size], lifetime: self, **)
|
|
72
|
-
byte_size = shape.reduce(::NDAV::ITEM_SIZES[format], :*)
|
|
73
|
-
ptr = ::Fiddle::Pointer.new(address, byte_size)
|
|
74
|
-
::NDAV.new(ptr, shape:, format:, lifetime:, **)
|
|
75
|
-
end
|
|
76
|
-
end
|
|
77
|
-
|
|
78
|
-
module Converter
|
|
79
|
-
def FFIPointer(array, *, **)
|
|
80
|
-
# use instance_of? instead of kind_of? beacuase MemoryPointer is subclass of Pointer
|
|
81
|
-
if array.instance_of? ::FFI::Pointer
|
|
82
|
-
array
|
|
83
|
-
elsif array.respond_to? :to_ffi_pointer
|
|
84
|
-
array.to_ffi_pointer(*, **)
|
|
85
|
-
elsif array.respond_to? :to_ndav
|
|
86
|
-
array.to_ndav(*, **).to_ffi_pointer
|
|
87
|
-
else
|
|
88
|
-
::NDAV.new(array, *, **).to_ffi_pointer
|
|
89
|
-
end
|
|
90
|
-
end
|
|
91
|
-
end
|
|
92
|
-
end
|
|
93
|
-
|
|
94
|
-
::NDAV.register ::FFI::MemoryPointer, MemoryPointer, name: :ffi_memory_pointer
|
|
95
|
-
::NDAV.register ::FFI::Pointer, Pointer, name: :ffi_pointer
|
|
96
|
-
end
|
|
97
|
-
end
|
data/test/test_ffi.rb
DELETED
|
@@ -1,117 +0,0 @@
|
|
|
1
|
-
require_relative "helper"
|
|
2
|
-
|
|
3
|
-
class TestFFI < TestBase
|
|
4
|
-
def test_ndav_from_ffi_memory_pointer
|
|
5
|
-
ptr = ::FFI::MemoryPointer.new(:int16, 3)
|
|
6
|
-
ptr.write_array_of_int16([1, 2, 3])
|
|
7
|
-
ndav = ::NDAV.from_ffi_memory_pointer(ptr, shape: [3], format: "s")
|
|
8
|
-
|
|
9
|
-
assert_int_array ndav
|
|
10
|
-
end
|
|
11
|
-
|
|
12
|
-
def test_ndav_from_ffi_pointer
|
|
13
|
-
str = [1, 2, 3].pack("s*")
|
|
14
|
-
address = ::Fiddle::Pointer[str].to_i
|
|
15
|
-
ptr = ::FFI::Pointer.new(:int16, address)
|
|
16
|
-
ndav = ::NDAV.from_ffi_pointer(ptr, shape: [3], format: "s", lifetime: str)
|
|
17
|
-
|
|
18
|
-
assert_int_array ndav
|
|
19
|
-
end
|
|
20
|
-
|
|
21
|
-
def test_ndav_to_ffi_memory_pointer
|
|
22
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
23
|
-
ptr = ndav.to_ffi_memory_pointer
|
|
24
|
-
|
|
25
|
-
assert_equal [1, 2, 3].pack("s*"), ptr.read_string(ndav.byte_size)
|
|
26
|
-
end
|
|
27
|
-
|
|
28
|
-
def test_ndav_to_ffi_pointer
|
|
29
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
30
|
-
ptr = ndav.to_ffi_pointer
|
|
31
|
-
|
|
32
|
-
assert_equal [1, 2, 3].pack("s*"), ptr.read_string(ndav.byte_size)
|
|
33
|
-
assert_equal ndav.to_ptr.to_i, ptr.address
|
|
34
|
-
end
|
|
35
|
-
|
|
36
|
-
def test_ffi_memory_pointer_from_ndav
|
|
37
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
38
|
-
ptr = ::FFI::MemoryPointer.from_ndav(ndav)
|
|
39
|
-
|
|
40
|
-
assert_equal [1, 2, 3].pack("s*"), ptr.read_string(ndav.byte_size)
|
|
41
|
-
end
|
|
42
|
-
|
|
43
|
-
def test_ffi_pointer_from_ndav
|
|
44
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
45
|
-
ptr = ::FFI::Pointer.from_ndav(ndav)
|
|
46
|
-
|
|
47
|
-
assert_equal [1, 2, 3].pack("s*"), ptr.read_string(ndav.byte_size)
|
|
48
|
-
assert_equal ndav.to_ptr.to_i, ptr.address
|
|
49
|
-
end
|
|
50
|
-
|
|
51
|
-
def test_memory_view_from_ffi_memory_pointer
|
|
52
|
-
ptr = ::FFI::MemoryPointer.new(:int16, 3)
|
|
53
|
-
ptr.write_array_of_int16([1, 2, 3])
|
|
54
|
-
fmv = Fiddle::MemoryView.new(ptr)
|
|
55
|
-
|
|
56
|
-
assert_int_array fmv
|
|
57
|
-
end
|
|
58
|
-
|
|
59
|
-
class TestConverter < self
|
|
60
|
-
def test_ndav_from_ffi_memory_pointer
|
|
61
|
-
ptr = ::FFI::MemoryPointer.new(:int16, 3)
|
|
62
|
-
ptr.write_array_of_int16([1, 2, 3])
|
|
63
|
-
ndav = ::NDAV::Converter::NDAV(ptr)
|
|
64
|
-
|
|
65
|
-
assert_int_array ndav
|
|
66
|
-
end
|
|
67
|
-
|
|
68
|
-
def test_ffi_memory_pointer_from_ndav
|
|
69
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
70
|
-
|
|
71
|
-
assert_instance_of ::FFI::MemoryPointer, ::NDAV::Converter::FFIMemoryPointer(ndav)
|
|
72
|
-
end
|
|
73
|
-
|
|
74
|
-
def test_FFIMemoryPointer
|
|
75
|
-
c = Class.new {
|
|
76
|
-
include ::NDAV::Converter
|
|
77
|
-
|
|
78
|
-
def to_ffi_memory_pointer(ndav)
|
|
79
|
-
FFIMemoryPointer(ndav)
|
|
80
|
-
end
|
|
81
|
-
}
|
|
82
|
-
|
|
83
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
84
|
-
|
|
85
|
-
assert_instance_of ::FFI::MemoryPointer, c.new.to_ffi_memory_pointer(ndav)
|
|
86
|
-
end
|
|
87
|
-
|
|
88
|
-
def test_ndav_from_ffi_pointer
|
|
89
|
-
str = [1, 2, 3].pack("s*")
|
|
90
|
-
address = ::Fiddle::Pointer[str].to_i
|
|
91
|
-
ptr = ::FFI::Pointer.new(:int16, address)
|
|
92
|
-
ndav = ::NDAV::Converter::NDAV(ptr, shape: [3])
|
|
93
|
-
|
|
94
|
-
assert_int_array ndav
|
|
95
|
-
end
|
|
96
|
-
|
|
97
|
-
def test_ffi_pointer_from_ndav
|
|
98
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
99
|
-
|
|
100
|
-
assert_instance_of ::FFI::Pointer, ::NDAV::Converter::FFIPointer(ndav)
|
|
101
|
-
end
|
|
102
|
-
|
|
103
|
-
def test_FFIPointer
|
|
104
|
-
c = Class.new {
|
|
105
|
-
include ::NDAV::Converter
|
|
106
|
-
|
|
107
|
-
def to_ffi_pointer(ndav)
|
|
108
|
-
FFIPointer(ndav)
|
|
109
|
-
end
|
|
110
|
-
}
|
|
111
|
-
|
|
112
|
-
ndav = ::NDAV.new(Arrow::Int16Array.new([1, 2, 3]))
|
|
113
|
-
|
|
114
|
-
assert_instance_of ::FFI::Pointer, c.new.to_ffi_pointer(ndav)
|
|
115
|
-
end
|
|
116
|
-
end
|
|
117
|
-
end
|