flok 0.0.41 → 0.0.42
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/app/kern/pagers/pg_dummy.js +18 -0
- data/app/kern/services/vm.rb +361 -34
- data/docs/services/vm.md +69 -40
- data/docs/services/vm/diff.md +156 -0
- data/docs/services/vm/pagers.md +3 -2
- data/lib/flok/user_compiler.rb +3 -0
- data/lib/flok/version.rb +1 -1
- data/spec/env/kern.rb +19 -0
- data/spec/etc/user_compiler/controller0bg.rb +13 -0
- data/spec/etc/user_compiler_spec.rb +19 -12
- data/spec/kern/assets/global_on_entry5.rb +28 -0
- data/spec/kern/assets/vm/controller0_diff.rb +47 -0
- data/spec/kern/assets/vm/pg_dummy/config.rb +10 -0
- data/spec/kern/assets/vm/pg_mem/write.rb +1 -1
- data/spec/kern/assets/vm/pg_mem/write2.rb +1 -1
- data/spec/kern/assets/vm/vm_commit_pages.js +108 -0
- data/spec/kern/assets/vm/vm_diff_pages.js +140 -0
- data/spec/kern/assets/vm/vm_transaction_diff_pages.js +150 -0
- data/spec/kern/controller_spec.rb +19 -0
- data/spec/kern/vm_service_functional_spec.rb +1111 -0
- data/spec/kern/vm_service_spec.rb +84 -185
- data/spec/kern/vm_transaction_spec.rb +375 -0
- metadata +22 -4
- data/spec/kern/vm_service_spec2.rb +0 -39
@@ -0,0 +1,156 @@
|
|
1
|
+
#VM Diff
|
2
|
+
Information on the diff system of the vm service.
|
3
|
+
|
4
|
+
##How a diff is created
|
5
|
+
`vm_diff` embodies the creation of a diff. Some aspects of a diff, like a changed `_head` or a changed `_next` are just
|
6
|
+
simple comparisons. Entry specific vm_diff_entry types, like `+`, need to be in a special order. The order is
|
7
|
+
|
8
|
+
* `vm_diff_entry` types where order is unimportant
|
9
|
+
* `M` - Modifications do not rely on index
|
10
|
+
* `HEAD_M` - Head can change whereever
|
11
|
+
* `NEXT_M` - Next can change wherever
|
12
|
+
* `vm_diff_entry` types in order they should be in the `vm_diff` log:
|
13
|
+
1. Deletions (`-`)
|
14
|
+
2. Moves (`>`)
|
15
|
+
3. Insetions (`+`)
|
16
|
+
|
17
|
+
The order is special; imagine that we want to calculate a diff for a `from` list to a `to` list
|
18
|
+
|
19
|
+
```ruby
|
20
|
+
#from to
|
21
|
+
#---# #---#
|
22
|
+
#-A-# #-A-#
|
23
|
+
#-B-# #-D-#
|
24
|
+
#-F-# #-C-#
|
25
|
+
#-D-# #-B-#
|
26
|
+
#---# #-E-#
|
27
|
+
#---#
|
28
|
+
```
|
29
|
+
|
30
|
+
First we remove all entries in the `to` list that are not on the `from` list. All these entries are `+` entries, we take note of the index they were removed from.
|
31
|
+
|
32
|
+
```ruby
|
33
|
+
#Removed C at index 2 and E and index 4
|
34
|
+
#to
|
35
|
+
#---#
|
36
|
+
#-A-#
|
37
|
+
#-D-#
|
38
|
+
#-B-#
|
39
|
+
#---#
|
40
|
+
```
|
41
|
+
|
42
|
+
Second we remove all entries in the `from` list that are not in the `to` list. All the removed entries are `-` deletions.
|
43
|
+
```ruby
|
44
|
+
#Removed F
|
45
|
+
#from
|
46
|
+
#---#
|
47
|
+
#-A-#
|
48
|
+
#-B-#
|
49
|
+
#-D-#
|
50
|
+
#---#
|
51
|
+
```
|
52
|
+
|
53
|
+
Last, we compare the `to` and `from` lists making note of how we could remove and re-insert items in the `to` list to re-create the `from` list
|
54
|
+
```ruby
|
55
|
+
#from to
|
56
|
+
#---# #---#
|
57
|
+
#-A-# #-A-#
|
58
|
+
#-B-# => #-D-#
|
59
|
+
#-D-# #-B-#
|
60
|
+
#---# #---#
|
61
|
+
|
62
|
+
#1. Remove B and Insert at index 2 [">", "b_id", 2]
|
63
|
+
#from
|
64
|
+
#---#
|
65
|
+
#-A-#
|
66
|
+
#-D-#
|
67
|
+
#-B-#
|
68
|
+
#---#
|
69
|
+
```
|
70
|
+
Now we have all the pieces of the diff. If played in the order `delete`, `move`, and then `insert`, the resulting list will always be the same. `modifications` are position independent so they can be done at any time.
|
71
|
+
|
72
|
+
####Example replay `from => to`
|
73
|
+
```ruby
|
74
|
+
#from # diff
|
75
|
+
#---# # (-) F
|
76
|
+
#-A-# # (>) b_id to index:2
|
77
|
+
#-B-# # (+) C @ index 2
|
78
|
+
#-F-# # (+) E @ index 4
|
79
|
+
#-D-#
|
80
|
+
#---#
|
81
|
+
|
82
|
+
#1) (-) F
|
83
|
+
#---#
|
84
|
+
#-A-#
|
85
|
+
#-B-#
|
86
|
+
#-D-#
|
87
|
+
#---#
|
88
|
+
|
89
|
+
#2) (>) b_id to index:2
|
90
|
+
#---#
|
91
|
+
#-A-#
|
92
|
+
#-D-#
|
93
|
+
#-B-#
|
94
|
+
#---#
|
95
|
+
|
96
|
+
#3) (+) C @ index 2
|
97
|
+
#---#
|
98
|
+
#-A-#
|
99
|
+
#-D-#
|
100
|
+
#-C-#
|
101
|
+
#-B-#
|
102
|
+
#---#
|
103
|
+
|
104
|
+
#4) (+) E @ index 4
|
105
|
+
#---#
|
106
|
+
#-A-#
|
107
|
+
#-D-#
|
108
|
+
#-C-#
|
109
|
+
#-B-#
|
110
|
+
#-E-#
|
111
|
+
#---#
|
112
|
+
|
113
|
+
Now `from` is the original `to`
|
114
|
+
```
|
115
|
+
|
116
|
+
##Helpers
|
117
|
+
###Functional Kernel
|
118
|
+
* `vm_diff(old_page, new_page)` - Returns an array of type `vm_diff` w.r.t to the old page. E.g. if A appears in `new_page`, but not `old_page`
|
119
|
+
then it is an insertion.
|
120
|
+
* `vm_diff_replay(page, diff)` - Will run the diff against the page; the page will be modified. This will have no effect on any changelists.
|
121
|
+
|
122
|
+
##Data Types
|
123
|
+
###`vm_diff`
|
124
|
+
```ruby
|
125
|
+
vm_diff_log_schema = [
|
126
|
+
<<vm_diff_entry>>,
|
127
|
+
<<vm_diff_entry>>,
|
128
|
+
...
|
129
|
+
]
|
130
|
+
```
|
131
|
+
|
132
|
+
###`vm_diff_entry`
|
133
|
+
Each `vm_diff_entry` is an array with the form `[type_str, *args]`. The types are:
|
134
|
+
```ruby
|
135
|
+
#Entry Insertion
|
136
|
+
#eindex - The index of the insertion.
|
137
|
+
#ehash - A hash that contains the entry.
|
138
|
+
["+", eindex, ehash]
|
139
|
+
|
140
|
+
#Entry Deletion
|
141
|
+
#eid - The id of the entry that was deleted.
|
142
|
+
["-", eid]
|
143
|
+
|
144
|
+
#Entry Modification
|
145
|
+
#ehash - A hash that contains the new entry to replace the old entry.
|
146
|
+
["M", ehash]
|
147
|
+
|
148
|
+
#Entry Move
|
149
|
+
#to_index - The index, an integer, that the entry should be insertad at
|
150
|
+
#eid - The id of the entry to be moved to the to_index
|
151
|
+
[">", to_index, eid]
|
152
|
+
|
153
|
+
#Head or next pointer changed
|
154
|
+
["HEAD_M", new_head_id]
|
155
|
+
["NEXT_M", new_next_id]
|
156
|
+
```
|
data/docs/services/vm/pagers.md
CHANGED
@@ -12,8 +12,6 @@ If you haven't already, read [VM Service](../vm.md) for context on pagers.
|
|
12
12
|
* `$NAME_write(page)` - You should write this page, e.g. to network, and/or write to `vm_cache_write`. Alternatively, you can write the page over the network and then let the response from that call `vm_cache_write` in what ever listening code you have.
|
13
13
|
* `page` - A fully constructed page with correctly calculated `_hash` and _sigs on entries.
|
14
14
|
|
15
|
-
|
16
|
-
|
17
15
|
##When are pagers invoked?
|
18
16
|
Pagers handle all requests from controllers except for the following conditions:
|
19
17
|
1. There is a `watch` request placed but a previous `watch` request already exists for the requested page. The pager is already aware of the page watch request and is already waiting for a response. Cached pages would have been returned to the controller that made the `watch` request.
|
@@ -56,3 +54,6 @@ This pager provides you with local memory that will be automatically cached to d
|
|
56
54
|
* `watch` - Does nothing
|
57
55
|
* `unwatch` - Does nothing
|
58
56
|
* `write` - Writes the given page to `vm_cache_write`
|
57
|
+
|
58
|
+
####Dummy pager | `pg_dummy0`
|
59
|
+
This pager doesn't do anything. Used by some specs which manually write to the vm_cache in leu of the pager
|
data/lib/flok/user_compiler.rb
CHANGED
data/lib/flok/version.rb
CHANGED
data/spec/env/kern.rb
CHANGED
@@ -27,6 +27,25 @@ shared_context "kern" do
|
|
27
27
|
|
28
28
|
return JSON.parse(json_res)
|
29
29
|
end
|
30
|
+
|
31
|
+
#Will return everything put into the 'dump' dictionary (pre-defined for your convenience)
|
32
|
+
def evald str
|
33
|
+
self.eval "dump = {}"
|
34
|
+
self.eval str
|
35
|
+
_dump = self.dump("dump")
|
36
|
+
|
37
|
+
return DumpHelper.new(_dump)
|
38
|
+
end
|
39
|
+
end
|
40
|
+
|
41
|
+
class DumpHelper
|
42
|
+
def initialize dump
|
43
|
+
@dump = dump
|
44
|
+
end
|
45
|
+
|
46
|
+
def [](index)
|
47
|
+
return @dump[index]
|
48
|
+
end
|
30
49
|
end
|
31
50
|
|
32
51
|
#Execute flok binary with a command
|
@@ -13,7 +13,7 @@ RSpec.describe "User compiler" do
|
|
13
13
|
js_res = compiler.compile(js_src(fn))
|
14
14
|
ctx = V8::Context.new
|
15
15
|
ctx.eval js_res
|
16
|
-
ctx
|
16
|
+
return ctx, js_res
|
17
17
|
end
|
18
18
|
|
19
19
|
#Get the source for a file in ./user_compiler/*.rb
|
@@ -29,59 +29,66 @@ RSpec.describe "User compiler" do
|
|
29
29
|
|
30
30
|
it "Can compile a controller and give up the root
|
31
31
|
iew" do
|
32
|
-
ctx = compile "controller0"
|
32
|
+
ctx, js_src = compile "controller0"
|
33
33
|
root_view = ctx.eval "ctable.my_controller.root_view"
|
34
34
|
expect(root_view).to eq("my_controller")
|
35
35
|
end
|
36
36
|
|
37
37
|
it "Can compile a controller and contain a list of actions" do
|
38
|
-
ctx = compile "controller0"
|
38
|
+
ctx, js_src = compile "controller0"
|
39
39
|
actions = ctx.eval "Object.keys(ctable.my_controller.actions).length"
|
40
40
|
expect(actions).to eq(1)
|
41
41
|
end
|
42
42
|
|
43
43
|
it "Can compile a controller and contain an __init__ function" do
|
44
|
-
ctx = compile "controller0"
|
44
|
+
ctx, js_src = compile "controller0"
|
45
45
|
actions = ctx.eval "ctable.my_controller.__init__"
|
46
46
|
expect(actions).not_to eq(nil)
|
47
47
|
end
|
48
48
|
|
49
49
|
it "Can compile a controller with an action that contains an on_entry" do
|
50
|
-
ctx = compile "controller0"
|
50
|
+
ctx, js_src = compile "controller0"
|
51
51
|
on_entry = ctx.eval "ctable.my_controller.actions.my_action.on_entry"
|
52
52
|
expect(on_entry).not_to eq(nil)
|
53
53
|
end
|
54
54
|
|
55
55
|
it "Can compile a controller with an action that does not contains an on_entry" do
|
56
|
-
ctx = compile "controller0b"
|
56
|
+
ctx, js_src = compile "controller0b"
|
57
57
|
on_entry = ctx.eval "ctable.my_controller.actions.my_action.on_entry"
|
58
58
|
expect(on_entry).not_to eq(nil)
|
59
59
|
end
|
60
60
|
|
61
|
+
it "Can compile a controller with a global on_entry" do
|
62
|
+
ctx, js_src = compile "controller0bg"
|
63
|
+
on_entry = ctx.eval "ctable.my_controller.on_entry"
|
64
|
+
expect(js_src).to include("global_on_entry")
|
65
|
+
end
|
66
|
+
|
67
|
+
|
61
68
|
it "on_entry controller has more code than non on_entry controller" do
|
62
|
-
ctx = compile "controller0"
|
69
|
+
ctx, js_src = compile "controller0"
|
63
70
|
on_entry = ctx.eval "ctable.my_controller.actions.my_action.on_entry"
|
64
71
|
|
65
|
-
ctx2 = compile "controller0b"
|
72
|
+
ctx2, js_src = compile "controller0b"
|
66
73
|
on_entry2 = ctx2.eval "ctable.my_controller.actions.my_action.on_entry"
|
67
74
|
expect(on_entry2.to_s.length).to be < on_entry.to_s.length
|
68
75
|
end
|
69
76
|
|
70
77
|
|
71
78
|
it "Can compile a controller with an action that contains the name" do
|
72
|
-
ctx = compile "controller0"
|
79
|
+
ctx, js_src = compile "controller0"
|
73
80
|
on_entry = ctx.eval "ctable.my_controller.name"
|
74
81
|
expect(on_entry).to eq("my_controller")
|
75
82
|
end
|
76
83
|
|
77
84
|
it "Can compile a controller with an action that contains an event that responds to hello" do
|
78
|
-
ctx = compile "controller0"
|
85
|
+
ctx, js_src = compile "controller0"
|
79
86
|
hello_event_function = ctx.eval "ctable.my_controller.actions.my_action.handlers.hello"
|
80
87
|
expect(hello_event_function).not_to eq(nil)
|
81
88
|
end
|
82
89
|
|
83
90
|
it "Can compile a controller with spot names" do
|
84
|
-
ctx = compile "controller0"
|
91
|
+
ctx, js_src = compile "controller0"
|
85
92
|
spot_names = JSON.parse(ctx.eval "JSON.stringify(ctable.my_controller.spots)")
|
86
93
|
expect(spot_names).to include "hello"
|
87
94
|
expect(spot_names).to include "world"
|
@@ -89,7 +96,7 @@ RSpec.describe "User compiler" do
|
|
89
96
|
end
|
90
97
|
|
91
98
|
it "Can compile a controller with an action containing a timer and set the appropriate every_handlers key" do
|
92
|
-
ctx = compile "controller0timer"
|
99
|
+
ctx, js_src = compile "controller0timer"
|
93
100
|
|
94
101
|
function_names = JSON.parse(ctx.eval "JSON.stringify(Object.keys(ctable.my_controller.actions.my_action.handlers))")
|
95
102
|
expect(function_names).to include("hello")
|
@@ -0,0 +1,28 @@
|
|
1
|
+
controller :my_controller do
|
2
|
+
services "vm"
|
3
|
+
|
4
|
+
on_entry %{
|
5
|
+
context.secret = "foo";
|
6
|
+
context.base = __base__;
|
7
|
+
|
8
|
+
var page = NewPage("array", "test");
|
9
|
+
var info = {
|
10
|
+
page: page,
|
11
|
+
ns: "spec0",
|
12
|
+
id: "test"
|
13
|
+
}
|
14
|
+
Request("vm", "write", info);
|
15
|
+
|
16
|
+
Request("vm", "watch", info);
|
17
|
+
}
|
18
|
+
|
19
|
+
action :index do
|
20
|
+
on_entry %{
|
21
|
+
Send("context", context);
|
22
|
+
}
|
23
|
+
|
24
|
+
on "read_res", %{
|
25
|
+
read_res_called = true;
|
26
|
+
}
|
27
|
+
end
|
28
|
+
end
|
@@ -0,0 +1,47 @@
|
|
1
|
+
controller :my_controller do
|
2
|
+
services :vm
|
3
|
+
|
4
|
+
on_entry %{
|
5
|
+
entry_move_params = [];
|
6
|
+
entry_modify_params = [];
|
7
|
+
entry_del_params = [];
|
8
|
+
entry_ins_params = [];
|
9
|
+
next_changed_params = [];
|
10
|
+
head_changed_params = [];
|
11
|
+
}
|
12
|
+
|
13
|
+
action :my_action do
|
14
|
+
on_entry %{
|
15
|
+
var info = {ns: "dummy", id: "default"};
|
16
|
+
Request("vm", "watch", info);
|
17
|
+
}
|
18
|
+
|
19
|
+
on "entry_move", %{
|
20
|
+
entry_move_called = true;
|
21
|
+
entry_move_params.push(params);
|
22
|
+
}
|
23
|
+
|
24
|
+
on "entry_modify", %{
|
25
|
+
entry_modify_called = true;
|
26
|
+
entry_modify_params.push(params);
|
27
|
+
}
|
28
|
+
|
29
|
+
on "entry_del", %{
|
30
|
+
entry_del_called = true;
|
31
|
+
entry_del_params.push(params);
|
32
|
+
}
|
33
|
+
|
34
|
+
on "entry_ins", %{
|
35
|
+
entry_ins_called = true;
|
36
|
+
entry_ins_params.push(params);
|
37
|
+
}
|
38
|
+
|
39
|
+
on "head_changed", %{
|
40
|
+
head_changed_params.push(params);
|
41
|
+
}
|
42
|
+
|
43
|
+
on "next_changed", %{
|
44
|
+
next_changed_params.push(params);
|
45
|
+
}
|
46
|
+
end
|
47
|
+
end
|
@@ -0,0 +1,108 @@
|
|
1
|
+
//Page Factor
|
2
|
+
//////////////////////////////////////////////////////////////////////////////////////////
|
3
|
+
function PageFactory(head, next) {
|
4
|
+
this.head = head;
|
5
|
+
this.next = next;
|
6
|
+
this.entries = [];
|
7
|
+
}
|
8
|
+
|
9
|
+
//Add an entry
|
10
|
+
PageFactory.prototype.addEntry = function(eid, value) {
|
11
|
+
this.entries.push({_id: eid, _sig: value, value: value});
|
12
|
+
}
|
13
|
+
|
14
|
+
//This adds up to four entrys that can be represented as a square:
|
15
|
+
//-------------
|
16
|
+
//| id0 | id1 |
|
17
|
+
//-------------
|
18
|
+
//| id2 | id3 |
|
19
|
+
//-------------
|
20
|
+
//Leaving out parts of the values array will not add those entries, e.g. ["Square", null, null, "Triangle"]
|
21
|
+
//--------------------|
|
22
|
+
//| Square | null |
|
23
|
+
//--------------------|
|
24
|
+
//| null | Triangle |
|
25
|
+
//--------------------|
|
26
|
+
//Where 'Square' is id0 and 'Triangle' is id3
|
27
|
+
PageFactory.prototype.addEntryFourSquare = function(values) {
|
28
|
+
if (values.length != 4) {
|
29
|
+
throw "FourSquare requires for values. Make values null if you don't need them"
|
30
|
+
}
|
31
|
+
|
32
|
+
for (var i = 0; i < values.length; ++i) {
|
33
|
+
if (values[i]) {
|
34
|
+
this.addEntry("id"+i, values[i]);
|
35
|
+
}
|
36
|
+
}
|
37
|
+
}
|
38
|
+
|
39
|
+
//Returns a page
|
40
|
+
PageFactory.prototype.compile = function() {
|
41
|
+
var page = vm_create_page("default");
|
42
|
+
page._head = this.head;
|
43
|
+
page._next = this.next;
|
44
|
+
|
45
|
+
page.entries = this.entries;
|
46
|
+
|
47
|
+
vm_rehash_page(page);
|
48
|
+
vm_reindex_page(page);
|
49
|
+
|
50
|
+
return page;
|
51
|
+
}
|
52
|
+
//////////////////////////////////////////////////////////////////////////////////////////
|
53
|
+
|
54
|
+
var pf = new PageFactory();
|
55
|
+
pf.addEntryFourSquare(["Triangle", "Square", "Z", null]);
|
56
|
+
triangle_square_z_null = pf.compile();
|
57
|
+
|
58
|
+
var pf = new PageFactory();
|
59
|
+
pf.addEntryFourSquare(["Triangle", "Circle", null, "Q"]);
|
60
|
+
triangle_circle_null_q = pf.compile();
|
61
|
+
|
62
|
+
var pf = new PageFactory();
|
63
|
+
pf.addEntryFourSquare(["Triangle", "Circle", null, "Q"]);
|
64
|
+
triangle_circle_null_q = pf.compile();
|
65
|
+
|
66
|
+
var pf = new PageFactory();
|
67
|
+
pf.addEntryFourSquare(["Q", null, "Circle", "Square"]);
|
68
|
+
q_null_circle_square = pf.compile();
|
69
|
+
|
70
|
+
var pf = new PageFactory();
|
71
|
+
pf.addEntryFourSquare(["P", "Circle", null, "Q"]);
|
72
|
+
p_circle_null_q = pf.compile();
|
73
|
+
|
74
|
+
var pf = new PageFactory();
|
75
|
+
pf.addEntryFourSquare(["P", "Circle", null, null]);
|
76
|
+
p_circle_null_null = pf.compile();
|
77
|
+
|
78
|
+
var pf = new PageFactory();
|
79
|
+
pf.addEntryFourSquare(["P", null, null, "Q"]);
|
80
|
+
p_null_null_q = pf.compile();
|
81
|
+
|
82
|
+
var pf = new PageFactory();
|
83
|
+
pf.addEntryFourSquare(["P", "Square", null, null]);
|
84
|
+
p_square_null_null = pf.compile();
|
85
|
+
|
86
|
+
var pf = new PageFactory();
|
87
|
+
pf.addEntryFourSquare(["Triangle", null, "A", "M"]);
|
88
|
+
triangle_null_a_m = pf.compile();
|
89
|
+
|
90
|
+
var pf = new PageFactory();
|
91
|
+
pf.addEntryFourSquare(["Triangle", "Square", null, null]);
|
92
|
+
triangle_square_null_null = pf.compile();
|
93
|
+
|
94
|
+
var pf = new PageFactory();
|
95
|
+
pf.addEntryFourSquare(["Triangle", "Z", "Q", null]);
|
96
|
+
triangle_z_q_null = pf.compile();
|
97
|
+
|
98
|
+
var pf = new PageFactory(null);
|
99
|
+
head_null = pf.compile();
|
100
|
+
|
101
|
+
var pf = new PageFactory("world");
|
102
|
+
head_world = pf.compile();
|
103
|
+
|
104
|
+
var pf = new PageFactory(null);
|
105
|
+
next_null = pf.compile();
|
106
|
+
|
107
|
+
var pf = new PageFactory(null, "world");
|
108
|
+
next_world = pf.compile();
|