evals-lab 0.1.4 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -0
- package/lab/VERSION +1 -1
- package/lab/demo/pipelines/demo-1.json +34 -24
- package/lab/demo/pipelines/demo-2.json +34 -24
- package/lab/evals-core.mjs +701 -284
- package/lab/run-evals.js +42 -16
- package/lab/server.py +251 -75
- package/lab/web/dist/assets/gallery-DsetJSXv.js +3 -0
- package/lab/web/dist/assets/{main-DjQQums6.css → main-B-VtDGxC.css} +1 -1
- package/lab/web/dist/assets/main-ERoD2Kll.js +19 -0
- package/lab/web/dist/assets/tokens-ClRQ7Mui.js +51 -0
- package/lab/web/dist/assets/tokens-D3C8O2Ib.css +1 -0
- package/lab/web/dist/gallery.html +3 -3
- package/lab/web/dist/index.html +4 -4
- package/package.json +1 -1
- package/lab/web/dist/assets/gallery-DipkRvqJ.js +0 -3
- package/lab/web/dist/assets/main-61NS6C4m.js +0 -18
- package/lab/web/dist/assets/tokens-B9intIuT.js +0 -51
- package/lab/web/dist/assets/tokens-s6I-RMVq.css +0 -1
package/README.md
CHANGED
|
@@ -126,6 +126,41 @@ at `127.0.0.1` reloads the page at `localhost`, the address registered above,
|
|
|
126
126
|
and carries on from there. Anything open in the tab, such as an Add source
|
|
127
127
|
dialog, does not carry over.
|
|
128
128
|
|
|
129
|
+
## Google Drive
|
|
130
|
+
|
|
131
|
+
The lab can sign in to Google Drive, so a Source can read files from a Drive
|
|
132
|
+
folder. The sign-in is held by the lab's server, on your machine, and it can
|
|
133
|
+
reach only the files you pick: the lab asks for `drive.file` access, never
|
|
134
|
+
your whole Drive.
|
|
135
|
+
|
|
136
|
+
Open **Setup › Connections**, then use the Google Drive row's menu ›
|
|
137
|
+
**Sign in**. If the row says *Not Configured*, give the lab a Google app
|
|
138
|
+
first, once. These steps are provisional: they haven't yet been tried
|
|
139
|
+
against a real Google project, and the Drive Source that uses the sign-in
|
|
140
|
+
is still to come.
|
|
141
|
+
|
|
142
|
+
1. At <https://console.cloud.google.com>, make a project. Enable the
|
|
143
|
+
**Google Drive API** and the **Google Picker API**.
|
|
144
|
+
2. **OAuth consent screen:** choose *External*. Give the app a name and your
|
|
145
|
+
email, add the scope `.../auth/drive.file`, then **Publish app**. It needs
|
|
146
|
+
no review from Google, because that scope is non-sensitive. Left in
|
|
147
|
+
*Testing*, a sign-in lapses after 7 days.
|
|
148
|
+
3. **Credentials › Create credentials › OAuth client ID**, of type
|
|
149
|
+
**Desktop app**. Download its JSON file.
|
|
150
|
+
4. **Credentials › Create credentials › API key.** Restrict it to the Google
|
|
151
|
+
Picker API.
|
|
152
|
+
5. In the lab, use the Google Drive row's menu › **Configure…**, then
|
|
153
|
+
**Load client file…**, and choose the file from step 3. Paste the key
|
|
154
|
+
from step 4 into **API key**.
|
|
155
|
+
|
|
156
|
+
A Desktop app needs no redirect registered: Google sends the browser back to
|
|
157
|
+
any port on this machine. A lab you reach at its own address, from another
|
|
158
|
+
machine, needs a *Web application* client instead. Choose that type in
|
|
159
|
+
Configure…, then register the **Redirect URI** it shows with the client.
|
|
160
|
+
|
|
161
|
+
Everything saves as you type. The client's secret is kept by the lab and
|
|
162
|
+
never shown again. The window says only that one is saved.
|
|
163
|
+
|
|
129
164
|
## Reaching it from another machine
|
|
130
165
|
|
|
131
166
|
By default only your own machine can reach the lab. That is deliberate: the
|
package/lab/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.2.0 (2026.10.02-323)
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"version":
|
|
2
|
+
"version": 10,
|
|
3
3
|
"id": "demo-1",
|
|
4
4
|
"name": "Demo 1",
|
|
5
5
|
"jobs": [
|
|
@@ -21,38 +21,46 @@
|
|
|
21
21
|
}
|
|
22
22
|
},
|
|
23
23
|
{
|
|
24
|
-
"type": "
|
|
25
|
-
"withImage": true,
|
|
26
|
-
"tokenMappings": []
|
|
24
|
+
"type": "attachImage"
|
|
27
25
|
},
|
|
28
26
|
{
|
|
29
27
|
"type": "readReply",
|
|
30
28
|
"out": {
|
|
31
29
|
"kind": "list",
|
|
32
30
|
"parse": "csv",
|
|
33
|
-
"preamble": false
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
31
|
+
"preamble": false
|
|
32
|
+
}
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
"type": "modifier",
|
|
36
|
+
"modifier": {
|
|
37
|
+
"type": "tidy"
|
|
38
|
+
}
|
|
39
|
+
},
|
|
40
|
+
{
|
|
41
|
+
"type": "modifier",
|
|
42
|
+
"modifier": {
|
|
43
|
+
"type": "dedupe"
|
|
44
|
+
}
|
|
45
|
+
},
|
|
46
|
+
{
|
|
47
|
+
"type": "modifier",
|
|
48
|
+
"modifier": {
|
|
49
|
+
"type": "keep-first",
|
|
50
|
+
"n": 12
|
|
51
|
+
}
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
"type": "modifier",
|
|
55
|
+
"modifier": {
|
|
56
|
+
"type": "case",
|
|
57
|
+
"mode": "LOWER"
|
|
50
58
|
}
|
|
51
59
|
}
|
|
52
60
|
]
|
|
53
61
|
}
|
|
54
62
|
],
|
|
55
|
-
"
|
|
63
|
+
"targets": [
|
|
56
64
|
{
|
|
57
65
|
"id": "s1",
|
|
58
66
|
"name": "",
|
|
@@ -60,8 +68,9 @@
|
|
|
60
68
|
"id": "vision",
|
|
61
69
|
"name": "Vision"
|
|
62
70
|
},
|
|
63
|
-
"
|
|
71
|
+
"steps": [
|
|
64
72
|
{
|
|
73
|
+
"type": "prompt",
|
|
65
74
|
"prompt": "List the things you can see in this image, as a comma-separated list of short items (one to three words each), most important first. Up to 12 items."
|
|
66
75
|
}
|
|
67
76
|
]
|
|
@@ -73,8 +82,9 @@
|
|
|
73
82
|
"id": "vision",
|
|
74
83
|
"name": "Vision"
|
|
75
84
|
},
|
|
76
|
-
"
|
|
85
|
+
"steps": [
|
|
77
86
|
{
|
|
87
|
+
"type": "prompt",
|
|
78
88
|
"prompt": "Name what is in this image. Reply with a comma-separated list only: up to 12 short items, most important first."
|
|
79
89
|
}
|
|
80
90
|
]
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"version":
|
|
2
|
+
"version": 10,
|
|
3
3
|
"id": "demo-2",
|
|
4
4
|
"name": "Demo 2",
|
|
5
5
|
"jobs": [
|
|
@@ -21,38 +21,46 @@
|
|
|
21
21
|
}
|
|
22
22
|
},
|
|
23
23
|
{
|
|
24
|
-
"type": "
|
|
25
|
-
"withImage": true,
|
|
26
|
-
"tokenMappings": []
|
|
24
|
+
"type": "attachImage"
|
|
27
25
|
},
|
|
28
26
|
{
|
|
29
27
|
"type": "readReply",
|
|
30
28
|
"out": {
|
|
31
29
|
"kind": "list",
|
|
32
30
|
"parse": "csv",
|
|
33
|
-
"preamble": false
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
31
|
+
"preamble": false
|
|
32
|
+
}
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
"type": "modifier",
|
|
36
|
+
"modifier": {
|
|
37
|
+
"type": "tidy"
|
|
38
|
+
}
|
|
39
|
+
},
|
|
40
|
+
{
|
|
41
|
+
"type": "modifier",
|
|
42
|
+
"modifier": {
|
|
43
|
+
"type": "dedupe"
|
|
44
|
+
}
|
|
45
|
+
},
|
|
46
|
+
{
|
|
47
|
+
"type": "modifier",
|
|
48
|
+
"modifier": {
|
|
49
|
+
"type": "keep-first",
|
|
50
|
+
"n": 12
|
|
51
|
+
}
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
"type": "modifier",
|
|
55
|
+
"modifier": {
|
|
56
|
+
"type": "case",
|
|
57
|
+
"mode": "LOWER"
|
|
50
58
|
}
|
|
51
59
|
}
|
|
52
60
|
]
|
|
53
61
|
}
|
|
54
62
|
],
|
|
55
|
-
"
|
|
63
|
+
"targets": [
|
|
56
64
|
{
|
|
57
65
|
"id": "s1",
|
|
58
66
|
"name": "",
|
|
@@ -60,8 +68,9 @@
|
|
|
60
68
|
"id": "vision",
|
|
61
69
|
"name": "Vision"
|
|
62
70
|
},
|
|
63
|
-
"
|
|
71
|
+
"steps": [
|
|
64
72
|
{
|
|
73
|
+
"type": "prompt",
|
|
65
74
|
"prompt": "List the things you can see in this image, as a comma-separated list of short items (one to three words each), most important first. Up to 12 items."
|
|
66
75
|
}
|
|
67
76
|
]
|
|
@@ -73,8 +82,9 @@
|
|
|
73
82
|
"id": "vision",
|
|
74
83
|
"name": "Vision"
|
|
75
84
|
},
|
|
76
|
-
"
|
|
85
|
+
"steps": [
|
|
77
86
|
{
|
|
87
|
+
"type": "prompt",
|
|
78
88
|
"prompt": "Name what is in this image. Reply with a comma-separated list only: up to 12 short items, most important first."
|
|
79
89
|
}
|
|
80
90
|
]
|