pxt-microbit 7.1.20 → 7.1.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/built/block-tests.js +1 -1
- package/built/target.js +1 -1
- package/built/target.json +1 -1
- package/built/targetlight.json +1 -1
- package/built/theme.json +1 -1
- package/built/web/react-common-authcode.css +1 -1
- package/built/web/react-common-multiplayer.css +1 -1
- package/built/web/react-common-skillmap.css +1 -1
- package/built/web/rtlreact-common-authcode.css +1 -1
- package/built/web/rtlreact-common-multiplayer.css +1 -1
- package/built/web/rtlreact-common-skillmap.css +1 -1
- package/built/web/rtlsemantic.css +1 -1
- package/built/web/semantic.css +1 -1
- package/docs/code-eval-tool.md +192 -0
- package/docs/teachertool/ai-faq.md +31 -0
- package/docs/teachertool.md +14 -8
- package/package.json +3 -3
- package/pxtarget.json +3 -0
- package/targetconfig.json +3 -9
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
# Code Evaluation Tool
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
The [Code Evaluation Tool]( https://microbit.makecode.com/--eval) is a mechanism for constructing a checklist of requirements for an assignment and running that list automatically against projects in quick succession. This allows teachers to build a checklist, then easily evaluate any number of projects based on that checklist. Projects are evaluated one at a time, but with auto-run enabled, you can update the loaded project by providing a new share link, at which point the rules will automatically be re-run on the new project.
|
|
6
|
+
|
|
7
|
+
## Code Evaluation Tool Features
|
|
8
|
+
|
|
9
|
+
### Creating, Editing, and Running a Checklist
|
|
10
|
+
|
|
11
|
+
#### 1. Creating a new checklist
|
|
12
|
+
|
|
13
|
+
Create a new checklist using the **New Checklist** card. If there is already an "in progress" checklist, a warning will appear asking if it is okay to overwrite it.
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
#### 2. Naming a checklist
|
|
20
|
+
|
|
21
|
+
The checklist is given a name.
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
#### 3. Add Criteria
|
|
26
|
+
|
|
27
|
+
One or more **_criteria_** are added from the catalog using the **Add Criteria** button.
|
|
28
|
+
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
Some criteria (like `[block] used [count] times`) can be added multiple times, others (like `Read a GPIO pin` can only be added once).
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
#### 4. Fill in Parameters
|
|
36
|
+
|
|
37
|
+
Parameters for the criteria item are filled in for a criteria item.
|
|
38
|
+
|
|
39
|
+
### ~ tip
|
|
40
|
+
|
|
41
|
+
#### Parameter types
|
|
42
|
+
|
|
43
|
+
From a technical perspective, criteria parameters have these types:
|
|
44
|
+
|
|
45
|
+
- **Numeric** parameters have a small input and only allow number inputs.
|
|
46
|
+
- **String** parameters can have medium and long sized inputs.
|
|
47
|
+
- **Block** parameters should open a block-picker modal.
|
|
48
|
+
- **Empty** parameters appear in an error state until they have values.
|
|
49
|
+
|
|
50
|
+
### ~
|
|
51
|
+
|
|
52
|
+
Here a block is selected and used 3 times:
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
Parameter options are displayed and then selected.
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
#### 5. Ask AI
|
|
63
|
+
|
|
64
|
+
You can also have an **Ask AI** question as a criteria item in the checklist. You are limited to up to 5 Ask AI questions per checklist.
|
|
65
|
+
|
|
66
|
+
|
|
67
|
+
|
|
68
|
+
### ~hint
|
|
69
|
+
|
|
70
|
+
#### AI usage
|
|
71
|
+
|
|
72
|
+
The use of AI in criteria items is further explained in the [AI FAQ](/teachertool/ai-faq).
|
|
73
|
+
|
|
74
|
+
### ~
|
|
75
|
+
|
|
76
|
+
#### 6. Remove Criteria
|
|
77
|
+
|
|
78
|
+
A criteria item is removed using the **trash** button.
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
#### 7. Load a project
|
|
83
|
+
|
|
84
|
+
Load a project into the project view by pasting in a share link or share ID.
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
The project will load in read-only mode with the project title appearing at the top of the project view.
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
#### 8. Run the checklist
|
|
93
|
+
|
|
94
|
+
With a project loaded, the checklist can run. The results are shown after clicking the **Run** button.
|
|
95
|
+
|
|
96
|
+
|
|
97
|
+
|
|
98
|
+
The results view lists each criteria with its outcome.
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
|
|
102
|
+
**Note**: the **Run** Button is disabled without loaded project.
|
|
103
|
+
|
|
104
|
+
### Editing Results
|
|
105
|
+
|
|
106
|
+
#### 1. Add feedback and notes
|
|
107
|
+
|
|
108
|
+
Feedback and notes are added using the **Add Notes** button. The feedback box should resize to fit its content as notes are added. The original feedback remains even if you re-run the rules using the **Run** button.
|
|
109
|
+
|
|
110
|
+
|
|
111
|
+
|
|
112
|
+
|
|
113
|
+
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
#### 2. Edit outcomes
|
|
117
|
+
|
|
118
|
+
An outcome is edited using the provided dropdown.
|
|
119
|
+
|
|
120
|
+
|
|
121
|
+
|
|
122
|
+
The new selected outcome.
|
|
123
|
+
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
### Result Clearing and Auto-Run
|
|
127
|
+
|
|
128
|
+
#### 1. Toggling Auto-run
|
|
129
|
+
|
|
130
|
+
Auto-run is toggled either **on** or **off** using the button in the menu.
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
#### 2. Auto-run disabled
|
|
135
|
+
|
|
136
|
+
If auto-run is **disabled**, a result's outcome (i.e. "Looks good", "Needs work", etc...) is set to "Not started" automatically if any of any of the following conditions are met:
|
|
137
|
+
|
|
138
|
+
- It is newly added (defaults to the "Not Started" state).
|
|
139
|
+
- A parameter in a rule is changed (only the affected rule enters the "Not Started" state).
|
|
140
|
+
- The loaded project changes (all rules are be set to "Not started").
|
|
141
|
+
|
|
142
|
+
#### 3. Auto-run enabled
|
|
143
|
+
|
|
144
|
+
If auto-run is **enabled**, any rules that enter the "Not started" state due to the conditions listed above are immediately and automatically re-run with their results updated.
|
|
145
|
+
|
|
146
|
+
### Loading/Importing/Exporting Checklists
|
|
147
|
+
|
|
148
|
+
#### 1. Pre-built checklists
|
|
149
|
+
|
|
150
|
+
There are pre-built checklists are available on the welcome page. If a selected checklist is already in-progress, an overwrite confirmation prompt is given.
|
|
151
|
+
|
|
152
|
+
|
|
153
|
+
|
|
154
|
+
#### 2. Export a checklist
|
|
155
|
+
|
|
156
|
+
A checklist is exported using the vertical "..." menu near the "auto-run" button. Only the checklist is exported, not the results. For a copy of the results, use the [print](#other) function in the Results view.
|
|
157
|
+
|
|
158
|
+
|
|
159
|
+
|
|
160
|
+
This will download a json file for the checklist.
|
|
161
|
+
|
|
162
|
+
|
|
163
|
+
|
|
164
|
+
#### 3. Import a checklist
|
|
165
|
+
|
|
166
|
+
User can import a checklist from a file using the same "..." menu, or from the card on the welcome page.
|
|
167
|
+
|
|
168
|
+
|
|
169
|
+
|
|
170
|
+
|
|
171
|
+
|
|
172
|
+
Checklist file is selected using "Browse" or dropped directly into the popup. An overwrite confirmation prompted if there is currently an in-progress checklist.
|
|
173
|
+
|
|
174
|
+
|
|
175
|
+
|
|
176
|
+
|
|
177
|
+
|
|
178
|
+
### Other
|
|
179
|
+
|
|
180
|
+
If the page is refreshed (or if the browser closes/re-opens), the current checklist preserved.
|
|
181
|
+
|
|
182
|
+
Use the print button to create a version of the results with the outcomes and feedback visible (the other UI elements are hidden).
|
|
183
|
+
|
|
184
|
+
|
|
185
|
+
|
|
186
|
+
The checklist-view/project-view splitter can be resized. It can also be reset to 50/50 split with double-click.
|
|
187
|
+
|
|
188
|
+
|
|
189
|
+
|
|
190
|
+
Slide the splitter to widen the view of the criteria and results.
|
|
191
|
+
|
|
192
|
+
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Microsoft MakeCode Code Evaluation Tool
|
|
2
|
+
|
|
3
|
+
## Responsible AI FAQ
|
|
4
|
+
|
|
5
|
+
### 1. What is the MakeCode Code Evaluation Tool?
|
|
6
|
+
|
|
7
|
+
The MakeCode Code Evaluation tool is an online tool for teachers to help them understand and evaluate student block-based coding programs. In addition to static analysis functionality, there is an optional AI component for teachers to provide additional feedback and recommendations to students. The teacher can ask specific questions about one student project at a time (i.e. "Do the variables in this program have meaningful names?"), and the AI will respond with an answer and reasoning.
|
|
8
|
+
|
|
9
|
+
### 2. What can the MakeCode Code Evaluation Tool do?
|
|
10
|
+
|
|
11
|
+
The MakeCode Code Evaluation tool will send the current student code with the teacher question to DeepPrompt (a Microsoft Azure LLM service) along with some contextual prompt information and return the resulting AI response back to the user.
|
|
12
|
+
|
|
13
|
+
### 3. What is MakeCode Code Evaluation Tool’s intended use(s)?
|
|
14
|
+
|
|
15
|
+
The MakeCode Code Evaluation tool is intended to help teachers expedite the process of giving feedback on student programs.
|
|
16
|
+
|
|
17
|
+
### 4. How was the MakeCode Code Evaluation Tool evaluated? What metrics are used to measure performance?
|
|
18
|
+
|
|
19
|
+
The system was evaluated with 1000+ prompts from multiple sources to ensure the responses are grounded and relevant to the educator’s task of assessing student code. We evaluated accuracy with red teaming and expert review of responses.
|
|
20
|
+
|
|
21
|
+
### 5. What are the limitations of the MakeCode Code Evaluation Tool? How can users minimize the impact of the Code Evaluation Tool’s limitations when using the system?
|
|
22
|
+
|
|
23
|
+
The system only supports educational scenarios related to student code. The system will not perform well for other scenarios or unrelated questions. When using this tool, educators should ask short, concise questions relating to the assessment of student code. Questions are limited to 5 per program, and 1000 characters per question. The MakeCode Code Evaluation tool cannot provide direct scores or grades for student work.
|
|
24
|
+
|
|
25
|
+
### ~reminder
|
|
26
|
+
|
|
27
|
+
#### Tool Beta
|
|
28
|
+
|
|
29
|
+
This tool is currently in Beta, and we value your feedback. Please click on the **Feedback** button to share your experiences and thoughts about the MakeCode Code Evaluation Tool.
|
|
30
|
+
|
|
31
|
+
### ~
|
package/docs/teachertool.md
CHANGED
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
## Overview
|
|
4
4
|
|
|
5
|
-
The [
|
|
5
|
+
The [Code Evaluation Tool]( https://microbit.makecode.com/--eval) is a mechanism for constructing a checklist of requirements for an assignment and running that list automatically against projects in quick succession. This allows teachers to build a checklist, then easily evaluate any number of projects based on that checklist. Projects are evaluated one at a time, but with auto-run enabled, you can update the loaded project by providing a new share link, at which point the rules will automatically be re-run on the new project.
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## Code Evaluation Tool Features
|
|
8
8
|
|
|
9
9
|
### Creating, Editing, and Running a Checklist
|
|
10
10
|
|
|
@@ -59,15 +59,21 @@ Parameter options are displayed and then selected.
|
|
|
59
59
|
|
|
60
60
|
|
|
61
61
|
|
|
62
|
-
#### 5.
|
|
62
|
+
#### 5. Ask AI
|
|
63
|
+
|
|
64
|
+
You can also have an **Ask AI** question as a criteria item in the checklist. You are limited to up to 5 Ask AI questions per checklist.
|
|
65
|
+
|
|
66
|
+
|
|
67
|
+
|
|
68
|
+
#### 6. Remove Criteria
|
|
63
69
|
|
|
64
70
|
A criteria item is removed using the **trash** button.
|
|
65
71
|
|
|
66
72
|
|
|
67
73
|
|
|
68
|
-
####
|
|
74
|
+
#### 7. Load a project
|
|
69
75
|
|
|
70
|
-
|
|
76
|
+
Load a project into the project view by pasting in a share link or share ID.
|
|
71
77
|
|
|
72
78
|
|
|
73
79
|
|
|
@@ -75,7 +81,7 @@ The project will load in read-only mode with the project title appearing at the
|
|
|
75
81
|
|
|
76
82
|
|
|
77
83
|
|
|
78
|
-
####
|
|
84
|
+
#### 8. Run the checklist
|
|
79
85
|
|
|
80
86
|
With a project loaded, the checklist can run. The results are shown after clicking the **Run** button.
|
|
81
87
|
|
|
@@ -133,13 +139,13 @@ If auto-run is **enabled**, any rules that enter the "Not started" state due to
|
|
|
133
139
|
|
|
134
140
|
#### 1. Pre-built checklists
|
|
135
141
|
|
|
136
|
-
There are pre-built checklists are available
|
|
142
|
+
There are pre-built checklists are available on the welcome page. If a selected checklist is already in-progress, an overwrite confirmation prompt is given.
|
|
137
143
|
|
|
138
144
|
|
|
139
145
|
|
|
140
146
|
#### 2. Export a checklist
|
|
141
147
|
|
|
142
|
-
A checklist is exported using the vertical "..." menu near the "auto-run" button.
|
|
148
|
+
A checklist is exported using the vertical "..." menu near the "auto-run" button. Only the checklist is exported, not the results. For a copy of the results, use the [print](#other) function in the Results view.
|
|
143
149
|
|
|
144
150
|
|
|
145
151
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pxt-microbit",
|
|
3
|
-
"version": "7.1.
|
|
3
|
+
"version": "7.1.21",
|
|
4
4
|
"description": "micro:bit target for Microsoft MakeCode (PXT)",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"JavaScript",
|
|
@@ -45,7 +45,7 @@
|
|
|
45
45
|
"typescript": "4.8.3"
|
|
46
46
|
},
|
|
47
47
|
"dependencies": {
|
|
48
|
-
"pxt-common-packages": "12.
|
|
49
|
-
"pxt-core": "11.3.
|
|
48
|
+
"pxt-common-packages": "12.2.2",
|
|
49
|
+
"pxt-core": "11.3.15"
|
|
50
50
|
}
|
|
51
51
|
}
|
package/pxtarget.json
CHANGED
package/targetconfig.json
CHANGED
|
@@ -480,20 +480,14 @@
|
|
|
480
480
|
},
|
|
481
481
|
"microbit-foundation/pxt-microbit-ml": {
|
|
482
482
|
"simx": {
|
|
483
|
-
"sha": "
|
|
483
|
+
"sha": "ba99fd9588e87eb994345d04b596adc51d2d8116",
|
|
484
484
|
"devUrl": "http://localhost:5173",
|
|
485
485
|
"aspectRatio": 3.45
|
|
486
486
|
}
|
|
487
487
|
},
|
|
488
|
-
"microbit-apps/
|
|
488
|
+
"microbit-apps/display-shield": {
|
|
489
489
|
"simx": {
|
|
490
|
-
"sha": "
|
|
491
|
-
"devUrl": "http://localhost:3000"
|
|
492
|
-
}
|
|
493
|
-
},
|
|
494
|
-
"microbit-apps/arcadeshield": {
|
|
495
|
-
"simx": {
|
|
496
|
-
"sha": "d0f7ccae711cff6dd61ffd0bd95c4bab913e3a99",
|
|
490
|
+
"sha": "0825d56f78528c57af100a96abdfe2162f59945a",
|
|
497
491
|
"devUrl": "http://localhost:3000"
|
|
498
492
|
}
|
|
499
493
|
}
|