@aotui/mobile-ai-native 0.1.0-alpha.0 → 0.1.0-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/GUIDE.md +79 -109
- package/LICENSE +201 -0
- package/README.md +106 -40
- package/dist/core/action/createActionRuntime.d.ts +6 -6
- package/dist/core/action/createActionRuntime.js +71 -18
- package/dist/core/action/defineAction.d.ts +4 -5
- package/dist/core/action/defineViewTypeTool.d.ts +5 -0
- package/dist/core/action/defineViewTypeTool.js +3 -0
- package/dist/core/effect/types.d.ts +21 -1
- package/dist/core/snapshot/createSnapshotBundle.d.ts +7 -4
- package/dist/core/snapshot/createSnapshotBundle.js +110 -3
- package/dist/core/snapshot/createSnapshotRegistry.d.ts +4 -0
- package/dist/core/snapshot/createSnapshotRegistry.js +52 -0
- package/dist/core/trace/createTraceStore.d.ts +2 -0
- package/dist/core/trace/createTraceStore.js +38 -0
- package/dist/core/trace/types.d.ts +21 -0
- package/dist/core/trace/types.js +1 -0
- package/dist/core/types.d.ts +61 -10
- package/dist/demo/inbox/InboxGUI.js +5 -3
- package/dist/demo/inbox/InboxTUI.d.ts +3 -1
- package/dist/demo/inbox/InboxTUI.js +73 -7
- package/dist/demo/inbox/actions.d.ts +2 -2
- package/dist/demo/inbox/actions.js +15 -4
- package/dist/demo/inbox/createInboxApp.d.ts +2 -5
- package/dist/demo/inbox/createInboxApp.js +4 -4
- package/dist/demo/inbox/effects.d.ts +3 -5
- package/dist/demo/inbox/effects.js +5 -0
- package/dist/demo/inbox/state.d.ts +3 -0
- package/dist/demo/inbox/state.js +16 -0
- package/dist/index.d.ts +17 -2
- package/dist/index.js +12 -2
- package/dist/projection/gui/AppProvider.d.ts +8 -12
- package/dist/projection/gui/AppProvider.js +77 -2
- package/dist/projection/gui/hooks.d.ts +2 -5
- package/dist/projection/gui/hooks.js +3 -12
- package/dist/projection/react/AppRuntimeProvider.d.ts +10 -0
- package/dist/projection/react/AppRuntimeProvider.js +15 -0
- package/dist/projection/react/createReactAppRuntime.d.ts +35 -0
- package/dist/projection/react/createReactAppRuntime.js +80 -0
- package/dist/projection/react/hooks.d.ts +8 -0
- package/dist/projection/react/hooks.js +18 -0
- package/dist/projection/tui/View.d.ts +8 -0
- package/dist/projection/tui/View.js +5 -0
- package/dist/projection/tui/createSnapshotAssembler.d.ts +2 -0
- package/dist/projection/tui/createSnapshotAssembler.js +14 -0
- package/dist/projection/tui/renderSnapshotDocument.d.ts +2 -0
- package/dist/projection/tui/renderSnapshotDocument.js +3 -0
- package/dist/projection/tui/renderTUI.d.ts +1 -1
- package/dist/projection/tui/renderTUI.js +13 -4
- package/dist/projection/tui/renderViewFragment.d.ts +10 -0
- package/dist/projection/tui/renderViewFragment.js +15 -0
- package/dist/tool/createToolBridge.d.ts +4 -9
- package/dist/tool/createToolBridge.js +55 -11
- package/dist/version.d.ts +1 -0
- package/dist/version.js +1 -0
- package/package.json +8 -9
package/GUIDE.md
CHANGED
|
@@ -1,11 +1,16 @@
|
|
|
1
1
|
# GUIDE: Building an Agent Native iOS Calendar App
|
|
2
2
|
|
|
3
|
-
This guide is for a developer who wants to build a high-quality
|
|
3
|
+
This guide is for a developer who wants to build a high-quality agent-native calendar app on iOS using `@aotui/mobile-ai-native`.
|
|
4
|
+
|
|
5
|
+
Important boundary:
|
|
6
|
+
|
|
7
|
+
- `@aotui/mobile-ai-native` is the runtime core
|
|
8
|
+
- `@aotui/mobile-ai-native-react-native` is the React Native / Expo host adapter
|
|
4
9
|
|
|
5
10
|
The goal is simple:
|
|
6
11
|
|
|
7
12
|
- humans use a normal GUI calendar
|
|
8
|
-
- the LLM uses tools through a
|
|
13
|
+
- the LLM uses tools through a snapshot
|
|
9
14
|
- both operate on the same app state
|
|
10
15
|
- the human can see the result of the LLM's actions
|
|
11
16
|
|
|
@@ -31,9 +36,26 @@ They must meet in the same place:
|
|
|
31
36
|
|
|
32
37
|
That is the whole trick.
|
|
33
38
|
|
|
34
|
-
## 2.
|
|
39
|
+
## 2. How The Snapshot Works
|
|
40
|
+
|
|
41
|
+
The LLM should not guess ids from pixels.
|
|
42
|
+
It should read the current `SnapshotBundle`.
|
|
43
|
+
|
|
44
|
+
In the current runtime, the snapshot is assembled from ordered `<View>` fragments:
|
|
45
|
+
|
|
46
|
+
- the first fragment is the static root navigation fragment with `type: "Root"`
|
|
47
|
+
- later fragments are mounted business views derived from current state
|
|
48
|
+
- `markup` is the composed xml+markdown snapshot
|
|
49
|
+
- `views` preserves the fragment order
|
|
50
|
+
- `refIndex` resolves semantic refs by exact key lookup
|
|
51
|
+
- `visibleTools` is the tool list for that same render tick
|
|
35
52
|
|
|
36
|
-
|
|
53
|
+
`snapshotId` is non-negotiable.
|
|
54
|
+
If the app changes after the LLM reads a snapshot, the tool call must still be tied to the exact snapshot the model saw.
|
|
55
|
+
|
|
56
|
+
## 3. Why This Architecture Matters
|
|
57
|
+
|
|
58
|
+
If the LLM drives the GUI by fake taps, the system becomes fragile.
|
|
37
59
|
|
|
38
60
|
Why?
|
|
39
61
|
|
|
@@ -50,30 +72,31 @@ Instead, this framework makes the LLM act on meaning:
|
|
|
50
72
|
- `changeCalendarView`
|
|
51
73
|
|
|
52
74
|
The GUI is one projection of state.
|
|
53
|
-
The
|
|
75
|
+
The snapshot is another projection of state.
|
|
54
76
|
|
|
55
|
-
The LLM should never
|
|
77
|
+
The LLM should never infer refs from the visual layout.
|
|
56
78
|
It should receive semantic refs from the current `SnapshotBundle`.
|
|
57
79
|
|
|
58
|
-
##
|
|
80
|
+
## 4. The Core Mental Model
|
|
59
81
|
|
|
60
82
|
Your calendar app should follow this loop:
|
|
61
83
|
|
|
62
84
|
`State -> GUI`
|
|
63
|
-
`State ->
|
|
85
|
+
`State -> Root view + mounted business views -> SnapshotBundle`
|
|
64
86
|
`Tool -> Action -> Event/Effect -> State`
|
|
65
87
|
`GUI Event -> Action -> Event/Effect -> State`
|
|
66
88
|
|
|
67
89
|
That means:
|
|
68
90
|
|
|
69
91
|
- GUI controls do not own business logic
|
|
70
|
-
-
|
|
92
|
+
- snapshot views do not own business logic
|
|
71
93
|
- tools do not own business logic
|
|
72
94
|
- `Action` is the one real business entry
|
|
95
|
+
- effects are framework-managed side effects, but the app still owns what they mean
|
|
73
96
|
|
|
74
|
-
##
|
|
97
|
+
## 5. Calendar App State
|
|
75
98
|
|
|
76
|
-
A good
|
|
99
|
+
A good agent-native calendar app should have state shaped more like this:
|
|
77
100
|
|
|
78
101
|
```ts
|
|
79
102
|
type CalendarState = {
|
|
@@ -95,26 +118,26 @@ Good rule:
|
|
|
95
118
|
|
|
96
119
|
- if it affects GUI, TUI, tool visibility, or trace, it belongs in framework state
|
|
97
120
|
- if it is only a tiny render helper, keep it local
|
|
121
|
+
- if it must be read by the host through a selector, keep it in the runtime store so `useRuntimeState()` can subscribe to it
|
|
98
122
|
|
|
99
|
-
##
|
|
123
|
+
## 6. Calendar Views And Tools
|
|
100
124
|
|
|
101
|
-
|
|
125
|
+
Model the app as a static root plus mounted runtime views:
|
|
102
126
|
|
|
103
|
-
- `
|
|
104
|
-
- `
|
|
105
|
-
- `
|
|
106
|
-
- `
|
|
107
|
-
- `changeCalendarView`
|
|
127
|
+
- `RootView` is the navigation map, currently emitted with `type: "Root"`
|
|
128
|
+
- `CalendarView` can be always mounted when the calendar shell is active
|
|
129
|
+
- `EventDetailView` can mount only when an event is selected
|
|
130
|
+
- `SearchResultView` can mount only while search results are present
|
|
108
131
|
|
|
109
|
-
|
|
110
|
-
You can always add more later.
|
|
132
|
+
The tool surface should follow the same semantic grouping.
|
|
111
133
|
|
|
112
|
-
|
|
134
|
+
- register tools against a `viewType`
|
|
135
|
+
- filter them with `visibility(state)`
|
|
136
|
+
- only expose tools for currently relevant view types
|
|
113
137
|
|
|
114
|
-
|
|
115
|
-
- not UI actions like `tapButton` or `scrollList`
|
|
138
|
+
That keeps the calendar agent surface aligned with the current screen reality without pretending the runtime is a desktop tree inspector.
|
|
116
139
|
|
|
117
|
-
##
|
|
140
|
+
## 7. Why Refs Matter In Calendar Apps
|
|
118
141
|
|
|
119
142
|
Calendars are full of structured data:
|
|
120
143
|
|
|
@@ -153,7 +176,7 @@ const [eventsRef, eventRef] = useArrayRef("event", visibleEvents, "events");
|
|
|
153
176
|
))}
|
|
154
177
|
```
|
|
155
178
|
|
|
156
|
-
##
|
|
179
|
+
## 8. Why `snapshotId` Is Non-Negotiable
|
|
157
180
|
|
|
158
181
|
The screen can change after the LLM reads it.
|
|
159
182
|
|
|
@@ -169,15 +192,11 @@ So tool execution must always be tied to the exact snapshot the LLM saw:
|
|
|
169
192
|
await bridge.executeTool("openEvent", { event: "events[0]" }, snapshotId);
|
|
170
193
|
```
|
|
171
194
|
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
That is a big deal.
|
|
175
|
-
It is the difference between:
|
|
195
|
+
That prevents the runtime from guessing against the latest UI.
|
|
176
196
|
|
|
177
|
-
|
|
178
|
-
- and a haunted house
|
|
197
|
+
In the hardened runtime, the snapshot registry distinguishes `SNAPSHOT_NOT_FOUND` from `SNAPSHOT_STALE`. A tool execution that mutates state marks its originating snapshot stale, even if the action returns a recoverable failure result. That forces the next reasoning turn to fetch a fresh snapshot.
|
|
179
198
|
|
|
180
|
-
##
|
|
199
|
+
## 9. How To Build The iOS App
|
|
181
200
|
|
|
182
201
|
### Step 1: Keep this package as the core
|
|
183
202
|
|
|
@@ -196,15 +215,18 @@ The React Native app should do only host work:
|
|
|
196
215
|
- host the agent session
|
|
197
216
|
- ask the framework for `SnapshotBundle`
|
|
198
217
|
- pass tool calls into `executeTool`
|
|
218
|
+
- subscribe to state and trace through the runtime hooks instead of copying framework state into local component state
|
|
199
219
|
|
|
200
|
-
|
|
220
|
+
In practice, that means mounting the core through `createReactNativeAppRuntime()` and `AppRuntimeProvider` from `@aotui/mobile-ai-native-react-native`.
|
|
201
221
|
|
|
202
|
-
|
|
222
|
+
### Step 3: Build GUI and snapshot views as separate projections
|
|
223
|
+
|
|
224
|
+
Do not auto-generate the snapshot from GUI.
|
|
203
225
|
|
|
204
226
|
For calendar apps this is especially important because:
|
|
205
227
|
|
|
206
228
|
- GUI cares about touch and spatial layout
|
|
207
|
-
-
|
|
229
|
+
- snapshot views care about semantic clarity for the model
|
|
208
230
|
|
|
209
231
|
### Step 4: Start with one vertical slice
|
|
210
232
|
|
|
@@ -223,20 +245,30 @@ Only after that is stable, add:
|
|
|
223
245
|
- move
|
|
224
246
|
- delete
|
|
225
247
|
- recurring events
|
|
248
|
+
- trace UI for recent AI actions
|
|
226
249
|
|
|
227
|
-
##
|
|
250
|
+
## 10. Suggested First Calendar Snapshot
|
|
228
251
|
|
|
229
|
-
Your first
|
|
252
|
+
Your first snapshot should be boring and clear, not clever:
|
|
230
253
|
|
|
231
254
|
```tsx
|
|
232
|
-
<
|
|
255
|
+
<View id="root" type="Root" name="Navigation">
|
|
256
|
+
## Calendar Navigation
|
|
257
|
+
- Root
|
|
258
|
+
- purpose: app navigation and view map
|
|
259
|
+
- Calendar
|
|
260
|
+
- enter: mounted by default after launch
|
|
261
|
+
- actions: open_event, search_events, change_calendar_view
|
|
262
|
+
- EventDetail
|
|
263
|
+
- enter: use open_event from Calendar
|
|
264
|
+
- actions: update_event, close_event
|
|
265
|
+
</View>
|
|
266
|
+
|
|
267
|
+
<View id="calendar" type="Calendar" name="Week Calendar">
|
|
233
268
|
<text>View: week</text>
|
|
234
269
|
<text>Date: 2026-03-24</text>
|
|
235
270
|
<text>{daysRef("Visible days")}</text>
|
|
236
|
-
|
|
237
|
-
<item key={event.id}>{eventRef(index, `${event.title} at ${event.startTime}`)}</item>
|
|
238
|
-
))}
|
|
239
|
-
</screen>
|
|
271
|
+
</View>
|
|
240
272
|
```
|
|
241
273
|
|
|
242
274
|
The LLM does not need visual beauty.
|
|
@@ -246,72 +278,10 @@ It needs:
|
|
|
246
278
|
- meaningful refs
|
|
247
279
|
- clear tool choices
|
|
248
280
|
|
|
249
|
-
##
|
|
250
|
-
|
|
251
|
-
Before you call the app "good", verify these:
|
|
252
|
-
|
|
253
|
-
### State correctness
|
|
254
|
-
|
|
255
|
-
- GUI and TUI always reflect the same event state
|
|
256
|
-
- tool calls only act on snapshot-scoped refs
|
|
257
|
-
|
|
258
|
-
### Tool correctness
|
|
259
|
-
|
|
260
|
-
- invisible tools are not callable
|
|
261
|
-
- stale `snapshotId` fails cleanly
|
|
262
|
-
- missing refs fail with explicit errors
|
|
263
|
-
|
|
264
|
-
### UX correctness
|
|
265
|
-
|
|
266
|
-
- human sees the result of AI actions
|
|
267
|
-
- recent AI action summary is visible
|
|
268
|
-
- event openings, searches, and edits are understandable
|
|
269
|
-
|
|
270
|
-
### Product correctness
|
|
271
|
-
|
|
272
|
-
- no tool is named after a button
|
|
273
|
-
- tools match domain intent
|
|
274
|
-
- TUI exposes enough semantic data without dumping noise
|
|
275
|
-
|
|
276
|
-
## 11. What This Package Does Not Give You Yet
|
|
277
|
-
|
|
278
|
-
Be honest with yourself:
|
|
279
|
-
|
|
280
|
-
this package is still an alpha core.
|
|
281
|
-
|
|
282
|
-
It does not yet give you:
|
|
283
|
-
|
|
284
|
-
- a full React Native adapter
|
|
285
|
-
- iOS simulator harness
|
|
286
|
-
- production persistence
|
|
287
|
-
- production networking
|
|
288
|
-
- voice or background agent integration
|
|
289
|
-
|
|
290
|
-
That is okay.
|
|
291
|
-
|
|
292
|
-
It gives you the hardest part first:
|
|
293
|
-
|
|
294
|
-
- the protocol spine
|
|
295
|
-
|
|
296
|
-
## 12. Recommended Build Order
|
|
297
|
-
|
|
298
|
-
If your friend is building the calendar app, this is the order I recommend:
|
|
299
|
-
|
|
300
|
-
1. build one RN screen shell
|
|
301
|
-
2. integrate framework state and tool bridge
|
|
302
|
-
3. implement week view with event refs
|
|
303
|
-
4. implement `openEvent`
|
|
304
|
-
5. implement `searchEvents`
|
|
305
|
-
6. add trace banner for recent AI action
|
|
306
|
-
7. add create/edit flows
|
|
307
|
-
8. only then add advanced calendar features
|
|
308
|
-
|
|
309
|
-
## 13. The One Sentence To Remember
|
|
310
|
-
|
|
311
|
-
An Agent Native iOS app is not:
|
|
312
|
-
|
|
313
|
-
- "AI controlling UI"
|
|
281
|
+
## 11. Quality Bar For A Good Agent Native Calendar App
|
|
314
282
|
|
|
315
|
-
|
|
283
|
+
A good app should feel like one system, not two loosely related interfaces.
|
|
316
284
|
|
|
317
|
-
|
|
285
|
+
If the GUI and snapshot disagree, the user will feel it immediately.
|
|
286
|
+
If tools are visible in the wrong view type, the agent will feel it immediately.
|
|
287
|
+
If refs drift from the current render tick, both paths become unreliable.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for reasonable and customary use in describing the
|
|
141
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
166
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
167
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
168
|
+
or other liability obligations and/or rights consistent with this
|
|
169
|
+
License. However, in accepting such obligations, You may act only
|
|
170
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
171
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
172
|
+
defend, and hold each Contributor harmless for any liability
|
|
173
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
174
|
+
of your accepting any such warranty or additional liability.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
|
177
|
+
|
|
178
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
179
|
+
|
|
180
|
+
To apply the Apache License to your work, attach the following
|
|
181
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
182
|
+
replaced with your own identifying information. (Don't include
|
|
183
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
184
|
+
comment syntax for the file format. We also recommend that a
|
|
185
|
+
file or class name and description of purpose be included on the
|
|
186
|
+
same "printed page" as the copyright notice for easier
|
|
187
|
+
identification within third-party archives.
|
|
188
|
+
|
|
189
|
+
Copyright 2026 Johnny Zhang
|
|
190
|
+
|
|
191
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
192
|
+
you may not use this file except in compliance with the License.
|
|
193
|
+
You may obtain a copy of the License at
|
|
194
|
+
|
|
195
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
196
|
+
|
|
197
|
+
Unless required by applicable law or agreed to in writing, software
|
|
198
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
199
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
200
|
+
See the License for the specific language governing permissions and
|
|
201
|
+
limitations under the License.
|
package/README.md
CHANGED
|
@@ -1,21 +1,93 @@
|
|
|
1
1
|
# @aotui/mobile-ai-native
|
|
2
2
|
|
|
3
|
-
This package is
|
|
3
|
+
This package is the hardened runtime core for building agent-native mobile apps.
|
|
4
4
|
|
|
5
|
-
It
|
|
5
|
+
It is the **core**, not the full React Native host layer.
|
|
6
|
+
React Native / Expo mounting now lives in [`@aotui/mobile-ai-native-react-native`](../packages/mobile-ai-native-react-native/README.md).
|
|
6
7
|
|
|
7
|
-
|
|
8
|
+
It provides the state store, action runtime, trace lifecycle, snapshot registry, and tool bridge needed for a real mobile host.
|
|
8
9
|
|
|
9
|
-
|
|
10
|
+
The current snapshot model is view-based:
|
|
11
|
+
|
|
12
|
+
`State -> static Root view + state-derived mounted views -> SnapshotBundle -> Tool Call(ref_id + snapshotId) -> Action -> Event/Effect -> State -> GUI/TUI refresh`
|
|
10
13
|
|
|
11
14
|
## What This Slice Proves
|
|
12
15
|
|
|
13
16
|
- one shared state system drives both GUI and TUI
|
|
14
|
-
-
|
|
17
|
+
- the root view is static navigation knowledge, not runtime state
|
|
18
|
+
- business views are mounted from current state and represent runtime reality
|
|
19
|
+
- tools are scoped by `viewType` and then filtered by `visibility(state)`
|
|
15
20
|
- the framework builds one atomic `SnapshotBundle`
|
|
21
|
+
- snapshot markup is composed from ordered `<View>` fragments
|
|
16
22
|
- tools execute against the exact `snapshotId` the LLM saw
|
|
17
23
|
- `refIndex` stores serializable snapshot payloads, not live object references
|
|
18
|
-
-
|
|
24
|
+
- mutated tool results stale the originating snapshot, even when the result is recoverable
|
|
25
|
+
- trace entries record the action lifecycle: `started`, `updated`, `succeeded`, and `failed`
|
|
26
|
+
|
|
27
|
+
## SnapshotBundle
|
|
28
|
+
|
|
29
|
+
The LLM-facing read model is:
|
|
30
|
+
|
|
31
|
+
```ts
|
|
32
|
+
type SnapshotBundle = {
|
|
33
|
+
snapshotId: string;
|
|
34
|
+
generatedAt: number;
|
|
35
|
+
markup: string;
|
|
36
|
+
views: readonly ViewFragment[];
|
|
37
|
+
tui: string;
|
|
38
|
+
refIndex: Record<string, RefIndexEntry>;
|
|
39
|
+
visibleTools: readonly ToolDefinition[];
|
|
40
|
+
};
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Current behavior to keep in mind:
|
|
44
|
+
|
|
45
|
+
- `markup` is the composed xml+markdown snapshot built from ordered `<View>` fragments
|
|
46
|
+
- `views` preserves the ordered fragment list, with the `Root` fragment first
|
|
47
|
+
- `tui` is retained as a compatibility readout and may differ from `markup`, but it should be produced from the same snapshot generation pass
|
|
48
|
+
- `refIndex` and `visibleTools` are produced and frozen alongside snapshot creation
|
|
49
|
+
|
|
50
|
+
The bundle is intended to be atomic:
|
|
51
|
+
|
|
52
|
+
- `markup`
|
|
53
|
+
- `views`
|
|
54
|
+
- `tui`
|
|
55
|
+
- `refIndex`
|
|
56
|
+
- `visibleTools`
|
|
57
|
+
|
|
58
|
+
Today the runtime hard-validates `markup` against `views`, then freezes the associated `tui`, `refIndex`, and `visibleTools` outputs generated on that same snapshot path.
|
|
59
|
+
|
|
60
|
+
## RootView And Mounted Views
|
|
61
|
+
|
|
62
|
+
`RootView` is the conceptual navigation role, and the current runtime emits it as a fragment with `type: "Root"`.
|
|
63
|
+
|
|
64
|
+
It explains:
|
|
65
|
+
|
|
66
|
+
- what view types exist
|
|
67
|
+
- how to enter them
|
|
68
|
+
- what each view type is for
|
|
69
|
+
|
|
70
|
+
It does not try to narrate runtime state.
|
|
71
|
+
|
|
72
|
+
Mounted business views are the runtime reality.
|
|
73
|
+
|
|
74
|
+
They are derived from current state and describe:
|
|
75
|
+
|
|
76
|
+
- which concrete views are mounted right now
|
|
77
|
+
- what state each mounted view is showing
|
|
78
|
+
- what refs and actions are relevant in that state
|
|
79
|
+
|
|
80
|
+
That means the LLM should read the static root first, then read the mounted business views for the live app state.
|
|
81
|
+
|
|
82
|
+
## Tool Model
|
|
83
|
+
|
|
84
|
+
Tools are defined against a semantic `viewType`, then filtered by current state.
|
|
85
|
+
|
|
86
|
+
That gives the runtime this rule:
|
|
87
|
+
|
|
88
|
+
`visibleTools = tools for currently relevant viewTypes + visibility(state)`
|
|
89
|
+
|
|
90
|
+
So a tool can exist in the app, be mounted to a view type, and still be hidden when the current state does not allow it.
|
|
19
91
|
|
|
20
92
|
## Core Contract
|
|
21
93
|
|
|
@@ -32,7 +104,20 @@ Then it calls a tool with:
|
|
|
32
104
|
await bridge.executeTool("openMessage", { message: "messages[0]" }, snapshotId);
|
|
33
105
|
```
|
|
34
106
|
|
|
35
|
-
|
|
107
|
+
For tools that declare `meta.supportsRefs === true`, the bridge resolves top-level string inputs as either exact `ref_id` values or canonical marker strings from that snapshot's `refIndex`.
|
|
108
|
+
It does not infer nested field refs or field-level ref metadata.
|
|
109
|
+
|
|
110
|
+
The low-level React-family host path inside the core is still present because the adapter composes it, but product code should prefer the dedicated RN adapter package.
|
|
111
|
+
|
|
112
|
+
The adapter-facing runtime path is:
|
|
113
|
+
|
|
114
|
+
- `createReactAppRuntime()` owns the store, action runtime, snapshot registry, trace store, and tool bridge
|
|
115
|
+
- `AppRuntimeProvider` publishes that runtime through context
|
|
116
|
+
- `useRuntimeState(selector)` subscribes to store updates with `useSyncExternalStore`
|
|
117
|
+
- `useRuntimeTrace(selector)` subscribes to the trace store the same way
|
|
118
|
+
- `createReactNativeAppRuntime()` in the adapter package wraps that core runtime and exposes `runtime.ai.getSnapshot()` / `runtime.ai.executeTool(...)` for host-safe tool execution
|
|
119
|
+
|
|
120
|
+
That means GUI consumers react to state and trace changes without pulling the whole runtime object into component state.
|
|
36
121
|
|
|
37
122
|
## Ref APIs
|
|
38
123
|
|
|
@@ -55,51 +140,22 @@ const [listRef, itemRef] = useArrayRef("message", messages, "messages");
|
|
|
55
140
|
<item>{itemRef(0, "Welcome back")}</item>;
|
|
56
141
|
```
|
|
57
142
|
|
|
58
|
-
## SnapshotBundle
|
|
59
|
-
|
|
60
|
-
The LLM-facing read model is:
|
|
61
|
-
|
|
62
|
-
```ts
|
|
63
|
-
type SnapshotBundle = {
|
|
64
|
-
snapshotId: string;
|
|
65
|
-
generatedAt: number;
|
|
66
|
-
tui: string;
|
|
67
|
-
refIndex: Record<string, { type: string; value: unknown }>;
|
|
68
|
-
visibleTools: ToolDefinition[];
|
|
69
|
-
};
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
This is intentionally atomic:
|
|
73
|
-
|
|
74
|
-
- `tui`
|
|
75
|
-
- `visibleTools`
|
|
76
|
-
- `refIndex`
|
|
77
|
-
|
|
78
|
-
must all come from the same render tick.
|
|
79
|
-
|
|
80
|
-
## Why `refIndex` Stores Serializable Payloads
|
|
81
|
-
|
|
82
|
-
Screens can change after the LLM reads them.
|
|
83
|
-
Live object references may already be gone.
|
|
84
|
-
|
|
85
|
-
So `refIndex` stores serializable snapshot payloads.
|
|
86
|
-
When the LLM later calls a tool, the framework reconstructs the action input from the payload attached to that `snapshotId`.
|
|
87
|
-
|
|
88
143
|
## Current Status
|
|
89
144
|
|
|
90
|
-
This is not
|
|
145
|
+
This package is the core, not the full app shell.
|
|
91
146
|
|
|
92
147
|
What it gives you today:
|
|
93
148
|
|
|
94
149
|
- shared state core
|
|
95
150
|
- semantic refs with `useDataRef` and `useArrayRef`
|
|
96
151
|
- atomic `SnapshotBundle`
|
|
152
|
+
- static root navigation plus state-derived mounted business views
|
|
97
153
|
- snapshot-scoped tool execution
|
|
98
|
-
- a
|
|
154
|
+
- a pure core that can now be mounted by the dedicated RN / Expo adapter
|
|
155
|
+
- structured trace and effect contracts
|
|
99
156
|
|
|
100
157
|
What you still need for a production iOS app:
|
|
101
158
|
|
|
102
|
-
- a thin React Native host adapter
|
|
103
159
|
- real GUI components
|
|
104
160
|
- model orchestration and networking
|
|
105
161
|
- product-level trace UI and persistence
|
|
@@ -112,3 +168,13 @@ The inbox demo exposes:
|
|
|
112
168
|
- `searchMessages`
|
|
113
169
|
|
|
114
170
|
and proves that GUI and TUI both refresh from the same state after tool execution.
|
|
171
|
+
|
|
172
|
+
## Runtime Boundaries
|
|
173
|
+
|
|
174
|
+
Keep the framework and app responsibilities separated:
|
|
175
|
+
|
|
176
|
+
- framework owns the store, action runtime, snapshot registry, trace runtime, tool bridge, and host adapter
|
|
177
|
+
- business apps own state shape, domain actions, handwritten TUI, and app-specific effect behavior
|
|
178
|
+
- actions can read state, emit events, run effects, and update trace summaries
|
|
179
|
+
- effects can read state, emit events, and report structured success or failure, but they do not write state directly
|
|
180
|
+
- snapshot coherence is enforced for the textual view representation, so `markup`, `views`, and `tui` must agree for the same render tick; `refIndex` and `visibleTools` are built and frozen alongside that snapshot path but are not yet cross-validated field-by-field
|