@dxos/assistant 0.8.4-main.b97322e → 0.8.4-main.bc674ce
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/lib/browser/chunk-7CDDG6LL.mjs +578 -0
- package/dist/lib/browser/chunk-7CDDG6LL.mjs.map +7 -0
- package/dist/lib/browser/chunk-J5LGTIGS.mjs +10 -0
- package/dist/lib/browser/chunk-J5LGTIGS.mjs.map +7 -0
- package/dist/lib/browser/extraction/index.mjs +225 -0
- package/dist/lib/browser/extraction/index.mjs.map +7 -0
- package/dist/lib/browser/index.mjs +723 -1704
- package/dist/lib/browser/index.mjs.map +4 -4
- package/dist/lib/browser/meta.json +1 -1
- package/dist/lib/browser/testing/index.mjs +32 -0
- package/dist/lib/browser/testing/index.mjs.map +7 -0
- package/dist/lib/node-esm/chunk-CC7ER6Q3.mjs +579 -0
- package/dist/lib/node-esm/chunk-CC7ER6Q3.mjs.map +7 -0
- package/dist/lib/node-esm/chunk-HSLMI22Q.mjs +11 -0
- package/dist/lib/node-esm/chunk-HSLMI22Q.mjs.map +7 -0
- package/dist/lib/node-esm/extraction/index.mjs +226 -0
- package/dist/lib/node-esm/extraction/index.mjs.map +7 -0
- package/dist/lib/node-esm/index.mjs +723 -1705
- package/dist/lib/node-esm/index.mjs.map +4 -4
- package/dist/lib/node-esm/meta.json +1 -1
- package/dist/lib/node-esm/testing/index.mjs +33 -0
- package/dist/lib/node-esm/testing/index.mjs.map +7 -0
- package/dist/types/src/conversation/context.d.ts +129 -0
- package/dist/types/src/conversation/context.d.ts.map +1 -0
- package/dist/types/src/conversation/context.test.d.ts +2 -0
- package/dist/types/src/conversation/context.test.d.ts.map +1 -0
- package/dist/types/src/conversation/conversation.d.ts +29 -26
- package/dist/types/src/conversation/conversation.d.ts.map +1 -1
- package/dist/types/src/conversation/conversation.test.d.ts +2 -0
- package/dist/types/src/conversation/conversation.test.d.ts.map +1 -0
- package/dist/types/src/conversation/index.d.ts +1 -0
- package/dist/types/src/conversation/index.d.ts.map +1 -1
- package/dist/types/src/diff.d.ts +15 -0
- package/dist/types/src/diff.d.ts.map +1 -0
- package/dist/types/src/diff.test.d.ts +2 -0
- package/dist/types/src/diff.test.d.ts.map +1 -0
- package/dist/types/src/errors.d.ts +47 -18
- package/dist/types/src/errors.d.ts.map +1 -1
- package/dist/types/src/execution-graph.d.ts +85 -0
- package/dist/types/src/execution-graph.d.ts.map +1 -0
- package/dist/types/src/execution-graph.test.d.ts +2 -0
- package/dist/types/src/execution-graph.test.d.ts.map +1 -0
- package/dist/types/src/extraction/extraction-llm-function/extraction-llm-function.d.ts +1 -1
- package/dist/types/src/extraction/extraction-llm-function/extraction-llm-function.d.ts.map +1 -1
- package/dist/types/src/extraction/extraction-ner-function/extraction-ner-function.d.ts +258 -213
- package/dist/types/src/extraction/extraction-ner-function/extraction-ner-function.d.ts.map +1 -1
- package/dist/types/src/extraction/extraction-ner-function/named-entity-recognition.d.ts.map +1 -1
- package/dist/types/src/extraction/extraction.d.ts +345 -270
- package/dist/types/src/extraction/extraction.d.ts.map +1 -1
- package/dist/types/src/extraction/quotes.d.ts +3 -3
- package/dist/types/src/extraction/quotes.d.ts.map +1 -1
- package/dist/types/src/functions/services.d.ts +18 -4
- package/dist/types/src/functions/services.d.ts.map +1 -1
- package/dist/types/src/functions.test.d.ts +2 -0
- package/dist/types/src/functions.test.d.ts.map +1 -0
- package/dist/types/src/index.d.ts +3 -3
- package/dist/types/src/index.d.ts.map +1 -1
- package/dist/types/src/session/GenericToolkit.d.ts +85 -0
- package/dist/types/src/session/GenericToolkit.d.ts.map +1 -0
- package/dist/types/src/session/artifact-diff.d.ts +27 -0
- package/dist/types/src/session/artifact-diff.d.ts.map +1 -0
- package/dist/types/src/session/error-handling.d.ts +4 -0
- package/dist/types/src/session/error-handling.d.ts.map +1 -0
- package/dist/types/src/session/format.d.ts +12 -0
- package/dist/types/src/session/format.d.ts.map +1 -0
- package/dist/types/src/session/format.test.d.ts +2 -0
- package/dist/types/src/session/format.test.d.ts.map +1 -0
- package/dist/types/src/session/index.d.ts +4 -0
- package/dist/types/src/session/index.d.ts.map +1 -1
- package/dist/types/src/session/observer.d.ts +23 -0
- package/dist/types/src/session/observer.d.ts.map +1 -0
- package/dist/types/src/session/session.d.ts +33 -237
- package/dist/types/src/session/session.d.ts.map +1 -1
- package/dist/types/src/session/toolkit.d.ts +14 -0
- package/dist/types/src/session/toolkit.d.ts.map +1 -0
- package/dist/types/src/session/version-pin.d.ts +4 -4
- package/dist/types/src/session/version-pin.d.ts.map +1 -1
- package/dist/types/src/templates/index.d.ts +1 -1
- package/dist/types/src/templates/system.d.ts +18 -0
- package/dist/types/src/templates/system.d.ts.map +1 -0
- package/dist/types/src/testing/index.d.ts +1 -2
- package/dist/types/src/testing/index.d.ts.map +1 -1
- package/dist/types/src/testing/layer.d.ts +21 -0
- package/dist/types/src/testing/layer.d.ts.map +1 -0
- package/dist/types/src/types/artifact.d.ts +28 -0
- package/dist/types/src/types/artifact.d.ts.map +1 -0
- package/dist/types/src/types/index.d.ts +2 -0
- package/dist/types/src/types/index.d.ts.map +1 -0
- package/dist/types/tsconfig.tsbuildinfo +1 -1
- package/package.json +56 -33
- package/src/conversation/context.test.ts +46 -0
- package/src/conversation/context.ts +303 -0
- package/src/conversation/conversation.test.ts +65 -0
- package/src/conversation/conversation.ts +93 -64
- package/src/conversation/index.ts +1 -0
- package/src/diff.test.ts +43 -0
- package/src/diff.ts +84 -0
- package/src/errors.ts +6 -1
- package/src/execution-graph.test.ts +740 -0
- package/src/execution-graph.ts +509 -0
- package/src/extraction/extraction-llm-function/extraction-llm-function.test.ts +10 -9
- package/src/extraction/extraction-llm-function/extraction-llm-function.ts +48 -56
- package/src/extraction/extraction-ner-function/extraction-ner-function.test.ts +22 -21
- package/src/extraction/extraction-ner-function/extraction-ner-function.ts +42 -43
- package/src/extraction/extraction-ner-function/named-entity-recognition.test.ts +12 -6
- package/src/extraction/extraction-ner-function/named-entity-recognition.ts +2 -0
- package/src/extraction/extraction.ts +10 -9
- package/src/extraction/quotes.test.ts +3 -3
- package/src/extraction/quotes.ts +6 -7
- package/src/functions/services.ts +138 -97
- package/src/functions.conversations.json +1 -0
- package/src/functions.test.ts +74 -0
- package/src/index.ts +3 -3
- package/src/session/GenericToolkit.ts +131 -0
- package/src/session/artifact-diff.ts +41 -0
- package/src/session/error-handling.ts +41 -0
- package/src/session/format.test.ts +124 -0
- package/src/session/format.ts +143 -0
- package/src/session/index.ts +5 -0
- package/src/session/observer.ts +61 -0
- package/src/session/session.conversations.json +1 -0
- package/src/session/session.test.ts +34 -201
- package/src/session/session.ts +146 -248
- package/src/session/toolkit.ts +39 -0
- package/src/session/version-pin.ts +8 -8
- package/src/templates/index.ts +1 -1
- package/src/templates/instructions/format.tpl +61 -0
- package/src/templates/instructions/system.tpl +48 -0
- package/src/templates/system.ts +41 -0
- package/src/testing/index.ts +2 -8
- package/src/testing/layer.ts +78 -0
- package/src/types/artifact.ts +82 -0
- package/src/types/index.ts +5 -0
- package/src/typings.d.ts +5 -0
- package/dist/types/bin/chat.d.ts +0 -2
- package/dist/types/bin/chat.d.ts.map +0 -1
- package/dist/types/src/context/binder.d.ts +0 -34
- package/dist/types/src/context/binder.d.ts.map +0 -1
- package/dist/types/src/context/binding.d.ts +0 -56
- package/dist/types/src/context/binding.d.ts.map +0 -1
- package/dist/types/src/context/index.d.ts +0 -3
- package/dist/types/src/context/index.d.ts.map +0 -1
- package/dist/types/src/research/exa.d.ts +0 -12
- package/dist/types/src/research/exa.d.ts.map +0 -1
- package/dist/types/src/research/graph.d.ts +0 -51
- package/dist/types/src/research/graph.d.ts.map +0 -1
- package/dist/types/src/research/graph.test.d.ts +0 -2
- package/dist/types/src/research/graph.test.d.ts.map +0 -1
- package/dist/types/src/research/index.d.ts +0 -4
- package/dist/types/src/research/index.d.ts.map +0 -1
- package/dist/types/src/research/research.d.ts +0 -10
- package/dist/types/src/research/research.d.ts.map +0 -1
- package/dist/types/src/research/research.test.d.ts +0 -2
- package/dist/types/src/research/research.test.d.ts.map +0 -1
- package/dist/types/src/session/version-pin.test.d.ts +0 -2
- package/dist/types/src/session/version-pin.test.d.ts.map +0 -1
- package/dist/types/src/summarization/summarization.test.d.ts +0 -2
- package/dist/types/src/summarization/summarization.test.d.ts.map +0 -1
- package/dist/types/src/templates/prompt.d.ts +0 -30
- package/dist/types/src/templates/prompt.d.ts.map +0 -1
- package/dist/types/src/testing/exa-search-1748337321991.d.ts +0 -38
- package/dist/types/src/testing/exa-search-1748337321991.d.ts.map +0 -1
- package/dist/types/src/testing/exa-search-1748337331526.d.ts +0 -37
- package/dist/types/src/testing/exa-search-1748337331526.d.ts.map +0 -1
- package/dist/types/src/testing/exa-search-1748337344119.d.ts +0 -58
- package/dist/types/src/testing/exa-search-1748337344119.d.ts.map +0 -1
- package/dist/types/src/types.d.ts +0 -13
- package/dist/types/src/types.d.ts.map +0 -1
- package/src/context/binder.ts +0 -103
- package/src/context/binding.ts +0 -32
- package/src/context/index.ts +0 -6
- package/src/research/exa.ts +0 -97
- package/src/research/graph.test.ts +0 -64
- package/src/research/graph.ts +0 -369
- package/src/research/index.ts +0 -7
- package/src/research/instructions-research.tpl +0 -98
- package/src/research/research.test.ts +0 -190
- package/src/research/research.ts +0 -87
- package/src/session/__snapshots__/session.test.ts.snap +0 -32
- package/src/session/version-pin.test.ts +0 -7
- package/src/summarization/summarization.test.ts +0 -208
- package/src/templates/assistant.tpl +0 -111
- package/src/templates/prompt.ts +0 -45
- package/src/testing/exa-search-1748337321991.ts +0 -131
- package/src/testing/exa-search-1748337331526.ts +0 -144
- package/src/testing/exa-search-1748337344119.ts +0 -133
- package/src/typedefs.d.ts +0 -8
- package/src/types.ts +0 -47
- /package/src/extraction/extraction-llm-function/{instructions.tpl → transcription-instructions.tpl} +0 -0
package/src/session/session.ts
CHANGED
|
@@ -2,42 +2,57 @@
|
|
|
2
2
|
// Copyright 2025 DXOS.org
|
|
3
3
|
//
|
|
4
4
|
|
|
5
|
-
import
|
|
6
|
-
import
|
|
7
|
-
import
|
|
5
|
+
import type * as AiError from '@effect/ai/AiError';
|
|
6
|
+
import * as LanguageModel from '@effect/ai/LanguageModel';
|
|
7
|
+
import type * as Tool from '@effect/ai/Tool';
|
|
8
|
+
import type * as Toolkit from '@effect/ai/Toolkit';
|
|
9
|
+
import * as Chunk from 'effect/Chunk';
|
|
10
|
+
import * as Effect from 'effect/Effect';
|
|
11
|
+
import type * as Schema from 'effect/Schema';
|
|
12
|
+
import * as Stream from 'effect/Stream';
|
|
8
13
|
|
|
9
14
|
import {
|
|
10
15
|
AiParser,
|
|
11
16
|
AiPreprocessor,
|
|
12
|
-
getToolCalls,
|
|
13
|
-
runTool,
|
|
14
|
-
ToolExecutionService,
|
|
15
|
-
ToolResolverService,
|
|
16
|
-
type AgentStatus,
|
|
17
|
-
type AiInputPreprocessingError,
|
|
18
17
|
type AiToolNotFoundError,
|
|
19
|
-
type
|
|
18
|
+
type PromptPreprocessingError,
|
|
19
|
+
type ToolExecutionService,
|
|
20
|
+
type ToolResolverService,
|
|
21
|
+
callTool,
|
|
22
|
+
getToolCalls,
|
|
20
23
|
} from '@dxos/ai';
|
|
21
|
-
import { Event } from '@dxos/async';
|
|
22
24
|
import { type Blueprint } from '@dxos/blueprints';
|
|
23
25
|
import { todo } from '@dxos/debug';
|
|
24
26
|
import { Obj } from '@dxos/echo';
|
|
25
|
-
import {
|
|
26
|
-
import { type ObjectId } from '@dxos/echo-schema';
|
|
27
|
-
import { DatabaseService } from '@dxos/functions';
|
|
27
|
+
import { type FunctionInvocationService, TracingService } from '@dxos/functions';
|
|
28
28
|
import { log } from '@dxos/log';
|
|
29
|
-
import {
|
|
29
|
+
import { Message } from '@dxos/types';
|
|
30
|
+
|
|
31
|
+
import { type AiAssistantError } from '../errors';
|
|
32
|
+
|
|
33
|
+
import { formatSystemPrompt, formatUserPrompt } from './format';
|
|
34
|
+
import { GenerationObserver } from './observer';
|
|
35
|
+
|
|
36
|
+
export type AiSessionRunError = AiError.AiError | PromptPreprocessingError | AiToolNotFoundError | AiAssistantError;
|
|
30
37
|
|
|
31
|
-
|
|
38
|
+
export type AiSessionRunRequirements =
|
|
39
|
+
| LanguageModel.LanguageModel
|
|
40
|
+
| ToolExecutionService
|
|
41
|
+
| ToolResolverService
|
|
42
|
+
| TracingService
|
|
43
|
+
| FunctionInvocationService;
|
|
32
44
|
|
|
33
45
|
export type AiSessionOptions = {};
|
|
34
46
|
|
|
35
|
-
export type
|
|
47
|
+
export type AiSessionRunProps<Tools extends Record<string, Tool.Any>> = {
|
|
36
48
|
prompt: string;
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
49
|
+
// TODO(wittjosiah): Rename to systemPrompt.
|
|
50
|
+
system?: string;
|
|
51
|
+
history?: Message.Message[];
|
|
52
|
+
objects?: Obj.Unknown[];
|
|
53
|
+
blueprints?: readonly Blueprint.Blueprint[];
|
|
54
|
+
toolkit?: Toolkit.WithHandler<Tools>;
|
|
55
|
+
observer?: GenerationObserver<Tools>;
|
|
41
56
|
};
|
|
42
57
|
|
|
43
58
|
/**
|
|
@@ -51,165 +66,150 @@ export type SessionRunOptions<Tools extends AiTool.Any> = {
|
|
|
51
66
|
* Could be personal or shared.
|
|
52
67
|
*/
|
|
53
68
|
export class AiSession {
|
|
54
|
-
|
|
69
|
+
/** Prevents concurrent execution of session. */
|
|
55
70
|
private readonly _semaphore = Effect.runSync(Effect.makeSemaphore(1));
|
|
56
71
|
|
|
57
|
-
/**
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
/** Current streaming response. */
|
|
61
|
-
private _stream: GenerationStream | undefined;
|
|
62
|
-
|
|
63
|
-
/** Prior history from queue. */
|
|
64
|
-
private _history: DataType.Message[] = [];
|
|
65
|
-
|
|
66
|
-
/**
|
|
67
|
-
* New message.
|
|
68
|
-
*/
|
|
69
|
-
public readonly message = new Event<DataType.Message>();
|
|
72
|
+
/** Message history from queue. */
|
|
73
|
+
// TODO(burdon): Evolve into supporting a git-like graph of messages.
|
|
74
|
+
private _history: Message.Message[] = [];
|
|
70
75
|
|
|
71
|
-
/**
|
|
72
|
-
|
|
73
|
-
*/
|
|
74
|
-
public readonly block = new Event<ContentBlock.Any>();
|
|
76
|
+
/** Pending messages for this session (incl. the current prompt). */
|
|
77
|
+
private _pending: Message.Message[] = [];
|
|
75
78
|
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
public readonly update = new Event<ContentBlock.Any>();
|
|
79
|
+
private _started = 0;
|
|
80
|
+
private _ended = 0;
|
|
81
|
+
private _toolCalls = 0;
|
|
80
82
|
|
|
81
|
-
|
|
82
|
-
* Unparsed events from the underlying generation stream.
|
|
83
|
-
*/
|
|
84
|
-
public readonly streamEvent = new Event<AiResponse.Part>();
|
|
83
|
+
constructor(private readonly _options: AiSessionOptions = {}) {}
|
|
85
84
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
public readonly userMessage = new Event<DataType.Message>();
|
|
85
|
+
get duration(): number {
|
|
86
|
+
return this._ended - this._started;
|
|
87
|
+
}
|
|
90
88
|
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
*/
|
|
95
|
-
public readonly statusReport = new Event<AgentStatus>();
|
|
89
|
+
get toolCalls(): number {
|
|
90
|
+
return this._toolCalls;
|
|
91
|
+
}
|
|
96
92
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
93
|
+
run = <Tools extends Record<string, Tool.Any>>({
|
|
94
|
+
prompt,
|
|
95
|
+
system: systemTemplate,
|
|
96
|
+
history = [],
|
|
97
|
+
objects = [],
|
|
98
|
+
blueprints = [],
|
|
99
|
+
toolkit,
|
|
100
|
+
observer = GenerationObserver.noop(),
|
|
101
|
+
}: AiSessionRunProps<Tools>): Effect.Effect<Message.Message[], AiSessionRunError, AiSessionRunRequirements> =>
|
|
102
|
+
Effect.gen(this, function* () {
|
|
103
|
+
this._started = Date.now();
|
|
104
|
+
this._history = [...history];
|
|
105
|
+
this._pending = [];
|
|
106
|
+
const pending = this._pending;
|
|
107
|
+
|
|
108
|
+
const submitMessage = Effect.fnUntraced(function* (message: Message.Message) {
|
|
109
|
+
pending.push(message);
|
|
110
|
+
yield* observer.onMessage(message);
|
|
111
|
+
yield* TracingService.emitConverationMessage(message);
|
|
112
|
+
return message;
|
|
113
|
+
});
|
|
101
114
|
|
|
102
|
-
|
|
115
|
+
// Submit the prompt.
|
|
116
|
+
// TODO(burdon): Remove if cancelled?
|
|
117
|
+
const promptMessage = yield* submitMessage(yield* formatUserPrompt({ prompt, history }));
|
|
103
118
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
* @param options - The session options.
|
|
107
|
-
* @returns The messages generated by the session, including the user's prompt.
|
|
108
|
-
*/
|
|
109
|
-
// TODO(dmaretskyi): Toolkit context doesn't get added to the effect type.
|
|
110
|
-
run = <Tools extends AiTool.Any>(
|
|
111
|
-
options: SessionRunOptions<Tools>,
|
|
112
|
-
): Effect.Effect<
|
|
113
|
-
DataType.Message[],
|
|
114
|
-
AiAssistantError | AiInputPreprocessingError | AiToolNotFoundError | AiError.AiError,
|
|
115
|
-
AiLanguageModel.AiLanguageModel | ToolResolverService | ToolExecutionService | AiTool.ToHandler<Tools>
|
|
116
|
-
> =>
|
|
117
|
-
Effect.gen(this, function* () {
|
|
118
|
-
this._history = [...options.history];
|
|
119
|
+
// Generate system and prompt messages.
|
|
120
|
+
const system = yield* formatSystemPrompt({ system: systemTemplate, blueprints, objects }).pipe(Effect.orDie);
|
|
119
121
|
|
|
120
|
-
|
|
121
|
-
this._pending = [promptMessages];
|
|
122
|
-
this.userMessage.emit(promptMessages);
|
|
122
|
+
// log('system', { prompt: system });
|
|
123
123
|
|
|
124
|
-
//
|
|
124
|
+
// Tool call loop.
|
|
125
125
|
do {
|
|
126
126
|
log('request', {
|
|
127
|
+
prompt: promptMessage,
|
|
128
|
+
system: { snippet: createSnippet(system), length: system.length },
|
|
127
129
|
pending: this._pending.length,
|
|
128
130
|
history: this._history.length,
|
|
129
|
-
|
|
131
|
+
objects: objects?.length ?? 0,
|
|
130
132
|
});
|
|
131
133
|
|
|
132
|
-
const prompt = yield* AiPreprocessor.
|
|
133
|
-
|
|
134
|
-
const blueprints = options.blueprints ?? [];
|
|
135
|
-
|
|
136
|
-
// Build system prompt from blueprint templates.
|
|
137
|
-
// TODO(dmaretskyi): Loading BP from the Database should be done at the higher level. We need a type for the resolved blueprint.
|
|
138
|
-
const systemPrompt = yield* pipe(
|
|
139
|
-
blueprints,
|
|
140
|
-
Effect.forEach((blueprint) => DatabaseService.loadRef(blueprint.instructions)),
|
|
141
|
-
Effect.flatMap(Effect.forEach((template) => DatabaseService.loadRef(template.source))),
|
|
142
|
-
Effect.map(Array.map((template) => `\n\n<blueprint>${template.content}</blueprint>`)),
|
|
143
|
-
Effect.map(Array.reduce(options.systemPrompt ?? '', String.concat)),
|
|
144
|
-
);
|
|
145
|
-
|
|
146
|
-
// Build a combined toolkit from the blueprint tools and the provided toolkit.
|
|
147
|
-
const blueprintToolkit = yield* ToolResolverService.resolveToolkit(blueprints.flatMap((bp) => bp.tools));
|
|
148
|
-
const blueprintToolkitHandler: Context.Context<AiTool.ToHandler<AiTool.Any>> =
|
|
149
|
-
yield* blueprintToolkit.toContext(yield* ToolExecutionService.handlersFor(blueprintToolkit));
|
|
150
|
-
const toolkit = options.toolkit != null ? AiToolkit.merge(options.toolkit, blueprintToolkit) : blueprintToolkit;
|
|
151
|
-
const toolkitWithBlueprintHandlers = yield* toolkit.pipe(
|
|
152
|
-
Effect.provide(blueprintToolkitHandler) as any,
|
|
153
|
-
) as Effect.Effect<AiToolkit.ToHandler<any>, never, AiTool.ToHandler<Tools>>;
|
|
154
|
-
|
|
155
|
-
log.info('run', {
|
|
156
|
-
systemPrompt: [systemPrompt.slice(0, 32), '...', systemPrompt.slice(-32), systemPrompt.length].join(' '),
|
|
157
|
-
tools: Object.keys(blueprintToolkit.tools),
|
|
158
|
-
});
|
|
134
|
+
const prompt = yield* AiPreprocessor.preprocessPrompt([...this._history, ...this._pending], { system });
|
|
159
135
|
|
|
160
|
-
|
|
136
|
+
// Execute the stream request.
|
|
137
|
+
// logDump('prompt', Prompt.Prompt.pipe(Schema.encodeSync)(prompt));
|
|
138
|
+
const blocks = yield* LanguageModel.streamText({
|
|
161
139
|
prompt,
|
|
162
|
-
|
|
163
|
-
toolkit: toolkitWithBlueprintHandlers,
|
|
140
|
+
toolkit,
|
|
164
141
|
disableToolCallResolution: true,
|
|
165
142
|
}).pipe(
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
this.block.emit(block);
|
|
173
|
-
}
|
|
174
|
-
}),
|
|
175
|
-
onPart: (part) =>
|
|
176
|
-
Effect.gen(this, function* () {
|
|
177
|
-
this.streamEvent.emit(part);
|
|
178
|
-
}),
|
|
143
|
+
// TOOD(dmaretskyi): Error mapping.
|
|
144
|
+
AiParser.parseResponse({
|
|
145
|
+
onBegin: () => observer.onBegin(),
|
|
146
|
+
onBlock: (block) => observer.onBlock(block),
|
|
147
|
+
onPart: (part) => observer.onPart(part),
|
|
148
|
+
onEnd: (summary) => observer.onEnd(summary),
|
|
179
149
|
}),
|
|
180
150
|
Stream.runCollect,
|
|
181
151
|
Effect.map(Chunk.toArray),
|
|
182
152
|
);
|
|
183
153
|
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
154
|
+
// log('blocks', { blocks });
|
|
155
|
+
|
|
156
|
+
// Create the response message.
|
|
157
|
+
const response = yield* submitMessage(
|
|
158
|
+
Obj.make(Message.Message, {
|
|
159
|
+
created: new Date().toISOString(),
|
|
160
|
+
sender: { role: 'assistant' },
|
|
161
|
+
blocks,
|
|
162
|
+
}),
|
|
163
|
+
);
|
|
191
164
|
|
|
165
|
+
// Parse the response for tool calls.
|
|
192
166
|
const toolCalls = getToolCalls(response);
|
|
193
167
|
if (toolCalls.length === 0) {
|
|
194
168
|
break;
|
|
169
|
+
} else if (!toolkit) {
|
|
170
|
+
throw new Error('No toolkit provided'); // TODO(burdon): Throw user error?
|
|
195
171
|
}
|
|
196
172
|
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
173
|
+
// Execute the tool calls.
|
|
174
|
+
// TODO(burdon): Retry errors? Write result when each completes individually?
|
|
175
|
+
const toolResults = yield* Effect.forEach(toolCalls, (toolCall) => {
|
|
176
|
+
return callTool(toolkit, toolCall).pipe(
|
|
177
|
+
Effect.provide(
|
|
178
|
+
TracingService.layerSubframe((context) => ({
|
|
179
|
+
...context,
|
|
180
|
+
parentMessage: response.id,
|
|
181
|
+
toolCallId: toolCall.toolCallId,
|
|
182
|
+
})),
|
|
183
|
+
),
|
|
184
|
+
);
|
|
185
|
+
});
|
|
186
|
+
|
|
187
|
+
// Add to queue and continue loop.
|
|
188
|
+
// TODO(wittjosiah): Sometimes tool error results are added to the queue before the tool agent statuses.
|
|
189
|
+
// This results in a broken execution graph.
|
|
190
|
+
yield* submitMessage(
|
|
191
|
+
Obj.make(Message.Message, {
|
|
200
192
|
created: new Date().toISOString(),
|
|
201
|
-
sender: { role: '
|
|
193
|
+
sender: { role: 'tool' },
|
|
202
194
|
blocks: toolResults,
|
|
203
195
|
}),
|
|
204
196
|
);
|
|
197
|
+
|
|
198
|
+
this._toolCalls++;
|
|
205
199
|
} while (true);
|
|
206
200
|
|
|
201
|
+
this._ended = Date.now();
|
|
202
|
+
log('done', { pending: this._pending.length, duration: this.duration, tools: this._toolCalls });
|
|
207
203
|
return this._pending;
|
|
208
204
|
}).pipe(this._semaphore.withPermits(1), Effect.withSpan('AiSession.run'));
|
|
209
205
|
|
|
206
|
+
/**
|
|
207
|
+
* @deprecated
|
|
208
|
+
*/
|
|
209
|
+
// TODO(burdon): Implement or remove.
|
|
210
210
|
async runStructured<S extends Schema.Schema.AnyNoContext>(
|
|
211
|
-
|
|
212
|
-
|
|
211
|
+
_schema: S,
|
|
212
|
+
_options: AiSessionRunProps<any>,
|
|
213
213
|
): Promise<Schema.Schema.Type<S>> {
|
|
214
214
|
return todo();
|
|
215
215
|
// const parser = structuredOutputParser(schema);
|
|
@@ -219,118 +219,16 @@ export class AiSession {
|
|
|
219
219
|
// });
|
|
220
220
|
// return parser.getResult(result);
|
|
221
221
|
}
|
|
222
|
-
|
|
223
|
-
private _formatUserPrompt = (prompt: string, history: DataType.Message[]) =>
|
|
224
|
-
Effect.gen(function* () {
|
|
225
|
-
const prelude: ContentBlock.Any[] = [];
|
|
226
|
-
|
|
227
|
-
// TODO(dmaretskyi): Evaluate other approaches as `serviceOption` isn't represented in the type system.
|
|
228
|
-
const artifactDiffResolver = yield* Effect.serviceOption(ArtifactDiffResolver);
|
|
229
|
-
if (Option.isSome(artifactDiffResolver)) {
|
|
230
|
-
const versions = gatherObjectVersions(history);
|
|
231
|
-
|
|
232
|
-
const artifactDiff = yield* Effect.tryPromise({
|
|
233
|
-
try: () =>
|
|
234
|
-
artifactDiffResolver.value.resolve(
|
|
235
|
-
[...versions.entries()].map(([id, version]) => ({ id, lastVersion: version })),
|
|
236
|
-
),
|
|
237
|
-
catch: AiAssistantError.wrap('Artifact diff resolution error'),
|
|
238
|
-
});
|
|
239
|
-
|
|
240
|
-
log.info('vision', {
|
|
241
|
-
artifactDiff,
|
|
242
|
-
versions,
|
|
243
|
-
});
|
|
244
|
-
|
|
245
|
-
for (const [id, { version }] of [...artifactDiff.entries()]) {
|
|
246
|
-
if (ObjectVersion.equals(version, versions.get(id)!)) {
|
|
247
|
-
artifactDiff.delete(id);
|
|
248
|
-
continue;
|
|
249
|
-
}
|
|
250
|
-
|
|
251
|
-
prelude.push({ _tag: 'anchor', objectId: id, version });
|
|
252
|
-
}
|
|
253
|
-
if (artifactDiff.size > 0) {
|
|
254
|
-
prelude.push(createArtifactUpdateBlock(artifactDiff));
|
|
255
|
-
}
|
|
256
|
-
}
|
|
257
|
-
|
|
258
|
-
return Obj.make(DataType.Message, {
|
|
259
|
-
created: new Date().toISOString(),
|
|
260
|
-
sender: { role: 'user' },
|
|
261
|
-
blocks: [...prelude, { _tag: 'text', text: prompt }],
|
|
262
|
-
});
|
|
263
|
-
});
|
|
264
|
-
|
|
265
|
-
abort(): void {
|
|
266
|
-
this._stream?.abort();
|
|
267
|
-
}
|
|
268
222
|
}
|
|
269
223
|
|
|
270
|
-
const
|
|
271
|
-
|
|
272
|
-
for (const message of messages) {
|
|
273
|
-
for (const block of message.blocks) {
|
|
274
|
-
if (block._tag === 'anchor') {
|
|
275
|
-
artifactIds.set(block.objectId, block.version as ObjectVersion);
|
|
276
|
-
}
|
|
277
|
-
}
|
|
278
|
-
}
|
|
279
|
-
|
|
280
|
-
return artifactIds;
|
|
281
|
-
};
|
|
224
|
+
const createSnippet = (text: string, len = 32) =>
|
|
225
|
+
text.length <= len * 2 ? text : [text.slice(0, len), '...', text.slice(-len)].join('');
|
|
282
226
|
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
text: `
|
|
291
|
-
The following artifacts have been updated since the last message:
|
|
292
|
-
${[...artifactDiff.entries()]
|
|
293
|
-
.map(([id, { diff }]) => `<changed-artifact id="${id}">${diff ? `\n${diff}` : ''}</changed-artifact>`)
|
|
294
|
-
.join('\n')}
|
|
295
|
-
`,
|
|
296
|
-
};
|
|
227
|
+
// TODO(dmaretskyi): Extract as a general util.
|
|
228
|
+
const logDump = (message: string, data: unknown) => {
|
|
229
|
+
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
|
230
|
+
const { writeFileSync } = require('node:fs');
|
|
231
|
+
const path = `/tmp/log-data-${Date.now()}.json`;
|
|
232
|
+
writeFileSync(path, JSON.stringify(data, null, 2));
|
|
233
|
+
log.info(message, { path });
|
|
297
234
|
};
|
|
298
|
-
|
|
299
|
-
/**
|
|
300
|
-
* Resolves artifact ids to their versions.
|
|
301
|
-
* Used to give the model a sense of the changes to the artifacts made by users during the conversation.
|
|
302
|
-
* The artifacts versions are pinned in the history, and whenever the artifact changes in-between assistant's steps,
|
|
303
|
-
* a diff is inserted into the conversation.
|
|
304
|
-
*
|
|
305
|
-
* Can be optionally provided to the session run call.
|
|
306
|
-
*/
|
|
307
|
-
// TODO(dmaretskyi): Convert to Context.Reference
|
|
308
|
-
export class ArtifactDiffResolver extends Context.Tag('ArtifactDiffResolver')<
|
|
309
|
-
ArtifactDiffResolver,
|
|
310
|
-
ArtifactDiffResolver.Service
|
|
311
|
-
>() {}
|
|
312
|
-
|
|
313
|
-
export namespace ArtifactDiffResolver {
|
|
314
|
-
export type Service = {
|
|
315
|
-
resolve: (artifacts: { id: ObjectId; lastVersion: ObjectVersion }[]) => Promise<
|
|
316
|
-
Map<
|
|
317
|
-
ObjectId,
|
|
318
|
-
{
|
|
319
|
-
version: ObjectVersion;
|
|
320
|
-
diff?: string;
|
|
321
|
-
}
|
|
322
|
-
>
|
|
323
|
-
>;
|
|
324
|
-
};
|
|
325
|
-
}
|
|
326
|
-
|
|
327
|
-
const runTools: <Tools extends AiTool.Any>(
|
|
328
|
-
toolCalls: ContentBlock.ToolCall[],
|
|
329
|
-
toolkit: AiToolkit.AiToolkit<Tools>,
|
|
330
|
-
) => Effect.Effect<ContentBlock.ToolResult[], AiError.AiError, AiTool.ToHandler<Tools>> = Effect.fn('runTools')(
|
|
331
|
-
function* (toolCalls, toolkit) {
|
|
332
|
-
const toolkitWithHandlers = Effect.isEffect(toolkit) ? yield* toolkit : toolkit;
|
|
333
|
-
|
|
334
|
-
return yield* Effect.forEach(toolCalls, (toolCall) => runTool(toolkitWithHandlers, toolCall));
|
|
335
|
-
},
|
|
336
|
-
);
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
//
|
|
2
|
+
// Copyright 2025 DXOS.org
|
|
3
|
+
//
|
|
4
|
+
|
|
5
|
+
import type * as Tool from '@effect/ai/Tool';
|
|
6
|
+
import * as Toolkit from '@effect/ai/Toolkit';
|
|
7
|
+
import * as Effect from 'effect/Effect';
|
|
8
|
+
|
|
9
|
+
import { type AiToolNotFoundError, ToolExecutionService, ToolResolverService } from '@dxos/ai';
|
|
10
|
+
import { type Blueprint } from '@dxos/blueprints';
|
|
11
|
+
import { isTruthy } from '@dxos/util';
|
|
12
|
+
|
|
13
|
+
export type CreateToolkitProps = {
|
|
14
|
+
toolkit?: Toolkit.Any;
|
|
15
|
+
blueprints?: readonly Blueprint.Blueprint[];
|
|
16
|
+
};
|
|
17
|
+
|
|
18
|
+
/**
|
|
19
|
+
* Build a combined toolkit from the blueprint tools and the provided toolkit.
|
|
20
|
+
*/
|
|
21
|
+
export const createToolkit = ({
|
|
22
|
+
toolkit: toolkitProp,
|
|
23
|
+
blueprints = [],
|
|
24
|
+
}: CreateToolkitProps): Effect.Effect<
|
|
25
|
+
Toolkit.WithHandler<any>,
|
|
26
|
+
AiToolNotFoundError,
|
|
27
|
+
ToolResolverService | ToolExecutionService | Tool.HandlersFor<any>
|
|
28
|
+
> =>
|
|
29
|
+
Effect.gen(function* () {
|
|
30
|
+
const blueprintToolkit = yield* ToolResolverService.resolveToolkit(blueprints.flatMap(({ tools }) => tools));
|
|
31
|
+
const blueprintToolHandler = yield* blueprintToolkit.toContext(ToolExecutionService.handlersFor(blueprintToolkit));
|
|
32
|
+
|
|
33
|
+
const toolkit = Toolkit.merge(...[toolkitProp, blueprintToolkit].filter(isTruthy));
|
|
34
|
+
return yield* toolkit.pipe(Effect.provide(blueprintToolHandler)) as any as Effect.Effect<
|
|
35
|
+
Toolkit.WithHandler<any>,
|
|
36
|
+
never,
|
|
37
|
+
Tool.HandlersFor<any>
|
|
38
|
+
>;
|
|
39
|
+
});
|
|
@@ -2,12 +2,12 @@
|
|
|
2
2
|
// Copyright 2025 DXOS.org
|
|
3
3
|
//
|
|
4
4
|
|
|
5
|
-
import
|
|
5
|
+
import * as Schema from 'effect/Schema';
|
|
6
6
|
|
|
7
|
-
import { type
|
|
8
|
-
import { getVersion } from '@dxos/echo-db';
|
|
9
|
-
import { ObjectId
|
|
10
|
-
import { type ContentBlock } from '@dxos/
|
|
7
|
+
import { type Obj } from '@dxos/echo';
|
|
8
|
+
import { type ObjectVersion, getVersion } from '@dxos/echo-db';
|
|
9
|
+
import { ObjectId } from '@dxos/keys';
|
|
10
|
+
import { type ContentBlock } from '@dxos/types';
|
|
11
11
|
|
|
12
12
|
// TODO(dmaretskyi): Extract.
|
|
13
13
|
const ObjectVersionSchema = Schema.Unknown as Schema.Schema<ObjectVersion>;
|
|
@@ -15,7 +15,7 @@ const ObjectVersionSchema = Schema.Unknown as Schema.Schema<ObjectVersion>;
|
|
|
15
15
|
const VersionPinSchema = Schema.Struct({
|
|
16
16
|
// TODO(dmaretskyi): Use Ref when those support encoding.
|
|
17
17
|
objectId: ObjectId,
|
|
18
|
-
// TODO(dmaretskyi): Could be opaque
|
|
18
|
+
// TODO(dmaretskyi): Could be opaque.
|
|
19
19
|
version: ObjectVersionSchema,
|
|
20
20
|
});
|
|
21
21
|
|
|
@@ -27,11 +27,11 @@ export interface VersionPin extends Schema.Schema.Type<typeof VersionPinSchema>
|
|
|
27
27
|
|
|
28
28
|
export const VersionPin: typeof VersionPinSchema & {
|
|
29
29
|
DISPOSITION: 'version-pin';
|
|
30
|
-
fromObject: (object:
|
|
30
|
+
fromObject: (object: Obj.Unknown) => VersionPin;
|
|
31
31
|
createBlock: (pin: VersionPin) => ContentBlock.Any;
|
|
32
32
|
} = class extends VersionPinSchema {
|
|
33
33
|
static readonly DISPOSITION = 'version-pin';
|
|
34
|
-
static fromObject(object:
|
|
34
|
+
static fromObject(object: Obj.Unknown): VersionPin {
|
|
35
35
|
return VersionPin.make({
|
|
36
36
|
objectId: object.id,
|
|
37
37
|
version: getVersion(object),
|
package/src/templates/index.ts
CHANGED
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
{{! System Core }}
|
|
2
|
+
|
|
3
|
+
## Input Formats
|
|
4
|
+
|
|
5
|
+
### Blueprints
|
|
6
|
+
|
|
7
|
+
You may be provided with Blueprint definitions that include specific instructions and tools for a given set of tasks.
|
|
8
|
+
Blue print instructions will be enclosed in a <blueprint> tag.
|
|
9
|
+
|
|
10
|
+
### Context objects
|
|
11
|
+
|
|
12
|
+
You may be provided with references to objects that are relevant to the current prompt.
|
|
13
|
+
Objects will be in the form of a <object> tag with a <dxn> element that is the unique key of the object required by many tools.
|
|
14
|
+
The <typename> element is the type of the object.
|
|
15
|
+
|
|
16
|
+
<object>
|
|
17
|
+
<dxn>object-id</dxn>
|
|
18
|
+
<typename>example.com/type/TypeName</typename>
|
|
19
|
+
</object>
|
|
20
|
+
|
|
21
|
+
## Response Formats
|
|
22
|
+
|
|
23
|
+
It is very important to respond in the correct format.
|
|
24
|
+
You response must be valid Markdown.
|
|
25
|
+
In some cases you should respond with well-formed XML tags along with the content.
|
|
26
|
+
Code snippets should use 2-space indents.
|
|
27
|
+
|
|
28
|
+
{{#if cot}}
|
|
29
|
+
Before responding, explain your reasoning and include your detailed chain-of-thought enclosed in a <cot> tag.
|
|
30
|
+
The <cot> tag should be the first thing in your response.
|
|
31
|
+
{{/if}}
|
|
32
|
+
|
|
33
|
+
## Block references
|
|
34
|
+
|
|
35
|
+
You can reply with a block reference for objects if you have the DXN. For example:
|
|
36
|
+
|
|
37
|
+
<object>
|
|
38
|
+
<dxn>dxn:queue:data:B6INSIBY3CBEF4M5VZRYBCMAHQMPYK5AJ:01K24XMVHSZHS97SG1VTVQDM5Z:01K24XPK464FSCKVQJAB2H662M</dxn>
|
|
39
|
+
</object>
|
|
40
|
+
<object>
|
|
41
|
+
<dxn>dxn:echo:@:01K24XPK464FSCKVQJAB2H662M</dxn>
|
|
42
|
+
</object>
|
|
43
|
+
|
|
44
|
+
### Suggestions
|
|
45
|
+
|
|
46
|
+
Each suggestion must in the form of a proposed action enclosed in a <suggestion> tag.
|
|
47
|
+
Example: <suggestion>Show on a map</suggestion>
|
|
48
|
+
Do not use bullets when listing suggestions.
|
|
49
|
+
|
|
50
|
+
### Multiple choice questions
|
|
51
|
+
|
|
52
|
+
You may ask multiple choice questions by wrapping consise options inside <option> tags inside a well formed <select> tag.
|
|
53
|
+
Example: <select><option>Yes</option><option>No</option></select>
|
|
54
|
+
|
|
55
|
+
## Toolkit
|
|
56
|
+
|
|
57
|
+
If the user asks for a list of tools, then just emit a single self-closing <toolkit/> tag instead of listing the tools.
|
|
58
|
+
|
|
59
|
+
## Context
|
|
60
|
+
|
|
61
|
+
The current date and time is {{DATETIME}}.
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
{{! System Prompt }}
|
|
2
|
+
|
|
3
|
+
- You are a helpful assistant called Kai.
|
|
4
|
+
- Your name is Kai and you are powered by different language models.
|
|
5
|
+
- You were created by DXOS and operate inside of Composer and on the DXOS EDGE network.
|
|
6
|
+
- In your initial greeting state your name and make some suggestions based on the current context objects.
|
|
7
|
+
- You are an advanced AI assistant capable of creating and managing artifacts from provided data and tools.
|
|
8
|
+
- Your task is to process user commands and questions and decide how best to respond.
|
|
9
|
+
- In general be concise and direct.
|
|
10
|
+
- Follow all instructions carefully.
|
|
11
|
+
|
|
12
|
+
## Planning
|
|
13
|
+
|
|
14
|
+
- Analyze the structure and type of the content in the user's message.
|
|
15
|
+
- Determine if you complete the task using the available blueprint definitions?
|
|
16
|
+
- If you can't complete the task using the available blueprint definitions, query the list of available blueprint definitions using the appropriate tool.
|
|
17
|
+
- Identify which blueprint definitions are relevant to the user's request.
|
|
18
|
+
- Evaluate the potential benefits of creating an artifact vs. normal processing for each identified element.
|
|
19
|
+
- Make a final decision on whether to create an artifact and explain your reasoning.
|
|
20
|
+
- Are the required blueprint definitions already available?
|
|
21
|
+
- If not, select which blueprint definition(s) will be the most relevant and require them using the require_artifact_definitions tool.
|
|
22
|
+
- The require'd artifact tools will be available for use after require.
|
|
23
|
+
- If creating an artifact, outline how you will structure it within the response.
|
|
24
|
+
- Decide if the artifact needs to be shown to the user.
|
|
25
|
+
- Call the show tool to show the artifact to the user.
|
|
26
|
+
- Your reasoning must include: whether to use artifacts or not, to create one or query, whether to show the artifact to the user, and how to structure the response.
|
|
27
|
+
|
|
28
|
+
## Blueprints and Artifacts
|
|
29
|
+
|
|
30
|
+
- Determine if the interaction involves an artifact. Prefer artifacts for tables, lists, spreadsheets, kanbans, games, images, and other structured data.
|
|
31
|
+
- Determine if the user is explicitly talking about creating a new artifact, or wants to use an existing artifact.
|
|
32
|
+
- If it is ambiguous, query for existing artifacts first and then decide.
|
|
33
|
+
- If you decide to create an artifact, call the associated tool to create the artifact.
|
|
34
|
+
- Artifacts are stored in the database. Tools are used to create and query artifacts.
|
|
35
|
+
- If you are unsure about creating an artifact ask the user for clarification.
|
|
36
|
+
- Artifacts are mutable objects that can change over the course of the conversation.
|
|
37
|
+
- Always re-query the artifact using the tool (like query or inspect) to get the latest state of the artifact before answering the user.
|
|
38
|
+
- You must never generate the id of the artifact; only recall the ids that are already in the history.
|
|
39
|
+
- Artifacts are created by requiring the specific artifact using the require_artifact tool and creating it by calling the associated tool.
|
|
40
|
+
- You can add suggested actions at the end of your response.
|
|
41
|
+
- Suggested actions should be very concise and start with a verb and be phrased as a command to an agent -- not a question to the user.
|
|
42
|
+
- Suggested actions must be in the form of a user instruction that you can follow.
|
|
43
|
+
- Suggested actions could include actions that create artifacts.
|
|
44
|
+
- After creating an artifact use a tool to add it to the chat context.
|
|
45
|
+
|
|
46
|
+
## Context
|
|
47
|
+
|
|
48
|
+
Today is {{DATETIME}}.
|