@dxos/assistant 0.8.4-main.c1de068 → 0.8.4-main.c85a9c8dae
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -0
- package/dist/lib/neutral/chunk-BE6SRJ4N.mjs +193 -0
- package/dist/lib/neutral/chunk-BE6SRJ4N.mjs.map +7 -0
- package/dist/lib/neutral/extraction/index.mjs +223 -0
- package/dist/lib/neutral/extraction/index.mjs.map +7 -0
- package/dist/lib/neutral/index.mjs +1328 -0
- package/dist/lib/neutral/index.mjs.map +7 -0
- package/dist/lib/neutral/meta.json +1 -0
- package/dist/lib/neutral/testing/index.mjs +38 -0
- package/dist/lib/neutral/testing/index.mjs.map +7 -0
- package/dist/types/src/conversation/context.d.ts +146 -0
- package/dist/types/src/conversation/context.d.ts.map +1 -0
- package/dist/types/src/conversation/context.test.d.ts +2 -0
- package/dist/types/src/conversation/context.test.d.ts.map +1 -0
- package/dist/types/src/conversation/conversation.d.ts +53 -25
- package/dist/types/src/conversation/conversation.d.ts.map +1 -1
- package/dist/types/src/conversation/conversation.test.d.ts +2 -0
- package/dist/types/src/conversation/conversation.test.d.ts.map +1 -0
- package/dist/types/src/conversation/index.d.ts +1 -0
- package/dist/types/src/conversation/index.d.ts.map +1 -1
- package/dist/types/src/diff.d.ts +17 -0
- package/dist/types/src/diff.d.ts.map +1 -0
- package/dist/types/src/diff.test.d.ts +2 -0
- package/dist/types/src/diff.test.d.ts.map +1 -0
- package/dist/types/src/errors.d.ts +47 -18
- package/dist/types/src/errors.d.ts.map +1 -1
- package/dist/types/src/execution-graph.d.ts +85 -0
- package/dist/types/src/execution-graph.d.ts.map +1 -0
- package/dist/types/src/execution-graph.test.d.ts +2 -0
- package/dist/types/src/execution-graph.test.d.ts.map +1 -0
- package/dist/types/src/extraction/extraction-llm-function/extraction-llm-function.d.ts +1 -1
- package/dist/types/src/extraction/extraction-llm-function/extraction-llm-function.d.ts.map +1 -1
- package/dist/types/src/extraction/extraction-ner-function/extraction-ner-function.d.ts +290 -237
- package/dist/types/src/extraction/extraction-ner-function/extraction-ner-function.d.ts.map +1 -1
- package/dist/types/src/extraction/extraction-ner-function/named-entity-recognition.d.ts.map +1 -1
- package/dist/types/src/extraction/extraction.d.ts +349 -270
- package/dist/types/src/extraction/extraction.d.ts.map +1 -1
- package/dist/types/src/extraction/quotes.d.ts +3 -3
- package/dist/types/src/extraction/quotes.d.ts.map +1 -1
- package/dist/types/src/functions/services.d.ts +17 -4
- package/dist/types/src/functions/services.d.ts.map +1 -1
- package/dist/types/src/functions.test.d.ts +2 -0
- package/dist/types/src/functions.test.d.ts.map +1 -0
- package/dist/types/src/index.d.ts +3 -3
- package/dist/types/src/index.d.ts.map +1 -1
- package/dist/types/src/session/artifact-diff.d.ts +27 -0
- package/dist/types/src/session/artifact-diff.d.ts.map +1 -0
- package/dist/types/src/session/error-handling.d.ts +4 -0
- package/dist/types/src/session/error-handling.d.ts.map +1 -0
- package/dist/types/src/session/format.d.ts +12 -0
- package/dist/types/src/session/format.d.ts.map +1 -0
- package/dist/types/src/session/format.test.d.ts +2 -0
- package/dist/types/src/session/format.test.d.ts.map +1 -0
- package/dist/types/src/session/index.d.ts +4 -0
- package/dist/types/src/session/index.d.ts.map +1 -1
- package/dist/types/src/session/observer.d.ts +23 -0
- package/dist/types/src/session/observer.d.ts.map +1 -0
- package/dist/types/src/session/session.d.ts +41 -240
- package/dist/types/src/session/session.d.ts.map +1 -1
- package/dist/types/src/session/toolkit.d.ts +18 -0
- package/dist/types/src/session/toolkit.d.ts.map +1 -0
- package/dist/types/src/session/version-pin.d.ts +4 -4
- package/dist/types/src/session/version-pin.d.ts.map +1 -1
- package/dist/types/src/templates/index.d.ts +1 -1
- package/dist/types/src/templates/system.d.ts +18 -0
- package/dist/types/src/templates/system.d.ts.map +1 -0
- package/dist/types/src/testing/index.d.ts +1 -2
- package/dist/types/src/testing/index.d.ts.map +1 -1
- package/dist/types/src/testing/layer.d.ts +22 -0
- package/dist/types/src/testing/layer.d.ts.map +1 -0
- package/dist/types/src/types/artifact.d.ts +28 -0
- package/dist/types/src/types/artifact.d.ts.map +1 -0
- package/dist/types/src/types/index.d.ts +2 -0
- package/dist/types/src/types/index.d.ts.map +1 -0
- package/dist/types/tsconfig.tsbuildinfo +1 -1
- package/package.json +57 -35
- package/src/conversation/context.test.ts +46 -0
- package/src/conversation/context.ts +330 -0
- package/src/conversation/conversation.test.ts +65 -0
- package/src/conversation/conversation.ts +214 -63
- package/src/conversation/index.ts +1 -0
- package/src/diff.test.ts +74 -0
- package/src/diff.ts +93 -0
- package/src/errors.ts +6 -1
- package/src/execution-graph.test.ts +740 -0
- package/src/execution-graph.ts +509 -0
- package/src/extraction/extraction-llm-function/extraction-llm-function.test.ts +10 -9
- package/src/extraction/extraction-llm-function/extraction-llm-function.ts +48 -56
- package/src/extraction/extraction-ner-function/extraction-ner-function.test.ts +23 -22
- package/src/extraction/extraction-ner-function/extraction-ner-function.ts +42 -43
- package/src/extraction/extraction-ner-function/named-entity-recognition.test.ts +12 -6
- package/src/extraction/extraction-ner-function/named-entity-recognition.ts +2 -0
- package/src/extraction/extraction.ts +10 -9
- package/src/extraction/quotes.test.ts +3 -3
- package/src/extraction/quotes.ts +6 -7
- package/src/functions/services.ts +165 -95
- package/src/functions.conversations.json +1 -0
- package/src/functions.test.ts +74 -0
- package/src/index.ts +3 -3
- package/src/session/artifact-diff.ts +41 -0
- package/src/session/error-handling.ts +41 -0
- package/src/session/format.test.ts +124 -0
- package/src/session/format.ts +143 -0
- package/src/session/index.ts +4 -0
- package/src/session/observer.ts +62 -0
- package/src/session/session.conversations.json +1 -0
- package/src/session/session.test.ts +81 -200
- package/src/session/session.ts +176 -261
- package/src/session/toolkit.ts +44 -0
- package/src/session/version-pin.ts +8 -8
- package/src/templates/index.ts +1 -1
- package/src/templates/instructions/format.tpl +57 -0
- package/src/templates/instructions/system.tpl +48 -0
- package/src/templates/system.ts +41 -0
- package/src/testing/index.ts +2 -8
- package/src/testing/layer.ts +85 -0
- package/src/types/artifact.ts +89 -0
- package/src/types/index.ts +5 -0
- package/src/typings.d.ts +5 -0
- package/dist/lib/browser/index.mjs +0 -1831
- package/dist/lib/browser/index.mjs.map +0 -7
- package/dist/lib/browser/meta.json +0 -1
- package/dist/lib/node-esm/index.mjs +0 -1833
- package/dist/lib/node-esm/index.mjs.map +0 -7
- package/dist/lib/node-esm/meta.json +0 -1
- package/dist/types/bin/chat.d.ts +0 -2
- package/dist/types/bin/chat.d.ts.map +0 -1
- package/dist/types/src/context/binder.d.ts +0 -34
- package/dist/types/src/context/binder.d.ts.map +0 -1
- package/dist/types/src/context/binding.d.ts +0 -56
- package/dist/types/src/context/binding.d.ts.map +0 -1
- package/dist/types/src/context/index.d.ts +0 -3
- package/dist/types/src/context/index.d.ts.map +0 -1
- package/dist/types/src/research/exa.d.ts +0 -12
- package/dist/types/src/research/exa.d.ts.map +0 -1
- package/dist/types/src/research/graph.d.ts +0 -51
- package/dist/types/src/research/graph.d.ts.map +0 -1
- package/dist/types/src/research/graph.test.d.ts +0 -2
- package/dist/types/src/research/graph.test.d.ts.map +0 -1
- package/dist/types/src/research/index.d.ts +0 -4
- package/dist/types/src/research/index.d.ts.map +0 -1
- package/dist/types/src/research/research.d.ts +0 -10
- package/dist/types/src/research/research.d.ts.map +0 -1
- package/dist/types/src/research/research.test.d.ts +0 -2
- package/dist/types/src/research/research.test.d.ts.map +0 -1
- package/dist/types/src/session/version-pin.test.d.ts +0 -2
- package/dist/types/src/session/version-pin.test.d.ts.map +0 -1
- package/dist/types/src/summarization/summarization.test.d.ts +0 -2
- package/dist/types/src/summarization/summarization.test.d.ts.map +0 -1
- package/dist/types/src/templates/prompt.d.ts +0 -30
- package/dist/types/src/templates/prompt.d.ts.map +0 -1
- package/dist/types/src/testing/exa-search-1748337321991.d.ts +0 -38
- package/dist/types/src/testing/exa-search-1748337321991.d.ts.map +0 -1
- package/dist/types/src/testing/exa-search-1748337331526.d.ts +0 -37
- package/dist/types/src/testing/exa-search-1748337331526.d.ts.map +0 -1
- package/dist/types/src/testing/exa-search-1748337344119.d.ts +0 -58
- package/dist/types/src/testing/exa-search-1748337344119.d.ts.map +0 -1
- package/dist/types/src/types.d.ts +0 -13
- package/dist/types/src/types.d.ts.map +0 -1
- package/src/context/binder.ts +0 -103
- package/src/context/binding.ts +0 -32
- package/src/context/index.ts +0 -6
- package/src/research/exa.ts +0 -97
- package/src/research/graph.test.ts +0 -64
- package/src/research/graph.ts +0 -369
- package/src/research/index.ts +0 -7
- package/src/research/instructions-research.tpl +0 -98
- package/src/research/research.test.ts +0 -190
- package/src/research/research.ts +0 -87
- package/src/session/__snapshots__/session.test.ts.snap +0 -32
- package/src/session/version-pin.test.ts +0 -7
- package/src/summarization/summarization.test.ts +0 -208
- package/src/templates/assistant.tpl +0 -111
- package/src/templates/prompt.ts +0 -45
- package/src/testing/exa-search-1748337321991.ts +0 -131
- package/src/testing/exa-search-1748337331526.ts +0 -144
- package/src/testing/exa-search-1748337344119.ts +0 -133
- package/src/typedefs.d.ts +0 -8
- package/src/types.ts +0 -47
- /package/src/extraction/extraction-llm-function/{instructions.tpl → transcription-instructions.tpl} +0 -0
package/src/session/session.ts
CHANGED
|
@@ -2,42 +2,67 @@
|
|
|
2
2
|
// Copyright 2025 DXOS.org
|
|
3
3
|
//
|
|
4
4
|
|
|
5
|
-
import
|
|
6
|
-
import
|
|
7
|
-
import
|
|
5
|
+
import type * as AiError from '@effect/ai/AiError';
|
|
6
|
+
import * as LanguageModel from '@effect/ai/LanguageModel';
|
|
7
|
+
import type * as Tool from '@effect/ai/Tool';
|
|
8
|
+
import type * as Toolkit from '@effect/ai/Toolkit';
|
|
9
|
+
import * as Chunk from 'effect/Chunk';
|
|
10
|
+
import * as Effect from 'effect/Effect';
|
|
11
|
+
import * as Stream from 'effect/Stream';
|
|
8
12
|
|
|
9
13
|
import {
|
|
10
14
|
AiParser,
|
|
11
15
|
AiPreprocessor,
|
|
12
|
-
|
|
13
|
-
runTool,
|
|
14
|
-
ToolExecutionService,
|
|
15
|
-
ToolResolverService,
|
|
16
|
-
type AgentStatus,
|
|
17
|
-
type AiInputPreprocessingError,
|
|
16
|
+
AiSummarizer,
|
|
18
17
|
type AiToolNotFoundError,
|
|
19
|
-
type
|
|
18
|
+
type PromptPreprocessingError,
|
|
19
|
+
type ToolExecutionService,
|
|
20
|
+
type ToolResolverService,
|
|
21
|
+
callTool,
|
|
22
|
+
withoutToolCallParising,
|
|
20
23
|
} from '@dxos/ai';
|
|
21
|
-
import { Event } from '@dxos/async';
|
|
22
24
|
import { type Blueprint } from '@dxos/blueprints';
|
|
23
|
-
import { todo } from '@dxos/debug';
|
|
24
25
|
import { Obj } from '@dxos/echo';
|
|
25
|
-
import {
|
|
26
|
-
import { type ObjectId } from '@dxos/echo-schema';
|
|
27
|
-
import { DatabaseService } from '@dxos/functions';
|
|
26
|
+
import { type FunctionInvocationService, TracingService } from '@dxos/functions';
|
|
28
27
|
import { log } from '@dxos/log';
|
|
29
|
-
import {
|
|
28
|
+
import { type ContentBlock, Message } from '@dxos/types';
|
|
30
29
|
|
|
31
|
-
import { AiAssistantError } from '../errors';
|
|
30
|
+
import { type AiAssistantError } from '../errors';
|
|
32
31
|
|
|
33
|
-
|
|
32
|
+
import { formatSystemPrompt, formatUserPrompt } from './format';
|
|
33
|
+
import { GenerationObserver } from './observer';
|
|
34
34
|
|
|
35
|
-
export type
|
|
35
|
+
export type AiSessionRunError = AiError.AiError | PromptPreprocessingError | AiToolNotFoundError | AiAssistantError;
|
|
36
|
+
|
|
37
|
+
export type AiSessionRunRequirements =
|
|
38
|
+
| LanguageModel.LanguageModel
|
|
39
|
+
| ToolExecutionService
|
|
40
|
+
| ToolResolverService
|
|
41
|
+
| TracingService
|
|
42
|
+
| FunctionInvocationService;
|
|
43
|
+
|
|
44
|
+
export type AiSessionOptions = {
|
|
45
|
+
/**
|
|
46
|
+
* Summarize before executing the prompt if the existing history exceeds this threshold.
|
|
47
|
+
*/
|
|
48
|
+
summarizationThreshold?: number;
|
|
49
|
+
};
|
|
50
|
+
|
|
51
|
+
export type AiSessionRunProps<Tools extends Record<string, Tool.Any>> = {
|
|
36
52
|
prompt: string;
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
53
|
+
// TODO(wittjosiah): Rename to systemPrompt.
|
|
54
|
+
system?: string;
|
|
55
|
+
history?: Message.Message[];
|
|
56
|
+
objects?: Obj.Unknown[];
|
|
57
|
+
blueprints?: readonly Blueprint.Blueprint[];
|
|
58
|
+
toolkit?: Toolkit.WithHandler<Tools>;
|
|
59
|
+
observer?: GenerationObserver<Tools>;
|
|
60
|
+
/**
|
|
61
|
+
* Callback for when a message is received from the user, model, or tool.
|
|
62
|
+
* This is useful for streaming the output to a queue.
|
|
63
|
+
*/
|
|
64
|
+
// TODO(dmaretskyi): This is duplicated by generation observer. Better solution would be to convert the return type from effect to stream.
|
|
65
|
+
onOutput?: (message: Message.Message) => Effect.Effect<void, never, never>;
|
|
41
66
|
};
|
|
42
67
|
|
|
43
68
|
/**
|
|
@@ -51,286 +76,176 @@ export type SessionRunOptions<Tools extends AiTool.Any> = {
|
|
|
51
76
|
* Could be personal or shared.
|
|
52
77
|
*/
|
|
53
78
|
export class AiSession {
|
|
54
|
-
|
|
79
|
+
/** Prevents concurrent execution of session. */
|
|
55
80
|
private readonly _semaphore = Effect.runSync(Effect.makeSemaphore(1));
|
|
56
81
|
|
|
57
|
-
/**
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
/** Current streaming response. */
|
|
61
|
-
private _stream: GenerationStream | undefined;
|
|
62
|
-
|
|
63
|
-
/** Prior history from queue. */
|
|
64
|
-
private _history: DataType.Message[] = [];
|
|
82
|
+
/** Message history from queue. */
|
|
83
|
+
// TODO(burdon): Evolve into supporting a git-like graph of messages.
|
|
84
|
+
private _history: Message.Message[] = [];
|
|
65
85
|
|
|
66
|
-
/**
|
|
67
|
-
|
|
68
|
-
*/
|
|
69
|
-
public readonly message = new Event<DataType.Message>();
|
|
86
|
+
/** Pending messages for this session (incl. the current prompt). */
|
|
87
|
+
private _pending: Message.Message[] = [];
|
|
70
88
|
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
public readonly block = new Event<ContentBlock.Any>();
|
|
89
|
+
private _started = 0;
|
|
90
|
+
private _ended = 0;
|
|
91
|
+
private _toolCalls = 0;
|
|
75
92
|
|
|
76
|
-
|
|
77
|
-
* Update partial block (while streaming).
|
|
78
|
-
*/
|
|
79
|
-
public readonly update = new Event<ContentBlock.Any>();
|
|
93
|
+
constructor(private readonly _options: AiSessionOptions = {}) {}
|
|
80
94
|
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
public readonly streamEvent = new Event<AiResponse.Part>();
|
|
95
|
+
get duration(): number {
|
|
96
|
+
return this._ended - this._started;
|
|
97
|
+
}
|
|
85
98
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
public readonly userMessage = new Event<DataType.Message>();
|
|
99
|
+
get toolCalls(): number {
|
|
100
|
+
return this._toolCalls;
|
|
101
|
+
}
|
|
90
102
|
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
103
|
+
run = <Tools extends Record<string, Tool.Any>>({
|
|
104
|
+
prompt,
|
|
105
|
+
system: systemTemplate,
|
|
106
|
+
history = [],
|
|
107
|
+
objects = [],
|
|
108
|
+
blueprints = [],
|
|
109
|
+
toolkit,
|
|
110
|
+
observer = GenerationObserver.noop(),
|
|
111
|
+
onOutput = () => Effect.void,
|
|
112
|
+
}: AiSessionRunProps<Tools>): Effect.Effect<Message.Message[], AiSessionRunError, AiSessionRunRequirements> =>
|
|
113
|
+
Effect.gen(this, function* () {
|
|
114
|
+
this._started = Date.now();
|
|
115
|
+
this._history = [...history];
|
|
116
|
+
this._pending = [];
|
|
117
|
+
|
|
118
|
+
const submitMessage = (message: Message.Message) =>
|
|
119
|
+
Effect.gen(this, function* () {
|
|
120
|
+
this._pending.push(message);
|
|
121
|
+
yield* observer.onMessage(message);
|
|
122
|
+
yield* TracingService.emitConverationMessage(message);
|
|
123
|
+
yield* onOutput(message);
|
|
124
|
+
return message;
|
|
125
|
+
});
|
|
96
126
|
|
|
97
|
-
|
|
98
|
-
* Emits when the session is done.
|
|
99
|
-
*/
|
|
100
|
-
public readonly done = new Event<void>();
|
|
127
|
+
const system = yield* formatSystemPrompt({ system: systemTemplate, blueprints, objects }).pipe(Effect.orDie);
|
|
101
128
|
|
|
102
|
-
|
|
129
|
+
if (this._options.summarizationThreshold !== undefined) {
|
|
130
|
+
const tokenCount = yield* AiPreprocessor.estimateTokens(
|
|
131
|
+
yield* AiPreprocessor.preprocessPrompt([...this._history], {
|
|
132
|
+
system,
|
|
133
|
+
}),
|
|
134
|
+
);
|
|
135
|
+
if (tokenCount > this._options.summarizationThreshold) {
|
|
136
|
+
const summary = yield* AiSummarizer.summarize([...this._history]);
|
|
137
|
+
yield* submitMessage(summary);
|
|
138
|
+
}
|
|
139
|
+
}
|
|
103
140
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
* @returns The messages generated by the session, including the user's prompt.
|
|
108
|
-
*/
|
|
109
|
-
// TODO(dmaretskyi): Toolkit context doesn't get added to the effect type.
|
|
110
|
-
run = <Tools extends AiTool.Any>(
|
|
111
|
-
options: SessionRunOptions<Tools>,
|
|
112
|
-
): Effect.Effect<
|
|
113
|
-
DataType.Message[],
|
|
114
|
-
AiAssistantError | AiInputPreprocessingError | AiToolNotFoundError | AiError.AiError,
|
|
115
|
-
AiLanguageModel.AiLanguageModel | ToolResolverService | ToolExecutionService | AiTool.ToHandler<Tools>
|
|
116
|
-
> =>
|
|
117
|
-
Effect.gen(this, function* () {
|
|
118
|
-
this._history = [...options.history];
|
|
141
|
+
// Submit the prompt.
|
|
142
|
+
// TODO(burdon): Remove if cancelled?
|
|
143
|
+
const promptMessage = yield* submitMessage(yield* formatUserPrompt({ prompt, history }));
|
|
119
144
|
|
|
120
|
-
|
|
121
|
-
this._pending = [promptMessages];
|
|
122
|
-
this.userMessage.emit(promptMessages);
|
|
145
|
+
// Generate system and prompt messages.
|
|
123
146
|
|
|
124
|
-
//
|
|
147
|
+
// Tool call loop.
|
|
125
148
|
do {
|
|
126
149
|
log('request', {
|
|
150
|
+
prompt: promptMessage,
|
|
151
|
+
system: { snippet: createSnippet(system), length: system.length },
|
|
127
152
|
pending: this._pending.length,
|
|
128
153
|
history: this._history.length,
|
|
129
|
-
|
|
154
|
+
objects: objects?.length ?? 0,
|
|
130
155
|
});
|
|
131
156
|
|
|
132
|
-
const prompt = yield* AiPreprocessor.
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
// Build system prompt from blueprint templates.
|
|
137
|
-
// TODO(dmaretskyi): Loading BP from the Database should be done at the higher level. We need a type for the resolved blueprint.
|
|
138
|
-
const systemPrompt = yield* pipe(
|
|
139
|
-
blueprints,
|
|
140
|
-
Effect.forEach((blueprint) => DatabaseService.loadRef(blueprint.instructions)),
|
|
141
|
-
Effect.flatMap(Effect.forEach((template) => DatabaseService.loadRef(template.source))),
|
|
142
|
-
Effect.map(Array.map((template) => `\n\n<blueprint>${template.content}</blueprint>`)),
|
|
143
|
-
Effect.map(Array.reduce(options.systemPrompt ?? '', String.concat)),
|
|
144
|
-
);
|
|
145
|
-
|
|
146
|
-
// Build a combined toolkit from the blueprint tools and the provided toolkit.
|
|
147
|
-
const blueprintToolkit = yield* ToolResolverService.resolveToolkit(blueprints.flatMap((bp) => bp.tools));
|
|
148
|
-
const blueprintToolkitHandler: Context.Context<AiTool.ToHandler<AiTool.Any>> =
|
|
149
|
-
yield* blueprintToolkit.toContext(yield* ToolExecutionService.handlersFor(blueprintToolkit));
|
|
150
|
-
const toolkit = options.toolkit != null ? AiToolkit.merge(options.toolkit, blueprintToolkit) : blueprintToolkit;
|
|
151
|
-
const toolkitWithBlueprintHandlers = yield* toolkit.pipe(
|
|
152
|
-
Effect.provide(blueprintToolkitHandler) as any,
|
|
153
|
-
) as Effect.Effect<AiToolkit.ToHandler<any>, never, AiTool.ToHandler<Tools>>;
|
|
154
|
-
|
|
155
|
-
log.info('run', {
|
|
156
|
-
systemPrompt: [systemPrompt.slice(0, 32), '...', systemPrompt.slice(-32), systemPrompt.length].join(' '),
|
|
157
|
-
tools: Object.keys(blueprintToolkit.tools),
|
|
157
|
+
const prompt = yield* AiPreprocessor.preprocessPrompt([...this._history, ...this._pending], {
|
|
158
|
+
system,
|
|
159
|
+
cacheControl: 'ephemeral',
|
|
158
160
|
});
|
|
159
161
|
|
|
160
|
-
|
|
162
|
+
// Execute the stream request.
|
|
163
|
+
const messages = yield* LanguageModel.streamText({
|
|
161
164
|
prompt,
|
|
162
|
-
|
|
163
|
-
toolkit: toolkitWithBlueprintHandlers,
|
|
165
|
+
toolkit,
|
|
164
166
|
disableToolCallResolution: true,
|
|
165
167
|
}).pipe(
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
}),
|
|
175
|
-
onPart: (part) =>
|
|
176
|
-
Effect.gen(this, function* () {
|
|
177
|
-
this.streamEvent.emit(part);
|
|
178
|
-
}),
|
|
168
|
+
// Disable tool call parsing so that we can handle parameter schema errors on a per-tool call basis.
|
|
169
|
+
withoutToolCallParising,
|
|
170
|
+
// TOOD(dmaretskyi): Error mapping.
|
|
171
|
+
AiParser.parseResponse({
|
|
172
|
+
onBegin: () => observer.onBegin(),
|
|
173
|
+
onBlock: (block) => observer.onBlock(block),
|
|
174
|
+
onPart: (part) => observer.onPart(part),
|
|
175
|
+
onEnd: (summary) => observer.onEnd(summary),
|
|
179
176
|
}),
|
|
177
|
+
Stream.mapEffect((block) =>
|
|
178
|
+
Effect.gen(function* () {
|
|
179
|
+
return yield* submitMessage(
|
|
180
|
+
Obj.make(Message.Message, {
|
|
181
|
+
created: new Date().toISOString(),
|
|
182
|
+
sender: { role: 'assistant' },
|
|
183
|
+
blocks: [block],
|
|
184
|
+
}),
|
|
185
|
+
);
|
|
186
|
+
}),
|
|
187
|
+
),
|
|
180
188
|
Stream.runCollect,
|
|
181
189
|
Effect.map(Chunk.toArray),
|
|
182
190
|
);
|
|
183
191
|
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
const toolCalls = getToolCalls(response);
|
|
192
|
+
// Parse the response for tool calls.
|
|
193
|
+
const toolCalls = messages.flatMap((message) =>
|
|
194
|
+
message.blocks
|
|
195
|
+
.filter(
|
|
196
|
+
(block): block is ContentBlock.ToolCall => block._tag === 'toolCall' && block.providerExecuted === false,
|
|
197
|
+
)
|
|
198
|
+
.map((block) => ({ block, message })),
|
|
199
|
+
);
|
|
193
200
|
if (toolCalls.length === 0) {
|
|
194
201
|
break;
|
|
202
|
+
} else if (!toolkit) {
|
|
203
|
+
throw new Error('No toolkit provided'); // TODO(burdon): Throw user error?
|
|
195
204
|
}
|
|
196
205
|
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
206
|
+
// Execute the tool calls.
|
|
207
|
+
// TODO(burdon): Retry errors? Write result when each completes individually?
|
|
208
|
+
const toolResults = yield* Effect.forEach(toolCalls, ({ block, message }) => {
|
|
209
|
+
return callTool(toolkit, block).pipe(
|
|
210
|
+
Effect.provide(
|
|
211
|
+
TracingService.layerSubframe((context) => ({
|
|
212
|
+
...context,
|
|
213
|
+
parentMessage: message.id,
|
|
214
|
+
toolCallId: block.toolCallId,
|
|
215
|
+
})),
|
|
216
|
+
),
|
|
217
|
+
);
|
|
218
|
+
});
|
|
219
|
+
|
|
220
|
+
// Add to queue and continue loop.
|
|
221
|
+
// TODO(wittjosiah): Sometimes tool error results are added to the queue before the tool agent statuses.
|
|
222
|
+
// This results in a broken execution graph.
|
|
223
|
+
// TODO(dmaretskyi): Stream tool results one by one.
|
|
224
|
+
yield* submitMessage(
|
|
225
|
+
Obj.make(Message.Message, {
|
|
200
226
|
created: new Date().toISOString(),
|
|
201
|
-
sender: { role: '
|
|
227
|
+
sender: { role: 'tool' },
|
|
202
228
|
blocks: toolResults,
|
|
203
229
|
}),
|
|
204
230
|
);
|
|
231
|
+
|
|
232
|
+
this._toolCalls++;
|
|
205
233
|
} while (true);
|
|
206
234
|
|
|
235
|
+
this._ended = Date.now();
|
|
236
|
+
log('done', { pending: this._pending.length, duration: this.duration, tools: this._toolCalls });
|
|
207
237
|
return this._pending;
|
|
208
238
|
}).pipe(this._semaphore.withPermits(1), Effect.withSpan('AiSession.run'));
|
|
209
|
-
|
|
210
|
-
async runStructured<S extends Schema.Schema.AnyNoContext>(
|
|
211
|
-
schema: S,
|
|
212
|
-
options: SessionRunOptions<AiTool.Any>,
|
|
213
|
-
): Promise<Schema.Schema.Type<S>> {
|
|
214
|
-
return todo();
|
|
215
|
-
// const parser = structuredOutputParser(schema);
|
|
216
|
-
// const result = await this.run({
|
|
217
|
-
// ...options,
|
|
218
|
-
// executableTools: [...(options.executableTools ?? []), parser.tool],
|
|
219
|
-
// });
|
|
220
|
-
// return parser.getResult(result);
|
|
221
|
-
}
|
|
222
|
-
|
|
223
|
-
private _formatUserPrompt = (prompt: string, history: DataType.Message[]) =>
|
|
224
|
-
Effect.gen(function* () {
|
|
225
|
-
const prelude: ContentBlock.Any[] = [];
|
|
226
|
-
|
|
227
|
-
// TODO(dmaretskyi): Evaluate other approaches as `serviceOption` isn't represented in the type system.
|
|
228
|
-
const artifactDiffResolver = yield* Effect.serviceOption(ArtifactDiffResolver);
|
|
229
|
-
if (Option.isSome(artifactDiffResolver)) {
|
|
230
|
-
const versions = gatherObjectVersions(history);
|
|
231
|
-
|
|
232
|
-
const artifactDiff = yield* Effect.tryPromise({
|
|
233
|
-
try: () =>
|
|
234
|
-
artifactDiffResolver.value.resolve(
|
|
235
|
-
[...versions.entries()].map(([id, version]) => ({ id, lastVersion: version })),
|
|
236
|
-
),
|
|
237
|
-
catch: AiAssistantError.wrap('Artifact diff resolution error'),
|
|
238
|
-
});
|
|
239
|
-
|
|
240
|
-
log.info('vision', {
|
|
241
|
-
artifactDiff,
|
|
242
|
-
versions,
|
|
243
|
-
});
|
|
244
|
-
|
|
245
|
-
for (const [id, { version }] of [...artifactDiff.entries()]) {
|
|
246
|
-
if (ObjectVersion.equals(version, versions.get(id)!)) {
|
|
247
|
-
artifactDiff.delete(id);
|
|
248
|
-
continue;
|
|
249
|
-
}
|
|
250
|
-
|
|
251
|
-
prelude.push({ _tag: 'anchor', objectId: id, version });
|
|
252
|
-
}
|
|
253
|
-
if (artifactDiff.size > 0) {
|
|
254
|
-
prelude.push(createArtifactUpdateBlock(artifactDiff));
|
|
255
|
-
}
|
|
256
|
-
}
|
|
257
|
-
|
|
258
|
-
return Obj.make(DataType.Message, {
|
|
259
|
-
created: new Date().toISOString(),
|
|
260
|
-
sender: { role: 'user' },
|
|
261
|
-
blocks: [...prelude, { _tag: 'text', text: prompt }],
|
|
262
|
-
});
|
|
263
|
-
});
|
|
264
|
-
|
|
265
|
-
abort(): void {
|
|
266
|
-
this._stream?.abort();
|
|
267
|
-
}
|
|
268
239
|
}
|
|
269
240
|
|
|
270
|
-
const
|
|
271
|
-
|
|
272
|
-
for (const message of messages) {
|
|
273
|
-
for (const block of message.blocks) {
|
|
274
|
-
if (block._tag === 'anchor') {
|
|
275
|
-
artifactIds.set(block.objectId, block.version as ObjectVersion);
|
|
276
|
-
}
|
|
277
|
-
}
|
|
278
|
-
}
|
|
241
|
+
const createSnippet = (text: string, len = 32) =>
|
|
242
|
+
text.length <= len * 2 ? text : [text.slice(0, len), '...', text.slice(-len)].join('');
|
|
279
243
|
|
|
280
|
-
|
|
244
|
+
// TODO(dmaretskyi): Extract as a general util.
|
|
245
|
+
const logDump = (message: string, data: unknown) => {
|
|
246
|
+
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
|
247
|
+
const { writeFileSync } = require('node:fs');
|
|
248
|
+
const path = `/tmp/log-data-${Date.now()}.json`;
|
|
249
|
+
writeFileSync(path, JSON.stringify(data, null, 2));
|
|
250
|
+
log.info(message, { path });
|
|
281
251
|
};
|
|
282
|
-
|
|
283
|
-
const createArtifactUpdateBlock = (
|
|
284
|
-
artifactDiff: Map<ObjectId, { version: ObjectVersion; diff?: string }>,
|
|
285
|
-
): ContentBlock.Any => {
|
|
286
|
-
return {
|
|
287
|
-
_tag: 'text',
|
|
288
|
-
// TODO(dmaretskyi): Does this need to be a special content-block?
|
|
289
|
-
disposition: 'artifact-update',
|
|
290
|
-
text: `
|
|
291
|
-
The following artifacts have been updated since the last message:
|
|
292
|
-
${[...artifactDiff.entries()]
|
|
293
|
-
.map(([id, { diff }]) => `<changed-artifact id="${id}">${diff ? `\n${diff}` : ''}</changed-artifact>`)
|
|
294
|
-
.join('\n')}
|
|
295
|
-
`,
|
|
296
|
-
};
|
|
297
|
-
};
|
|
298
|
-
|
|
299
|
-
/**
|
|
300
|
-
* Resolves artifact ids to their versions.
|
|
301
|
-
* Used to give the model a sense of the changes to the artifacts made by users during the conversation.
|
|
302
|
-
* The artifacts versions are pinned in the history, and whenever the artifact changes in-between assistant's steps,
|
|
303
|
-
* a diff is inserted into the conversation.
|
|
304
|
-
*
|
|
305
|
-
* Can be optionally provided to the session run call.
|
|
306
|
-
*/
|
|
307
|
-
// TODO(dmaretskyi): Convert to Context.Reference
|
|
308
|
-
export class ArtifactDiffResolver extends Context.Tag('ArtifactDiffResolver')<
|
|
309
|
-
ArtifactDiffResolver,
|
|
310
|
-
ArtifactDiffResolver.Service
|
|
311
|
-
>() {}
|
|
312
|
-
|
|
313
|
-
export namespace ArtifactDiffResolver {
|
|
314
|
-
export type Service = {
|
|
315
|
-
resolve: (artifacts: { id: ObjectId; lastVersion: ObjectVersion }[]) => Promise<
|
|
316
|
-
Map<
|
|
317
|
-
ObjectId,
|
|
318
|
-
{
|
|
319
|
-
version: ObjectVersion;
|
|
320
|
-
diff?: string;
|
|
321
|
-
}
|
|
322
|
-
>
|
|
323
|
-
>;
|
|
324
|
-
};
|
|
325
|
-
}
|
|
326
|
-
|
|
327
|
-
const runTools: <Tools extends AiTool.Any>(
|
|
328
|
-
toolCalls: ContentBlock.ToolCall[],
|
|
329
|
-
toolkit: AiToolkit.AiToolkit<Tools>,
|
|
330
|
-
) => Effect.Effect<ContentBlock.ToolResult[], AiError.AiError, AiTool.ToHandler<Tools>> = Effect.fn('runTools')(
|
|
331
|
-
function* (toolCalls, toolkit) {
|
|
332
|
-
const toolkitWithHandlers = Effect.isEffect(toolkit) ? yield* toolkit : toolkit;
|
|
333
|
-
|
|
334
|
-
return yield* Effect.forEach(toolCalls, (toolCall) => runTool(toolkitWithHandlers, toolCall));
|
|
335
|
-
},
|
|
336
|
-
);
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
//
|
|
2
|
+
// Copyright 2025 DXOS.org
|
|
3
|
+
//
|
|
4
|
+
|
|
5
|
+
import type * as Tool from '@effect/ai/Tool';
|
|
6
|
+
import * as Toolkit from '@effect/ai/Toolkit';
|
|
7
|
+
import * as Effect from 'effect/Effect';
|
|
8
|
+
|
|
9
|
+
import { type AiToolNotFoundError, GenericToolkit, ToolExecutionService, ToolResolverService } from '@dxos/ai';
|
|
10
|
+
import { type Blueprint } from '@dxos/blueprints';
|
|
11
|
+
import { isTruthy } from '@dxos/util';
|
|
12
|
+
|
|
13
|
+
export type CreateToolkitProps = {
|
|
14
|
+
toolkit?: Toolkit.Any;
|
|
15
|
+
blueprints?: readonly Blueprint.Blueprint[];
|
|
16
|
+
/**
|
|
17
|
+
* Self-contained with handlers toolkits.
|
|
18
|
+
*/
|
|
19
|
+
genericToolkits?: readonly GenericToolkit.GenericToolkit[];
|
|
20
|
+
};
|
|
21
|
+
|
|
22
|
+
/**
|
|
23
|
+
* Build a combined toolkit from the blueprint tools and the provided toolkit.
|
|
24
|
+
*/
|
|
25
|
+
export const createToolkit = ({
|
|
26
|
+
toolkit: toolkitProp,
|
|
27
|
+
blueprints = [],
|
|
28
|
+
genericToolkits = [],
|
|
29
|
+
}: CreateToolkitProps): Effect.Effect<
|
|
30
|
+
Toolkit.WithHandler<any>,
|
|
31
|
+
AiToolNotFoundError,
|
|
32
|
+
ToolResolverService | ToolExecutionService | Tool.HandlersFor<any>
|
|
33
|
+
> =>
|
|
34
|
+
Effect.gen(function* () {
|
|
35
|
+
const blueprintToolkit = yield* ToolResolverService.resolveToolkit(blueprints.flatMap(({ tools }) => tools));
|
|
36
|
+
const blueprintToolHandler = yield* blueprintToolkit.toContext(ToolExecutionService.handlersFor(blueprintToolkit));
|
|
37
|
+
const genericToolkit = GenericToolkit.merge(...genericToolkits);
|
|
38
|
+
|
|
39
|
+
const toolkit = Toolkit.merge(...[toolkitProp, blueprintToolkit, genericToolkit.toolkit].filter(isTruthy));
|
|
40
|
+
return yield* toolkit.pipe(
|
|
41
|
+
Effect.provide(blueprintToolHandler),
|
|
42
|
+
Effect.provide(genericToolkit.layer),
|
|
43
|
+
) as any as Effect.Effect<Toolkit.WithHandler<any>, never, Tool.HandlersFor<any>>;
|
|
44
|
+
});
|
|
@@ -2,12 +2,12 @@
|
|
|
2
2
|
// Copyright 2025 DXOS.org
|
|
3
3
|
//
|
|
4
4
|
|
|
5
|
-
import
|
|
5
|
+
import * as Schema from 'effect/Schema';
|
|
6
6
|
|
|
7
|
-
import { type
|
|
8
|
-
import { getVersion } from '@dxos/echo-db';
|
|
9
|
-
import { ObjectId
|
|
10
|
-
import { type ContentBlock } from '@dxos/
|
|
7
|
+
import { type Obj } from '@dxos/echo';
|
|
8
|
+
import { type ObjectVersion, getVersion } from '@dxos/echo-db';
|
|
9
|
+
import { ObjectId } from '@dxos/keys';
|
|
10
|
+
import { type ContentBlock } from '@dxos/types';
|
|
11
11
|
|
|
12
12
|
// TODO(dmaretskyi): Extract.
|
|
13
13
|
const ObjectVersionSchema = Schema.Unknown as Schema.Schema<ObjectVersion>;
|
|
@@ -15,7 +15,7 @@ const ObjectVersionSchema = Schema.Unknown as Schema.Schema<ObjectVersion>;
|
|
|
15
15
|
const VersionPinSchema = Schema.Struct({
|
|
16
16
|
// TODO(dmaretskyi): Use Ref when those support encoding.
|
|
17
17
|
objectId: ObjectId,
|
|
18
|
-
// TODO(dmaretskyi): Could be opaque
|
|
18
|
+
// TODO(dmaretskyi): Could be opaque.
|
|
19
19
|
version: ObjectVersionSchema,
|
|
20
20
|
});
|
|
21
21
|
|
|
@@ -27,11 +27,11 @@ export interface VersionPin extends Schema.Schema.Type<typeof VersionPinSchema>
|
|
|
27
27
|
|
|
28
28
|
export const VersionPin: typeof VersionPinSchema & {
|
|
29
29
|
DISPOSITION: 'version-pin';
|
|
30
|
-
fromObject: (object:
|
|
30
|
+
fromObject: (object: Obj.Unknown) => VersionPin;
|
|
31
31
|
createBlock: (pin: VersionPin) => ContentBlock.Any;
|
|
32
32
|
} = class extends VersionPinSchema {
|
|
33
33
|
static readonly DISPOSITION = 'version-pin';
|
|
34
|
-
static fromObject(object:
|
|
34
|
+
static fromObject(object: Obj.Unknown): VersionPin {
|
|
35
35
|
return VersionPin.make({
|
|
36
36
|
objectId: object.id,
|
|
37
37
|
version: getVersion(object),
|
package/src/templates/index.ts
CHANGED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
{{! System Core }}
|
|
2
|
+
|
|
3
|
+
## Input Formats
|
|
4
|
+
|
|
5
|
+
### Blueprints
|
|
6
|
+
|
|
7
|
+
You may be provided with Blueprint definitions that include specific instructions and tools for a given set of tasks.
|
|
8
|
+
Blue print instructions will be enclosed in a <blueprint> tag.
|
|
9
|
+
|
|
10
|
+
### Context objects
|
|
11
|
+
|
|
12
|
+
You may be provided with references to objects that are relevant to the current prompt.
|
|
13
|
+
Objects will be in the form of a <object> tag with a <dxn> element that is the unique key of the object required by many tools.
|
|
14
|
+
The <typename> element is the type of the object.
|
|
15
|
+
|
|
16
|
+
<object>
|
|
17
|
+
<dxn>object-id</dxn>
|
|
18
|
+
<typename>example.com/type/TypeName</typename>
|
|
19
|
+
</object>
|
|
20
|
+
|
|
21
|
+
## Response Formats
|
|
22
|
+
|
|
23
|
+
It is very important to respond in the correct format.
|
|
24
|
+
You response must be valid Markdown.
|
|
25
|
+
In some cases you should respond with well-formed XML tags along with the content.
|
|
26
|
+
Code snippets should use 2-space indents.
|
|
27
|
+
|
|
28
|
+
{{#if cot}}
|
|
29
|
+
Before responding, explain your reasoning and include your detailed chain-of-thought enclosed in a <cot> tag.
|
|
30
|
+
The <cot> tag should be the first thing in your response.
|
|
31
|
+
{{/if}}
|
|
32
|
+
|
|
33
|
+
## Block references
|
|
34
|
+
|
|
35
|
+
You can reply with a block reference for objects if you have the DXN. For example:
|
|
36
|
+
|
|
37
|
+
<object>
|
|
38
|
+
<dxn>dxn:queue:data:B6INSIBY3CBEF4M5VZRYBCMAHQMPYK5AJ:01K24XMVHSZHS97SG1VTVQDM5Z:01K24XPK464FSCKVQJAB2H662M</dxn>
|
|
39
|
+
</object>
|
|
40
|
+
<object>
|
|
41
|
+
<dxn>dxn:echo:@:01K24XPK464FSCKVQJAB2H662M</dxn>
|
|
42
|
+
</object>
|
|
43
|
+
|
|
44
|
+
### Suggestions
|
|
45
|
+
|
|
46
|
+
Each suggestion must in the form of a proposed action enclosed in a <suggestion> tag.
|
|
47
|
+
Example: <suggestion>Show on a map</suggestion>
|
|
48
|
+
Do not use bullets when listing suggestions.
|
|
49
|
+
|
|
50
|
+
### Multiple choice questions
|
|
51
|
+
|
|
52
|
+
You may ask multiple choice questions by wrapping consise options inside <option> tags inside a well formed <select> tag.
|
|
53
|
+
Example: <select><option>Yes</option><option>No</option></select>
|
|
54
|
+
|
|
55
|
+
## Context
|
|
56
|
+
|
|
57
|
+
The current date and time is {{DATETIME}}.
|