@lobehub/chat 1.80.0 → 1.80.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,712 +1,463 @@
1
- # Complete Guide to LobeChat Feature Development
1
+ # LobeChat Feature Development Complete Guide
2
2
 
3
- This document aims to guide developers on how to develop a complete feature requirement in LobeChat.
3
+ This document aims to guide developers on how to develop a complete feature in LobeChat.
4
4
 
5
- We will use the implementation of sessionGroup as an example: [✨ feat: add session group manager](https://github.com/lobehub/lobe-chat/pull/1055), and explain the complete implementation process through the following six main sections:
5
+ We will use [RFC 021 - Custom Assistant Opening Guidance](https://github.com/lobehub/lobe-chat/discussions/891) as an example to illustrate the complete implementation process.
6
6
 
7
- 1. [Data Model / Database Definition](#1-data-model--database-definition)
8
- 2. [Service Implementation / Model Implementation](#2-service-implementation--model-implementation)
9
- 3. [Frontend Data Flow Store Implementation](#3-frontend-data-flow-store-implementation)
10
- 4. [UI Implementation and Action Binding](#4-ui-implementation-and-action-binding)
11
- 5. [Data Migration](#5-data-migration)
12
- 6. [Data Import and Export](#6-data-import-and-export)
7
+ ## 1. Update Schema
13
8
 
14
- ## 1. Data Model / Database Definition
9
+ lobe-chat uses a postgres database, with the browser-side local database using [pglite](https://pglite.dev/) (wasm version of postgres). The project also uses [drizzle](https://orm.drizzle.team/) ORM to operate the database.
15
10
 
16
- To implementation the Session Group feature, it is necessary to define the relevant data model and indexes at the database level.
11
+ Compared to the old solution where the browser side used indexDB, having both the browser side and server side use postgres has the advantage that the model layer code can be completely reused.
17
12
 
18
- Define a new sessionGroup table in 3 steps:
19
-
20
- ### 1. Establish Data Model Schema
21
-
22
- Define the data model of `DB_SessionGroup` in `src/database/schema/sessionGroup.ts`:
23
-
24
- ```ts
25
- import { z } from 'zod';
26
-
27
- export const DB_SessionGroupSchema = z.object({
28
- name: z.string(),
29
- sort: z.number().optional(),
30
- });
31
-
32
- export type DB_SessionGroup = z.infer<typeof DB_SessionGroupSchema>;
33
- ```
34
-
35
- ### 2. Create Database Indexes
36
-
37
- Since a new table needs to be added, an index needs to be added to the database schema for the `sessionGroup` table.
38
-
39
- Add `dbSchemaV4` in `src/database/core/schema.ts`:
40
-
41
- ```diff
42
- // ... previous implementations
43
-
44
- // ************************************** //
45
- // ******* Version 3 - 2023-12-06 ******* //
46
- // ************************************** //
47
- // - Added `plugin` table
48
-
49
- export const dbSchemaV3 = {
50
- ...dbSchemaV2,
51
- plugins:
52
- '&identifier, type, manifest.type, manifest.meta.title, manifest.meta.description, manifest.meta.author, createdAt, updatedAt',
53
- };
54
-
55
- + // ************************************** //
56
- + // ******* Version 4 - 2024-01-21 ******* //
57
- + // ************************************** //
58
- + // - Added `sessionGroup` table
59
-
60
- + export const dbSchemaV4 = {
61
- + ...dbSchemaV3,
62
- + sessionGroups: '&id, name, sort, createdAt, updatedAt',
63
- + sessions: '&id, type, group, pinned, meta.title, meta.description, meta.tags, createdAt, updatedAt',
64
- };
65
- ```
66
-
67
- > \[!Note]
68
- >
69
- > In addition to `sessionGroups`, the definition of `sessions` has also been modified here due to data migration. However, as this section only focuses on schema definition and does not delve into the implementation of data migration, please refer to section five for details.
70
-
71
- > \[!Important]
72
- >
73
- > If you are unfamiliar with the need to create indexes here and the syntax of schema definition, you may need to familiarize yourself with the basics of Dexie.js.
74
-
75
- ### 3. Add the sessionGroups Table to the Local DB
76
-
77
- Extend the local database class to include the new `sessionGroups` table:
13
+ All schemas are uniformly placed in `src/database/schemas`. We need to adjust the `agents` table to add two fields corresponding to the configuration items:
78
14
 
79
15
  ```diff
16
+ // src/database/schemas/agent.ts
17
+ export const agents = pgTable(
18
+ 'agents',
19
+ {
20
+ id: text('id')
21
+ .primaryKey()
22
+ .$defaultFn(() => idGenerator('agents'))
23
+ .notNull(),
24
+ avatar: text('avatar'),
25
+ backgroundColor: text('background_color'),
26
+ plugins: jsonb('plugins').$type<string[]>().default([]),
27
+ // ...
28
+ tts: jsonb('tts').$type<LobeAgentTTSConfig>(),
29
+
30
+ + openingMessage: text('opening_message'),
31
+ + openingQuestions: text('opening_questions').array().default([]),
32
+
33
+ ...timestamps,
34
+ },
35
+ (t) => ({
36
+ // ...
37
+ // !: update index here
38
+ }),
39
+ );
80
40
 
81
- import { dbSchemaV1, dbSchemaV2, dbSchemaV3, dbSchemaV4 } from './schemas';
82
-
83
- interface LobeDBSchemaMap {
84
- files: DB_File;
85
- messages: DB_Message;
86
- plugins: DB_Plugin;
87
- + sessionGroups: DB_SessionGroup;
88
- sessions: DB_Session;
89
- topics: DB_Topic;
90
- }
91
-
92
- // Define a local DB
93
- export class LocalDB extends Dexie {
94
- public files: LobeDBTable<'files'>;
95
- public sessions: LobeDBTable<'sessions'>;
96
- public messages: LobeDBTable<'messages'>;
97
- public topics: LobeDBTable<'topics'>;
98
- public plugins: LobeDBTable<'plugins'>;
99
- + public sessionGroups: LobeDBTable<'sessionGroups'>;
100
-
101
- constructor() {
102
- super(LOBE_CHAT_LOCAL_DB_NAME);
103
- this.version(1).stores(dbSchemaV1);
104
- this.version(2).stores(dbSchemaV2);
105
- this.version(3).stores(dbSchemaV3);
106
- + this.version(4).stores(dbSchemaV4);
107
-
108
- this.files = this.table('files');
109
- this.sessions = this.table('sessions');
110
- this.messages = this.table('messages');
111
- this.topics = this.table('topics');
112
- this.plugins = this.table('plugins');
113
- + this.sessionGroups = this.table('sessionGroups');
114
- }
115
- }
116
41
  ```
117
42
 
118
- As a result, you can now view the `sessionGroups` table in the `LOBE_CHAT_DB` in `Application` -> `Storage` -> `IndexedDB`.
119
-
120
- ![](https://github.com/lobehub/lobe-chat/assets/28616219/aea50f66-4060-4a32-88c8-b3c672d05be8)
43
+ Note that sometimes we may also need to update the index, but for this feature, we don't have any related performance bottleneck issues, so we don't need to update the index.
121
44
 
122
- ## 2. Service Implementation / Model Implementation
45
+ After adjusting the schema, we need to run `npm run db:generate` to use drizzle-kit's built-in database migration capability to generate the corresponding SQL code for migrating to the latest schema. After execution, four files will be generated:
123
46
 
124
- ### Define Model
47
+ - src/database/migrations/meta/\_journal.json: Saves information about each migration
48
+ - src/database/migrations/0021\_add\_agent\_opening\_settings.sql: SQL commands for this migration
49
+ - src/database/client/migrations.json: SQL commands for this migration used by pglite
50
+ - src/database/migrations/meta/0021\_snapshot.json: The current latest complete database snapshot
125
51
 
126
- When building the LobeChat application, the Model is responsible for interacting with the database. It defines how to read, insert, update, and delete data from the database, as well as defining specific business logic.
52
+ Note that the migration SQL filename generated by the script by default is not semantically clear like `0021_add_agent_opening_settings.sql`. You need to manually rename it and update `src/database/migrations/meta/_journal.json`.
127
53
 
128
- In `src/database/model/sessionGroup.ts`, the `SessionGroupModel` is defined as follows:
54
+ Previously, client-side storage using indexDB made database migration relatively complicated. Now with pglite on the local side, database migration is a simple command, very quick and easy. You can also check if there's any room for optimization in the generated migration SQL and make manual adjustments.
129
55
 
130
- ```ts
131
- import { BaseModel } from '@/database/client/core';
132
- import { DB_SessionGroup, DB_SessionGroupSchema } from '@/database/client/schemas/sessionGroup';
133
- import { nanoid } from '@/utils/uuid';
56
+ ## 2. Update Data Model
134
57
 
135
- class _SessionGroupModel extends BaseModel {
136
- constructor() {
137
- super('sessions', DB_SessionGroupSchema);
138
- }
58
+ Data models used in our project are defined in `src/types`. We don't directly use the types exported from the drizzle schema, such as `export type NewAgent = typeof agents.$inferInsert;`, but instead define corresponding data models based on frontend requirements and data types of the corresponding fields in the db schema definition.
139
59
 
140
- async create(name: string, sort?: number, id = nanoid()) {
141
- return this._add({ name, sort }, id);
142
- }
60
+ Data model definitions are placed in the `src/types` folder. Update the `LobeAgentConfig` type in `src/types/agent/index.ts`:
143
61
 
144
- // ... Implementation of other CRUD methods
145
- }
146
-
147
- export const SessionGroupModel = new _SessionGroupModel();
148
- ```
149
-
150
- ### Service Implementation
151
-
152
- In LobeChat, the Service layer is mainly responsible for communicating with the backend service, encapsulating business logic, and providing data to other layers in the frontend. `SessionService` is a service class specifically handling business logic related to sessions. It encapsulates operations such as creating sessions, querying sessions, and updating sessions.
153
-
154
- To maintain code maintainability and extensibility, we place the logic related to session grouping in the `SessionService`. This helps to keep the business logic of the session domain cohesive. When business requirements increase or change, it becomes easier to modify and extend within this domain.
155
-
156
- `SessionService` implements session group-related request logic by calling methods from `SessionGroupModel`. The following is the implementation of Session Group-related request logic in `sessionService`:
62
+ ```diff
63
+ export interface LobeAgentConfig {
64
+ // ...
65
+ chatConfig: LobeAgentChatConfig;
66
+ /**
67
+ * The language model used by the agent
68
+ * @default gpt-4o-mini
69
+ */
70
+ model: string;
157
71
 
158
- ```ts
159
- class SessionService {
160
- // ... Omitted session business logic
72
+ + /**
73
+ + * Opening message
74
+ + */
75
+ + openingMessage?: string;
76
+ + /**
77
+ + * Opening questions
78
+ + */
79
+ + openingQuestions?: string[];
161
80
 
162
- // ************************************** //
163
- // *********** SessionGroup *********** //
164
- // ************************************** //
81
+ /**
82
+ * Language model parameters
83
+ */
84
+ params: LLMParams;
85
+ /**
86
+ * Enabled plugins
87
+ */
88
+ plugins?: string[];
165
89
 
166
- async createSessionGroup(name: string, sort?: number) {
167
- const item = await SessionGroupModel.create(name, sort);
168
- if (!item) {
169
- throw new Error('session group create Error');
170
- }
90
+ /**
91
+ * Model provider
92
+ */
93
+ provider?: string;
171
94
 
172
- return item.id;
173
- }
95
+ /**
96
+ * System role
97
+ */
98
+ systemRole: string;
174
99
 
175
- // ... Other SessionGroup related implementations
100
+ /**
101
+ * Text-to-speech service
102
+ */
103
+ tts: LobeAgentTTSConfig;
176
104
  }
177
105
  ```
178
106
 
179
- ## 3. Frontend Data Flow Store Implementation
107
+ ## 3. Service Implementation / Model Implementation
180
108
 
181
- In the LobeChat application, the Store module is used to manage the frontend state of the application. The Actions within it are functions that trigger state updates, usually by calling methods in the service layer to perform actual data processing operations and then updating the state in the Store. We use `zustand` as the underlying dependency for the Store module. For a detailed practical introduction to state management, you can refer to [📘 Best Practices for State Management](../state-management/state-management-intro).
109
+ - The `model` layer encapsulates reusable operations on the DB
110
+ - The `service` layer implements application business logic
182
111
 
183
- ### sessionGroup CRUD
112
+ Both have corresponding top-level folders in the `src` directory.
184
113
 
185
- CRUD operations for session groups are the core behaviors for managing session group data. In `src/store/session/slice/sessionGroup`, we will implement the state logic related to session groups, including adding, deleting, updating session groups, and their sorting.
114
+ We need to check if we need to update their implementation. Agent configuration in the frontend is abstracted as session configuration. In `src/services/session/server.ts` we can see a service function `updateSessionConfig`:
186
115
 
187
- The following are the methods of the `SessionGroupAction` interface that need to be implemented in the `action.ts` file:
188
-
189
- ```ts
190
- export interface SessionGroupAction {
191
- // Add session group
192
- addSessionGroup: (name: string) => Promise<string>;
193
- // Remove session group
194
- removeSessionGroup: (id: string) => Promise<void>;
195
- // Update session group ID for a session
196
- updateSessionGroupId: (sessionId: string, groupId: string) => Promise<void>;
197
- // Update session group name
198
- updateSessionGroupName: (id: string, name: string) => Promise<void>;
199
- // Update session group sorting
200
- updateSessionGroupSort: (items: SessionGroupItem[]) => Promise<void>;
116
+ ```typescript
117
+ export class ServerService implements ISessionService {
118
+ // ...
119
+ updateSessionConfig: ISessionService['updateSessionConfig'] = (id, config, signal) => {
120
+ return lambdaClient.session.updateSessionConfig.mutate({ id, value: config }, { signal });
121
+ };
201
122
  }
202
123
  ```
203
124
 
204
- Taking the `addSessionGroup` method as an example, we first call the `createSessionGroup` method of `sessionService` to create a new session group, and then use the `refreshSessions` method to refresh the sessions state:
205
-
206
- ```ts
207
- export const createSessionGroupSlice: StateCreator<
208
- SessionStore,
209
- [['zustand/devtools', never]],
210
- [],
211
- SessionGroupAction
212
- > = (set, get) => ({
213
- // Implement the logic for adding a session group
214
- addSessionGroup: async (name) => {
215
- // Call the createSessionGroup method in the service layer and pass in the session group name
216
- const id = await sessionService.createSessionGroup(name);
217
- // Call the get method to get the current Store state and execute the refreshSessions method to refresh the session data
218
- await get().refreshSessions();
219
- // Return the ID of the newly created session group
220
- return id;
221
- },
222
- // ... Other action implementations
223
- });
224
- ```
225
-
226
- With the above implementation, we can ensure that after adding a new session group, the application's state will be updated in a timely manner, and the relevant components will receive the latest state and re-render. This approach improves the predictability and maintainability of the data flow, while also simplifying communication between components.
227
-
228
- ### Sessions Group Logic Refactoring
229
-
230
- This requirement involves upgrading the Sessions feature to transform it from a single list to three different groups: `pinnedSessions` (pinned list), `customSessionGroups` (custom groups), and `defaultSessions` (default list).
231
-
232
- To handle these groups, we need to refactor the implementation logic of `useFetchSessions`. Here are the key changes:
233
-
234
- 1. Use the `sessionService.getGroupedSessions` method to call the backend API and retrieve the grouped session data.
235
- 2. Save the retrieved data into three different state fields: `pinnedSessions`, `customSessionGroups`, and `defaultSessions`.
236
-
237
- #### `useFetchSessions` Method
238
-
239
- This method is defined in `createSessionSlice` as follows:
240
-
241
- ```ts
242
- export const createSessionSlice: StateCreator<
243
- SessionStore,
244
- [['zustand/devtools', never]],
245
- [],
246
- SessionAction
247
- > = (set, get) => ({
248
- // ... other methods
249
- useFetchSessions: () =>
250
- useSWR<ChatSessionList>(FETCH_SESSIONS_KEY, sessionService.getGroupedSessions, {
251
- onSuccess: (data) => {
252
- set(
253
- {
254
- customSessionGroups: data.customGroup,
255
- defaultSessions: data.default,
256
- isSessionsFirstFetchFinished: true,
257
- pinnedSessions: data.pinned,
258
- sessions: data.all,
259
- },
260
- false,
261
- n('useFetchSessions/onSuccess', data),
262
- );
263
- },
125
+ Jumping to the implementation of `lambdaClient.session.updateSessionConfig`, we find that it simply **merges** the new config with the old config.
126
+
127
+ ```typescript
128
+ export const sessionRouter = router({
129
+ // ..
130
+ updateSessionConfig: sessionProcedure
131
+ .input(
132
+ z.object({
133
+ id: z.string(),
134
+ value: z.object({}).passthrough().partial(),
135
+ }),
136
+ )
137
+ .mutation(async ({ input, ctx }) => {
138
+ const session = await ctx.sessionModel.findByIdOrSlug(input.id);
139
+ // ...
140
+ const mergedValue = merge(session.agent, input.value);
141
+ return ctx.sessionModel.updateConfig(session.agent.id, mergedValue);
264
142
  }),
265
143
  });
266
144
  ```
267
145
 
268
- After successfully retrieving the data, we use the `set` method to update the `customSessionGroups`, `defaultSessions`, `pinnedSessions`, and `sessions` states. This ensures that the states are synchronized with the latest session data.
269
-
270
- #### `sessionService.getGroupedSessions` Method
271
-
272
- The `sessionService.getGroupedSessions` method is responsible for calling the backend API `SessionModel.queryWithGroups()`.
273
-
274
- ```ts
275
- class SessionService {
276
- // ... other SessionGroup related implementations
277
-
278
- async getGroupedSessions(): Promise<ChatSessionList> {
279
- return SessionModel.queryWithGroups();
280
- }
281
- }
282
- ```
283
-
284
- #### `SessionModel.queryWithGroups` Method
146
+ Foreseeably, the frontend will add two inputs, calling updateSessionConfig upon user modification. As the current implementation lacks field-level granularity for the config update, the service and model layers remain unaffected.
285
147
 
286
- This method is the core method called by `sessionService.getGroupedSessions`, and it is responsible for querying and organizing session data. The code is as follows:
148
+ ## 4. Frontend Implementation
287
149
 
288
- ```ts
289
- class _SessionModel extends BaseModel {
290
- // ... other methods
150
+ ### Data Flow Store Implementation
291
151
 
292
- /**
293
- * Query session data and categorize sessions based on groups.
294
- * @returns {Promise<ChatSessionList>} An object containing all sessions and categorized session lists.
295
- */
296
- async queryWithGroups(): Promise<ChatSessionList> {
297
- // Query session group data
298
- const groups = await SessionGroupModel.query();
299
- // Query custom session groups based on session group IDs
300
- const customGroups = await this.queryByGroupIds(groups.map((item) => item.id));
301
- // Query default session list
302
- const defaultItems = await this.querySessionsByGroupId(SessionDefaultGroup.Default);
303
- // Query pinned sessions
304
- const pinnedItems = await this.getPinnedSessions();
305
-
306
- // Query all sessions
307
- const all = await this.query();
308
- // Combine and return all sessions and their group information
309
- return {
310
- all, // Array containing all sessions
311
- customGroup: groups.map((group) => ({ ...group, children: customGroups[group.id] })), // Custom groups
312
- default: defaultItems, // Default session list
313
- pinned: pinnedItems, // Pinned session list
314
- };
315
- }
316
- }
317
- ```
318
-
319
- The `queryWithGroups` method first queries all session groups, then based on the IDs of these groups, it queries custom session groups, as well as default and pinned sessions. Finally, it returns an object containing all sessions and categorized session lists.
152
+ lobe-chat uses [zustand](https://zustand.docs.pmnd.rs/getting-started/introduction) as the global state management framework. For detailed practices on state management, you can refer to [📘 State Management Best Practices](/docs/development/state-management/state-management-intro).
320
153
 
321
- ### Adjusting sessions selectors
154
+ There are two stores related to the agent:
322
155
 
323
- Due to changes in the logic of grouping within sessions, we need to adjust the logic of the `sessions` selectors to ensure they can correctly handle the new data structure.
156
+ - `src/features/AgentSetting/store` serves the local store for agent settings
157
+ - `src/store/agent` is used to get the current session agent's store
324
158
 
325
- Original selectors:
159
+ The latter listens for and updates the current session's agent configuration through the `onConfigChange` in the `AgentSettings` component in `src/features/AgentSetting/AgentSettings.tsx`.
326
160
 
327
- ```ts
328
- // Default group
329
- const defaultSessions = (s: SessionStore): LobeSessions => s.sessions;
161
+ #### Update AgentSetting/store
330
162
 
331
- // Pinned group
332
- const pinnedSessionList = (s: SessionStore) =>
333
- defaultSessions(s).filter((s) => s.group === SessionGroupDefaultKeys.Pinned);
163
+ First, we update the initialState. After reading `src/features/AgentSetting/store/initialState.ts`, we learn that the initial agent configuration is saved in `DEFAULT_AGENT_CONFIG` in `src/const/settings/agent.ts`:
334
164
 
335
- // Unpinned group
336
- const unpinnedSessionList = (s: SessionStore) =>
337
- defaultSessions(s).filter((s) => s.group === SessionGroupDefaultKeys.Default);
338
- ```
339
-
340
- Revised:
341
-
342
- ```ts
343
- const defaultSessions = (s: SessionStore): LobeSessions => s.defaultSessions;
344
- const pinnedSessions = (s: SessionStore): LobeSessions => s.pinnedSessions;
345
- const customSessionGroups = (s: SessionStore): CustomSessionGroup[] => s.customSessionGroups;
165
+ ```diff
166
+ export const DEFAULT_AGENT_CONFIG: LobeAgentConfig = {
167
+ chatConfig: DEFAULT_AGENT_CHAT_CONFIG,
168
+ model: DEFAULT_MODEL,
169
+ + openingQuestions: [],
170
+ params: {
171
+ frequency_penalty: 0,
172
+ presence_penalty: 0,
173
+ temperature: 1,
174
+ top_p: 1,
175
+ },
176
+ plugins: [],
177
+ provider: DEFAULT_PROVIDER,
178
+ systemRole: '',
179
+ tts: DEFAUTT_AGENT_TTS_CONFIG,
180
+ };
346
181
  ```
347
182
 
348
- Since all data retrieval in the UI is implemented using syntax like `useSessionStore(sessionSelectors.defaultSessions)`, we only need to modify the selector implementation of `defaultSessions` to complete the data structure change. The data retrieval code in the UI layer does not need to be changed at all, which can greatly reduce the cost and risk of refactoring.
349
-
350
- > !\[Important]
351
- >
352
- > If you are not familiar with the concept and functionality of selectors, you can refer to the section [📘 Data Storage and Retrieval Module](./State-Management-Selectors.en-US) for relevant information.
183
+ Actually, you don't even need to update this since the `openingQuestions` type is already optional. I'm not updating `openingMessage` here.
353
184
 
354
- ## 4. UI Implementation and Action Binding
185
+ Because we've added two new fields, to facilitate access by components in the `src/features/AgentSetting/AgentOpening` folder and for performance optimization, we add related selectors in `src/features/AgentSetting/store/selectors.ts`:
355
186
 
356
- Bind Store Action in the UI component to implement interactive logic, for example `CreateGroupModal`:
187
+ ```diff
188
+ import { DEFAULT_AGENT_CHAT_CONFIG } from '@/const/settings';
189
+ import { LobeAgentChatConfig } from '@/types/agent';
357
190
 
358
- ```tsx
359
- const CreateGroupModal = () => {
360
- // ... Other logic
191
+ import { Store } from './action';
361
192
 
362
- const [updateSessionGroup, addCustomGroup] = useSessionStore((s) => [
363
- s.updateSessionGroupId,
364
- s.addSessionGroup,
365
- ]);
193
+ const chatConfig = (s: Store): LobeAgentChatConfig =>
194
+ s.config.chatConfig || DEFAULT_AGENT_CHAT_CONFIG;
366
195
 
367
- return (
368
- <Modal
369
- onOk={async () => {
370
- // ... Other logic
371
- const groupId = await addCustomGroup(name);
372
- await updateSessionGroup(sessionId, groupId);
373
- }}
374
- >
375
- {/* ... */}
376
- </Modal>
377
- );
196
+ +export const DEFAULT_OPENING_QUESTIONS: string[] = [];
197
+ export const selectors = {
198
+ chatConfig,
199
+ + openingMessage: (s: Store) => s.config.openingMessage,
200
+ + openingQuestions: (s: Store) => s.config.openingQuestions || DEFAULT_OPENING_QUESTIONS,
378
201
  };
379
202
  ```
380
203
 
381
- ## 5. Data Migration
382
-
383
- In the process of software development, data migration is an inevitable issue, especially when the existing data structure cannot meet the new business requirements. For this iteration of SessionGroup, we need to handle the migration of the `group` field in the `session`, which is a typical data migration case.
384
-
385
- ### Issues with the Old Data Structure
386
-
387
- In the old data structure, the `group` field was used to mark whether the session was "pinned" or belonged to a "default" group. However, when support for multiple session groups is needed, the original data structure becomes inflexible.
388
-
389
- For example:
204
+ We won't add additional actions to update the agent config here, as I've observed that other existing code also directly uses the unified `setChatConfig` in the existing code:
390
205
 
206
+ ```typescript
207
+ export const store: StateCreator<Store, [['zustand/devtools', never]]> = (set, get) => ({
208
+ setAgentConfig: (config) => {
209
+ get().dispatchConfig({ config, type: 'update' });
210
+ },
211
+ });
391
212
  ```
392
- before pin: group = abc
393
- after pin: group = pinned
394
- after unpin: group = default
395
- ```
396
-
397
- From the above example, it can be seen that once a session is unpinned from the "pinned" state, the `group` field cannot be restored to its original `abc` value. This is because we do not have a separate field to maintain the pinned state. Therefore, we have introduced a new field `pinned` to indicate whether the session is pinned, while the `group` field will be used solely to identify the session group.
398
-
399
- ### Migration Strategy
400
213
 
401
- The core logic of this migration is as follows:
214
+ #### Update store/agent
402
215
 
403
- - When the user's `group` field is `pinned`, set their `pinned` field to `true`, and set the group to `default`.
216
+ In the component `src/app/[variants]/(main)/chat/(workspace)/@conversation/features/ChatList/WelcomeChatItem/WelcomeMessage.tsx`, we use this store to get the current agent configuration to render user-customized opening messages and guiding questions.
404
217
 
405
- However, data migration in LobeChat typically involves two parts: **configuration file migration** and **database migration**. Therefore, the above logic will need to be implemented separately in these two areas.
218
+ Since we only need to read two configuration items, we'll simply add two selectors:
406
219
 
407
- #### Configuration File Migration
408
-
409
- For configuration file migration, we recommend performing it before database migration, as configuration file migration is usually easier to test and validate. LobeChat's file migration configuration is located in the `src/migrations/index.ts` file, which defines the various versions of configuration file migration and their corresponding migration scripts.
220
+ Update `src/store/agent/slices/chat/selectors/agent.ts`:
410
221
 
411
222
  ```diff
412
- // Current latest version number
413
- - export const CURRENT_CONFIG_VERSION = 2;
414
- + export const CURRENT_CONFIG_VERSION = 3;
415
-
416
- // Historical version upgrade module
417
- const ConfigMigrations = [
418
- + /**
419
- + * 2024.01.22
420
- + * from `group = pinned` to `pinned:true`
421
- + */
422
- + MigrationV2ToV3,
423
- /**
424
- * 2023.11.27
425
- * Migrate from single key database to dexie-based relational structure
426
- */
427
- MigrationV1ToV2,
428
- /**
429
- * 2023.07.11
430
- * just the first version, Nothing to do
431
- */
432
- MigrationV0ToV1,
433
- ];
223
+ // ...
224
+ +const openingQuestions = (s: AgentStoreState) =>
225
+ + currentAgentConfig(s).openingQuestions || DEFAULT_OPENING_QUESTIONS;
226
+ +const openingMessage = (s: AgentStoreState) => currentAgentConfig(s).openingMessage || '';
227
+
228
+ export const agentSelectors = {
229
+ // ...
230
+ isInboxSession,
231
+ + openingMessage,
232
+ + openingQuestions,
233
+ };
434
234
  ```
435
235
 
436
- The logic for this configuration file migration is defined in `src/migrations/FromV2ToV3/index.ts`, simplified as follows:
437
-
438
- ```ts
439
- export class MigrationV2ToV3 implements Migration {
440
- // Specify the version from which to upgrade
441
- version = 2;
442
-
443
- migrate(data: MigrationData<V2ConfigState>): MigrationData<V3ConfigState> {
444
- const { sessions } = data.state;
445
-
446
- return {
447
- ...data,
448
- state: {
449
- ...data.state,
450
- sessions: sessions.map((s) => this.migrateSession(s)),
451
- },
452
- };
453
- }
454
-
455
- migrateSession = (session: V2Session): V3Session => {
456
- return {
457
- ...session,
458
- group: 'default',
459
- pinned: session.group === 'pinned',
460
- };
461
- };
236
+ ### UI Implementation and Action Binding
237
+
238
+ We're adding a new category of settings this time. In `src/features/AgentSetting`, various UI components for agent settings are defined. This time we're adding a setting type: opening settings. We'll add a folder `AgentOpening` to store opening settings-related components. The project uses:
239
+
240
+ - [ant-design](https://ant.design/) and [lobe-ui](https://github.com/lobehub/lobe-ui): component libraries
241
+ - [antd-style](https://ant-design.github.io/antd-style): css-in-js solution
242
+ - [react-layout-kit](https://github.com/arvinxx/react-layout-kit): responsive layout components
243
+ - [@ant-design/icons](https://ant.design/components/icon-cn) and [lucide](https://lucide.dev/icons/): icon libraries
244
+ - [react-i18next](https://github.com/i18next/react-i18next) and [lobe-i18n](https://github.com/lobehub/lobe-cli-toolbox/tree/master/packages/lobe-i18n): i18n framework and multi-language automatic translation tool
245
+
246
+ lobe-chat is an internationalized project, so newly added text needs to update the default `locale` file: `src/locales/default/setting.ts`.
247
+
248
+ Let's take the subcomponent `OpeningQuestion.tsx` as an example. Component implementation:
249
+
250
+ ```typescript
251
+ // src/features/AgentSetting/AgentOpening/OpeningQuestions.tsx
252
+ 'use client';
253
+
254
+ import { DeleteOutlined, PlusOutlined } from '@ant-design/icons';
255
+ import { SortableList } from '@lobehub/ui';
256
+ import { Button, Empty, Input } from 'antd';
257
+ import { createStyles } from 'antd-style';
258
+ import { memo, useCallback, useMemo, useState } from 'react';
259
+ import { useTranslation } from 'react-i18next';
260
+ import { Flexbox } from 'react-layout-kit';
261
+
262
+ import { useStore } from '../store';
263
+ import { selectors } from '../store/selectors';
264
+
265
+ const useStyles = createStyles(({ css, token }) => ({
266
+ empty: css`
267
+ margin-block: 24px;
268
+ margin-inline: 0;
269
+ `,
270
+ questionItemContainer: css`
271
+ margin-block-end: 8px;
272
+ padding-block: 2px;
273
+ padding-inline: 10px 0;
274
+ background: ${token.colorBgContainer};
275
+ `,
276
+ questionItemContent: css`
277
+ flex: 1;
278
+ `,
279
+ questionsList: css`
280
+ width: 100%;
281
+ margin-block-start: 16px;
282
+ `,
283
+ repeatError: css`
284
+ margin: 0;
285
+ color: ${token.colorErrorText};
286
+ `,
287
+ }));
288
+
289
+ interface QuestionItem {
290
+ content: string;
291
+ id: number;
462
292
  }
463
- ```
464
-
465
- It can be seen that the migration implementation is very simple. However, it is important to ensure the correctness of the migration, so corresponding test cases need to be written in `src/migrations/FromV2ToV3/migrations.test.ts`:
466
-
467
- ```ts
468
- import { MigrationData, VersionController } from '@/migrations/VersionController';
469
-
470
- import { MigrationV1ToV2 } from '../FromV1ToV2';
471
- import inputV1Data from '../FromV1ToV2/fixtures/input-v1-session.json';
472
- import inputV2Data from './fixtures/input-v2-session.json';
473
- import outputV3DataFromV1 from './fixtures/output-v3-from-v1.json';
474
- import outputV3Data from './fixtures/output-v3.json';
475
- import { MigrationV2ToV3 } from './index';
476
-
477
- describe('MigrationV2ToV3', () => {
478
- let migrations;
479
- let versionController: VersionController<any>;
480
-
481
- beforeEach(() => {
482
- migrations = [MigrationV2ToV3];
483
- versionController = new VersionController(migrations, 3);
484
- });
485
293
 
486
- it('should migrate data correctly through multiple versions', () => {
487
- const data: MigrationData = inputV2Data;
488
-
489
- const migratedData = versionController.migrate(data);
294
+ const OpeningQuestions = memo(() => {
295
+ const { t } = useTranslation('setting');
296
+ const { styles } = useStyles();
297
+ const [questionInput, setQuestionInput] = useState('');
298
+
299
+ // Use selector to access corresponding configuration
300
+ const openingQuestions = useStore(selectors.openingQuestions);
301
+ // Use action to update configuration
302
+ const updateConfig = useStore((s) => s.setAgentConfig);
303
+ const setQuestions = useCallback(
304
+ (questions: string[]) => {
305
+ updateConfig({ openingQuestions: questions });
306
+ },
307
+ [updateConfig],
308
+ );
490
309
 
491
- expect(migratedData.version).toEqual(outputV3Data.version);
492
- expect(migratedData.state.sessions).toEqual(outputV3Data.state.sessions);
493
- expect(migratedData.state.topics).toEqual(outputV3Data.state.topics);
494
- expect(migratedData.state.messages).toEqual(outputV3Data.state.messages);
495
- });
310
+ const addQuestion = useCallback(() => {
311
+ if (!questionInput.trim()) return;
312
+
313
+ setQuestions([...openingQuestions, questionInput.trim()]);
314
+ setQuestionInput('');
315
+ }, [openingQuestions, questionInput, setQuestions]);
316
+
317
+ const removeQuestion = useCallback(
318
+ (content: string) => {
319
+ const newQuestions = [...openingQuestions];
320
+ const index = newQuestions.indexOf(content);
321
+ newQuestions.splice(index, 1);
322
+ setQuestions(newQuestions);
323
+ },
324
+ [openingQuestions, setQuestions],
325
+ );
496
326
 
497
- it('should work correct from v1 to v3', () => {
498
- const data: MigrationData = inputV1Data;
327
+ // Handle logic after drag and drop sorting
328
+ const handleSortEnd = useCallback(
329
+ (items: QuestionItem[]) => {
330
+ setQuestions(items.map((item) => item.content));
331
+ },
332
+ [setQuestions],
333
+ );
499
334
 
500
- versionController = new VersionController([MigrationV2ToV3, MigrationV1ToV2], 3);
335
+ const items: QuestionItem[] = useMemo(() => {
336
+ return openingQuestions.map((item, index) => ({
337
+ content: item,
338
+ id: index,
339
+ }));
340
+ }, [openingQuestions]);
501
341
 
502
- const migratedData = versionController.migrate(data);
342
+ const isRepeat = openingQuestions.includes(questionInput.trim());
503
343
 
504
- expect(migratedData.version).toEqual(outputV3DataFromV1.version);
505
- expect(migratedData.state.sessions).toEqual(outputV3DataFromV1.state.sessions);
506
- expect(migratedData.state.topics).toEqual(outputV3DataFromV1.state.topics);
507
- expect(migratedData.state.messages).toEqual(outputV3DataFromV1.state.messages);
508
- });
344
+ return (
345
+ <Flexbox gap={8}>
346
+ <Flexbox gap={4}>
347
+ <Input
348
+ addonAfter={
349
+ <Button
350
+ // don't allow repeat
351
+ disabled={openingQuestions.includes(questionInput.trim())}
352
+ icon={<PlusOutlined />}
353
+ onClick={addQuestion}
354
+ size="small"
355
+ type="text"
356
+ />
357
+ }
358
+ onChange={(e) => setQuestionInput(e.target.value)}
359
+ onPressEnter={addQuestion}
360
+ placeholder={t('settingOpening.openingQuestions.placeholder')}
361
+ value={questionInput}
362
+ />
363
+
364
+ {isRepeat && (
365
+ <p className={styles.repeatError}>{t('settingOpening.openingQuestions.repeat')}</p>
366
+ )}
367
+ </Flexbox>
368
+
369
+ <div className={styles.questionsList}>
370
+ {openingQuestions.length > 0 ? (
371
+ <SortableList
372
+ items={items}
373
+ onChange={handleSortEnd}
374
+ renderItem={(item) => (
375
+ <SortableList.Item className={styles.questionItemContainer} id={item.id}>
376
+ <SortableList.DragHandle />
377
+ <div className={styles.questionItemContent}>{item.content}</div>
378
+ <Button
379
+ icon={<DeleteOutlined />}
380
+ onClick={() => removeQuestion(item.content)}
381
+ type="text"
382
+ />
383
+ </SortableList.Item>
384
+ )}
385
+ />
386
+ ) : (
387
+ <Empty
388
+ className={styles.empty}
389
+ description={t('settingOpening.openingQuestions.empty')}
390
+ />
391
+ )}
392
+ </div>
393
+ </Flexbox>
394
+ );
509
395
  });
510
- ```
511
396
 
512
- ```markdown
397
+ export default OpeningQuestions;
513
398
  ```
514
399
 
515
- Unit tests require the use of `fixtures` to fix the test data. The test cases include verification logic for two parts: 1) the correctness of a single migration (v2 -> v3) and 2) the correctness of a complete migration (v1 -> v3).
400
+ At the same time, we need to display the opening configuration set by the user, which is on the chat page. The corresponding component is in `src/app/[variants]/(main)/chat/(workspace)/@conversation/features/ChatList/WelcomeChatItem/WelcomeMessage.tsx`:
516
401
 
517
- > \[!Important]
518
- >
519
- > The version number in the configuration file may not match the database version number, as database version updates do not always involve changes to the data structure (such as adding tables or fields), while configuration file version updates usually involve data migration.
402
+ ```typescript
403
+ const WelcomeMessage = () => {
404
+ const { t } = useTranslation('chat');
520
405
 
521
- ````
406
+ // Get current opening configuration
407
+ const openingMessage = useAgentStore(agentSelectors.openingMessage);
408
+ const openingQuestions = useAgentStore(agentSelectors.openingQuestions);
522
409
 
523
- #### Database Migration
410
+ const meta = useSessionStore(sessionMetaSelectors.currentAgentMeta, isEqual);
411
+ const { isAgentEditable } = useServerConfigStore(featureFlagsSelectors);
412
+ const activeId = useChatStore((s) => s.activeId);
524
413
 
525
- Database migration needs to be implemented in the `LocalDB` class, which is defined in the `src/database/core/db.ts` file. The migration process involves adding a new `pinned` field for each record in the `sessions` table and resetting the `group` field:
526
-
527
- ```diff
528
- export class LocalDB extends Dexie {
529
- public files: LobeDBTable<'files'>;
530
- public sessions: LobeDBTable<'sessions'>;
531
- public messages: LobeDBTable<'messages'>;
532
- public topics: LobeDBTable<'topics'>;
533
- public plugins: LobeDBTable<'plugins'>;
534
- public sessionGroups: LobeDBTable<'sessionGroups'>;
535
-
536
- constructor() {
537
- super(LOBE_CHAT_LOCAL_DB_NAME);
538
- this.version(1).stores(dbSchemaV1);
539
- this.version(2).stores(dbSchemaV2);
540
- this.version(3).stores(dbSchemaV3);
541
- this.version(4)
542
- .stores(dbSchemaV4)
543
- + .upgrade((trans) => this.upgradeToV4(trans));
544
-
545
- this.files = this.table('files');
546
- this.sessions = this.table('sessions');
547
- this.messages = this.table('messages');
548
- this.topics = this.table('topics');
549
- this.plugins = this.table('plugins');
550
- this.sessionGroups = this.table('sessionGroups');
551
- }
552
-
553
- + /**
554
- + * 2024.01.22
555
- + *
556
- + * DB V3 to V4
557
- + * from `group = pinned` to `pinned:true`
558
- + */
559
- + upgradeToV4 = async (trans: Transaction) => {
560
- + const sessions = trans.table('sessions');
561
- + await sessions.toCollection().modify((session) => {
562
- + // translate boolean to number
563
- + session.pinned = session.group === 'pinned' ? 1 : 0;
564
- session.group = 'default';
565
- });
566
- + };
567
- }
568
- ````
569
-
570
- This is our data migration strategy. When performing the migration, it is essential to ensure the correctness of the migration script and validate the migration results through thorough testing.
571
-
572
- ## 6. Data Import and Export
573
-
574
- In LobeChat, the data import and export feature is designed to ensure that users can migrate their data between different devices. This includes session, topic, message, and settings data. In the implementation of the Session Group feature, we also need to handle data import and export to ensure that the complete exported data can be restored exactly the same on other devices.
575
-
576
- The core implementation of data import and export is in the `ConfigService` in `src/service/config.ts`, with key methods as follows:
577
-
578
- | Method Name | Description |
579
- | --------------------- | -------------------------- |
580
- | `importConfigState` | Import configuration data |
581
- | `exportAgents` | Export all agent data |
582
- | `exportSessions` | Export all session data |
583
- | `exportSingleSession` | Export single session data |
584
- | `exportSingleAgent` | Export single agent data |
585
- | `exportSettings` | Export settings data |
586
- | `exportAll` | Export all data |
587
-
588
- ### Data Export
589
-
590
- In LobeChat, when a user chooses to export data, the current session, topic, message, and settings data are packaged into a JSON file and provided for download. The standard structure of this JSON file is as follows:
591
-
592
- ```json
593
- {
594
- "exportType": "sessions",
595
- "state": {
596
- "sessions": [],
597
- "topics": [],
598
- "messages": []
599
- },
600
- "version": 3
601
- }
602
- ```
603
-
604
- Where:
605
-
606
- - `exportType`: Identifies the type of data being exported, currently including `sessions`, `agent`, `settings`, and `all`.
607
- - `state`: Stores the actual data, with different data types for different `exportType`.
608
- - `version`: Indicates the data version.
609
-
610
- In the implementation of the Session Group feature, we need to add `sessionGroups` data to the `state` field. This way, when users export data, their Session Group data will also be included.
611
-
612
- For example, when exporting sessions, the relevant implementation code modification is as follows:
613
-
614
- ```diff
615
- class ConfigService {
616
- // ... Other code omitted
414
+ const agentSystemRoleMsg = t('agentDefaultMessageWithSystemRole', {
415
+ name: meta.title || t('defaultAgent'),
416
+ systemRole: meta.description,
417
+ });
617
418
 
618
- exportSessions = async () => {
619
- const sessions = await sessionService.getAllSessions();
620
- + const sessionGroups = await sessionService.getSessionGroups();
621
- const messages = await messageService.getAllMessages();
622
- const topics = await topicService.getAllTopics();
419
+ const agentMsg = t(isAgentEditable ? 'agentDefaultMessage' : 'agentDefaultMessageWithoutEdit', {
420
+ name: meta.title || t('defaultAgent'),
421
+ url: `/chat/settings?session=${activeId}`,
422
+ });
623
423
 
624
- - const config = createConfigFile('sessions', { messages, sessions, topics });
625
- + const config = createConfigFile('sessions', { messages, sessionGroups, sessions, topics });
424
+ const message = useMemo(() => {
425
+ // Use user-set message if available
426
+ if (openingMessage) return openingMessage;
427
+ return !!meta.description ? agentSystemRoleMsg : agentMsg;
428
+ }, [openingMessage, agentSystemRoleMsg, agentMsg, meta.description]);
429
+
430
+ const chatItem = (
431
+ <ChatItem
432
+ avatar={meta}
433
+ editing={false}
434
+ message={message}
435
+ placement={'left'}
436
+ type={type === 'chat' ? 'block' : 'pure'}
437
+ />
438
+ );
626
439
 
627
- exportConfigFile(config, 'sessions');
628
- };
629
- }
440
+ return openingQuestions.length > 0 ? (
441
+ <Flexbox>
442
+ {chatItem}
443
+ {/* Render guiding questions */}
444
+ <OpeningQuestions mobile={mobile} questions={openingQuestions} />
445
+ </Flexbox>
446
+ ) : (
447
+ chatItem
448
+ );
449
+ };
450
+ export default WelcomeMessage;
630
451
  ```
631
452
 
632
- ### Data Import
633
-
634
- The data import functionality is implemented through `ConfigService.importConfigState`. When users choose to import data, they need to provide a JSON file that conforms to the above structure specification. The `importConfigState` method accepts the data of the configuration file and imports it into the application.
635
-
636
- In the implementation of the Session Group feature, we need to handle the `sessionGroups` data during the data import process. This way, when users import data, their Session Group data will also be imported correctly.
637
-
638
- The following is the modified code for the import implementation in `importConfigState`:
453
+ ## 5. Testing
639
454
 
640
- ```diff
641
- class ConfigService {
642
- // ... Other code omitted
643
-
644
- + importSessionGroups = async (sessionGroups: SessionGroupItem[]) => {
645
- + return sessionService.batchCreateSessionGroups(sessionGroups);
646
- + };
647
-
648
- importConfigState = async (config: ConfigFile): Promise<ImportResults | undefined> => {
649
- switch (config.exportType) {
650
- case 'settings': {
651
- await this.importSettings(config.state.settings);
652
-
653
- break;
654
- }
655
-
656
- case 'agents': {
657
- + const sessionGroups = await this.importSessionGroups(config.state.sessionGroups);
658
-
659
- const data = await this.importSessions(config.state.sessions);
660
- return {
661
- + sessionGroups: this.mapImportResult(sessionGroups),
662
- sessions: this.mapImportResult(data),
663
- };
664
- }
665
-
666
- case 'all': {
667
- await this.importSettings(config.state.settings);
668
-
669
- + const sessionGroups = await this.importSessionGroups(config.state.sessionGroups);
670
-
671
- const [sessions, messages, topics] = await Promise.all([
672
- this.importSessions(config.state.sessions),
673
- this.importMessages(config.state.messages),
674
- this.importTopics(config.state.topics),
675
- ]);
676
-
677
- return {
678
- messages: this.mapImportResult(messages),
679
- + sessionGroups: this.mapImportResult(sessionGroups),
680
- sessions: this.mapImportResult(sessions),
681
- topics: this.mapImportResult(topics),
682
- };
683
- }
684
-
685
- case 'sessions': {
686
- + const sessionGroups = await this.importSessionGroups(config.state.sessionGroups);
687
-
688
- const [sessions, messages, topics] = await Promise.all([
689
- this.importSessions(config.state.sessions),
690
- this.importMessages(config.state.messages),
691
- this.importTopics(config.state.topics),
692
- ]);
693
-
694
- return {
695
- messages: this.mapImportResult(messages),
696
- + sessionGroups: this.mapImportResult(sessionGroups),
697
- sessions: this.mapImportResult(sessions),
698
- topics: this.mapImportResult(topics),
699
- };
700
- }
701
- }
702
- };
703
- }
704
- ```
455
+ The project uses vitest for unit testing.
705
456
 
706
- One key point of the above modification is to import sessionGroup first, because if sessions are imported first and the corresponding SessionGroup Id is not found in the current database, the group of this session will default to be modified to the default value. This will prevent the correct association of the sessionGroup's ID with the session.
457
+ Since our two new configuration fields are both optional, theoretically you could pass the tests without updating them. However, since we added the `openingQuestions` field to the `DEFAULT_AGENT_CONFIG` mentioned earlier, this causes many tests to calculate configurations that include this field, so we still need to update some test snapshots.
707
458
 
708
- This is the implementation of the LobeChat Session Group feature in the data import and export process. This approach ensures that users' Session Group data is correctly handled during the import and export process.
459
+ For the current scenario, I recommend running the tests locally to see which tests fail, and then update them as needed. For example, for the test file `src/store/agent/slices/chat/selectors/agent.test.ts`, you need to run `npx vitest -u src/store/agent/slices/chat/selectors/agent.test.ts` to update the snapshot.
709
460
 
710
461
  ## Summary
711
462
 
712
- The above is the complete implementation process of the LobeChat Session Group feature. Developers can refer to this document for the development and testing of related functionalities.
463
+ The above is the complete implementation process for the LobeChat opening settings feature. Developers can refer to this document for the development and testing of related features.