@aigne/doc-smith 0.8.12-beta.3 → 0.8.12-beta.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.aigne/doc-smith/config.yaml +9 -6
- package/.aigne/doc-smith/output/structure-plan.json +123 -109
- package/.aigne/doc-smith/upload-cache.yaml +48 -0
- package/.github/workflows/publish-docs.yml +4 -7
- package/.release-please-manifest.json +1 -1
- package/CHANGELOG.md +20 -0
- package/agents/clear/choose-contents.mjs +22 -5
- package/agents/clear/clear-auth-tokens.mjs +2 -4
- package/agents/clear/clear-deployment-config.mjs +49 -0
- package/agents/clear/clear-document-config.mjs +2 -5
- package/agents/clear/clear-document-structure.mjs +2 -6
- package/agents/clear/clear-generated-docs.mjs +0 -1
- package/agents/generate/check-need-generate-structure.mjs +15 -60
- package/agents/generate/document-structure-tools/generate-sub-structure.mjs +131 -0
- package/agents/generate/generate-structure-without-tools.yaml +65 -0
- package/agents/generate/generate-structure.yaml +7 -1
- package/agents/generate/index.yaml +0 -3
- package/agents/generate/update-document-structure.yaml +3 -0
- package/agents/generate/user-review-document-structure.mjs +7 -5
- package/agents/init/index.mjs +15 -15
- package/agents/publish/publish-docs.mjs +130 -111
- package/agents/translate/index.yaml +1 -1
- package/agents/update/batch-generate-document.yaml +1 -1
- package/agents/update/batch-update-document.yaml +1 -1
- package/agents/update/check-document.mjs +4 -19
- package/agents/update/user-review-document.mjs +4 -3
- package/agents/utils/load-sources.mjs +54 -151
- package/agents/utils/transform-detail-datasources.mjs +14 -18
- package/aigne.yaml +2 -0
- package/biome.json +1 -1
- package/docs/_sidebar.md +13 -15
- package/docs/configuration-initial-setup.ja.md +179 -0
- package/docs/configuration-initial-setup.md +179 -0
- package/docs/configuration-initial-setup.zh-TW.md +179 -0
- package/docs/configuration-initial-setup.zh.md +179 -0
- package/docs/configuration-managing-preferences.ja.md +100 -0
- package/docs/configuration-managing-preferences.md +100 -0
- package/docs/configuration-managing-preferences.zh-TW.md +100 -0
- package/docs/configuration-managing-preferences.zh.md +100 -0
- package/docs/configuration.ja.md +68 -184
- package/docs/configuration.md +62 -178
- package/docs/configuration.zh-TW.md +70 -186
- package/docs/configuration.zh.md +67 -183
- package/docs/getting-started.ja.md +46 -78
- package/docs/getting-started.md +46 -78
- package/docs/getting-started.zh-TW.md +47 -79
- package/docs/getting-started.zh.md +47 -79
- package/docs/guides-cleaning-up.ja.md +50 -0
- package/docs/guides-cleaning-up.md +50 -0
- package/docs/guides-cleaning-up.zh-TW.md +50 -0
- package/docs/guides-cleaning-up.zh.md +50 -0
- package/docs/guides-evaluating-documents.ja.md +66 -0
- package/docs/guides-evaluating-documents.md +66 -0
- package/docs/guides-evaluating-documents.zh-TW.md +66 -0
- package/docs/guides-evaluating-documents.zh.md +66 -0
- package/docs/guides-generating-documentation.ja.md +149 -0
- package/docs/guides-generating-documentation.md +149 -0
- package/docs/guides-generating-documentation.zh-TW.md +149 -0
- package/docs/guides-generating-documentation.zh.md +149 -0
- package/docs/guides-interactive-chat.ja.md +85 -0
- package/docs/guides-interactive-chat.md +85 -0
- package/docs/guides-interactive-chat.zh-TW.md +85 -0
- package/docs/guides-interactive-chat.zh.md +85 -0
- package/docs/guides-managing-history.ja.md +51 -0
- package/docs/guides-managing-history.md +51 -0
- package/docs/guides-managing-history.zh-TW.md +51 -0
- package/docs/guides-managing-history.zh.md +51 -0
- package/docs/guides-publishing-your-docs.ja.md +78 -0
- package/docs/guides-publishing-your-docs.md +78 -0
- package/docs/guides-publishing-your-docs.zh-TW.md +78 -0
- package/docs/guides-publishing-your-docs.zh.md +78 -0
- package/docs/guides-translating-documentation.ja.md +95 -0
- package/docs/guides-translating-documentation.md +95 -0
- package/docs/guides-translating-documentation.zh-TW.md +95 -0
- package/docs/guides-translating-documentation.zh.md +95 -0
- package/docs/guides-updating-documentation.ja.md +77 -0
- package/docs/guides-updating-documentation.md +77 -0
- package/docs/guides-updating-documentation.zh-TW.md +77 -0
- package/docs/guides-updating-documentation.zh.md +77 -0
- package/docs/guides.ja.md +32 -0
- package/docs/guides.md +32 -0
- package/docs/guides.zh-TW.md +32 -0
- package/docs/guides.zh.md +32 -0
- package/docs/overview.ja.md +39 -60
- package/docs/overview.md +39 -60
- package/docs/overview.zh-TW.md +39 -60
- package/docs/overview.zh.md +39 -60
- package/docs/release-notes.ja.md +255 -0
- package/docs/release-notes.md +255 -0
- package/docs/release-notes.zh-TW.md +255 -0
- package/docs/release-notes.zh.md +255 -0
- package/package.json +4 -2
- package/prompts/common/document/content-rules-core.md +1 -0
- package/prompts/common/document-structure/document-structure-rules.md +8 -9
- package/prompts/common/document-structure/output-constraints.md +1 -1
- package/prompts/structure/document-rules.md +8 -2
- package/prompts/structure/generate/system-prompt.md +27 -2
- package/prompts/structure/generate/user-prompt.md +18 -0
- package/prompts/structure/update/system-prompt.md +12 -0
- package/prompts/structure/update/user-prompt.md +3 -0
- package/tests/agents/clear/choose-contents.test.mjs +8 -4
- package/tests/agents/generate/check-need-generate-structure.test.mjs +53 -63
- package/tests/agents/generate/document-structure-tools/generate-sub-structure.test.mjs +277 -0
- package/tests/agents/init/init.test.mjs +18 -18
- package/tests/agents/publish/publish-docs.test.mjs +79 -0
- package/tests/agents/update/check-document.test.mjs +7 -67
- package/tests/agents/utils/load-sources.test.mjs +90 -90
- package/tests/agents/utils/transform-detail-datasources.test.mjs +153 -196
- package/tests/utils/file-utils.test.mjs +309 -1
- package/utils/auth-utils.mjs +9 -3
- package/utils/constants/index.mjs +3 -1
- package/utils/file-utils.mjs +315 -0
- package/utils/utils.mjs +89 -50
- package/docs/advanced-how-it-works.ja.md +0 -101
- package/docs/advanced-how-it-works.md +0 -101
- package/docs/advanced-how-it-works.zh-TW.md +0 -101
- package/docs/advanced-how-it-works.zh.md +0 -101
- package/docs/advanced-quality-assurance.ja.md +0 -92
- package/docs/advanced-quality-assurance.md +0 -92
- package/docs/advanced-quality-assurance.zh-TW.md +0 -92
- package/docs/advanced-quality-assurance.zh.md +0 -92
- package/docs/advanced.ja.md +0 -20
- package/docs/advanced.md +0 -20
- package/docs/advanced.zh-TW.md +0 -20
- package/docs/advanced.zh.md +0 -20
- package/docs/changelog.ja.md +0 -486
- package/docs/changelog.md +0 -486
- package/docs/changelog.zh-TW.md +0 -486
- package/docs/changelog.zh.md +0 -486
- package/docs/cli-reference.ja.md +0 -311
- package/docs/cli-reference.md +0 -311
- package/docs/cli-reference.zh-TW.md +0 -311
- package/docs/cli-reference.zh.md +0 -311
- package/docs/configuration-interactive-setup.ja.md +0 -138
- package/docs/configuration-interactive-setup.md +0 -138
- package/docs/configuration-interactive-setup.zh-TW.md +0 -138
- package/docs/configuration-interactive-setup.zh.md +0 -138
- package/docs/configuration-language-support.ja.md +0 -64
- package/docs/configuration-language-support.md +0 -64
- package/docs/configuration-language-support.zh-TW.md +0 -64
- package/docs/configuration-language-support.zh.md +0 -64
- package/docs/configuration-llm-setup.ja.md +0 -56
- package/docs/configuration-llm-setup.md +0 -56
- package/docs/configuration-llm-setup.zh-TW.md +0 -56
- package/docs/configuration-llm-setup.zh.md +0 -56
- package/docs/configuration-preferences.ja.md +0 -144
- package/docs/configuration-preferences.md +0 -144
- package/docs/configuration-preferences.zh-TW.md +0 -144
- package/docs/configuration-preferences.zh.md +0 -144
- package/docs/features-generate-documentation.ja.md +0 -95
- package/docs/features-generate-documentation.md +0 -95
- package/docs/features-generate-documentation.zh-TW.md +0 -95
- package/docs/features-generate-documentation.zh.md +0 -95
- package/docs/features-publish-your-docs.ja.md +0 -130
- package/docs/features-publish-your-docs.md +0 -130
- package/docs/features-publish-your-docs.zh-TW.md +0 -130
- package/docs/features-publish-your-docs.zh.md +0 -130
- package/docs/features-translate-documentation.ja.md +0 -90
- package/docs/features-translate-documentation.md +0 -90
- package/docs/features-translate-documentation.zh-TW.md +0 -90
- package/docs/features-translate-documentation.zh.md +0 -90
- package/docs/features-update-and-refine.ja.md +0 -142
- package/docs/features-update-and-refine.md +0 -142
- package/docs/features-update-and-refine.zh-TW.md +0 -143
- package/docs/features-update-and-refine.zh.md +0 -142
- package/docs/features.ja.md +0 -62
- package/docs/features.md +0 -62
- package/docs/features.zh-TW.md +0 -62
- package/docs/features.zh.md +0 -62
package/utils/file-utils.mjs
CHANGED
|
@@ -2,6 +2,10 @@ import { execSync } from "node:child_process";
|
|
|
2
2
|
import { access, readFile, stat } from "node:fs/promises";
|
|
3
3
|
import path from "node:path";
|
|
4
4
|
import { glob } from "glob";
|
|
5
|
+
import { isBinaryFile } from "isbinaryfile";
|
|
6
|
+
import { encode } from "gpt-tokenizer";
|
|
7
|
+
import { isGlobPattern } from "./utils.mjs";
|
|
8
|
+
import { INTELLIGENT_SUGGESTION_TOKEN_THRESHOLD } from "./constants/index.mjs";
|
|
5
9
|
|
|
6
10
|
/**
|
|
7
11
|
* Check if a directory is inside a git repository using git command
|
|
@@ -243,3 +247,314 @@ export function resolveToAbsolute(value) {
|
|
|
243
247
|
if (!value) return undefined;
|
|
244
248
|
return path.isAbsolute(value) ? value : path.resolve(process.cwd(), value);
|
|
245
249
|
}
|
|
250
|
+
|
|
251
|
+
/**
|
|
252
|
+
* Load files from sourcesPath array
|
|
253
|
+
* Supports file paths, directory paths, and glob patterns
|
|
254
|
+
* @param {string|string[]} sourcesPath - Single path or array of paths
|
|
255
|
+
* @param {object} options - Options for file loading
|
|
256
|
+
* @param {string|string[]} options.includePatterns - Include patterns for directories
|
|
257
|
+
* @param {string|string[]} options.excludePatterns - Exclude patterns for directories
|
|
258
|
+
* @param {boolean} options.useDefaultPatterns - Whether to use default patterns (default: true)
|
|
259
|
+
* @param {string[]} options.defaultIncludePatterns - Default include patterns
|
|
260
|
+
* @param {string[]} options.defaultExcludePatterns - Default exclude patterns
|
|
261
|
+
* @returns {Promise<string[]>} Array of absolute file paths
|
|
262
|
+
*/
|
|
263
|
+
export async function loadFilesFromPaths(sourcesPath, options = {}) {
|
|
264
|
+
const {
|
|
265
|
+
includePatterns,
|
|
266
|
+
excludePatterns,
|
|
267
|
+
useDefaultPatterns = true,
|
|
268
|
+
defaultIncludePatterns = [],
|
|
269
|
+
defaultExcludePatterns = [],
|
|
270
|
+
} = options;
|
|
271
|
+
|
|
272
|
+
const paths = Array.isArray(sourcesPath) ? sourcesPath : [sourcesPath];
|
|
273
|
+
let allFiles = [];
|
|
274
|
+
|
|
275
|
+
for (const dir of paths) {
|
|
276
|
+
try {
|
|
277
|
+
if (typeof dir !== "string") {
|
|
278
|
+
console.warn(`Invalid source path: ${dir}`);
|
|
279
|
+
continue;
|
|
280
|
+
}
|
|
281
|
+
|
|
282
|
+
// First try to access as a file or directory
|
|
283
|
+
const stats = await stat(dir);
|
|
284
|
+
|
|
285
|
+
if (stats.isFile()) {
|
|
286
|
+
// If it's a file, add it directly without filtering
|
|
287
|
+
allFiles.push(dir);
|
|
288
|
+
} else if (stats.isDirectory()) {
|
|
289
|
+
// If it's a directory, use the existing glob logic
|
|
290
|
+
// Load .gitignore for this directory
|
|
291
|
+
const gitignorePatterns = await loadGitignore(dir);
|
|
292
|
+
|
|
293
|
+
// Prepare patterns
|
|
294
|
+
let finalIncludePatterns = null;
|
|
295
|
+
let finalExcludePatterns = null;
|
|
296
|
+
|
|
297
|
+
if (useDefaultPatterns) {
|
|
298
|
+
// Merge with default patterns
|
|
299
|
+
const userInclude = includePatterns
|
|
300
|
+
? Array.isArray(includePatterns)
|
|
301
|
+
? includePatterns
|
|
302
|
+
: [includePatterns]
|
|
303
|
+
: [];
|
|
304
|
+
const userExclude = excludePatterns
|
|
305
|
+
? Array.isArray(excludePatterns)
|
|
306
|
+
? excludePatterns
|
|
307
|
+
: [excludePatterns]
|
|
308
|
+
: [];
|
|
309
|
+
|
|
310
|
+
finalIncludePatterns = [...defaultIncludePatterns, ...userInclude];
|
|
311
|
+
finalExcludePatterns = [...defaultExcludePatterns, ...userExclude];
|
|
312
|
+
} else {
|
|
313
|
+
// Use only user patterns
|
|
314
|
+
if (includePatterns) {
|
|
315
|
+
finalIncludePatterns = Array.isArray(includePatterns)
|
|
316
|
+
? includePatterns
|
|
317
|
+
: [includePatterns];
|
|
318
|
+
}
|
|
319
|
+
if (excludePatterns) {
|
|
320
|
+
finalExcludePatterns = Array.isArray(excludePatterns)
|
|
321
|
+
? excludePatterns
|
|
322
|
+
: [excludePatterns];
|
|
323
|
+
}
|
|
324
|
+
}
|
|
325
|
+
|
|
326
|
+
// Get files using glob
|
|
327
|
+
const filesInDir = await getFilesWithGlob(
|
|
328
|
+
dir,
|
|
329
|
+
finalIncludePatterns,
|
|
330
|
+
finalExcludePatterns,
|
|
331
|
+
gitignorePatterns,
|
|
332
|
+
);
|
|
333
|
+
allFiles = allFiles.concat(filesInDir);
|
|
334
|
+
}
|
|
335
|
+
} catch (err) {
|
|
336
|
+
if (err.code === "ENOENT") {
|
|
337
|
+
// Path doesn't exist as file or directory, try as glob pattern
|
|
338
|
+
try {
|
|
339
|
+
// Check if it looks like a glob pattern
|
|
340
|
+
const isGlobPatternResult = isGlobPattern(dir);
|
|
341
|
+
|
|
342
|
+
if (isGlobPatternResult) {
|
|
343
|
+
// Use glob to find matching files from current working directory
|
|
344
|
+
const matchedFiles = await glob(dir, {
|
|
345
|
+
absolute: true,
|
|
346
|
+
nodir: true, // Only files, not directories
|
|
347
|
+
dot: false, // Don't include hidden files
|
|
348
|
+
});
|
|
349
|
+
|
|
350
|
+
if (matchedFiles.length > 0) {
|
|
351
|
+
allFiles = allFiles.concat(matchedFiles);
|
|
352
|
+
}
|
|
353
|
+
}
|
|
354
|
+
} catch (globErr) {
|
|
355
|
+
console.warn(`Failed to process glob pattern "${dir}": ${globErr.message}`);
|
|
356
|
+
}
|
|
357
|
+
} else {
|
|
358
|
+
throw err;
|
|
359
|
+
}
|
|
360
|
+
}
|
|
361
|
+
}
|
|
362
|
+
|
|
363
|
+
return allFiles;
|
|
364
|
+
}
|
|
365
|
+
|
|
366
|
+
/**
|
|
367
|
+
* Check if a file is likely a text file by checking if it's binary
|
|
368
|
+
* @param {string} filePath - File path to check
|
|
369
|
+
* @returns {Promise<boolean>} True if file appears to be a text file
|
|
370
|
+
*/
|
|
371
|
+
async function isTextFile(filePath) {
|
|
372
|
+
try {
|
|
373
|
+
const isBinary = await isBinaryFile(filePath);
|
|
374
|
+
return !isBinary;
|
|
375
|
+
} catch (_error) {
|
|
376
|
+
// If we can't read the file, assume it might be binary to be safe
|
|
377
|
+
return false;
|
|
378
|
+
}
|
|
379
|
+
}
|
|
380
|
+
|
|
381
|
+
/**
|
|
382
|
+
* Read and parse file contents from an array of file paths
|
|
383
|
+
* @param {string[]} files - Array of file paths to read
|
|
384
|
+
* @param {string} baseDir - Base directory for calculating relative paths (defaults to cwd)
|
|
385
|
+
* @param {object} options - Options for reading files
|
|
386
|
+
* @param {boolean} options.skipBinaryFiles - Whether to skip binary files (default: true)
|
|
387
|
+
* @returns {Promise<{sourceId: string, content: string}[]>} Array of file objects with sourceId and content
|
|
388
|
+
*/
|
|
389
|
+
export async function readFileContents(files, baseDir = process.cwd(), options = {}) {
|
|
390
|
+
const { skipBinaryFiles = true } = options;
|
|
391
|
+
|
|
392
|
+
const results = await Promise.all(
|
|
393
|
+
files.map(async (file) => {
|
|
394
|
+
// Skip binary files if enabled
|
|
395
|
+
if (skipBinaryFiles) {
|
|
396
|
+
const isText = await isTextFile(file);
|
|
397
|
+
if (!isText) {
|
|
398
|
+
return null;
|
|
399
|
+
}
|
|
400
|
+
}
|
|
401
|
+
|
|
402
|
+
try {
|
|
403
|
+
const content = await readFile(file, "utf8");
|
|
404
|
+
const relativePath = path.relative(baseDir, file);
|
|
405
|
+
return {
|
|
406
|
+
sourceId: relativePath,
|
|
407
|
+
content,
|
|
408
|
+
};
|
|
409
|
+
} catch (error) {
|
|
410
|
+
// If reading as text fails (e.g., binary file), skip it
|
|
411
|
+
console.warn(`Failed to read file as text: ${file} - ${error.message}`);
|
|
412
|
+
return null;
|
|
413
|
+
}
|
|
414
|
+
}),
|
|
415
|
+
);
|
|
416
|
+
|
|
417
|
+
// Filter out null results
|
|
418
|
+
return results.filter((result) => result !== null);
|
|
419
|
+
}
|
|
420
|
+
|
|
421
|
+
/**
|
|
422
|
+
* Calculate total lines and tokens from file contents
|
|
423
|
+
* @param {Array<{content: string}>} sourceFiles - Array of objects containing content property
|
|
424
|
+
* @returns {{totalTokens: number, totalLines: number}} Object with totalTokens and totalLines
|
|
425
|
+
*/
|
|
426
|
+
export function calculateFileStats(sourceFiles) {
|
|
427
|
+
let totalTokens = 0;
|
|
428
|
+
let totalLines = 0;
|
|
429
|
+
|
|
430
|
+
for (const source of sourceFiles) {
|
|
431
|
+
const { content } = source;
|
|
432
|
+
if (content) {
|
|
433
|
+
// Count tokens using gpt-tokenizer
|
|
434
|
+
const tokens = encode(content);
|
|
435
|
+
totalTokens += tokens.length;
|
|
436
|
+
|
|
437
|
+
// Count lines (excluding empty lines)
|
|
438
|
+
totalLines += content.split("\n").filter((line) => line.trim() !== "").length;
|
|
439
|
+
}
|
|
440
|
+
}
|
|
441
|
+
|
|
442
|
+
return { totalTokens, totalLines };
|
|
443
|
+
}
|
|
444
|
+
|
|
445
|
+
/**
|
|
446
|
+
* Build sources content string based on context size
|
|
447
|
+
* For large contexts, only include core project files to avoid token limit issues
|
|
448
|
+
* @param {Array<{sourceId: string, content: string}>} sourceFiles - Array of source file objects
|
|
449
|
+
* @param {boolean} isLargeContext - Whether the context is large
|
|
450
|
+
* @returns {string} Concatenated sources content with sourceId comments
|
|
451
|
+
*/
|
|
452
|
+
export function buildSourcesContent(sourceFiles, isLargeContext = false) {
|
|
453
|
+
// Define core file patterns that represent project structure and key information
|
|
454
|
+
const coreFilePatterns = [
|
|
455
|
+
// Configuration files
|
|
456
|
+
/package\.json$/,
|
|
457
|
+
/tsconfig\.json$/,
|
|
458
|
+
/jsconfig\.json$/,
|
|
459
|
+
/\.env\.example$/,
|
|
460
|
+
/Cargo\.toml$/,
|
|
461
|
+
/go\.mod$/,
|
|
462
|
+
/pom\.xml$/,
|
|
463
|
+
/build\.gradle$/,
|
|
464
|
+
/Gemfile$/,
|
|
465
|
+
/requirements\.txt$/,
|
|
466
|
+
/Pipfile$/,
|
|
467
|
+
/composer\.json$/,
|
|
468
|
+
/pyproject\.toml$/,
|
|
469
|
+
|
|
470
|
+
// Documentation
|
|
471
|
+
/README\.md$/i,
|
|
472
|
+
/CHANGELOG\.md$/i,
|
|
473
|
+
/CONTRIBUTING\.md$/i,
|
|
474
|
+
/\.github\/.*\.md$/i,
|
|
475
|
+
|
|
476
|
+
// Entry points and main files
|
|
477
|
+
/index\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
478
|
+
/main\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
479
|
+
/app\.(js|ts|jsx|tsx|py)$/,
|
|
480
|
+
/server\.(js|ts|jsx|tsx|py)$/,
|
|
481
|
+
|
|
482
|
+
// API definitions
|
|
483
|
+
/api\/.*\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
484
|
+
/routes\/.*\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
485
|
+
/controllers\/.*\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
486
|
+
|
|
487
|
+
// Type definitions and schemas
|
|
488
|
+
/types\.(ts|d\.ts)$/,
|
|
489
|
+
/schema\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
490
|
+
/.*\.d\.ts$/,
|
|
491
|
+
|
|
492
|
+
// Core utilities
|
|
493
|
+
/utils\/.*\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
494
|
+
/lib\/.*\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
495
|
+
/helpers\/.*\.(js|ts|jsx|tsx|py|go|rs|java|rb|php)$/,
|
|
496
|
+
];
|
|
497
|
+
|
|
498
|
+
// Function to check if a file is a core file
|
|
499
|
+
const isCoreFile = (filePath) => {
|
|
500
|
+
return coreFilePatterns.some((pattern) => pattern.test(filePath));
|
|
501
|
+
};
|
|
502
|
+
|
|
503
|
+
// Build sources string
|
|
504
|
+
let allSources = "";
|
|
505
|
+
|
|
506
|
+
if (isLargeContext) {
|
|
507
|
+
// Only include core files for large contexts
|
|
508
|
+
const coreFiles = sourceFiles.filter((source) => isCoreFile(source.sourceId));
|
|
509
|
+
|
|
510
|
+
// Determine which files to use and set appropriate message
|
|
511
|
+
const filesToInclude = coreFiles.length > 0 ? coreFiles : sourceFiles;
|
|
512
|
+
const noteMessage =
|
|
513
|
+
coreFiles.length > 0
|
|
514
|
+
? "// Note: Context is large, showing only core project files.\n"
|
|
515
|
+
: "// Note: Context is large, showing a sample of files.\n";
|
|
516
|
+
|
|
517
|
+
allSources += noteMessage;
|
|
518
|
+
let accumulatedTokens = 0;
|
|
519
|
+
|
|
520
|
+
for (const source of filesToInclude) {
|
|
521
|
+
const fileContent = `// sourceId: ${source.sourceId}\n${source.content}\n`;
|
|
522
|
+
const fileTokens = encode(fileContent);
|
|
523
|
+
|
|
524
|
+
// Check if adding this file would exceed the token limit
|
|
525
|
+
if (accumulatedTokens + fileTokens.length > INTELLIGENT_SUGGESTION_TOKEN_THRESHOLD) {
|
|
526
|
+
break;
|
|
527
|
+
}
|
|
528
|
+
|
|
529
|
+
allSources += fileContent;
|
|
530
|
+
accumulatedTokens += fileTokens.length;
|
|
531
|
+
}
|
|
532
|
+
} else {
|
|
533
|
+
// Include all files for normal contexts
|
|
534
|
+
for (const source of sourceFiles) {
|
|
535
|
+
allSources += `// sourceId: ${source.sourceId}\n${source.content}\n`;
|
|
536
|
+
}
|
|
537
|
+
}
|
|
538
|
+
|
|
539
|
+
return allSources;
|
|
540
|
+
}
|
|
541
|
+
|
|
542
|
+
/**
|
|
543
|
+
* Get doc-smith configuration file path
|
|
544
|
+
* @param {string} workDir - Working directory (defaults to current directory)
|
|
545
|
+
* @returns {string} Absolute path to config.yaml
|
|
546
|
+
*/
|
|
547
|
+
export function getConfigFilePath(workDir) {
|
|
548
|
+
const cwd = workDir || process.cwd();
|
|
549
|
+
return path.join(cwd, ".aigne", "doc-smith", "config.yaml");
|
|
550
|
+
}
|
|
551
|
+
|
|
552
|
+
/**
|
|
553
|
+
* Get doc-smith structure plan file path
|
|
554
|
+
* @param {string} workDir - Working directory (defaults to current directory)
|
|
555
|
+
* @returns {string} Absolute path to structure-plan.json
|
|
556
|
+
*/
|
|
557
|
+
export function getStructurePlanPath(workDir) {
|
|
558
|
+
const cwd = workDir || process.cwd();
|
|
559
|
+
return path.join(cwd, ".aigne", "doc-smith", "output", "structure-plan.json");
|
|
560
|
+
}
|
package/utils/utils.mjs
CHANGED
|
@@ -873,6 +873,81 @@ export async function getProjectInfo() {
|
|
|
873
873
|
};
|
|
874
874
|
}
|
|
875
875
|
|
|
876
|
+
/**
|
|
877
|
+
* Process document purpose configuration and generate purpose rules
|
|
878
|
+
* @param {Array<string>} documentPurpose - Array of document purpose keys
|
|
879
|
+
* @returns {Object} Object containing purposes string and rules content
|
|
880
|
+
*/
|
|
881
|
+
export function processDocumentPurpose(documentPurpose) {
|
|
882
|
+
if (!documentPurpose || !Array.isArray(documentPurpose)) {
|
|
883
|
+
return { purposes: "" };
|
|
884
|
+
}
|
|
885
|
+
|
|
886
|
+
const purposeRules = documentPurpose
|
|
887
|
+
.map((key) => {
|
|
888
|
+
const style = DOCUMENT_STYLES[key];
|
|
889
|
+
if (!style) return null;
|
|
890
|
+
return `Document Purpose - ${style.name}:\n${style.description}\n${style.content}`;
|
|
891
|
+
})
|
|
892
|
+
.filter(Boolean);
|
|
893
|
+
|
|
894
|
+
if (purposeRules.length === 0) {
|
|
895
|
+
return { purposes: "" };
|
|
896
|
+
}
|
|
897
|
+
|
|
898
|
+
const purposes = purposeRules.join("\n\n");
|
|
899
|
+
return {
|
|
900
|
+
purposes,
|
|
901
|
+
};
|
|
902
|
+
}
|
|
903
|
+
|
|
904
|
+
/**
|
|
905
|
+
* Process target audience configuration and generate audience rules and names
|
|
906
|
+
* @param {Array<string>} targetAudienceTypes - Array of target audience type keys
|
|
907
|
+
* @param {string} existingTargetAudience - Existing target audience content
|
|
908
|
+
* @returns {Object} Object containing audiences string, targetAudience string, and rules content
|
|
909
|
+
*/
|
|
910
|
+
export function processTargetAudience(targetAudienceTypes, existingTargetAudience = "") {
|
|
911
|
+
if (!targetAudienceTypes || !Array.isArray(targetAudienceTypes)) {
|
|
912
|
+
return { audiences: "", targetAudience: existingTargetAudience || "" };
|
|
913
|
+
}
|
|
914
|
+
|
|
915
|
+
// Get structured content for rules
|
|
916
|
+
const audienceRules = targetAudienceTypes
|
|
917
|
+
.map((key) => {
|
|
918
|
+
const audience = TARGET_AUDIENCES[key];
|
|
919
|
+
if (!audience) return null;
|
|
920
|
+
return `Target Audience - ${audience.name}:\n${audience.description}\n${audience.content}`;
|
|
921
|
+
})
|
|
922
|
+
.filter(Boolean);
|
|
923
|
+
|
|
924
|
+
let audiences = "";
|
|
925
|
+
if (audienceRules.length > 0) {
|
|
926
|
+
audiences = audienceRules.join("\n\n");
|
|
927
|
+
}
|
|
928
|
+
|
|
929
|
+
// Get names for targetAudience field
|
|
930
|
+
const audienceNames = targetAudienceTypes
|
|
931
|
+
.map((key) => TARGET_AUDIENCES[key]?.name)
|
|
932
|
+
.filter(Boolean)
|
|
933
|
+
.join(", ");
|
|
934
|
+
|
|
935
|
+
let targetAudience = existingTargetAudience || "";
|
|
936
|
+
if (audienceNames) {
|
|
937
|
+
const existingTargetAudienceTrimmed = existingTargetAudience?.trim();
|
|
938
|
+
if (existingTargetAudienceTrimmed) {
|
|
939
|
+
targetAudience = `${existingTargetAudienceTrimmed}\n\n${audienceNames}`;
|
|
940
|
+
} else {
|
|
941
|
+
targetAudience = audienceNames;
|
|
942
|
+
}
|
|
943
|
+
}
|
|
944
|
+
|
|
945
|
+
return {
|
|
946
|
+
audiences,
|
|
947
|
+
targetAudience,
|
|
948
|
+
};
|
|
949
|
+
}
|
|
950
|
+
|
|
876
951
|
/**
|
|
877
952
|
* Process configuration fields - convert keys to actual content
|
|
878
953
|
* @param {Object} config - Parsed configuration
|
|
@@ -924,59 +999,23 @@ export function processConfigFields(config) {
|
|
|
924
999
|
}
|
|
925
1000
|
|
|
926
1001
|
// Process document purpose (array)
|
|
927
|
-
|
|
928
|
-
|
|
929
|
-
|
|
930
|
-
|
|
931
|
-
if (!style) return null;
|
|
932
|
-
return `Document Purpose - ${style.name}:\n${style.description}\n${style.content}`;
|
|
933
|
-
})
|
|
934
|
-
.filter(Boolean);
|
|
935
|
-
|
|
936
|
-
if (purposeRules.length > 0) {
|
|
937
|
-
const purposes = purposeRules.join("\n\n");
|
|
938
|
-
allRulesContent.push(purposes);
|
|
939
|
-
|
|
940
|
-
processed.purposes = purposes;
|
|
941
|
-
}
|
|
1002
|
+
const documentPurposeResult = processDocumentPurpose(config.documentPurpose);
|
|
1003
|
+
if (documentPurposeResult.purposes) {
|
|
1004
|
+
allRulesContent.push(documentPurposeResult.purposes);
|
|
1005
|
+
processed.purposes = documentPurposeResult.purposes;
|
|
942
1006
|
}
|
|
943
1007
|
|
|
944
1008
|
// Process target audience types (array)
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
|
|
951
|
-
|
|
952
|
-
|
|
953
|
-
|
|
954
|
-
|
|
955
|
-
|
|
956
|
-
if (audienceRules.length > 0) {
|
|
957
|
-
const audiences = audienceRules.join("\n\n");
|
|
958
|
-
allRulesContent.push(audiences);
|
|
959
|
-
|
|
960
|
-
processed.audiences = audiences;
|
|
961
|
-
}
|
|
962
|
-
|
|
963
|
-
// Get names for targetAudience field
|
|
964
|
-
audienceNames = config.targetAudienceTypes
|
|
965
|
-
.map((key) => TARGET_AUDIENCES[key]?.name)
|
|
966
|
-
.filter(Boolean)
|
|
967
|
-
.join(", ");
|
|
968
|
-
|
|
969
|
-
if (audienceNames) {
|
|
970
|
-
// Check if original targetAudience field has content
|
|
971
|
-
const existingTargetAudience = config.targetAudience?.trim();
|
|
972
|
-
const newAudienceNames = audienceNames;
|
|
973
|
-
|
|
974
|
-
if (existingTargetAudience) {
|
|
975
|
-
processed.targetAudience = `${existingTargetAudience}\n\n${newAudienceNames}`;
|
|
976
|
-
} else {
|
|
977
|
-
processed.targetAudience = newAudienceNames;
|
|
978
|
-
}
|
|
979
|
-
}
|
|
1009
|
+
const targetAudienceResult = processTargetAudience(
|
|
1010
|
+
config.targetAudienceTypes,
|
|
1011
|
+
config.targetAudience,
|
|
1012
|
+
);
|
|
1013
|
+
if (targetAudienceResult.audiences) {
|
|
1014
|
+
allRulesContent.push(targetAudienceResult.audiences);
|
|
1015
|
+
processed.audiences = targetAudienceResult.audiences;
|
|
1016
|
+
}
|
|
1017
|
+
if (targetAudienceResult.targetAudience) {
|
|
1018
|
+
processed.targetAudience = targetAudienceResult.targetAudience;
|
|
980
1019
|
}
|
|
981
1020
|
|
|
982
1021
|
// Process reader knowledge level (single value)
|
|
@@ -1,101 +0,0 @@
|
|
|
1
|
-
# 仕組み
|
|
2
|
-
|
|
3
|
-
AIGNE DocSmithは、マルチagentシステムで動作します。単一のモノリシックなプロセスではなく、専門のAI agentのパイプラインを編成し、各agentが特定のタスクを担当します。このアプローチにより、ソースコードを完全なドキュメントに変換するための構造化されたモジュラーなプロセスが可能になります。
|
|
4
|
-
|
|
5
|
-
このツールは、AIアプリケーションを開発・展開するためのプラットフォームを提供する、より大きなAIGNEエコシステムの不可欠な部分です。
|
|
6
|
-
|
|
7
|
-

|
|
8
|
-
|
|
9
|
-
## ドキュメント生成パイプライン
|
|
10
|
-
|
|
11
|
-
DocSmithの中核は、ソースコードをいくつかの異なるステージを通して処理するパイプラインです。各ステージは1つ以上の専用agentによって管理されます。通常、`aigne doc generate`コマンドによって開始される主要なワークフローは、次のように視覚化できます。
|
|
12
|
-
|
|
13
|
-
```d2
|
|
14
|
-
direction: down
|
|
15
|
-
|
|
16
|
-
Input: {
|
|
17
|
-
label: "ソースコードと設定"
|
|
18
|
-
shape: rectangle
|
|
19
|
-
}
|
|
20
|
-
|
|
21
|
-
Pipeline: {
|
|
22
|
-
label: "コア生成パイプライン"
|
|
23
|
-
shape: rectangle
|
|
24
|
-
grid-columns: 1
|
|
25
|
-
grid-gap: 40
|
|
26
|
-
|
|
27
|
-
Structure-Planning: {
|
|
28
|
-
label: "1. 構造計画"
|
|
29
|
-
shape: rectangle
|
|
30
|
-
}
|
|
31
|
-
|
|
32
|
-
Content-Generation: {
|
|
33
|
-
label: "2. コンテンツ生成"
|
|
34
|
-
shape: rectangle
|
|
35
|
-
}
|
|
36
|
-
|
|
37
|
-
Saving: {
|
|
38
|
-
label: "3. ドキュメント保存"
|
|
39
|
-
shape: rectangle
|
|
40
|
-
}
|
|
41
|
-
}
|
|
42
|
-
|
|
43
|
-
User-Feedback: {
|
|
44
|
-
label: "ユーザーフィードバックループ\n(--feedbackフラグ経由)"
|
|
45
|
-
shape: rectangle
|
|
46
|
-
}
|
|
47
|
-
|
|
48
|
-
Optional-Steps: {
|
|
49
|
-
label: "オプションの生成後ステップ"
|
|
50
|
-
shape: rectangle
|
|
51
|
-
grid-columns: 2
|
|
52
|
-
grid-gap: 40
|
|
53
|
-
|
|
54
|
-
Translation: {
|
|
55
|
-
label: "翻訳\n(aigne doc translate)"
|
|
56
|
-
shape: rectangle
|
|
57
|
-
}
|
|
58
|
-
|
|
59
|
-
Publishing: {
|
|
60
|
-
label: "公開\n(aigne doc publish)"
|
|
61
|
-
shape: rectangle
|
|
62
|
-
}
|
|
63
|
-
}
|
|
64
|
-
|
|
65
|
-
Input -> Pipeline.Structure-Planning
|
|
66
|
-
Pipeline.Structure-Planning -> Pipeline.Content-Generation
|
|
67
|
-
Pipeline.Content-Generation -> Pipeline.Saving
|
|
68
|
-
Pipeline.Saving -> Optional-Steps
|
|
69
|
-
|
|
70
|
-
User-Feedback -> Pipeline.Structure-Planning: "構造の改良"
|
|
71
|
-
User-Feedback -> Pipeline.Content-Generation: "コンテンツの再生成"
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
1. **入力分析**: このプロセスは、agentがソースコードとプロジェクト設定(`aigne.yaml`)を読み込むことから始まります。
|
|
75
|
-
|
|
76
|
-
2. **構造計画**: あるagentがコードベースを分析し、論理的なドキュメント構造を提案します。プロジェクトの構成と指定されたルールに基づいてアウトラインを作成します。
|
|
77
|
-
|
|
78
|
-
3. **コンテンツ生成**: 構造が決定されると、コンテンツ生成agentがドキュメントプランの各セクションに詳細なテキスト、コード例、説明を埋め込んでいきます。
|
|
79
|
-
|
|
80
|
-
4. **改良と更新**: `aigne doc update`または`aigne doc generate --feedback`を介して入力が提供されると、特定のagentがアクティブになり、個々のドキュメントを更新したり、全体の構造を調整したりします。
|
|
81
|
-
|
|
82
|
-
5. **翻訳と公開**: 主要なコンテンツが生成された後、オプションのagentが多言語翻訳や最終的なドキュメントをウェブプラットフォームに公開するなどのタスクを処理します。
|
|
83
|
-
|
|
84
|
-
## 主要なAI Agent
|
|
85
|
-
|
|
86
|
-
DocSmithの機能は、プロジェクトの設定で定義されたagentのコレクションによって提供されます。各agentには特定の役割があります。以下の表は、主要なagentとその機能の一部をリストアップしたものです。
|
|
87
|
-
|
|
88
|
-
| 機能的役割 | 主要なAgentファイル | 説明 |
|
|
89
|
-
| :--- | :--- | :--- |
|
|
90
|
-
| **構造計画** | `generate/generate-structure.yaml` | ソースコードを分析し、最初のドキュメントアウトラインを提案します。 |
|
|
91
|
-
| **構造の改良** | `generate/refine-document-structure.yaml` | ユーザーのフィードバックに基づいてドキュメントの構造を修正します。 |
|
|
92
|
-
| **コンテンツ生成** | `update/batch-generate-document.yaml`, `update/generate-document.yaml` | ドキュメント構造の各セクションに詳細なコンテンツを埋め込みます。 |
|
|
93
|
-
| **翻訳** | `translate/translate-document.yaml`, `translate/translate-multilingual.yaml` | 生成されたドキュメントを複数のターゲット言語に翻訳します。 |
|
|
94
|
-
| **公開** | `publish/publish-docs.mjs` | ドキュメントをDiscuss Kitインスタンスに公開するプロセスを管理します。 |
|
|
95
|
-
| **データI/O** | `utils/load-sources.mjs`, `utils/save-docs.mjs` | ソースファイルの読み込みと、最終的なマークダウンドキュメントのディスクへの書き込みを担当します。 |
|
|
96
|
-
|
|
97
|
-
このagentベースのアーキテクチャにより、ドキュメント化プロセスの各ステップを専門のツールで処理できるため、構造化され、保守可能なワークフローが保証されます。
|
|
98
|
-
|
|
99
|
-
---
|
|
100
|
-
|
|
101
|
-
DocSmithが出力の正確性と形式を保証するために講じている対策を理解するには、[品質保証](./advanced-quality-assurance.md)のセクションに進んでください。
|
|
@@ -1,101 +0,0 @@
|
|
|
1
|
-
# How It Works
|
|
2
|
-
|
|
3
|
-
AIGNE DocSmith operates on a multi-agent system. Instead of a single monolithic process, it orchestrates a pipeline of specialized AI agents, where each agent is responsible for a specific task. This approach allows for a structured and modular process that transforms source code into complete documentation.
|
|
4
|
-
|
|
5
|
-
The tool is an integral part of the larger AIGNE ecosystem, which provides a platform for developing and deploying AI applications.
|
|
6
|
-
|
|
7
|
-

|
|
8
|
-
|
|
9
|
-
## The Documentation Generation Pipeline
|
|
10
|
-
|
|
11
|
-
The core of DocSmith is a pipeline that processes your source code through several distinct stages. Each stage is managed by one or more dedicated agents. The primary workflow, typically initiated by the `aigne doc generate` command, can be visualized as follows:
|
|
12
|
-
|
|
13
|
-
```d2
|
|
14
|
-
direction: down
|
|
15
|
-
|
|
16
|
-
Input: {
|
|
17
|
-
label: "Source Code & Config"
|
|
18
|
-
shape: rectangle
|
|
19
|
-
}
|
|
20
|
-
|
|
21
|
-
Pipeline: {
|
|
22
|
-
label: "Core Generation Pipeline"
|
|
23
|
-
shape: rectangle
|
|
24
|
-
grid-columns: 1
|
|
25
|
-
grid-gap: 40
|
|
26
|
-
|
|
27
|
-
Structure-Planning: {
|
|
28
|
-
label: "1. Structure Planning"
|
|
29
|
-
shape: rectangle
|
|
30
|
-
}
|
|
31
|
-
|
|
32
|
-
Content-Generation: {
|
|
33
|
-
label: "2. Content Generation"
|
|
34
|
-
shape: rectangle
|
|
35
|
-
}
|
|
36
|
-
|
|
37
|
-
Saving: {
|
|
38
|
-
label: "3. Save Documents"
|
|
39
|
-
shape: rectangle
|
|
40
|
-
}
|
|
41
|
-
}
|
|
42
|
-
|
|
43
|
-
User-Feedback: {
|
|
44
|
-
label: "User Feedback Loop\n(via --feedback flag)"
|
|
45
|
-
shape: rectangle
|
|
46
|
-
}
|
|
47
|
-
|
|
48
|
-
Optional-Steps: {
|
|
49
|
-
label: "Optional Post-Generation Steps"
|
|
50
|
-
shape: rectangle
|
|
51
|
-
grid-columns: 2
|
|
52
|
-
grid-gap: 40
|
|
53
|
-
|
|
54
|
-
Translation: {
|
|
55
|
-
label: "Translate\n(aigne doc translate)"
|
|
56
|
-
shape: rectangle
|
|
57
|
-
}
|
|
58
|
-
|
|
59
|
-
Publishing: {
|
|
60
|
-
label: "Publish\n(aigne doc publish)"
|
|
61
|
-
shape: rectangle
|
|
62
|
-
}
|
|
63
|
-
}
|
|
64
|
-
|
|
65
|
-
Input -> Pipeline.Structure-Planning
|
|
66
|
-
Pipeline.Structure-Planning -> Pipeline.Content-Generation
|
|
67
|
-
Pipeline.Content-Generation -> Pipeline.Saving
|
|
68
|
-
Pipeline.Saving -> Optional-Steps
|
|
69
|
-
|
|
70
|
-
User-Feedback -> Pipeline.Structure-Planning: "Refine Structure"
|
|
71
|
-
User-Feedback -> Pipeline.Content-Generation: "Regenerate Content"
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
1. **Input Analysis**: The process begins when agents load your source code and project configuration (`aigne.yaml`).
|
|
75
|
-
|
|
76
|
-
2. **Structure Planning**: An agent analyzes the codebase to propose a logical documentation structure. It creates an outline based on the project's composition and any specified rules.
|
|
77
|
-
|
|
78
|
-
3. **Content Generation**: With the structure in place, content generation agents populate each section of the document plan with detailed text, code examples, and explanations.
|
|
79
|
-
|
|
80
|
-
4. **Refinement and Updates**: When you provide input via `aigne doc update` or `aigne doc generate --feedback`, specific agents are activated to update individual documents or adjust the overall structure.
|
|
81
|
-
|
|
82
|
-
5. **Translation and Publishing**: After the primary content is generated, optional agents handle tasks like multi-language translation and publishing the final documentation to a web platform.
|
|
83
|
-
|
|
84
|
-
## Key AI Agents
|
|
85
|
-
|
|
86
|
-
DocSmith's functionality is provided by a collection of agents defined in the project's configuration. Each agent has a specific role. The table below lists some of the key agents and their functions.
|
|
87
|
-
|
|
88
|
-
| Functional Role | Key Agent Files | Description |
|
|
89
|
-
| :--- | :--- | :--- |
|
|
90
|
-
| **Structure Planning** | `generate/generate-structure.yaml` | Analyzes source code to propose the initial document outline. |
|
|
91
|
-
| **Structure Refinement** | `generate/refine-document-structure.yaml` | Modifies the documentation structure based on user feedback. |
|
|
92
|
-
| **Content Generation** | `update/batch-generate-document.yaml`, `update/generate-document.yaml` | Populates the documentation structure with detailed content for each section. |
|
|
93
|
-
| **Translation** | `translate/translate-document.yaml`, `translate/translate-multilingual.yaml` | Translates generated documentation into multiple target languages. |
|
|
94
|
-
| **Publishing** | `publish/publish-docs.mjs` | Manages the process of publishing documents to Discuss Kit instances. |
|
|
95
|
-
| **Data I/O** | `utils/load-sources.mjs`, `utils/save-docs.mjs` | Responsible for reading source files and writing the final markdown documents to disk. |
|
|
96
|
-
|
|
97
|
-
This agent-based architecture allows each step of the documentation process to be handled by a specialized tool, ensuring a structured and maintainable workflow.
|
|
98
|
-
|
|
99
|
-
---
|
|
100
|
-
|
|
101
|
-
To understand the measures DocSmith takes to ensure the accuracy and format of the output, proceed to the [Quality Assurance](./advanced-quality-assurance.md) section.
|