@todoist/doistbot-cli 0.0.0-stage → 1.0.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/actions/auth.js +225 -0
- package/dist/actions/doctor.js +39 -0
- package/dist/actions/review-usage.js +25 -0
- package/dist/actions/review.js +311 -0
- package/dist/actions/setup.js +14 -0
- package/dist/auth.js +202 -0
- package/dist/config.js +145 -0
- package/dist/env.js +10 -0
- package/dist/file-store.js +112 -0
- package/dist/index.js +164 -0
- package/dist/input.js +40 -0
- package/dist/runtime.js +105 -0
- package/dist/secure-store.js +148 -0
- package/dist/terminal.js +185 -0
- package/dist/test-utils/secure-store-mocks.js +29 -0
- package/dist/update-cache-entrypoint.js +16 -0
- package/dist/update-check.js +235 -0
- package/package.json +65 -5
- package/sandbox/_node_modules/doistbot-logging/dist/index.d.ts +23 -0
- package/sandbox/_node_modules/doistbot-logging/dist/index.js +51 -0
- package/sandbox/_node_modules/doistbot-logging/package.json +12 -0
- package/sandbox/_node_modules/doistbot-repo-config/dist/index.d.ts +93 -0
- package/sandbox/_node_modules/doistbot-repo-config/dist/index.js +197 -0
- package/sandbox/_node_modules/doistbot-repo-config/package.json +12 -0
- package/sandbox/dist/core/autofix-resources.js +58 -0
- package/sandbox/dist/core/autofix-service-query.js +69 -0
- package/sandbox/dist/core/autofix-test-policy.js +11 -0
- package/sandbox/dist/core/autofix-verification.js +355 -0
- package/sandbox/dist/core/bot-logins.js +14 -0
- package/sandbox/dist/core/check-run.js +158 -0
- package/sandbox/dist/core/datadog-metrics.js +113 -0
- package/sandbox/dist/core/environment.js +49 -0
- package/sandbox/dist/core/git-auth.js +17 -0
- package/sandbox/dist/core/github-history.js +4 -0
- package/sandbox/dist/core/github-issues.js +18 -0
- package/sandbox/dist/core/label-parsing.js +1 -0
- package/sandbox/dist/core/langfuse-observation.js +33 -0
- package/sandbox/dist/core/langfuse-session.js +39 -0
- package/sandbox/dist/core/logger-span-processor.js +90 -0
- package/sandbox/dist/core/logging.js +18 -0
- package/sandbox/dist/core/pi-autofix-extension.js +188 -0
- package/sandbox/dist/core/pi-execution-limits-extension.js +79 -0
- package/sandbox/dist/core/pi-progress-logging.js +51 -0
- package/sandbox/dist/core/pi-tool-output-filter-extension.js +65 -0
- package/sandbox/dist/core/pi.js +870 -0
- package/sandbox/dist/core/pii-scrubber.js +31 -0
- package/sandbox/dist/core/prompt-template.js +63 -0
- package/sandbox/dist/core/reactions.js +36 -0
- package/sandbox/dist/core/regexp.js +3 -0
- package/sandbox/dist/core/repository-map.js +1 -0
- package/sandbox/dist/core/repository-resolver.js +1 -0
- package/sandbox/dist/core/retry.js +29 -0
- package/sandbox/dist/core/s3-utils.js +42 -0
- package/sandbox/dist/core/shared.js +124 -0
- package/sandbox/dist/core/snapshot-helpers.js +17 -0
- package/sandbox/dist/core/span-test-helpers.js +13 -0
- package/sandbox/dist/core/tool-environment.js +83 -0
- package/sandbox/dist/core/tool-output.js +18 -0
- package/sandbox/dist/core/tracing-setup.js +92 -0
- package/sandbox/dist/core/tracing.js +81 -0
- package/sandbox/dist/main.js +51 -0
- package/sandbox/dist/providers/config.js +23 -0
- package/sandbox/dist/providers/credentials.js +40 -0
- package/sandbox/dist/providers/gemini.js +28 -0
- package/sandbox/dist/providers/helpers.js +253 -0
- package/sandbox/dist/providers/index.js +27 -0
- package/sandbox/dist/providers/openai.js +23 -0
- package/sandbox/dist/providers/openrouter-pi.js +25 -0
- package/sandbox/dist/providers/pi-review.js +301 -0
- package/sandbox/dist/providers/process.js +34 -0
- package/sandbox/dist/providers/schemas.js +61 -0
- package/sandbox/dist/providers/types.js +4 -0
- package/sandbox/dist/tasks/chat/chat.js +440 -0
- package/sandbox/dist/tasks/chat/prompt.js +20 -0
- package/sandbox/dist/tasks/issue-fix-retry/fix-retry.js +304 -0
- package/sandbox/dist/tasks/issue-summarize/context.js +21 -0
- package/sandbox/dist/tasks/issue-summarize/media.js +335 -0
- package/sandbox/dist/tasks/issue-summarize/model.js +230 -0
- package/sandbox/dist/tasks/issue-summarize/output.js +45 -0
- package/sandbox/dist/tasks/issue-summarize/prompt.js +11 -0
- package/sandbox/dist/tasks/issue-summarize/summarize.js +238 -0
- package/sandbox/dist/tasks/issue-summarize/thread.js +123 -0
- package/sandbox/dist/tasks/issue-summarize/types.js +1 -0
- package/sandbox/dist/tasks/issue-triage/assignee-review.js +114 -0
- package/sandbox/dist/tasks/issue-triage/autofix-capabilities.js +22 -0
- package/sandbox/dist/tasks/issue-triage/autofix-checkpoint.js +122 -0
- package/sandbox/dist/tasks/issue-triage/autofix-policy.js +36 -0
- package/sandbox/dist/tasks/issue-triage/clone.js +40 -0
- package/sandbox/dist/tasks/issue-triage/context.js +57 -0
- package/sandbox/dist/tasks/issue-triage/diff-validator.js +102 -0
- package/sandbox/dist/tasks/issue-triage/fix-attempt.js +617 -0
- package/sandbox/dist/tasks/issue-triage/fix-context.js +5 -0
- package/sandbox/dist/tasks/issue-triage/fix-dispatch.js +263 -0
- package/sandbox/dist/tasks/issue-triage/fix-failure-reasons.js +2 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/effects.js +526 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/effects.types.js +1 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/fake-effects.js +65 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/promotion-task.js +108 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/recovery.js +512 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/runners.js +233 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/state-machine.js +171 -0
- package/sandbox/dist/tasks/issue-triage/github-request.js +2 -0
- package/sandbox/dist/tasks/issue-triage/hero-group-map.js +206 -0
- package/sandbox/dist/tasks/issue-triage/markers.js +6 -0
- package/sandbox/dist/tasks/issue-triage/model.js +124 -0
- package/sandbox/dist/tasks/issue-triage/output.js +164 -0
- package/sandbox/dist/tasks/issue-triage/pr-creator.js +585 -0
- package/sandbox/dist/tasks/issue-triage/pr-readiness.js +25 -0
- package/sandbox/dist/tasks/issue-triage/prompt.js +63 -0
- package/sandbox/dist/tasks/issue-triage/repo-config.js +47 -0
- package/sandbox/dist/tasks/issue-triage/repo-prompts.js +77 -0
- package/sandbox/dist/tasks/issue-triage/report-fixture.js +53 -0
- package/sandbox/dist/tasks/issue-triage/report-history.js +113 -0
- package/sandbox/dist/tasks/issue-triage/report-schema.js +63 -0
- package/sandbox/dist/tasks/issue-triage/report.js +111 -0
- package/sandbox/dist/tasks/issue-triage/resolution.js +30 -0
- package/sandbox/dist/tasks/issue-triage/routing.js +169 -0
- package/sandbox/dist/tasks/issue-triage/sandbox-environment.js +59 -0
- package/sandbox/dist/tasks/issue-triage/shared.js +86 -0
- package/sandbox/dist/tasks/issue-triage/side-effects.js +102 -0
- package/sandbox/dist/tasks/issue-triage/triage.js +442 -0
- package/sandbox/dist/tasks/issue-triage/types.js +1 -0
- package/sandbox/dist/tasks/persist-logs/context.js +19 -0
- package/sandbox/dist/tasks/persist-logs/dd-proxy.js +116 -0
- package/sandbox/dist/tasks/persist-logs/llm-query-planner.js +156 -0
- package/sandbox/dist/tasks/persist-logs/persist-logs.js +491 -0
- package/sandbox/dist/tasks/refresh-s3-urls/refresh-s3-urls.js +214 -0
- package/sandbox/dist/tasks/review/check-run.js +326 -0
- package/sandbox/dist/tasks/review/comment-mapper.js +46 -0
- package/sandbox/dist/tasks/review/conversation-context.js +477 -0
- package/sandbox/dist/tasks/review/diff.js +95 -0
- package/sandbox/dist/tasks/review/engines/dedupe.js +291 -0
- package/sandbox/dist/tasks/review/engines/multi-focus.js +419 -0
- package/sandbox/dist/tasks/review/engines/shared.js +87 -0
- package/sandbox/dist/tasks/review/engines/types.js +1 -0
- package/sandbox/dist/tasks/review/finding-output.js +18 -0
- package/sandbox/dist/tasks/review/format.js +17 -0
- package/sandbox/dist/tasks/review/github-comment-format.js +94 -0
- package/sandbox/dist/tasks/review/local-git.js +374 -0
- package/sandbox/dist/tasks/review/local.js +118 -0
- package/sandbox/dist/tasks/review/multi-focus-prompt.js +59 -0
- package/sandbox/dist/tasks/review/prompt.js +81 -0
- package/sandbox/dist/tasks/review/review-feedback.js +48 -0
- package/sandbox/dist/tasks/review/review-mode-github.js +58 -0
- package/sandbox/dist/tasks/review/review-mode-policy.js +232 -0
- package/sandbox/dist/tasks/review/review-mode-types.js +1 -0
- package/sandbox/dist/tasks/review/review-mode.js +296 -0
- package/sandbox/dist/tasks/review/review-summary.js +123 -0
- package/sandbox/dist/tasks/review/review.js +391 -0
- package/sandbox/dist/tasks/review/runner.js +255 -0
- package/sandbox/dist/tasks/review/severity.js +5 -0
- package/sandbox/dist/tasks/review/summary-model.js +271 -0
- package/sandbox/dist/tasks/review/thinking-level.js +14 -0
- package/sandbox/dist/tasks/review/translation-review-policy.js +37 -0
- package/sandbox/dist/tasks/review/types.js +1 -0
- package/sandbox/dist/tasks/review-slice-plan/context.js +66 -0
- package/sandbox/dist/tasks/review-slice-plan/plan.js +210 -0
- package/sandbox/dist/tasks/review-slice-plan/prompt.js +59 -0
- package/sandbox/dist/tasks/review-slice-plan/review-slice-plan.js +367 -0
- package/sandbox/dist/tasks/sentry-triage/store.js +133 -0
- package/sandbox/dist/tasks/sentry-triage/triage.js +217 -0
- package/sandbox/node_modules/doistbot-logging/dist/index.d.ts +23 -0
- package/sandbox/node_modules/doistbot-logging/dist/index.js +51 -0
- package/sandbox/node_modules/doistbot-logging/package.json +12 -0
- package/sandbox/node_modules/doistbot-repo-config/dist/index.d.ts +93 -0
- package/sandbox/node_modules/doistbot-repo-config/dist/index.js +197 -0
- package/sandbox/node_modules/doistbot-repo-config/package.json +12 -0
- package/sandbox/src/review/prompts/ai-internal-tools.md +254 -0
- package/sandbox/src/review/prompts/backend-repository-standards.md +102 -0
- package/sandbox/src/review/prompts/observability-standard.md +169 -0
- package/sandbox/src/review/prompts/outline-map.json +26 -0
- package/sandbox/src/review/prompts/secrets-management.md +286 -0
- package/sandbox/src/review/prompts/service-production-readiness.md +122 -0
- package/sandbox/src/review/prompts/workload-placement.md +345 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/efficiency.md +16 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/general.md +32 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/quality.md +20 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/reuse.md +13 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/security.md +18 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/standards.md +21 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/tests.md +19 -0
- package/sandbox/src/tasks/review/prompts/review-mode.md +40 -0
- package/sandbox/src/tasks/review/prompts/review-multi-focus-base-prompt.md +142 -0
- package/scripts/restore-runtime-assets.mjs +21 -0
- package/README.md +0 -3
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
import fs from 'fs';
|
|
2
|
+
import path from 'path';
|
|
3
|
+
import { load as yamlLoad } from 'js-yaml';
|
|
4
|
+
export const CONFIG_FILE_PATH = '.github/doistbot.yml';
|
|
5
|
+
export const DEFAULT_REVIEW_MODELS = ['openai', 'deepseek', 'zai'];
|
|
6
|
+
export class RepoFileContentError extends Error {
|
|
7
|
+
reason;
|
|
8
|
+
constructor(reason) {
|
|
9
|
+
super(`Repository file content is ${reason}`);
|
|
10
|
+
this.reason = reason;
|
|
11
|
+
this.name = 'RepoFileContentError';
|
|
12
|
+
}
|
|
13
|
+
}
|
|
14
|
+
function isMapping(value) {
|
|
15
|
+
return (typeof value === 'object' &&
|
|
16
|
+
value !== null &&
|
|
17
|
+
(Object.getPrototypeOf(value) === Object.prototype || Object.getPrototypeOf(value) === null));
|
|
18
|
+
}
|
|
19
|
+
function hasControlCharacters(value) {
|
|
20
|
+
return [...value].some((character) => {
|
|
21
|
+
const code = character.charCodeAt(0);
|
|
22
|
+
return code < 0x20 || code === 0x7f;
|
|
23
|
+
});
|
|
24
|
+
}
|
|
25
|
+
function parseReviewModeConfig(value) {
|
|
26
|
+
if (!isMapping(value)) {
|
|
27
|
+
throw new Error('review.mode must be a mapping');
|
|
28
|
+
}
|
|
29
|
+
const mode = {};
|
|
30
|
+
if (value.guidance_path !== undefined) {
|
|
31
|
+
const guidancePath = value.guidance_path;
|
|
32
|
+
if (typeof guidancePath !== 'string' ||
|
|
33
|
+
!guidancePath.trim() ||
|
|
34
|
+
path.posix.isAbsolute(guidancePath.trim()) ||
|
|
35
|
+
path.win32.isAbsolute(guidancePath.trim()) ||
|
|
36
|
+
/^[a-z][a-z\d+.-]*:/i.test(guidancePath.trim()) ||
|
|
37
|
+
guidancePath.includes('\\') ||
|
|
38
|
+
hasControlCharacters(guidancePath) ||
|
|
39
|
+
guidancePath.trim().split('/').includes('..') ||
|
|
40
|
+
!/\.(md|markdown)$/i.test(guidancePath.trim())) {
|
|
41
|
+
throw new Error('review.mode.guidance_path must be a repository-relative Markdown path');
|
|
42
|
+
}
|
|
43
|
+
mode.guidance_path = path.posix.normalize(guidancePath.trim());
|
|
44
|
+
}
|
|
45
|
+
if (value.labels !== undefined) {
|
|
46
|
+
if (!isMapping(value.labels)) {
|
|
47
|
+
throw new Error('review.mode.labels must be a mapping');
|
|
48
|
+
}
|
|
49
|
+
mode.labels = {};
|
|
50
|
+
for (const category of ['ask', 'show', 'ship']) {
|
|
51
|
+
const labels = value.labels[category];
|
|
52
|
+
if (labels === undefined)
|
|
53
|
+
continue;
|
|
54
|
+
if (!Array.isArray(labels) ||
|
|
55
|
+
labels.some((label) => typeof label !== 'string' || !label.trim() || hasControlCharacters(label))) {
|
|
56
|
+
throw new Error(`review.mode.labels.${category} must contain non-empty label names`);
|
|
57
|
+
}
|
|
58
|
+
mode.labels[category] = labels.map((label) => label.trim());
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
return mode;
|
|
62
|
+
}
|
|
63
|
+
/** Review-mode validation is opt-in so it cannot block unrelated config callers. */
|
|
64
|
+
export function parseRepoConfigContent(content, { validateReviewMode = false } = {}) {
|
|
65
|
+
const config = yamlLoad(content);
|
|
66
|
+
if (!validateReviewMode)
|
|
67
|
+
return config;
|
|
68
|
+
if (!isMapping(config)) {
|
|
69
|
+
throw new Error('Repository configuration must be a mapping');
|
|
70
|
+
}
|
|
71
|
+
if (config.review !== undefined) {
|
|
72
|
+
if (!isMapping(config.review)) {
|
|
73
|
+
throw new Error('review must be a mapping');
|
|
74
|
+
}
|
|
75
|
+
if (config.review.mode !== undefined) {
|
|
76
|
+
config.review.mode = parseReviewModeConfig(config.review.mode);
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
return config;
|
|
80
|
+
}
|
|
81
|
+
export async function fetchFileContent({ octokit, owner, repo, path, ref, signal, logger, strict = false, maxBytes, }) {
|
|
82
|
+
const params = {
|
|
83
|
+
owner,
|
|
84
|
+
repo,
|
|
85
|
+
path,
|
|
86
|
+
};
|
|
87
|
+
if (ref) {
|
|
88
|
+
params.ref = ref;
|
|
89
|
+
}
|
|
90
|
+
if (signal) {
|
|
91
|
+
params.request = { signal };
|
|
92
|
+
}
|
|
93
|
+
let response;
|
|
94
|
+
try {
|
|
95
|
+
response = await octokit.rest.repos.getContent(params);
|
|
96
|
+
}
|
|
97
|
+
catch (error) {
|
|
98
|
+
if (error instanceof Error && 'status' in error && error.status === 404) {
|
|
99
|
+
logger?.info('File not found', {
|
|
100
|
+
component: 'repo-config',
|
|
101
|
+
repository: `${owner}/${repo}`,
|
|
102
|
+
path,
|
|
103
|
+
});
|
|
104
|
+
return null;
|
|
105
|
+
}
|
|
106
|
+
throw error;
|
|
107
|
+
}
|
|
108
|
+
const { data } = response;
|
|
109
|
+
if (!('type' in data) || data.type !== 'file' || (strict && 'target' in data)) {
|
|
110
|
+
if (strict)
|
|
111
|
+
throw new RepoFileContentError('not_file');
|
|
112
|
+
logger?.warn('Path exists but is not a file', {
|
|
113
|
+
component: 'repo-config',
|
|
114
|
+
repository: `${owner}/${repo}`,
|
|
115
|
+
path,
|
|
116
|
+
});
|
|
117
|
+
return null;
|
|
118
|
+
}
|
|
119
|
+
if (maxBytes !== undefined && 'size' in data && data.size > maxBytes) {
|
|
120
|
+
throw new RepoFileContentError('too_large');
|
|
121
|
+
}
|
|
122
|
+
if (!('content' in data) ||
|
|
123
|
+
typeof data.content !== 'string' ||
|
|
124
|
+
(strict && 'encoding' in data && data.encoding !== 'base64')) {
|
|
125
|
+
if (strict)
|
|
126
|
+
throw new RepoFileContentError('unavailable');
|
|
127
|
+
return null;
|
|
128
|
+
}
|
|
129
|
+
if (!strict && !data.content)
|
|
130
|
+
return null;
|
|
131
|
+
const content = Buffer.from(data.content, 'base64').toString('utf8');
|
|
132
|
+
if (maxBytes !== undefined && Buffer.byteLength(content, 'utf8') > maxBytes) {
|
|
133
|
+
throw new RepoFileContentError('too_large');
|
|
134
|
+
}
|
|
135
|
+
return content;
|
|
136
|
+
}
|
|
137
|
+
/**
|
|
138
|
+
* Fetches the repository-specific configuration from .github/doistbot.yml.
|
|
139
|
+
* Returns null when the file is genuinely absent (404) or the path is not a file.
|
|
140
|
+
* Throws on transport / authorization errors so callers can distinguish "no
|
|
141
|
+
* config" from "could not load config" — important when an opt-out flag would
|
|
142
|
+
* otherwise be silently ignored.
|
|
143
|
+
*/
|
|
144
|
+
export async function getRepoConfig({ octokit, owner, repo, ref, signal, logger, }) {
|
|
145
|
+
const content = await fetchFileContent({
|
|
146
|
+
octokit,
|
|
147
|
+
owner,
|
|
148
|
+
repo,
|
|
149
|
+
path: CONFIG_FILE_PATH,
|
|
150
|
+
ref,
|
|
151
|
+
signal,
|
|
152
|
+
logger,
|
|
153
|
+
});
|
|
154
|
+
if (content === null) {
|
|
155
|
+
return null;
|
|
156
|
+
}
|
|
157
|
+
const config = parseRepoConfigContent(content);
|
|
158
|
+
logger.info('Loaded repo config', {
|
|
159
|
+
component: 'repo-config',
|
|
160
|
+
repository: `${owner}/${repo}`,
|
|
161
|
+
path: CONFIG_FILE_PATH,
|
|
162
|
+
});
|
|
163
|
+
return config;
|
|
164
|
+
}
|
|
165
|
+
export function getLocalRepoConfig({ workspaceDir, logger, strict = false, }) {
|
|
166
|
+
const configPath = path.join(workspaceDir, CONFIG_FILE_PATH);
|
|
167
|
+
try {
|
|
168
|
+
const config = parseRepoConfigContent(fs.readFileSync(configPath, 'utf8'));
|
|
169
|
+
logger.info('Loaded local repo config', {
|
|
170
|
+
component: 'repo-config',
|
|
171
|
+
path: CONFIG_FILE_PATH,
|
|
172
|
+
});
|
|
173
|
+
return config;
|
|
174
|
+
}
|
|
175
|
+
catch (error) {
|
|
176
|
+
if (error instanceof Error && 'code' in error && error.code === 'ENOENT')
|
|
177
|
+
return null;
|
|
178
|
+
if (strict)
|
|
179
|
+
throw error;
|
|
180
|
+
logger.warn('Failed to parse local repo config', {
|
|
181
|
+
component: 'repo-config',
|
|
182
|
+
path: CONFIG_FILE_PATH,
|
|
183
|
+
error: error instanceof Error ? error.message : String(error),
|
|
184
|
+
});
|
|
185
|
+
return null;
|
|
186
|
+
}
|
|
187
|
+
}
|
|
188
|
+
export function getDefaultReviewModels() {
|
|
189
|
+
return [...DEFAULT_REVIEW_MODELS];
|
|
190
|
+
}
|
|
191
|
+
export function parseReviewModelsEnabled(value) {
|
|
192
|
+
const models = value
|
|
193
|
+
?.split(',')
|
|
194
|
+
.map((model) => model.trim())
|
|
195
|
+
.filter(Boolean) ?? [];
|
|
196
|
+
return models.length > 0 ? [...new Set(models)] : getDefaultReviewModels();
|
|
197
|
+
}
|
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
# Standard: Internal AI Tools
|
|
2
|
+
|
|
3
|
+
| 📄 Document Type | Standard |
|
|
4
|
+
| :--------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
5
|
+
| 🎯 Status | Approved |
|
|
6
|
+
| 👤 Owner | Luciano Facchinelli |
|
|
7
|
+
| 📅 Last Reviewed | 2026-03-06 |
|
|
8
|
+
| 🗂️ References | [Data Classification Policy](https://handbook.doist.com/doc/data-classification-policy-t7mCBRgaec) [Encryption Policy](https://handbook.doist.com/doc/encryption-policy-zJ5ZApYT30) |
|
|
9
|
+
| 📌 Version | 1.0 |
|
|
10
|
+
|
|
11
|
+
## 1. Purpose
|
|
12
|
+
|
|
13
|
+
As we build internal tools that leverage LLMs (e.g., for debugging, support, analytics), we need clear guardrails to prevent accidental data exposure. PII sent to third-party AI providers is a data leak waiting to happen.
|
|
14
|
+
|
|
15
|
+
This standard exists to:
|
|
16
|
+
|
|
17
|
+
- Prevent PII from reaching local developer machines or unauthorized third parties
|
|
18
|
+
- Define what data can (and cannot) be sent to LLM providers
|
|
19
|
+
- Establish an approval process for new AI-powered internal tools
|
|
20
|
+
- Ensure auditability of AI tool usage
|
|
21
|
+
|
|
22
|
+
## 2. Scope
|
|
23
|
+
|
|
24
|
+
This standard applies to:
|
|
25
|
+
|
|
26
|
+
- All internal tools that invoke external AI/LLM APIs (OpenAI, Anthropic, Google, etc.), whether directly or indirectly
|
|
27
|
+
- Tools that process, display, or transmit user data (PII, task content, account info)
|
|
28
|
+
- Both production services and developer tooling
|
|
29
|
+
|
|
30
|
+
This does **not** apply to:
|
|
31
|
+
|
|
32
|
+
- User-facing AI features in Doist products, such as Todoist/Twist (covered by product specs)
|
|
33
|
+
- Purely local development tools that never touch real user data
|
|
34
|
+
|
|
35
|
+
## 3. Quick Reference
|
|
36
|
+
|
|
37
|
+
This is a list of requirements all internal AI tools should follow:
|
|
38
|
+
|
|
39
|
+
- Tools must only be accessible through our secure internal network (Tailscale)
|
|
40
|
+
- All processing happens on the server-side.
|
|
41
|
+
- Remove or mask personal information before sending data to AI services
|
|
42
|
+
- Keep records of all AI tool usage so we can review them if needed
|
|
43
|
+
|
|
44
|
+
## 4. Standard
|
|
45
|
+
|
|
46
|
+
### 4.1 Server-Side Only
|
|
47
|
+
|
|
48
|
+
**All AI-powered internal tools that access user data MUST run server-side.**
|
|
49
|
+
|
|
50
|
+
- No PII on local machines.
|
|
51
|
+
- Tools must authenticate users and enforce access controls (via Tailscale, preferably)
|
|
52
|
+
- Data flows: Production DB → Server-side service → approved LLM API → answer returned to local CLI/tool
|
|
53
|
+
- Local clients may only display results, never raw user data
|
|
54
|
+
|
|
55
|
+
### 4.2 External Service Credentials
|
|
56
|
+
|
|
57
|
+
The server-side service will call third-party APIs on behalf of internal users. How credentials are managed depends on how the external service enforces access control.
|
|
58
|
+
|
|
59
|
+
**Pattern A — Shared service token** (e.g. Zendesk, HiBob)
|
|
60
|
+
|
|
61
|
+
A single API key or service-account token is stored securely on the server (e.g. in a secrets manager, never in source code). All requests are made using this one credential, regardless of who triggered the request.
|
|
62
|
+
|
|
63
|
+
Use this pattern when the external service does not enforce per-user access controls. In this case, the internal service layer is responsible for filtering what each caller is allowed to see before returning a response.
|
|
64
|
+
|
|
65
|
+
**Pattern B — Per-user delegated token / OAuth** (e.g. Google Workspace)
|
|
66
|
+
|
|
67
|
+
Each user authenticates with the external service independently. Their OAuth token is stored server-side, scoped to their identity, and refreshed as needed. When a user makes a request, the server calls the external API on their behalf using their own token.
|
|
68
|
+
|
|
69
|
+
Use this pattern when the external service enforces its own per-user permissions that the internal service cannot replicate, or when acting as the user is required by the service's terms.
|
|
70
|
+
|
|
71
|
+
**Decision rule:** Does the external service itself enforce per-user access controls? If yes → use per-user delegated tokens. If no → use a shared service token with internal filtering.
|
|
72
|
+
|
|
73
|
+
> Credentials must never be hard-coded in source code or committed to version control. Please follow our [standard secrets management practices](https://handbook.doist.com/doc/standard-secrets-management-wbKmIfrtgr)
|
|
74
|
+
|
|
75
|
+
### 4.3 Data Filtering Before LLM Calls
|
|
76
|
+
|
|
77
|
+
Where possible, sensitive data should be only used on a need basis.
|
|
78
|
+
|
|
79
|
+
Examples:
|
|
80
|
+
|
|
81
|
+
- Ensure prompts/data flows prefer user IDs over their email addresses
|
|
82
|
+
- Error messages, stack traces (with user ID, not email)
|
|
83
|
+
- Schema information, code snippets
|
|
84
|
+
|
|
85
|
+
Things tool builders should actively avoid sending (feeding to the context window):
|
|
86
|
+
|
|
87
|
+
- User email addresses, names, or other PII
|
|
88
|
+
- Task/project content (user-generated text)
|
|
89
|
+
- Authentication tokens, credentials
|
|
90
|
+
- Payment information
|
|
91
|
+
|
|
92
|
+
### 4.4 Approved Providers Only
|
|
93
|
+
|
|
94
|
+
Only pre-approved LLM providers may be used and any interaction should happen through our organization's accounts.
|
|
95
|
+
|
|
96
|
+
Avoid using any browser extension or fancy AI tool under the sun that lack the required scrutiny in terms of the LLMs they use behind the scenes, as well as data retention.
|
|
97
|
+
|
|
98
|
+
Per our Privacy Policy, here are the ones we communicate publicly:
|
|
99
|
+
|
|
100
|
+
- OpenAI
|
|
101
|
+
- Anthropic
|
|
102
|
+
- Google Vertex AI
|
|
103
|
+
- AWS Bedrock
|
|
104
|
+
|
|
105
|
+
Loop in Platform team review if there’s any new tool.
|
|
106
|
+
|
|
107
|
+
> The example uses AWS Bedrock rather than the Anthropic API directly because Bedrock is likely already covered under Doist’s AWS agreement, keeping data within existing infrastructure. Any of the approved providers are valid — the key is using the organisation’s account, not a personal one.
|
|
108
|
+
|
|
109
|
+
### 4.5 Audit Logging
|
|
110
|
+
|
|
111
|
+
All LLM API calls SHOULD be logged.
|
|
112
|
+
|
|
113
|
+
Logs to include:
|
|
114
|
+
|
|
115
|
+
- Timestamp
|
|
116
|
+
- Tool/service name
|
|
117
|
+
- User who initiated the request
|
|
118
|
+
- What data context was accessed (user IDs, not content)
|
|
119
|
+
- LLM provider used
|
|
120
|
+
|
|
121
|
+
Logs must NOT include:
|
|
122
|
+
|
|
123
|
+
- Actual prompts containing user data
|
|
124
|
+
- LLM responses containing user data
|
|
125
|
+
|
|
126
|
+
### 4.6 Approval Process
|
|
127
|
+
|
|
128
|
+
New AI-powered internal tools that access user data require explicit approval.
|
|
129
|
+
|
|
130
|
+
Process:
|
|
131
|
+
|
|
132
|
+
1. Create a brief spec describing: what the tool does, what data it accesses (or is capable of accessing), if and how data is filtered
|
|
133
|
+
2. Create a Github Issue on Platform-backlog repo for Review
|
|
134
|
+
3. Get sign-off from at least one Dev head team member. Also remember to include Platform Team as Reviewers
|
|
135
|
+
4. Document the tool in [TBD: internal tools registry]
|
|
136
|
+
|
|
137
|
+
## 5. Rationale
|
|
138
|
+
|
|
139
|
+
Why server-side only?
|
|
140
|
+
|
|
141
|
+
- Local machines are harder to audit and secure
|
|
142
|
+
- Reduces blast radius if a developer's machine is compromised, or person leaves Doist
|
|
143
|
+
- Easier to enforce consistent filtering and logging
|
|
144
|
+
|
|
145
|
+
Why attempt to filter before LLM calls?
|
|
146
|
+
|
|
147
|
+
- Some third-party providers may not be covered by our DPA for user data processing
|
|
148
|
+
- Even "private" API calls may be logged, used for training, or accessed by provider staff
|
|
149
|
+
- Regulatory compliance (GDPR, SOC 2) requires controlling data flows and applying the principle of minimization
|
|
150
|
+
|
|
151
|
+
Why approved providers only?
|
|
152
|
+
|
|
153
|
+
- Ensures we have appropriate agreements in place
|
|
154
|
+
- Maintains consistency in security posture
|
|
155
|
+
- Simplifies compliance audits
|
|
156
|
+
|
|
157
|
+
**Decision Tree: Can I Build This?**
|
|
158
|
+
|
|
159
|
+
```text
|
|
160
|
+
Does your tool access user data?
|
|
161
|
+
├── No → Build it, no special approval needed
|
|
162
|
+
└── Yes → Does it send data to an LLM?
|
|
163
|
+
├── No → Follow standard data access policies
|
|
164
|
+
└── Yes →
|
|
165
|
+
├── Is it server-side only?
|
|
166
|
+
│ └── No → STOP. Redesign as server-side.
|
|
167
|
+
├── Is PII filtered before LLM calls?
|
|
168
|
+
│ └── No → STOP. Add filtering.
|
|
169
|
+
└── Yes to both → Get Platform approval, add logging, ship it.
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
## Examples
|
|
173
|
+
|
|
174
|
+
```mermaid
|
|
175
|
+
flowchart TD
|
|
176
|
+
subgraph LOCAL["🖥️ Local Computer"]
|
|
177
|
+
Agent["Claude Code / Agent"]
|
|
178
|
+
Tool["Local CLI"]
|
|
179
|
+
Agent --> Tool
|
|
180
|
+
end
|
|
181
|
+
|
|
182
|
+
subgraph SERVER["☁️ Server Side"]
|
|
183
|
+
BedrockAPI["Bedrock API"]
|
|
184
|
+
|
|
185
|
+
subgraph DataSources["Data Sources"]
|
|
186
|
+
DD["DD"]
|
|
187
|
+
Sentry["Sentry"]
|
|
188
|
+
Zendesk["Zendesk"]
|
|
189
|
+
end
|
|
190
|
+
|
|
191
|
+
Service["Service"]
|
|
192
|
+
Service --> DataSources
|
|
193
|
+
end
|
|
194
|
+
|
|
195
|
+
subgraph NonSensitive["🔓 Non-Sensitive Data Sources"]
|
|
196
|
+
Github["Github"]
|
|
197
|
+
Twist["Twist"]
|
|
198
|
+
Outline["Outline"]
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
Tool --> Service --> BedrockAPI
|
|
202
|
+
Agent --> NonSensitive
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
> Note: see [`examples/internal-ai-service/`](https://github.com/Doist/standards/tree/main/examples/internal-ai-service/) for the server-side service and [`examples/local-cli/`](https://github.com/Doist/standards/tree/main/examples/local-cli/) for the thin local CLI. The example implements Zendesk only; the same pattern applies to Sentry/Datadog.
|
|
206
|
+
|
|
207
|
+
### How to read this diagram
|
|
208
|
+
|
|
209
|
+
Think of the diagram as three separate rooms, each with different access rules:
|
|
210
|
+
|
|
211
|
+
- **Your laptop** — where you work. The AI assistant lives here, but it can only see non-sensitive information directly.
|
|
212
|
+
- **Your local CLI** — a thin relay that sends your question to the secure server and prints the answer. It never touches raw user data.
|
|
213
|
+
- **The secure server** — a locked room in the cloud. All sensitive data and AI calls happen here, never on anyone's personal machine.
|
|
214
|
+
- **Non-sensitive sources** — services like Github or Outline whose content is safe to access directly, since they don't hold user personal information.
|
|
215
|
+
|
|
216
|
+
> Note: the developer's free-text question is forwarded as-is. Avoid including PII (e.g. email addresses) in your questions.
|
|
217
|
+
|
|
218
|
+
**How a request travels:**
|
|
219
|
+
|
|
220
|
+
- When a task involves user data (e.g. "what's going on with this support ticket?"), it goes: your laptop → secure server → data is fetched and cleaned → AI is asked → answer comes back to your laptop.
|
|
221
|
+
- When a task doesn't involve user data (e.g. "find the relevant code for this feature"), the AI reads Github or Outline directly — no server needed.
|
|
222
|
+
|
|
223
|
+
> Authentication in this example is handled entirely by Tailscale — only machines on the internal network can reach the service. No additional API key is needed.
|
|
224
|
+
|
|
225
|
+
---
|
|
226
|
+
|
|
227
|
+
### Migration example: from local PII access to the compliant architecture
|
|
228
|
+
|
|
229
|
+
**Scenario:** You have Claude Code set up on your laptop so it can query Zendesk, Sentry, and Datadog directly. It's convenient, but it's not compliant — user emails, names, and ticket content are flowing through your personal machine and being sent to an external AI without any filtering.
|
|
230
|
+
|
|
231
|
+
**Before (non-compliant):** The AI on your laptop connects directly to Zendesk and other sensitive tools, reads raw data including personal details, and sends all of it to an external AI service.
|
|
232
|
+
|
|
233
|
+
**After (compliant):** The AI on your laptop connects only to an internal service. That service fetches the data, removes any personal details, and handles the AI call — your laptop only ever sees the final, cleaned-up answer.
|
|
234
|
+
|
|
235
|
+
**What to change:**
|
|
236
|
+
|
|
237
|
+
1. **Remove** the direct connections to Zendesk, Sentry, and Datadog from your local AI setup.
|
|
238
|
+
|
|
239
|
+
2. **Set up an internal service** (the Platform team can help) that fetches and cleans data before passing it to the AI — see [`examples/internal-ai-service/main.go`](https://github.com/Doist/standards/tree/main/examples/internal-ai-service/main.go) for a working reference. It should only return what's needed to answer the question — never raw personal information.
|
|
240
|
+
|
|
241
|
+
3. **Point your local tool at that internal service** instead of at the third-party tools directly — see [`examples/local-cli/main.go`](https://github.com/Doist/standards/tree/main/examples/local-cli/main.go) for a working reference.
|
|
242
|
+
|
|
243
|
+
4. **Keep your Github, Twist, and Outline connections as-is** — those don't hold user personal data, so they're fine to use directly.
|
|
244
|
+
|
|
245
|
+
The result: your AI assistant works just as well, but sensitive data never leaves the secure server — and there's a full audit trail of everything that happened.
|
|
246
|
+
|
|
247
|
+
---
|
|
248
|
+
|
|
249
|
+
## 6. References
|
|
250
|
+
|
|
251
|
+
- [Data Classification Policy](https://handbook.doist.com/doc/data-classification-policy-t7mCBRgaec)
|
|
252
|
+
- [Encryption Policy](https://handbook.doist.com/doc/encryption-policy-zJ5ZApYT30)
|
|
253
|
+
|
|
254
|
+
(add list of approved LLMs)
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Standard: Backend Repositories
|
|
2
|
+
|
|
3
|
+
| 📄 Document Type | Team Standard (Backend) |
|
|
4
|
+
| :--------------- | :------------------------------------------------------------------------------------------------------------------------ |
|
|
5
|
+
| 🎯 Status | Draft |
|
|
6
|
+
| 👤 Owner | Felipe Rodrigues |
|
|
7
|
+
| 📅 Last Reviewed | 2026-09-08 |
|
|
8
|
+
| 🗂️ References | [Standard: Service Production Readiness](https://handbook.doist.com/doc/standard-service-production-readiness-qgNRng8TJx) |
|
|
9
|
+
| 📌 Version | 0.2 |
|
|
10
|
+
|
|
11
|
+
This standard applies to **all backend repositories at Doist, regardless of language or current ownership.** Today that means `Todoist` and other repositories already owned by the Backend team as well as newer backend services such as `todoist-id` and `automations` - currently developed by squads, and expected to come under Backend team ownership over time.
|
|
12
|
+
|
|
13
|
+
Meeting this standard is part of what makes that handover possible.
|
|
14
|
+
|
|
15
|
+
## Tests
|
|
16
|
+
|
|
17
|
+
Every repository must have automated tests. A repository with no tests is not production-ready.
|
|
18
|
+
|
|
19
|
+
- Tests must run in CI on every pull request.
|
|
20
|
+
- Tests must run in CI on every merge to `main`.
|
|
21
|
+
- Deployments must be gated on tests passing. In GitHub Actions, the deploy job must depend on the test job — via `needs: test` in the same workflow, or by invoking the test workflow as a reusable workflow and depending on that job. A deploy that races its own test run is not gated.
|
|
22
|
+
|
|
23
|
+
## Branch Protection
|
|
24
|
+
|
|
25
|
+
Tests that exist but are never enforced provide false confidence. The `main` branch must be protected:
|
|
26
|
+
|
|
27
|
+
- Changes land via pull request — no direct pushes to `main`.
|
|
28
|
+
- Test, lint, and type-check jobs are configured as **required status checks**, so a PR cannot merge while they fail or haven't run.
|
|
29
|
+
|
|
30
|
+
## Code Quality Hooks
|
|
31
|
+
|
|
32
|
+
- Pre-commit hooks must be used to enforce formatting, linting, and repo hygiene. The runner is language-specific: [prek](https://prek.j178.dev) in Python repositories, [husky](https://typicode.github.io/husky/) in JavaScript/TypeScript ones.
|
|
33
|
+
- The same checks must also run in CI on every pull request and merge to `main` (e.g. `prek run --all-files`, or the `npm` scripts the hooks call). Hooks that only run locally are advisory; developers can skip or misconfigure them.
|
|
34
|
+
|
|
35
|
+
## README
|
|
36
|
+
|
|
37
|
+
Every repository must have a `README.md` with at least:
|
|
38
|
+
|
|
39
|
+
- Project description
|
|
40
|
+
- How to run the project locally
|
|
41
|
+
- How to run tests
|
|
42
|
+
- How to deploy the project
|
|
43
|
+
|
|
44
|
+
## Dependency Updates
|
|
45
|
+
|
|
46
|
+
Automated dependency updates (Renovate or Dependabot) must be enabled. Update PRs go through the same CI gates as any other change.
|
|
47
|
+
|
|
48
|
+
## Python Requirements
|
|
49
|
+
|
|
50
|
+
Python repositories must use the following stack (mirroring the main Todoist codebase):
|
|
51
|
+
|
|
52
|
+
| Concern | Tool |
|
|
53
|
+
| ------------------------------- | ---------------------------- |
|
|
54
|
+
| Project & dependency management | `uv` |
|
|
55
|
+
| Minimum Python version | 3.13 |
|
|
56
|
+
| Tests | `pytest` |
|
|
57
|
+
| Linting & formatting | `ruff check` + `ruff format` |
|
|
58
|
+
| Git hooks | `prek` |
|
|
59
|
+
| Type checking | `mypy` and `pyrefly` |
|
|
60
|
+
| Dead code detection | `vulture` |
|
|
61
|
+
|
|
62
|
+
- Dependencies in `pyproject.toml` must be pinned with upper bounds (`>=X.Y,<X+1`).
|
|
63
|
+
- Type hints are mandatory on production code.
|
|
64
|
+
|
|
65
|
+
## JavaScript / TypeScript Requirements
|
|
66
|
+
|
|
67
|
+
JavaScript and TypeScript repositories must use the following stack (mirroring `automations`, our newest backend TS project):
|
|
68
|
+
|
|
69
|
+
| Concern | Tool |
|
|
70
|
+
| ------------------------------- | ----------------------------------------------- |
|
|
71
|
+
| Language | TypeScript |
|
|
72
|
+
| Runtime | Node.js — current LTS major |
|
|
73
|
+
| Project & dependency management | `npm` (workspaces for monorepos) |
|
|
74
|
+
| Tests | `vitest` |
|
|
75
|
+
| Linting & formatting | `oxlint` + `oxfmt` |
|
|
76
|
+
| Git hooks | `husky` |
|
|
77
|
+
| Type checking | `tsc --noEmit`, exposed as `npm run type-check` |
|
|
78
|
+
|
|
79
|
+
- Production code must be TypeScript. Plain JavaScript only where a tool requires it.
|
|
80
|
+
- The Node version must be pinned in `engines` and in a version file (`.node-version`, which `actions/setup-node` reads via `node-version-file`; `.nvmrc` alongside it if people use `nvm`). Version files pin Node only, so pin the npm major in `engines` as well.
|
|
81
|
+
- `engine-strict=true` must be set in `.npmrc`. Without it npm only warns on an `engines` mismatch, and the pins above do not actually stop local, CI and container builds from drifting apart.
|
|
82
|
+
- Dependencies in `package.json` must be pinned to exact versions, with the lockfile committed.
|
|
83
|
+
- The TypeScript config must extend the shared `@doist/tsconfig` base and keep `strict` on.
|
|
84
|
+
|
|
85
|
+
## Related Documents
|
|
86
|
+
|
|
87
|
+
This standard complements the Doist-wide [Service Production Readiness](https://handbook.doist.com/doc/standard-service-production-readiness-qgNRng8TJx) standard: that one covers how a service must _behave_ in production while this one covers the baseline health a _repository_ must have before its code reaches production.
|
|
88
|
+
|
|
89
|
+
Related engineering-wide guidelines:
|
|
90
|
+
|
|
91
|
+
- [GitHub Repository Guidelines](https://handbook.doist.com/doc/github-repository-guidelines-iJNZ0D2vLN)
|
|
92
|
+
- [Automate linting and formatting](https://handbook.doist.com/doc/automate-linting-and-formatting-omSECLKiJI)
|
|
93
|
+
- [Keeping dependencies up-to-date](https://handbook.doist.com/doc/keeping-dependencies-up-to-date-Qf28E929N5)
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Open questions/Possible future expansions to the Standard
|
|
98
|
+
|
|
99
|
+
- Type checkers: Todoist currently runs `mypy`, `ty`, and `pyrefly` in parallel, but that is an evaluation state, not a recommendation (see [Choosing type checkers for Todoist](https://comms.todoist.com/69/ch/BH4VumxwScG1NAiomtcBq/t/CbVia3jW38koSRmHWXSeV/)). A June 2026 attempt to remove mypy was reverted after it left real gaps (pyrefly's legacy config disabled core checks; ty lacks Pydantic support). Current direction: pyrefly in strict mode is the candidate primary, mypy stays as the mature baseline, ty is a secondary bet pending Pydantic support. `mypy` + `pyrefly` therefore remains the right baseline for new repos; revisit when ty matures.
|
|
100
|
+
- Maybe create a [Copier](https://copier.readthedocs.io/en/stable/) template for Python repos — and an equivalent for TS ones — so new services start compliant? Or even a small template repo with “sane defaults” for our tooling.
|
|
101
|
+
- Dead code detection has no JS/TS entry yet. Python repos run `vulture`; `knip` is the closest equivalent, but nothing runs it today.
|
|
102
|
+
- Define the baseline ruff rule set / prek hook list based on the Todoist codebase (its `prek.toml` is a good starting point: ruff, standard pre-commit-hooks, shellcheck, hadolint, mypy/type checkers, vulture on pre-push).
|