@todoist/doistbot-cli 0.0.0-stage → 1.0.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/actions/auth.js +225 -0
- package/dist/actions/doctor.js +39 -0
- package/dist/actions/review-usage.js +25 -0
- package/dist/actions/review.js +311 -0
- package/dist/actions/setup.js +14 -0
- package/dist/auth.js +202 -0
- package/dist/config.js +145 -0
- package/dist/env.js +10 -0
- package/dist/file-store.js +112 -0
- package/dist/index.js +164 -0
- package/dist/input.js +40 -0
- package/dist/runtime.js +105 -0
- package/dist/secure-store.js +148 -0
- package/dist/terminal.js +185 -0
- package/dist/test-utils/secure-store-mocks.js +29 -0
- package/dist/update-cache-entrypoint.js +16 -0
- package/dist/update-check.js +235 -0
- package/package.json +65 -5
- package/sandbox/_node_modules/doistbot-logging/dist/index.d.ts +23 -0
- package/sandbox/_node_modules/doistbot-logging/dist/index.js +51 -0
- package/sandbox/_node_modules/doistbot-logging/package.json +12 -0
- package/sandbox/_node_modules/doistbot-repo-config/dist/index.d.ts +93 -0
- package/sandbox/_node_modules/doistbot-repo-config/dist/index.js +197 -0
- package/sandbox/_node_modules/doistbot-repo-config/package.json +12 -0
- package/sandbox/dist/core/autofix-resources.js +58 -0
- package/sandbox/dist/core/autofix-service-query.js +69 -0
- package/sandbox/dist/core/autofix-test-policy.js +11 -0
- package/sandbox/dist/core/autofix-verification.js +355 -0
- package/sandbox/dist/core/bot-logins.js +14 -0
- package/sandbox/dist/core/check-run.js +158 -0
- package/sandbox/dist/core/datadog-metrics.js +113 -0
- package/sandbox/dist/core/environment.js +49 -0
- package/sandbox/dist/core/git-auth.js +17 -0
- package/sandbox/dist/core/github-history.js +4 -0
- package/sandbox/dist/core/github-issues.js +18 -0
- package/sandbox/dist/core/label-parsing.js +1 -0
- package/sandbox/dist/core/langfuse-observation.js +33 -0
- package/sandbox/dist/core/langfuse-session.js +39 -0
- package/sandbox/dist/core/logger-span-processor.js +90 -0
- package/sandbox/dist/core/logging.js +18 -0
- package/sandbox/dist/core/pi-autofix-extension.js +188 -0
- package/sandbox/dist/core/pi-execution-limits-extension.js +79 -0
- package/sandbox/dist/core/pi-progress-logging.js +51 -0
- package/sandbox/dist/core/pi-tool-output-filter-extension.js +65 -0
- package/sandbox/dist/core/pi.js +870 -0
- package/sandbox/dist/core/pii-scrubber.js +31 -0
- package/sandbox/dist/core/prompt-template.js +63 -0
- package/sandbox/dist/core/reactions.js +36 -0
- package/sandbox/dist/core/regexp.js +3 -0
- package/sandbox/dist/core/repository-map.js +1 -0
- package/sandbox/dist/core/repository-resolver.js +1 -0
- package/sandbox/dist/core/retry.js +29 -0
- package/sandbox/dist/core/s3-utils.js +42 -0
- package/sandbox/dist/core/shared.js +124 -0
- package/sandbox/dist/core/snapshot-helpers.js +17 -0
- package/sandbox/dist/core/span-test-helpers.js +13 -0
- package/sandbox/dist/core/tool-environment.js +83 -0
- package/sandbox/dist/core/tool-output.js +18 -0
- package/sandbox/dist/core/tracing-setup.js +92 -0
- package/sandbox/dist/core/tracing.js +81 -0
- package/sandbox/dist/main.js +51 -0
- package/sandbox/dist/providers/config.js +23 -0
- package/sandbox/dist/providers/credentials.js +40 -0
- package/sandbox/dist/providers/gemini.js +28 -0
- package/sandbox/dist/providers/helpers.js +253 -0
- package/sandbox/dist/providers/index.js +27 -0
- package/sandbox/dist/providers/openai.js +23 -0
- package/sandbox/dist/providers/openrouter-pi.js +25 -0
- package/sandbox/dist/providers/pi-review.js +301 -0
- package/sandbox/dist/providers/process.js +34 -0
- package/sandbox/dist/providers/schemas.js +61 -0
- package/sandbox/dist/providers/types.js +4 -0
- package/sandbox/dist/tasks/chat/chat.js +440 -0
- package/sandbox/dist/tasks/chat/prompt.js +20 -0
- package/sandbox/dist/tasks/issue-fix-retry/fix-retry.js +304 -0
- package/sandbox/dist/tasks/issue-summarize/context.js +21 -0
- package/sandbox/dist/tasks/issue-summarize/media.js +335 -0
- package/sandbox/dist/tasks/issue-summarize/model.js +230 -0
- package/sandbox/dist/tasks/issue-summarize/output.js +45 -0
- package/sandbox/dist/tasks/issue-summarize/prompt.js +11 -0
- package/sandbox/dist/tasks/issue-summarize/summarize.js +238 -0
- package/sandbox/dist/tasks/issue-summarize/thread.js +123 -0
- package/sandbox/dist/tasks/issue-summarize/types.js +1 -0
- package/sandbox/dist/tasks/issue-triage/assignee-review.js +114 -0
- package/sandbox/dist/tasks/issue-triage/autofix-capabilities.js +22 -0
- package/sandbox/dist/tasks/issue-triage/autofix-checkpoint.js +122 -0
- package/sandbox/dist/tasks/issue-triage/autofix-policy.js +36 -0
- package/sandbox/dist/tasks/issue-triage/clone.js +40 -0
- package/sandbox/dist/tasks/issue-triage/context.js +57 -0
- package/sandbox/dist/tasks/issue-triage/diff-validator.js +102 -0
- package/sandbox/dist/tasks/issue-triage/fix-attempt.js +617 -0
- package/sandbox/dist/tasks/issue-triage/fix-context.js +5 -0
- package/sandbox/dist/tasks/issue-triage/fix-dispatch.js +263 -0
- package/sandbox/dist/tasks/issue-triage/fix-failure-reasons.js +2 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/effects.js +526 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/effects.types.js +1 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/fake-effects.js +65 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/promotion-task.js +108 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/recovery.js +512 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/runners.js +233 -0
- package/sandbox/dist/tasks/issue-triage/fix-loop/state-machine.js +171 -0
- package/sandbox/dist/tasks/issue-triage/github-request.js +2 -0
- package/sandbox/dist/tasks/issue-triage/hero-group-map.js +206 -0
- package/sandbox/dist/tasks/issue-triage/markers.js +6 -0
- package/sandbox/dist/tasks/issue-triage/model.js +124 -0
- package/sandbox/dist/tasks/issue-triage/output.js +164 -0
- package/sandbox/dist/tasks/issue-triage/pr-creator.js +585 -0
- package/sandbox/dist/tasks/issue-triage/pr-readiness.js +25 -0
- package/sandbox/dist/tasks/issue-triage/prompt.js +63 -0
- package/sandbox/dist/tasks/issue-triage/repo-config.js +47 -0
- package/sandbox/dist/tasks/issue-triage/repo-prompts.js +77 -0
- package/sandbox/dist/tasks/issue-triage/report-fixture.js +53 -0
- package/sandbox/dist/tasks/issue-triage/report-history.js +113 -0
- package/sandbox/dist/tasks/issue-triage/report-schema.js +63 -0
- package/sandbox/dist/tasks/issue-triage/report.js +111 -0
- package/sandbox/dist/tasks/issue-triage/resolution.js +30 -0
- package/sandbox/dist/tasks/issue-triage/routing.js +169 -0
- package/sandbox/dist/tasks/issue-triage/sandbox-environment.js +59 -0
- package/sandbox/dist/tasks/issue-triage/shared.js +86 -0
- package/sandbox/dist/tasks/issue-triage/side-effects.js +102 -0
- package/sandbox/dist/tasks/issue-triage/triage.js +442 -0
- package/sandbox/dist/tasks/issue-triage/types.js +1 -0
- package/sandbox/dist/tasks/persist-logs/context.js +19 -0
- package/sandbox/dist/tasks/persist-logs/dd-proxy.js +116 -0
- package/sandbox/dist/tasks/persist-logs/llm-query-planner.js +156 -0
- package/sandbox/dist/tasks/persist-logs/persist-logs.js +491 -0
- package/sandbox/dist/tasks/refresh-s3-urls/refresh-s3-urls.js +214 -0
- package/sandbox/dist/tasks/review/check-run.js +326 -0
- package/sandbox/dist/tasks/review/comment-mapper.js +46 -0
- package/sandbox/dist/tasks/review/conversation-context.js +477 -0
- package/sandbox/dist/tasks/review/diff.js +95 -0
- package/sandbox/dist/tasks/review/engines/dedupe.js +291 -0
- package/sandbox/dist/tasks/review/engines/multi-focus.js +419 -0
- package/sandbox/dist/tasks/review/engines/shared.js +87 -0
- package/sandbox/dist/tasks/review/engines/types.js +1 -0
- package/sandbox/dist/tasks/review/finding-output.js +18 -0
- package/sandbox/dist/tasks/review/format.js +17 -0
- package/sandbox/dist/tasks/review/github-comment-format.js +94 -0
- package/sandbox/dist/tasks/review/local-git.js +374 -0
- package/sandbox/dist/tasks/review/local.js +118 -0
- package/sandbox/dist/tasks/review/multi-focus-prompt.js +59 -0
- package/sandbox/dist/tasks/review/prompt.js +81 -0
- package/sandbox/dist/tasks/review/review-feedback.js +48 -0
- package/sandbox/dist/tasks/review/review-mode-github.js +58 -0
- package/sandbox/dist/tasks/review/review-mode-policy.js +232 -0
- package/sandbox/dist/tasks/review/review-mode-types.js +1 -0
- package/sandbox/dist/tasks/review/review-mode.js +296 -0
- package/sandbox/dist/tasks/review/review-summary.js +123 -0
- package/sandbox/dist/tasks/review/review.js +391 -0
- package/sandbox/dist/tasks/review/runner.js +255 -0
- package/sandbox/dist/tasks/review/severity.js +5 -0
- package/sandbox/dist/tasks/review/summary-model.js +271 -0
- package/sandbox/dist/tasks/review/thinking-level.js +14 -0
- package/sandbox/dist/tasks/review/translation-review-policy.js +37 -0
- package/sandbox/dist/tasks/review/types.js +1 -0
- package/sandbox/dist/tasks/review-slice-plan/context.js +66 -0
- package/sandbox/dist/tasks/review-slice-plan/plan.js +210 -0
- package/sandbox/dist/tasks/review-slice-plan/prompt.js +59 -0
- package/sandbox/dist/tasks/review-slice-plan/review-slice-plan.js +367 -0
- package/sandbox/dist/tasks/sentry-triage/store.js +133 -0
- package/sandbox/dist/tasks/sentry-triage/triage.js +217 -0
- package/sandbox/node_modules/doistbot-logging/dist/index.d.ts +23 -0
- package/sandbox/node_modules/doistbot-logging/dist/index.js +51 -0
- package/sandbox/node_modules/doistbot-logging/package.json +12 -0
- package/sandbox/node_modules/doistbot-repo-config/dist/index.d.ts +93 -0
- package/sandbox/node_modules/doistbot-repo-config/dist/index.js +197 -0
- package/sandbox/node_modules/doistbot-repo-config/package.json +12 -0
- package/sandbox/src/review/prompts/ai-internal-tools.md +254 -0
- package/sandbox/src/review/prompts/backend-repository-standards.md +102 -0
- package/sandbox/src/review/prompts/observability-standard.md +169 -0
- package/sandbox/src/review/prompts/outline-map.json +26 -0
- package/sandbox/src/review/prompts/secrets-management.md +286 -0
- package/sandbox/src/review/prompts/service-production-readiness.md +122 -0
- package/sandbox/src/review/prompts/workload-placement.md +345 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/efficiency.md +16 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/general.md +32 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/quality.md +20 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/reuse.md +13 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/security.md +18 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/standards.md +21 -0
- package/sandbox/src/tasks/review/prompts/review-focus-prompts/tests.md +19 -0
- package/sandbox/src/tasks/review/prompts/review-mode.md +40 -0
- package/sandbox/src/tasks/review/prompts/review-multi-focus-base-prompt.md +142 -0
- package/scripts/restore-runtime-assets.mjs +21 -0
- package/README.md +0 -3
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
# Standard: Observability
|
|
2
|
+
|
|
3
|
+
| 📄 Document Type | Standard |
|
|
4
|
+
| :--------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
5
|
+
| 🎯 Status | Approved |
|
|
6
|
+
| 👤 Owner | Luciano Facchinelli |
|
|
7
|
+
| 📅 Last Reviewed | 2026-03-20 |
|
|
8
|
+
| 🗂️ References | [Datadog Documentation](https://docs.datadoghq.com/), [Sentry Documentation](https://docs.sentry.io/), [OpenTelemetry Specification](https://opentelemetry.io/docs/) |
|
|
9
|
+
| 📌 Version | 1.0 |
|
|
10
|
+
|
|
11
|
+
This document defines the standard approach for implementing observability in new projects at Doist. It is based on patterns and tools established across our production systems including Todoist, Twist, and Agentist.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Overview
|
|
16
|
+
|
|
17
|
+
Observability at Doist is built on three core pillars: **Logs**, **Metrics**, and **Traces**. We supplement these with **Error Tracking** to provide complete visibility into our systems.
|
|
18
|
+
|
|
19
|
+
### Guiding Principles
|
|
20
|
+
|
|
21
|
+
- **Consistency**: Use the same tools and patterns across all projects
|
|
22
|
+
- **Structured Data**: Always use structured logging (JSON format)
|
|
23
|
+
- **Correlation**: Ensure logs, metrics, and traces can be correlated via trace IDs
|
|
24
|
+
- **Actionable Alerts**: Only alert on conditions that require human intervention or can be automated by leveraging LLMs
|
|
25
|
+
- **Privacy First**: Never log PII or sensitive user data. Reuse existing sanitization patterns (see [Existing Implementations](#existing-pii-sanitization-implementations))
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
## Core Pillars
|
|
30
|
+
|
|
31
|
+
### 1. Logging
|
|
32
|
+
|
|
33
|
+
Structured logging is mandatory for all services. Logs should:
|
|
34
|
+
|
|
35
|
+
- Use JSON format for machine parsability (single-line output is preferred where the log shipper supports it, but not strictly required)
|
|
36
|
+
- Include correlation IDs for request tracing
|
|
37
|
+
- Use appropriate log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
|
|
38
|
+
- Do not include timestamps — these are added by the logging system
|
|
39
|
+
- Avoid color codes, emojis, and decorative output
|
|
40
|
+
- Never contain sensitive data (passwords, tokens, PII) — use existing sanitization libraries where available (see [Existing Implementations](#existing-pii-sanitization-implementations))
|
|
41
|
+
|
|
42
|
+
### 2. Metrics
|
|
43
|
+
|
|
44
|
+
Metrics provide quantitative measurements of system behavior:
|
|
45
|
+
|
|
46
|
+
- **RED Metrics** (for services): Rate, Errors, Duration
|
|
47
|
+
- **USE Metrics** (for resources): Utilization, Saturation, Errors
|
|
48
|
+
- **Custom business metrics as needed**
|
|
49
|
+
|
|
50
|
+
### 3. Traces
|
|
51
|
+
|
|
52
|
+
Distributed tracing tracks requests across service boundaries:
|
|
53
|
+
|
|
54
|
+
- All inter-service calls must propagate trace context
|
|
55
|
+
- Use OpenTelemetry-compatible instrumentation like OpenTelemetry SDKs or Datadog APM libraries
|
|
56
|
+
- Sample appropriately for high-traffic services
|
|
57
|
+
|
|
58
|
+
> **Note**: For services running on AWS, the canonical trace entry point is the Application Load Balancer (ALB). The ALB automatically adds the `X-Amzn-Trace-Id` header to every incoming request, which your service should read and propagate to downstream calls. This header is compatible with OpenTelemetry. For services running on GCP, the canonical trace entry point will be defined in a future version of this standard.
|
|
59
|
+
|
|
60
|
+
### 4. Error Tracking
|
|
61
|
+
|
|
62
|
+
Dedicated error tracking provides detailed crash and exception analysis:
|
|
63
|
+
|
|
64
|
+
- Capture stack traces with source maps
|
|
65
|
+
- Group similar errors automatically
|
|
66
|
+
- Track error frequency and affected users
|
|
67
|
+
|
|
68
|
+
### 5. Real User Monitoring (RUM) _(optional)_
|
|
69
|
+
|
|
70
|
+
Datadog RUM is available but carries significant cost and is not recommended for most applications. Prefer lighter-weight alternatives already in use at Doist:
|
|
71
|
+
|
|
72
|
+
- **Web apps**: Use log-based metrics in Datadog (see [interaction-trace](https://github.com/Doist/interaction-trace) for reference)
|
|
73
|
+
- **Landing pages**: Use [RequestMetrics](https://requestmetrics.com) for Core Web Vitals and performance tracking
|
|
74
|
+
- **Mobile**: Evaluate tooling based on platform needs
|
|
75
|
+
|
|
76
|
+
If adopted, RUM should cover page/screen load times, user interactions, and Core Web Vitals (web) or app performance (mobile).
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Tool Stack
|
|
81
|
+
|
|
82
|
+
### Primary Tools
|
|
83
|
+
|
|
84
|
+
| Purpose | Tool | Usage |
|
|
85
|
+
| ------------------ | ------------------------ | ------------------------------------------ |
|
|
86
|
+
| APM & Tracing | **Datadog** | Distributed tracing, APM, metrics |
|
|
87
|
+
| Error Tracking | **Sentry** | Exception monitoring, crash reporting |
|
|
88
|
+
| Metrics Collection | **Datadog + CloudWatch** | Infrastructure and application metrics |
|
|
89
|
+
| Log Aggregation | **Datadog Logs** | Centralized log management |
|
|
90
|
+
| Feature Flags | **Flagsmith** | Feature flag management with observability |
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Configuration Standards
|
|
95
|
+
|
|
96
|
+
### Naming Conventions
|
|
97
|
+
|
|
98
|
+
- **Service Names**: lowercase, hyphen-separated (e.g., `todoist-api`, `twist-web`)
|
|
99
|
+
- **Metric Names**: lowercase, dot-separated (e.g., `api.request.duration`)
|
|
100
|
+
- **Log Fields**: snake_case (e.g., `user_id`, `request_id`)
|
|
101
|
+
|
|
102
|
+
### Required Tags/Labels
|
|
103
|
+
|
|
104
|
+
All metrics and traces must include:
|
|
105
|
+
|
|
106
|
+
```
|
|
107
|
+
service: <service-name>
|
|
108
|
+
env: <environment>
|
|
109
|
+
version: <app-version>
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Use the following canonical values for `env`:
|
|
113
|
+
|
|
114
|
+
| Environment | Tag value |
|
|
115
|
+
| ----------- | ------------- |
|
|
116
|
+
| Production | `production` |
|
|
117
|
+
| Staging | `staging` |
|
|
118
|
+
| Development | `development` |
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## Health Checks
|
|
123
|
+
|
|
124
|
+
Health check requirements are defined in the Service Production Readiness standard, which is the canonical reference. Follow that standard for `/health` endpoint definitions, and startup grace periods.
|
|
125
|
+
|
|
126
|
+
## Existing PII Sanitization Implementations
|
|
127
|
+
|
|
128
|
+
Before writing custom data-scrubbing logic, check these existing implementations first — they cover the most common patterns and are already battle-tested in production.
|
|
129
|
+
|
|
130
|
+
If you implement a new sanitizer or modify an existing one with improvements, please consider contributing it back to the original repository for reuse across projects.
|
|
131
|
+
|
|
132
|
+
### Python — `ist_libs` (shared library)
|
|
133
|
+
|
|
134
|
+
[`python/parts/logs/sanitizers.py`](https://github.com/Doist/ist_libs/blob/main/python/parts/logs/sanitizers.py) in `Doist/ist_libs`
|
|
135
|
+
|
|
136
|
+
### TypeScript — `todoist-web`
|
|
137
|
+
|
|
138
|
+
[`src/05-common/logger/utils.ts`](https://github.com/Doist/todoist-web/blob/main/src/05-common/logger/utils.ts) and [`sentry.ts`](https://github.com/Doist/todoist-web/blob/main/src/05-common/logger/sentry.ts) in `Doist/todoist-web`
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Common Metrics Reference
|
|
143
|
+
|
|
144
|
+
Real metric names from existing Doist services to use as inspiration when naming metrics in new services.
|
|
145
|
+
|
|
146
|
+
### Backend — `Doist/Todoist` (Datadog StatsD)
|
|
147
|
+
|
|
148
|
+
- `todoist.sync_api.all_sync_lengths` (Distribution)
|
|
149
|
+
- `todoist.sqs_workers.processing_duration` / `message_age_seconds` / `first_receive_latency_seconds` (Distributions)
|
|
150
|
+
- `todoist.project_health.analysis.*` — `success`, `error`, `queued`, `rescheduled`, `abandoned`
|
|
151
|
+
- `todoist.rate_limit.sync.<group>.activated`
|
|
152
|
+
- `todoist.audit_log.change` (tagged with `table` / `column` / `action`)
|
|
153
|
+
|
|
154
|
+
### Mobile — `Doist/todoist-ios` (Firebase GA4)
|
|
155
|
+
|
|
156
|
+
- `task_added` / `task_completed`
|
|
157
|
+
- `login` / `sign_up`
|
|
158
|
+
- `screen_view` / `dialog_view`
|
|
159
|
+
- `notification_opt_in` / `notification_opt_out`
|
|
160
|
+
- `widget_add` / `widget_remove`
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## References
|
|
165
|
+
|
|
166
|
+
- [Datadog Documentation](https://docs.datadoghq.com/)
|
|
167
|
+
- [Sentry Documentation](https://docs.sentry.io/)
|
|
168
|
+
- [OpenTelemetry Specification](https://opentelemetry.io/docs/)
|
|
169
|
+
- [Flagsmith Documentation](https://docs.flagsmith.com/)
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
{
|
|
2
|
+
"workload-placement.md": {
|
|
3
|
+
"id": "",
|
|
4
|
+
"url": "https://handbook.doist.com/doc/standard-workload-placement-"
|
|
5
|
+
},
|
|
6
|
+
"secrets-management.md": {
|
|
7
|
+
"id": "",
|
|
8
|
+
"url": "https://handbook.doist.com/doc/standard-secrets-management-"
|
|
9
|
+
},
|
|
10
|
+
"ai-internal-tools.md": {
|
|
11
|
+
"id": "",
|
|
12
|
+
"url": "https://handbook.doist.com/doc/standard-internal-ai-tools-"
|
|
13
|
+
},
|
|
14
|
+
"service-production-readiness.md": {
|
|
15
|
+
"id": "",
|
|
16
|
+
"url": "https://handbook.doist.com/doc/standard-service-production-readiness-"
|
|
17
|
+
},
|
|
18
|
+
"observability-standard.md": {
|
|
19
|
+
"id": "",
|
|
20
|
+
"url": "https://handbook.doist.com/doc/standard-observability-"
|
|
21
|
+
},
|
|
22
|
+
"backend-repository-standards.md": {
|
|
23
|
+
"id": "",
|
|
24
|
+
"url": "https://handbook.doist.com/doc/standard-backend-repositories-"
|
|
25
|
+
}
|
|
26
|
+
}
|
|
@@ -0,0 +1,286 @@
|
|
|
1
|
+
# Standard: Secrets Management
|
|
2
|
+
|
|
3
|
+
| 📄 Document Type | Standard |
|
|
4
|
+
| :--------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
5
|
+
| 🎯 Status | Approved |
|
|
6
|
+
| 👤 Owner | Luciano Facchinelli |
|
|
7
|
+
| 📅 Last Reviewed | 2026-03-06 |
|
|
8
|
+
| 🗂️ References | [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/) [GitHub Actions OIDC with AWS](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services) |
|
|
9
|
+
| 📌 Version | 1.0 |
|
|
10
|
+
|
|
11
|
+
## Overview
|
|
12
|
+
|
|
13
|
+
This document defines the standard approach for managing secrets across all environments at Doist. Secrets include API keys, database credentials, encryption keys, tokens, certificates, and any other sensitive configuration data that must not be exposed in source code or logs.
|
|
14
|
+
|
|
15
|
+
### Principles
|
|
16
|
+
|
|
17
|
+
1. **Never commit secrets to version control** - All secrets must be excluded from repositories
|
|
18
|
+
2. **Least privilege access** - Services should only have access to the secrets they need
|
|
19
|
+
3. **Environment isolation** - Production secrets must never be used in development or testing
|
|
20
|
+
4. **Rotation capability** - Secrets should be stored in a way that allows rotation without code changes
|
|
21
|
+
5. **Audit trail** - Access to production secrets should be logged and auditable
|
|
22
|
+
|
|
23
|
+
### Scope
|
|
24
|
+
|
|
25
|
+
This standard applies to:
|
|
26
|
+
|
|
27
|
+
- Application configuration secrets
|
|
28
|
+
- Database credentials
|
|
29
|
+
- API keys and tokens (internal and third-party)
|
|
30
|
+
- Encryption keys and certificates
|
|
31
|
+
- Service-to-service authentication tokens
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## Security Requirements
|
|
36
|
+
|
|
37
|
+
### Prohibited Practices
|
|
38
|
+
|
|
39
|
+
- Committing unencrypted secrets to version control (even in private repositories)
|
|
40
|
+
- Logging secret values independently of their encoding
|
|
41
|
+
- Passing secrets via command-line arguments (visible in process lists)
|
|
42
|
+
- Storing production secrets in development environments
|
|
43
|
+
- Sharing unencrypted secrets via unencrypted channels (email, Slack, Twist)
|
|
44
|
+
- Using production secrets for testing
|
|
45
|
+
|
|
46
|
+
### Required Practices
|
|
47
|
+
|
|
48
|
+
- Use environment-specific secrets (dev, staging, prod)
|
|
49
|
+
- Implement secret rotation procedures
|
|
50
|
+
- Monitor for secret exposure (e.g., [GitHub secret scanning](https://github.com/Doist/scan-github-secrets/blob/main/README.md))
|
|
51
|
+
- Use short-lived credentials where possible
|
|
52
|
+
- Encrypt secrets in transit and at rest
|
|
53
|
+
- Document secret dependencies for each service
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Production Environment
|
|
58
|
+
|
|
59
|
+
All production secrets MUST be stored in a cloud-provider secret service. For AWS this is [AWS Secret Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html), for Google this is [Google Cloud secrets](https://docs.cloud.google.com/secret-manager/docs/overview).
|
|
60
|
+
|
|
61
|
+
This provides:
|
|
62
|
+
|
|
63
|
+
- Encryption at rest
|
|
64
|
+
- Fine-grained access control
|
|
65
|
+
- Automatic rotation capabilities
|
|
66
|
+
- For AWS Secret Manager, we can configure [automatic rotation for our secrets using AWS Lambda functions](https://docs.aws.amazon.com/secretsmanager/latest/userguide/reference_available-rotation-templates.html)
|
|
67
|
+
- For Google Cloud Secret Manager, we can [set up a rotation schedule and provide a Cloud Function to handle the rotation process](https://docs.cloud.google.com/secret-manager/docs/secret-rotation)
|
|
68
|
+
- Audit logging
|
|
69
|
+
|
|
70
|
+
Although we can have each of our secrets stored independently, in general we aim to bundle all of a service's secrets into a single JSON-structured secret. With this we can reduce complexity.
|
|
71
|
+
|
|
72
|
+
The only exemption to this rule is database secrets.
|
|
73
|
+
|
|
74
|
+
### Secret Naming Convention
|
|
75
|
+
|
|
76
|
+
When we create secrets for our apps, we need to ensure they follow a logic structure that allows us to easily identify them and their purpose.
|
|
77
|
+
|
|
78
|
+
**Examples:**
|
|
79
|
+
|
|
80
|
+
#### AWS Secret Manager
|
|
81
|
+
|
|
82
|
+
| Secret Path | Description |
|
|
83
|
+
| ------------------------------- | ------------------------------- |
|
|
84
|
+
| `prod/todoist/secrets` | Main application secrets bundle |
|
|
85
|
+
| `prod/todoist/rds-main` | Primary database credentials |
|
|
86
|
+
| `prod/todoist/rds-shard01` | Shard database credentials |
|
|
87
|
+
| `prod/todoist/apple-signin-key` | Apple Sign-In private key |
|
|
88
|
+
| `staging/todoist/secrets` | Staging secrets bundle |
|
|
89
|
+
|
|
90
|
+
#### Google Cloud Secret Manager
|
|
91
|
+
|
|
92
|
+
| Secret Path | Description |
|
|
93
|
+
| ------------------------------- | ------------------------------- |
|
|
94
|
+
| `prod-todoist-secrets` | Main application secrets bundle |
|
|
95
|
+
| `prod-todoist-rds-main` | Primary database credentials |
|
|
96
|
+
| `prod-todoist-rds-shard01` | Shard database credentials |
|
|
97
|
+
| `prod-todoist-apple-signin-key` | Apple Sign-In private key |
|
|
98
|
+
| `staging-todoist-secrets` | Staging secrets bundle |
|
|
99
|
+
|
|
100
|
+
### Bundled Secrets Structure
|
|
101
|
+
|
|
102
|
+
For bundled application secrets, group related secrets:
|
|
103
|
+
|
|
104
|
+
```json
|
|
105
|
+
{
|
|
106
|
+
"STRIPE_SECRET_KEY": "sk_live_...",
|
|
107
|
+
"STRIPE_WEBHOOK_SECRET": "whsec_...",
|
|
108
|
+
"COOKIE_SECRET_KEY": "base64-encoded-key",
|
|
109
|
+
"API_TOKEN_SIGNATURE_KEY": "base64-encoded-key"
|
|
110
|
+
}
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Limits and tags
|
|
114
|
+
|
|
115
|
+
When creating secrets, we need to be aware of the limits of the cloud provider secret manager:
|
|
116
|
+
|
|
117
|
+
| Limit | AWS Secrets Manager | Google Cloud Secret Manager |
|
|
118
|
+
| ------------------ | ------------------- | --------------------------- |
|
|
119
|
+
| Secret value size | 64 KB per secret | 64 KB per secret version |
|
|
120
|
+
| Secret name length | 512 characters | 255 characters |
|
|
121
|
+
|
|
122
|
+
For tags, we should use them to add relevant information about the secret, such as the environment, the application it belongs to -or product-, and any other relevant information that can help us identify the secret and its purpose.
|
|
123
|
+
|
|
124
|
+
For more information about our tagging strategy, please refer to the [Tagging policy](https://handbook.doist.com/doc/tagging-aws-resources-Fp1quAqmda?q=aws%20tag)
|
|
125
|
+
|
|
126
|
+
### Accessing Secrets in Application Code
|
|
127
|
+
|
|
128
|
+
Once our secrets are stored following our standard, the next step is to consume those secrets. For this there are 2 approaches:
|
|
129
|
+
|
|
130
|
+
- Feed to our application as environment variables
|
|
131
|
+
- Consumed from the application using the cloud provider SDK
|
|
132
|
+
|
|
133
|
+
Each approach has its own pros and cons, and although consuming secrets as environment variables is the most straightforward process, it is also the most prone to expose those secrets.
|
|
134
|
+
|
|
135
|
+
For instance, another process in our virtual machine or container could potentially see the content of the environment variable. Moreover, crash dumps, container metadata and logging frameworks tend to dump all environment variables values for debugging, which would inadvertently leak our secrets to the world.
|
|
136
|
+
|
|
137
|
+
Consuming secrets from the application using cloud provider SDK allows us to avoid all of this, although requires more code lines to consume them.
|
|
138
|
+
|
|
139
|
+
With all this information laid out, for production environments our recommended approach is to consume the secrets from the application using the cloud provider SDK.
|
|
140
|
+
|
|
141
|
+
## Development Environment
|
|
142
|
+
|
|
143
|
+
Unlike production environments, local or development environment secrets can be stored locally.
|
|
144
|
+
It is important to ensure those files are not shared unencrypted and the secrets' values are not the same as production. This includes the permission level or access level they provide.
|
|
145
|
+
|
|
146
|
+
Those secrets should be stored in a separate file (i.e `.env`) from our application and later consumed by it. This ensures no secret lives inside our application source code.
|
|
147
|
+
|
|
148
|
+
For encryption of these secret files, we recommend the usage of [SOPS](https://github.com/getsops/sops). SOPS is an application that implements the mechanics to encrypt our secret file using an encryption key. With it, we can safely share our encrypted secret file over the internet to our colleagues and, in the event of someone leaking the secret file to the internet, it won't be possible to decrypt the secrets unless the bad actor has access to the decryption key.
|
|
149
|
+
|
|
150
|
+
At Doist, all devs have automatic access to a shared encryption-decryption key that is used exclusively for development secrets. Please refer to the following guide to learn how to use SOPS with our shared key: [SOPS with Doist's shared key](https://handbook.doist.com/doc/sharing-development-secrets-nh7WApeEZR?q=SOPS)
|
|
151
|
+
|
|
152
|
+
### Sharing Secrets Between Developers
|
|
153
|
+
|
|
154
|
+
For secret sharing between developers — such as onboarding credentials, service accounts, or any secret that doesn't belong in a SOPS-encrypted file — use [Bitwarden](https://bitwarden.com/).
|
|
155
|
+
|
|
156
|
+
Bitwarden is Doist's approved password manager. All developers have access to shared Bitwarden collections organized by team and project. When sharing secrets through Bitwarden:
|
|
157
|
+
|
|
158
|
+
- **Use shared collections**, not individual sends, so secrets remain accessible to the team and not tied to one person
|
|
159
|
+
- **Never duplicate production secrets** — Bitwarden is for development and internal credentials only
|
|
160
|
+
- **Add descriptive names** to entries so others understand the secret's purpose and scope
|
|
161
|
+
- **Remove entries that are no longer needed** to keep collections clean and reduce exposure
|
|
162
|
+
|
|
163
|
+
As stated in the previous chapter, for secrets that need to live in a repository's `.env` file and be shared with the full team, use SOPS.
|
|
164
|
+
|
|
165
|
+
Use Bitwarden for everything else — one-off sharing, service account passwords, and credentials that don't map to a specific repo.
|
|
166
|
+
|
|
167
|
+
### Docker Build Secrets
|
|
168
|
+
|
|
169
|
+
For build-time secrets (e.g., private package repositories), use [BuildKit secrets](https://docs.docker.com/reference/dockerfile#run---mounttypesecret):
|
|
170
|
+
|
|
171
|
+
```dockerfile
|
|
172
|
+
# Dockerfile
|
|
173
|
+
# syntax = docker/dockerfile:1.3
|
|
174
|
+
|
|
175
|
+
RUN --mount=type=secret,id=netrc,target=/root/.netrc \
|
|
176
|
+
pip install -r requirements.txt
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
Build command:
|
|
180
|
+
|
|
181
|
+
```bash
|
|
182
|
+
docker build --secret id=netrc,src=$HOME/.netrc -t myimage .
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## CI/CD Environment
|
|
188
|
+
|
|
189
|
+
### GitHub Actions Secrets
|
|
190
|
+
|
|
191
|
+
For CI/CD workflows, we store secrets in GitHub on each repository. This feature is available on the settings page of each repository, under `Security -> Secrets and Variables`
|
|
192
|
+
|
|
193
|
+
CI/CD Secrets that should be shared across all repositories should be added at the GitHub organization level instead.
|
|
194
|
+
|
|
195
|
+
Be aware that anyone can fork our public repositories and create pull requests against our main branch. By default GitHub Actions will not run on pull requests from forks, but if we enable it, we need to ensure that no secrets are exposed to those pull requests. For this reason, we should never use secrets in workflows that are triggered by pull requests from forks. Moreover, those forks could potentially be used to exfiltrate secrets if we are not careful enough.
|
|
196
|
+
|
|
197
|
+
To solve this, we should configure our GitHub public repositories to not run GitHub Actions on pull requests from forks. This can be done in the repository settings, under `Actions -> General -> Approval for running fork pull request workflows from contributors`, by selecting the option `Require approval for all external contributors`.
|
|
198
|
+
|
|
199
|
+
For more information about GitHub Actions Permissions configuration, please refer to the [About GitHub Actions permissions for your repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#about-github-actions-permissions-for-your-repository)
|
|
200
|
+
|
|
201
|
+
## Secret Types and Handling
|
|
202
|
+
|
|
203
|
+
### Classification
|
|
204
|
+
|
|
205
|
+
| Type | Storage | Example |
|
|
206
|
+
| -------------------------- | ---------------------------- | ------------------ |
|
|
207
|
+
| **Database credentials** | Secrets Manager (individual) | RDS passwords |
|
|
208
|
+
| **API keys (internal)** | Secrets Manager (bundled) | Service tokens |
|
|
209
|
+
| **API keys (third-party)** | Secrets Manager (bundled) | Stripe, Sendgrid |
|
|
210
|
+
| **Encryption keys** | Secrets Manager or KMS | Cookie signing |
|
|
211
|
+
| **Certificates** | ACM or Secrets Manager | TLS, Apple signing |
|
|
212
|
+
| **Webhook secrets** | Secrets Manager (bundled) | Stripe webhooks |
|
|
213
|
+
|
|
214
|
+
---
|
|
215
|
+
|
|
216
|
+
## Secret Rotation
|
|
217
|
+
|
|
218
|
+
Rotation limits the blast radius of a compromised secret by reducing the window of time it remains valid. Even without a known breach, periodic rotation reduces risk from undetected exposure.
|
|
219
|
+
|
|
220
|
+
### Rotation Strategy
|
|
221
|
+
|
|
222
|
+
Not all secrets can or should be rotated the same way. Choose the approach based on the secret type:
|
|
223
|
+
|
|
224
|
+
| Strategy | When to Use | Example |
|
|
225
|
+
| ------------------------- | ------------------------------------------ | ----------------------------------------------------------- |
|
|
226
|
+
| **Automatic (scheduled)** | Cloud-native secrets with provider support | Database credentials via AWS/GCP rotation lambdas/functions |
|
|
227
|
+
| **Dual-key (overlap)** | Secrets shared across multiple consumers | API signing keys, webhook secrets |
|
|
228
|
+
| **On-demand** | Third-party keys with manual regeneration | Stripe API keys, Sendgrid tokens |
|
|
229
|
+
|
|
230
|
+
### Dual-Key Rotation
|
|
231
|
+
|
|
232
|
+
For secrets consumed by multiple services or external parties, use a dual-key (overlap) approach to avoid downtime:
|
|
233
|
+
|
|
234
|
+
1. Generate the new secret and store it alongside the current one
|
|
235
|
+
2. Update consumers to accept **both** the old and new secret
|
|
236
|
+
3. Roll out the new secret to all producers
|
|
237
|
+
4. Once all traffic uses the new secret, remove the old one
|
|
238
|
+
|
|
239
|
+
This is especially relevant for webhook verification secrets and API token signing keys, where the producer and consumer may deploy at different times.
|
|
240
|
+
|
|
241
|
+
### Rotation Cadence
|
|
242
|
+
|
|
243
|
+
| Secret Type | Recommended Cadence | Notes |
|
|
244
|
+
| ----------------------- | -------------------------------------- | ------------------------------------------- |
|
|
245
|
+
| Database credentials | Annually | Use automatic rotation via Secrets Manager |
|
|
246
|
+
| Internal API keys | Annually | Coordinate across dependent services |
|
|
247
|
+
| Third-party API keys | On provider recommendation or annually | Some providers enforce their own schedule |
|
|
248
|
+
| Encryption/signing keys | Annually | Requires careful key versioning |
|
|
249
|
+
| Compromised secrets | Immediately | See [Incident Response](#incident-response) |
|
|
250
|
+
|
|
251
|
+
### Requirements
|
|
252
|
+
|
|
253
|
+
- Rotation **must not** require code changes or redeployments. Secrets should be fetched at runtime, not baked into builds or container images.
|
|
254
|
+
- Services consuming secrets via the cloud provider SDK (our recommended approach) should handle rotation transparently by re-fetching on a reasonable interval or on authentication failure.
|
|
255
|
+
- Each secret in Secrets Manager should have a **rotation reminder** configured, even if automatic rotation is not yet enabled, so that the owning team receives a notification when rotation is due.
|
|
256
|
+
- Rotation procedures for each service should be **documented and tested** before going to production. An untested rotation procedure is effectively no procedure at all.
|
|
257
|
+
|
|
258
|
+
---
|
|
259
|
+
|
|
260
|
+
## Incident Response
|
|
261
|
+
|
|
262
|
+
If a secret is compromised:
|
|
263
|
+
|
|
264
|
+
1. **Immediately rotate** the compromised secret
|
|
265
|
+
2. **Incident response** team should be notified to investigate the breach. Please create an incident thread in our [Doist incident channel](https://twist.com/a/1585/ch/255325/) and follow the [Incident Response Checklist](https://twist.com/a/1585/ch/255325/t/2359936/).
|
|
266
|
+
3. **Audit** access logs for unauthorized usage
|
|
267
|
+
4. **Notify** the platform team
|
|
268
|
+
5. **Review** and revoke any sessions or tokens that may have been compromised
|
|
269
|
+
6. **Document** the incident and update procedures if needed
|
|
270
|
+
|
|
271
|
+
---
|
|
272
|
+
|
|
273
|
+
## Implementation Checklist
|
|
274
|
+
|
|
275
|
+
Before deploying a service to production, verify that your secrets management aligns with this standard.
|
|
276
|
+
The full set of production readiness requirements — including but not limited to secrets — will be covered in our [Production Readiness Standard](./production-readiness.md).
|
|
277
|
+
That standard will define the baseline criteria every application or service must meet before being deployed to production, spanning areas such as observability, reliability, security, etc.
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
281
|
+
## References
|
|
282
|
+
|
|
283
|
+
- [AWS Secrets Manager Documentation](https://docs.aws.amazon.com/secretsmanager/)
|
|
284
|
+
- [GitHub Actions OIDC with AWS](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services)
|
|
285
|
+
- [Pydantic Settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/)
|
|
286
|
+
- [Docker BuildKit Secrets](https://docs.docker.com/build/building/secrets/)
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# Standard: Service Production Readiness
|
|
2
|
+
|
|
3
|
+
<!--
|
|
4
|
+
past attempts:
|
|
5
|
+
https://github.com/Doist/handbook/pull/81 (useful feedback)
|
|
6
|
+
https://medium.com/google-cloud/production-guideline-9d5d10c8f1e
|
|
7
|
+
|
|
8
|
+
https://github.com/Doist/platform-backlog/issues/283 (can close)
|
|
9
|
+
-->
|
|
10
|
+
|
|
11
|
+
| 📄 Document Type | Standard |
|
|
12
|
+
| :--------------- | :--------------- |
|
|
13
|
+
| 🎯 Status | Approved |
|
|
14
|
+
| 👤 Owner | Artyom Pervukhin |
|
|
15
|
+
| 📅 Last Reviewed | 2026-03-06 |
|
|
16
|
+
| 🗂️ References | — |
|
|
17
|
+
| 📌 Version | 1.0 |
|
|
18
|
+
|
|
19
|
+
This standard applies to all new services deployed at Doist.
|
|
20
|
+
|
|
21
|
+
A _service_ is a server-side application that performs a defined function. Common types include:
|
|
22
|
+
|
|
23
|
+
- Request-driven services (e.g., HTTP APIs).
|
|
24
|
+
- Background or scheduled workers (queues, batch jobs, reports).
|
|
25
|
+
|
|
26
|
+
## Health Checks
|
|
27
|
+
|
|
28
|
+
Services that accept network traffic must expose health checks.
|
|
29
|
+
|
|
30
|
+
- HTTP services should provide an endpoint (suggested path: `/health`) that returns `200 OK` HTTP status code when healthy (response body is not important).
|
|
31
|
+
- TCP services are considered healthy when they accept connections.
|
|
32
|
+
|
|
33
|
+
Health checks must reflect only the correctness of the deployed code.
|
|
34
|
+
|
|
35
|
+
They should pass if and only if the current version can operate correctly compared to the previous version. If a deployment fails, automated rollback must restore a previously healthy version.
|
|
36
|
+
|
|
37
|
+
Do **not** include external dependencies (e.g. database availability) in health checks. External outages must not prevent rollback. If database connectivity is checked during health checks, an external database outage could prevent automated rollback to a previously working version.
|
|
38
|
+
|
|
39
|
+
A service should report healthy only after full initialization. If startup regularly exceeds one minute, the Platform team must configure the [`HealthCheckGracePeriodSeconds`](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-ecs-service.html#cfn-ecs-service-healthcheckgraceperiodseconds) property on the ECS service resource to prevent premature health check failures from triggering rollbacks during deployment.
|
|
40
|
+
|
|
41
|
+
## Observability
|
|
42
|
+
|
|
43
|
+
All services must produce structured, meaningful logs.
|
|
44
|
+
|
|
45
|
+
- Prefer one JSON object per line.
|
|
46
|
+
- Avoid color codes, emojis, and decorative output.
|
|
47
|
+
- Do not include timestamps (added by the logging system).
|
|
48
|
+
- Do not include sensitive or personal information in logs.
|
|
49
|
+
|
|
50
|
+
Logs should support operational debugging and monitoring.
|
|
51
|
+
|
|
52
|
+
See [the Observability standard for more details](https://handbook.doist.com/doc/standard-observability-vWRaOfitho).
|
|
53
|
+
|
|
54
|
+
## Configuration Management
|
|
55
|
+
|
|
56
|
+
Configuration must be provided via environment variables.
|
|
57
|
+
|
|
58
|
+
- Command-line flags are acceptable for a small number of parameters.
|
|
59
|
+
- Env files are allowed only if variables can be overridden by the environment.
|
|
60
|
+
- Secrets must come from environment variables or read directly from a secrets management service (AWS Secrets Manager).
|
|
61
|
+
|
|
62
|
+
See [the Secrets Management standard for more details](https://handbook.doist.com/doc/standard-secrets-management-wbKmIfrtgr).
|
|
63
|
+
|
|
64
|
+
## State Persistence
|
|
65
|
+
|
|
66
|
+
Services run in containerized environments with ephemeral local storage (AWS ECS Fargate is the default platform).
|
|
67
|
+
|
|
68
|
+
Any state that must survive restarts must be stored externally, e.g.:
|
|
69
|
+
|
|
70
|
+
- Redis/Valkey, Memcached
|
|
71
|
+
- DynamoDB, MySQL
|
|
72
|
+
- S3, EFS
|
|
73
|
+
|
|
74
|
+
Local disk may be used only for temporary data.
|
|
75
|
+
|
|
76
|
+
Durable state must be backed up. DynamoDB tables must have [point-in-time recovery](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Point-in-time-recovery.html) enabled and carry the account's `Backup` tag so the AWS Backup plan selects them. Caches, lock tables and short-lived session tables with a TTL are exempt.
|
|
77
|
+
|
|
78
|
+
## Resource Limits
|
|
79
|
+
|
|
80
|
+
Define expected CPU, memory, network, and local disk usage.
|
|
81
|
+
|
|
82
|
+
Local development environments are often significantly more powerful than production instances. Profile and test accordingly.
|
|
83
|
+
|
|
84
|
+
Estimate local disk requirements for temporary files. Fargate provides 20 GiB of ephemeral storage by default; contact the Platform team if more is needed.
|
|
85
|
+
|
|
86
|
+
## Concurrency and Scaling
|
|
87
|
+
|
|
88
|
+
Assume services will run with multiple parallel instances.
|
|
89
|
+
|
|
90
|
+
- Do not rely on single-instance behavior.
|
|
91
|
+
- Requests from the same client may reach different instances.
|
|
92
|
+
- Shared state must be externalized.
|
|
93
|
+
|
|
94
|
+
## Error Handling
|
|
95
|
+
|
|
96
|
+
Services must handle non-happy-path scenarios:
|
|
97
|
+
|
|
98
|
+
- Apply timeouts to all network calls.
|
|
99
|
+
- Implement retries with backoff.
|
|
100
|
+
- Support graceful degradation (e.g. returning cached or degraded responses when a downstream service is unavailable).
|
|
101
|
+
- Use circuit breakers where appropriate (see [Martin Fowler's circuit breaker pattern](https://martinfowler.com/bliki/CircuitBreaker.html)).
|
|
102
|
+
|
|
103
|
+
## Deployment Behavior
|
|
104
|
+
|
|
105
|
+
Deployments use rolling updates: old and new versions run simultaneously.
|
|
106
|
+
|
|
107
|
+
Design for version overlap:
|
|
108
|
+
|
|
109
|
+
- Cache formats
|
|
110
|
+
- Database schemas
|
|
111
|
+
- Persisted data
|
|
112
|
+
|
|
113
|
+
Multiple versions must coexist without causing errors. In practice, this means being flexible on the reading side: new versions must tolerate data written by old versions. Adding fields to cache formats, schemas, or persisted data is safe; removing or renaming fields requires a multi-step migration.
|
|
114
|
+
|
|
115
|
+
## Building and Packaging
|
|
116
|
+
|
|
117
|
+
Services must be built into container images.
|
|
118
|
+
|
|
119
|
+
- Service code repository must include a Dockerfile.
|
|
120
|
+
- All dependencies must be installed at build time.
|
|
121
|
+
|
|
122
|
+
Services must not make unintended network calls at startup or runtime — for example, dependency update checks or telemetry to third-party build tools. Intentional integrations (e.g. Datadog, Sentry) are expected and not affected by this requirement.
|