@meyicloud/meyi-cost-server 1.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +197 -0
- package/README.md +454 -0
- package/cur.js +2 -0
- package/index.js +2 -0
- package/package.json +36 -0
- package/src/controllers/budget.controller.js +39 -0
- package/src/controllers/cost-analysis.controller.js +28 -0
- package/src/controllers/cost.controller.js +144 -0
- package/src/cur-discovery/cur-discovery.aws.js +82 -0
- package/src/cur-discovery/cur-discovery.repository.js +177 -0
- package/src/cur-discovery/cur-discovery.service.js +112 -0
- package/src/cur-discovery/cur-discovery.worker.js +57 -0
- package/src/cur-discovery/schema.js +48 -0
- package/src/lib/cost-analysis.js +116 -0
- package/src/lib/cost-utils.js +98 -0
- package/src/lib/llm-provider.js +239 -0
- package/src/models/budget.model.js +26 -0
- package/src/models/cur-data-status.model.js +17 -0
- package/src/models/cur-ingestion.model.js +9 -0
- package/src/models/customer-aws-context.model.js +15 -0
- package/src/models/saas-cur-context.model.js +10 -0
- package/src/plugin.js +82 -0
- package/src/repositories/aws-onboarding.repository.js +134 -0
- package/src/repositories/budget-alert.repository.js +37 -0
- package/src/repositories/budget.repository.js +33 -0
- package/src/repositories/cost-analysis.repository.js +50 -0
- package/src/routes/index.js +31 -0
- package/src/schema/cost-analysis.schema.js +7 -0
- package/src/schema/cost-budget.schema.js +9 -0
- package/src/services/aws-context.service.js +1 -0
- package/src/services/budget-alert.service.js +47 -0
- package/src/services/budget.service.js +32 -0
- package/src/services/cost-analysis-data.service.js +39 -0
- package/src/services/cost-analysis.service.js +190 -0
- package/src/services/cur-provider.service.js +95 -0
- package/src/services/cur.service.js +354 -0
- package/src/services/customer-aws-context.service.js +26 -0
- package/src/services/saas-athena-context.service.js +45 -0
package/AGENTS.md
ADDED
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
# Insight Cost Server Agent Guide
|
|
2
|
+
|
|
3
|
+
## Scope
|
|
4
|
+
|
|
5
|
+
This package is a tenant-aware Express plugin for AWS cost overview, reports,
|
|
6
|
+
budgets, alert dismissals, Cost Explorer, and optional CUR/Athena data. It is
|
|
7
|
+
designed to be mounted inside a host backend; it does not own login, tenant
|
|
8
|
+
onboarding, or the host plugin registry.
|
|
9
|
+
|
|
10
|
+
Never weaken tenant scoping or expose AWS credentials. A management/payer role
|
|
11
|
+
may return organization and member-account costs. A member role normally only
|
|
12
|
+
returns the costs visible to that member account.
|
|
13
|
+
|
|
14
|
+
## Folder structure
|
|
15
|
+
|
|
16
|
+
```text
|
|
17
|
+
insight-cost-server/
|
|
18
|
+
|-- src/
|
|
19
|
+
| |-- controllers/ # Maps validated HTTP input to service calls
|
|
20
|
+
| |-- routes/ # Express paths and common error handling
|
|
21
|
+
| |-- models/ # Immutable customer, SaaS CUR, and status models
|
|
22
|
+
| |-- repositories/ # Tenant-scoped onboarding persistence reads
|
|
23
|
+
| |-- services/ # Cost Explorer, SaaS Athena, context, budgets
|
|
24
|
+
| |-- schema/ # Plugin-owned database tables and installation
|
|
25
|
+
| |-- lib/ # Stateless date, cost, and filter helpers
|
|
26
|
+
| `-- plugin.js # Dependency composition and lifecycle
|
|
27
|
+
|-- index.js # Public createInsightCost export
|
|
28
|
+
|-- cur.js # CUR compatibility export
|
|
29
|
+
|-- README.md # Runtime and AWS configuration
|
|
30
|
+
|-- package.json
|
|
31
|
+
`-- AGENTS.md
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Keep HTTP concerns in routes/controllers, AWS and database behavior in services,
|
|
35
|
+
schema creation in `schema`, and pure calculations in `lib`. `index.js` and
|
|
36
|
+
`cur.js` must remain thin compatibility entry points.
|
|
37
|
+
|
|
38
|
+
## AI provider abstraction
|
|
39
|
+
|
|
40
|
+
`src/lib/llm-provider.js` is the only place that knows which vendor answers.
|
|
41
|
+
`createLlmProvider({ provider, modelId, apiKey, region, ... })` returns an
|
|
42
|
+
object with a single `converse({ system, messages, maxTokens, temperature,
|
|
43
|
+
topP })` method resolving to `{ text }`. Bedrock, Anthropic and OpenAI are
|
|
44
|
+
supported; `normalizeProviderName` matches loosely so UI labels like
|
|
45
|
+
"AWS Bedrock" resolve correctly.
|
|
46
|
+
|
|
47
|
+
`CostAnalysisService.resolveProvider(tenantId)` picks between a host-supplied
|
|
48
|
+
`providerResolver` and the `COST_AI_*` environment defaults. Two rules matter:
|
|
49
|
+
the analysis fingerprint is computed from the **effective** model so switching
|
|
50
|
+
provider cannot serve a cached answer from the previous one, and a resolver
|
|
51
|
+
failure degrades to the environment rather than failing the request.
|
|
52
|
+
|
|
53
|
+
When adding a provider, add it here and to `test/llm-provider.test.js`; no
|
|
54
|
+
other file should need to change.
|
|
55
|
+
|
|
56
|
+
## Request and data flow
|
|
57
|
+
|
|
58
|
+
```mermaid
|
|
59
|
+
flowchart LR
|
|
60
|
+
Host[Host Express app] --> Auth[Host authentication middleware]
|
|
61
|
+
Auth --> Enabled[Tenant plugin-enabled check]
|
|
62
|
+
Enabled --> Router[/api/v1/cost router]
|
|
63
|
+
Router --> Controller[Cost controller]
|
|
64
|
+
Controller --> Customer[Customer AWS context service]
|
|
65
|
+
Customer -->|Customer role| CE[AWS Cost Explorer]
|
|
66
|
+
Controller --> Source{Configured source}
|
|
67
|
+
Source -->|CUR| SaaS[SaaS Athena context service]
|
|
68
|
+
SaaS --> Athena[Central tenant CUR partition]
|
|
69
|
+
Source -->|auto fallback| CE
|
|
70
|
+
Controller --> Budget[Budget and dismissal services]
|
|
71
|
+
Budget --> DB[(Tenant-scoped PostgreSQL)]
|
|
72
|
+
Athena --> Controller
|
|
73
|
+
CE --> Controller
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
The plugin currently owns these route groups under `${apiBaseUri}/cost`:
|
|
77
|
+
|
|
78
|
+
- `GET /accounts`, `/data-status`, `/overview`, `/filter-options`, `/reports`, `/tags`
|
|
79
|
+
- `GET /budgets` and `POST /budgets`
|
|
80
|
+
- `DELETE /budgets/:id`
|
|
81
|
+
- `GET /budget-alert-dismissals` and `POST /budget-alert-dismissals`
|
|
82
|
+
|
|
83
|
+
## Integrating with another Express application
|
|
84
|
+
|
|
85
|
+
Install from the registry after publishing, or use a local file dependency:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
npm install @meyicloud/meyi-cost-server
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
```js
|
|
92
|
+
import express from "express";
|
|
93
|
+
import { drizzle } from "drizzle-orm/node-postgres";
|
|
94
|
+
import { Pool } from "pg";
|
|
95
|
+
import { createInsightCost } from "@meyicloud/meyi-cost-server";
|
|
96
|
+
|
|
97
|
+
const app = express();
|
|
98
|
+
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
|
|
99
|
+
const db = drizzle(pool);
|
|
100
|
+
|
|
101
|
+
// Register host authentication before starting the cost plugin.
|
|
102
|
+
app.use("/api/v1/cost", authenticateRequest);
|
|
103
|
+
|
|
104
|
+
const costPlugin = createInsightCost({
|
|
105
|
+
app,
|
|
106
|
+
db,
|
|
107
|
+
apiBaseUri: "/api/v1",
|
|
108
|
+
logger: console,
|
|
109
|
+
});
|
|
110
|
+
|
|
111
|
+
await costPlugin.install();
|
|
112
|
+
await costPlugin.start();
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Call `install()` during plugin installation/migration so plugin-owned tables
|
|
116
|
+
exist, then call `start()` once per process to mount routes. Call `stop()` from
|
|
117
|
+
the host lifecycle when supported.
|
|
118
|
+
|
|
119
|
+
The host authentication layer must populate the authenticated user and tenant
|
|
120
|
+
context expected by the adapter (`req.user.id`, `req.user.tenant_id`, or the
|
|
121
|
+
approved tenant header). The host must verify that this plugin is enabled for
|
|
122
|
+
that tenant before requests reach its router. Do not trust an arbitrary tenant
|
|
123
|
+
ID without authenticated host validation.
|
|
124
|
+
|
|
125
|
+
The current Meyi Connect adapter is
|
|
126
|
+
`meyi-connect/backend/src/plugins/cost/index.mjs`. It supplies the Drizzle
|
|
127
|
+
database, authentication, tenant plugin checks, and host lifecycle wiring.
|
|
128
|
+
|
|
129
|
+
## AWS and database configuration
|
|
130
|
+
|
|
131
|
+
Use onboarding role metadata in production. Environment credentials are an
|
|
132
|
+
explicit local-testing fallback and must stay backend-only. See `README.md` for
|
|
133
|
+
the supported Cost Explorer, assume-role, CUR/Athena, and local-test variables.
|
|
134
|
+
|
|
135
|
+
Important invariants:
|
|
136
|
+
|
|
137
|
+
- Resolve the customer AWS role per tenant for Cost Explorer/account metadata.
|
|
138
|
+
- Use only the SaaS runtime identity or `COST_CUR_ROLE_ARN` for Athena. Never
|
|
139
|
+
pass `CustomerAwsContext.credentials` into an Athena client.
|
|
140
|
+
- Keep central catalog configuration in SaaS environment settings. A customer
|
|
141
|
+
onboarding record may supply source-bucket and tenant-partition metadata, but
|
|
142
|
+
cannot override the central catalog unless the explicit legacy compatibility
|
|
143
|
+
switch is enabled.
|
|
144
|
+
- Scope account records, budgets, and alert dismissals by tenant.
|
|
145
|
+
- Preserve payer/organization access when showing linked member accounts.
|
|
146
|
+
- Keep CUR/Athena optional; `auto` may fall back to Cost Explorer, while `cur`
|
|
147
|
+
must surface CUR configuration/access errors.
|
|
148
|
+
- Do not send access keys, secret keys, or session tokens in API responses or
|
|
149
|
+
logs.
|
|
150
|
+
- Keep SQL parameterized and schema-qualified through the existing helpers.
|
|
151
|
+
|
|
152
|
+
## Development and validation
|
|
153
|
+
|
|
154
|
+
This package is native Node.js ESM and has no transpilation build step:
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
npm install
|
|
158
|
+
node --check index.js
|
|
159
|
+
node --check src/plugin.js
|
|
160
|
+
npm test
|
|
161
|
+
npm pack --dry-run
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
For any change, syntax-check every changed JavaScript file and exercise route
|
|
165
|
+
creation with an Express app and a stub database. Changes to AWS queries should
|
|
166
|
+
also test empty results, missing permissions, assumed-role failures, and both
|
|
167
|
+
Cost Explorer and CUR source selection. Never use real customer credentials in
|
|
168
|
+
automated tests.
|
|
169
|
+
|
|
170
|
+
When packaging in Docker, copy `index.js`, `cur.js`, and the complete `src`
|
|
171
|
+
directory. Copying only the entry point will produce runtime module-not-found
|
|
172
|
+
errors after the package was split into layers.
|
|
173
|
+
|
|
174
|
+
## Version and npm publishing
|
|
175
|
+
|
|
176
|
+
Publishing changes external state. Only publish when explicitly authorized:
|
|
177
|
+
|
|
178
|
+
```bash
|
|
179
|
+
npm version patch
|
|
180
|
+
npm pack --dry-run
|
|
181
|
+
npm publish --access public
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
Confirm that the dry-run archive contains both entry points, all `src` layers,
|
|
185
|
+
`README.md`, and `AGENTS.md`. Do not publish `.env` files, AWS credentials,
|
|
186
|
+
database dumps, logs, screenshots, or host application source.
|
|
187
|
+
|
|
188
|
+
## Change checklist
|
|
189
|
+
|
|
190
|
+
1. Preserve `createInsightCost({ app, db, apiBaseUri, logger })` and its
|
|
191
|
+
`install`, `start`, and `stop` lifecycle.
|
|
192
|
+
2. Add or change routes in the router, controller, service, and shared types as
|
|
193
|
+
appropriate; do not place the whole feature in one file.
|
|
194
|
+
3. Validate tenant isolation for every query and mutation.
|
|
195
|
+
4. Keep response shapes compatible with `@meyicloud/meyi-cost-ui`, or update both packages
|
|
196
|
+
and the host adapter in the same release.
|
|
197
|
+
5. Run syntax checks and package dry-run, then build the consuming host backend.
|
package/README.md
ADDED
|
@@ -0,0 +1,454 @@
|
|
|
1
|
+
# Meyi Insight Cost Server
|
|
2
|
+
|
|
3
|
+
Tenant-aware Express plugin that provides AWS cost overview, reports, dynamic
|
|
4
|
+
filters, budgets, and budget-alert dismissals from centralized Cost and Usage
|
|
5
|
+
Report (CUR) data queried through Athena. CUR is mandatory; the package does not
|
|
6
|
+
call or fall back to AWS Cost Explorer.
|
|
7
|
+
|
|
8
|
+
The package is designed to be mounted by a host backend. The host owns login,
|
|
9
|
+
token verification, tenant/plugin enablement, PostgreSQL connection creation,
|
|
10
|
+
and AWS onboarding. The plugin owns cost routes and its budget-related tables.
|
|
11
|
+
|
|
12
|
+
## Package contract
|
|
13
|
+
|
|
14
|
+
- Package name: `@meyicloud/meyi-cost-server`
|
|
15
|
+
- Runtime: Node.js 20 or newer, ESM
|
|
16
|
+
- Framework: Express 4
|
|
17
|
+
- Database API: injected Drizzle PostgreSQL instance
|
|
18
|
+
- Peer dependency: `pg >= 8`
|
|
19
|
+
- Default route: `/api/v1/cost`
|
|
20
|
+
- Public factory: `createInsightCost({ app, db, apiBaseUri, logger })`
|
|
21
|
+
|
|
22
|
+
## Features
|
|
23
|
+
|
|
24
|
+
- Tenant-isolated payer/organization and member-account cost views
|
|
25
|
+
- Monthly and yearly overview data
|
|
26
|
+
- Service, account, region, resource, and cost-allocation-tag reports
|
|
27
|
+
- Dynamic service, region, linked-account, and tag filter options
|
|
28
|
+
- Standard and comparison report data used by `@meyicloud/meyi-cost-ui`
|
|
29
|
+
- CUR/Athena active-resource counts and detailed cost data
|
|
30
|
+
- Explicit CUR setup, pending-data, and unavailable states
|
|
31
|
+
- Tenant-scoped budget rules
|
|
32
|
+
- User-, tenant-, month-, and status-scoped budget-alert dismissals
|
|
33
|
+
|
|
34
|
+
Budgets are application rules stored in PostgreSQL; they are not AWS Budgets
|
|
35
|
+
resources. The consuming application evaluates them against current cost and
|
|
36
|
+
shows the notification UI.
|
|
37
|
+
|
|
38
|
+
## Architecture and request flow
|
|
39
|
+
|
|
40
|
+
```mermaid
|
|
41
|
+
flowchart LR
|
|
42
|
+
Host[External Express application] --> Auth[Authentication]
|
|
43
|
+
Auth --> Enabled[Tenant plugin check]
|
|
44
|
+
Enabled --> Router[/api/v1/cost]
|
|
45
|
+
Router --> Controller[Cost controller]
|
|
46
|
+
Controller --> Customer[Tenant CUR metadata]
|
|
47
|
+
Controller --> SaaS[SaaS Athena context]
|
|
48
|
+
SaaS --> Athena[Central tenant-partitioned CUR]
|
|
49
|
+
Controller --> Budget[Budget services]
|
|
50
|
+
Budget --> DB[(PostgreSQL)]
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
```text
|
|
54
|
+
src/
|
|
55
|
+
|-- controllers/ HTTP-to-service orchestration
|
|
56
|
+
|-- routes/ Express routes and error handling
|
|
57
|
+
|-- models/ Normalized immutable domain models
|
|
58
|
+
|-- repositories/ Tenant onboarding persistence access
|
|
59
|
+
|-- services/ AWS CUR/Athena contexts and budgets
|
|
60
|
+
|-- schema/ Budget and dismissal table installation
|
|
61
|
+
|-- lib/ Cost, date, filter, and SQL helpers
|
|
62
|
+
`-- plugin.js Dependency composition and lifecycle
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Local development and validation
|
|
66
|
+
|
|
67
|
+
Requirements:
|
|
68
|
+
|
|
69
|
+
- Node.js 20 or newer
|
|
70
|
+
- npm
|
|
71
|
+
- PostgreSQL available to the host used for integration testing
|
|
72
|
+
|
|
73
|
+
Install dependencies:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
npm install
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
This package is native JavaScript ESM and has no transpilation build step.
|
|
80
|
+
Validate syntax and package contents with:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
node --check index.js
|
|
84
|
+
node --check cur.js
|
|
85
|
+
node --check src/plugin.js
|
|
86
|
+
npm pack --dry-run
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
When files below `src` change, syntax-check every changed `.js` file. The npm
|
|
90
|
+
archive must include `index.js`, `cur.js`, the complete `src` directory,
|
|
91
|
+
`README.md`, and `AGENTS.md`.
|
|
92
|
+
|
|
93
|
+
For a local package installation test:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
npm pack
|
|
97
|
+
npm install /path/to/meyicloud-meyi-cost-server-1.4.1.tgz
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
## Publish to npm
|
|
101
|
+
|
|
102
|
+
Publishing changes the external registry. Run these commands only after release
|
|
103
|
+
authorization:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
npm login
|
|
107
|
+
npm whoami
|
|
108
|
+
npm version patch
|
|
109
|
+
npm pack --dry-run
|
|
110
|
+
npm publish --access public
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Use `npm version minor` or `npm version major` when appropriate. This server
|
|
114
|
+
package currently has no compile step or `prepublishOnly` script, so syntax and
|
|
115
|
+
integration validation must be completed before publishing.
|
|
116
|
+
|
|
117
|
+
After publishing:
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
npm view @meyicloud/meyi-cost-server version
|
|
121
|
+
npm install @meyicloud/meyi-cost-server@<published-version>
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
## Integrate with an external Express application
|
|
125
|
+
|
|
126
|
+
Install the package and its PostgreSQL peer dependency:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
npm install @meyicloud/meyi-cost-server pg
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
For local development, use a file dependency:
|
|
133
|
+
|
|
134
|
+
```json
|
|
135
|
+
{
|
|
136
|
+
"dependencies": {
|
|
137
|
+
"@meyicloud/meyi-cost-server": "file:../../meyi-market-places/insight-cost-server"
|
|
138
|
+
}
|
|
139
|
+
}
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Create the plugin using the host's Express app and Drizzle database:
|
|
143
|
+
|
|
144
|
+
```js
|
|
145
|
+
import express from "express";
|
|
146
|
+
import { drizzle } from "drizzle-orm/node-postgres";
|
|
147
|
+
import { Pool } from "pg";
|
|
148
|
+
import { createInsightCost } from "@meyicloud/meyi-cost-server";
|
|
149
|
+
|
|
150
|
+
const app = express();
|
|
151
|
+
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
|
|
152
|
+
const db = drizzle(pool);
|
|
153
|
+
|
|
154
|
+
app.use(express.json());
|
|
155
|
+
|
|
156
|
+
// These host middlewares must be registered before plugin.start().
|
|
157
|
+
app.use("/api/v1/cost", authenticateRequest);
|
|
158
|
+
app.use("/api/v1/cost", requireCostPluginForTenant);
|
|
159
|
+
|
|
160
|
+
const costPlugin = createInsightCost({
|
|
161
|
+
app,
|
|
162
|
+
db,
|
|
163
|
+
apiBaseUri: "/api/v1",
|
|
164
|
+
logger: console,
|
|
165
|
+
});
|
|
166
|
+
|
|
167
|
+
await costPlugin.install();
|
|
168
|
+
await costPlugin.start();
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Lifecycle behavior:
|
|
172
|
+
|
|
173
|
+
- `install()` creates or verifies plugin-owned budget and dismissal tables.
|
|
174
|
+
- `start()` mounts the router once at `${apiBaseUri}/cost`.
|
|
175
|
+
- `stop()` is currently a no-op but is available for host lifecycle symmetry.
|
|
176
|
+
- `router` is also returned for hosts that need custom mounting.
|
|
177
|
+
|
|
178
|
+
`app` is optional only when the host mounts the returned `router` itself. `db`
|
|
179
|
+
is required.
|
|
180
|
+
|
|
181
|
+
## Host authentication and tenant contract
|
|
182
|
+
|
|
183
|
+
Before a request reaches the plugin, the host should authenticate it and set:
|
|
184
|
+
|
|
185
|
+
```js
|
|
186
|
+
req.user = {
|
|
187
|
+
id: "authenticated-user-id",
|
|
188
|
+
tenant_id: "authenticated-tenant-id",
|
|
189
|
+
};
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
The tenant resolver checks `req.user.tenant_id`, then `req.user.tenantId`, then
|
|
193
|
+
`x-tenant-id`, then `DEFAULT_TENANT_ID`. In production, prefer an authenticated
|
|
194
|
+
`req.user` tenant and do not accept an untrusted tenant header.
|
|
195
|
+
|
|
196
|
+
The Marketplace onboarding integration loads tenant access from:
|
|
197
|
+
|
|
198
|
+
- `<DB_SCHEMA>.aws_connections`
|
|
199
|
+
- `<DB_SCHEMA>.aws_accounts`
|
|
200
|
+
- `<DB_SCHEMA>.cost_cur_config`
|
|
201
|
+
- `<DB_SCHEMA>.cost_cur_discovery_jobs`
|
|
202
|
+
- `<DB_SCHEMA>.cost_cur_discovery_job_logs`
|
|
203
|
+
|
|
204
|
+
Only a connected row containing `Cost` with verified Cost access is accepted.
|
|
205
|
+
Connection metadata supplies account display names and CUR source metadata; the
|
|
206
|
+
dedicated Cost CUR table supplies the tenant partition and delivery/readiness
|
|
207
|
+
state. Customer credentials are never passed to the central Athena client.
|
|
208
|
+
|
|
209
|
+
CUR discovery runs as one shared scheduler with independent tenant job rows.
|
|
210
|
+
Each job checks only its configured S3 prefix, finds a tenant-safe Glue table,
|
|
211
|
+
and verifies the tenant partition through Athena. A successful job becomes
|
|
212
|
+
`READY` and is no longer scheduled. Job events are retained separately in
|
|
213
|
+
`cost_cur_discovery_job_logs` and also include tenant and job IDs in application
|
|
214
|
+
logs.
|
|
215
|
+
|
|
216
|
+
## Data-source behavior
|
|
217
|
+
|
|
218
|
+
### Central CUR through SaaS Athena
|
|
219
|
+
|
|
220
|
+
CUR applies `line_item_unblended_cost > 0` for displayed gross cost and can also
|
|
221
|
+
calculate net cost, credits/adjustments, active resources, detailed resource
|
|
222
|
+
rows, account/region relationships, and allocation-tag data.
|
|
223
|
+
|
|
224
|
+
The SaaS runtime identity, or the dedicated role configured by
|
|
225
|
+
`COST_CUR_ROLE_ARN`, queries the SaaS-owned Athena/Glue/S3 resources. Customer
|
|
226
|
+
onboarding creates a CUR 2.0 Data Export directly into a tenant prefix in the
|
|
227
|
+
Meyi-owned bucket; customer AWS credentials are not reused for Athena.
|
|
228
|
+
|
|
229
|
+
CUR is mandatory. `/data-status` reports whether configuration, discovery, and
|
|
230
|
+
tenant rows are ready. Data endpoints return HTTP 503 with a structured state
|
|
231
|
+
and action when CUR is not configured, still waiting for rows, or unavailable.
|
|
232
|
+
|
|
233
|
+
## Environment variables
|
|
234
|
+
|
|
235
|
+
All variables below belong to the **backend runtime**. Never place AWS or
|
|
236
|
+
database secrets in `@meyicloud/meyi-cost-ui`, a browser environment, source control, or a
|
|
237
|
+
Docker image layer. Supply them at container/process runtime.
|
|
238
|
+
|
|
239
|
+
### Database and tenant variables
|
|
240
|
+
|
|
241
|
+
| Variable | Default | Required | Purpose |
|
|
242
|
+
| --- | --- | --- | --- |
|
|
243
|
+
| `DB_SCHEMA` | `meyiconnect` | No | PostgreSQL schema containing onboarding and plugin-owned tables. |
|
|
244
|
+
| `DEFAULT_TENANT_ID` | `default` | No | Final tenant fallback when authenticated request context is absent. Prefer authenticated tenant context in production. |
|
|
245
|
+
| `DATABASE_URL` | None | Host-specific | Not read by the package directly. The host commonly uses it to create the injected `db` connection. |
|
|
246
|
+
|
|
247
|
+
Production tenants should use per-tenant onboarding role metadata. Environment
|
|
248
|
+
credentials are process-wide and therefore unsuitable as the normal credential
|
|
249
|
+
source for a multi-tenant service.
|
|
250
|
+
|
|
251
|
+
When customer environment credentials are enabled, central Athena is disabled
|
|
252
|
+
unless `COST_CUR_ROLE_ARN` is also configured. This prevents the AWS SDK default
|
|
253
|
+
credential chain from accidentally using a customer's local test key against
|
|
254
|
+
SaaS Athena resources.
|
|
255
|
+
|
|
256
|
+
### AI analysis variables
|
|
257
|
+
|
|
258
|
+
AI analysis is optional and invokes Amazon Bedrock from the backend runtime.
|
|
259
|
+
Only aggregated cost facts are sent to the model; tenant IDs, AWS credentials,
|
|
260
|
+
resource IDs, and raw CUR rows are excluded.
|
|
261
|
+
|
|
262
|
+
| Variable | Default | Required | Purpose |
|
|
263
|
+
| --- | --- | --- | --- |
|
|
264
|
+
| `COST_AI_ENABLED` | `false` | No | Enables the AI analysis endpoints and Bedrock client. |
|
|
265
|
+
| `COST_AI_REGION` | `AWS_REGION` or `us-east-1` | When enabled | Region used by the Bedrock Runtime client. |
|
|
266
|
+
| `COST_AI_MODEL_ID` | Global Claude Sonnet inference profile | When enabled | Bedrock model or inference-profile ID. |
|
|
267
|
+
| `COST_AI_MAX_TOKENS` | `1600` | No | Maximum response tokens, clamped from 600 to 3000. |
|
|
268
|
+
| `COST_AI_CACHE_TTL_MS` | `21600000` | No | Tenant analysis cache duration; defaults to six hours. |
|
|
269
|
+
| `COST_AI_HOURLY_LIMIT` | `6` | No | Maximum non-cached generations per tenant per hour. |
|
|
270
|
+
| `COST_AI_TIMEOUT_MS` | `120000` | No | Per-request timeout for key-based providers; Bedrock uses the SDK default. |
|
|
271
|
+
|
|
272
|
+
### Choosing a provider
|
|
273
|
+
|
|
274
|
+
The `COST_AI_*` variables above configure the deployment default, which is
|
|
275
|
+
Bedrock. A host application can additionally supply `providerResolver` to
|
|
276
|
+
`createInsightCost` to select a provider per tenant:
|
|
277
|
+
|
|
278
|
+
```js
|
|
279
|
+
createInsightCost({
|
|
280
|
+
app, db,
|
|
281
|
+
providerResolver: async (tenantId) => ({
|
|
282
|
+
provider: "anthropic", // bedrock | anthropic | openai
|
|
283
|
+
modelId: "claude-sonnet-4-5",
|
|
284
|
+
apiKey: "...", // required for anthropic and openai
|
|
285
|
+
region: "ap-south-2", // bedrock only
|
|
286
|
+
accessKeyId: "...", // bedrock only; omit to use ambient credentials
|
|
287
|
+
secretAccessKey: "...",
|
|
288
|
+
}),
|
|
289
|
+
})
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
Returning `null` falls back to the environment configuration, so a host that
|
|
293
|
+
does not pass a resolver behaves exactly as before. A resolver that throws is
|
|
294
|
+
logged and also falls back, so a credential-store outage degrades the feature
|
|
295
|
+
rather than failing the request.
|
|
296
|
+
|
|
297
|
+
Why this exists: Bedrock authenticates with SigV4 and is subject to
|
|
298
|
+
account-level Anthropic model access. The key-based providers are not, so a
|
|
299
|
+
deployment blocked on Bedrock model access can still run analysis. It also
|
|
300
|
+
lets a customer bring their own model account.
|
|
301
|
+
|
|
302
|
+
`GET /analysis/status` reports the provider that would serve the caller's
|
|
303
|
+
tenant, including a `source` of `tenant-configuration` or `environment`.
|
|
304
|
+
**Host applications are responsible for encrypting stored credentials.**
|
|
305
|
+
|
|
306
|
+
The runtime role needs `bedrock:InvokeModel` for the selected model or
|
|
307
|
+
inference profile. Analysis results are stored in the tenant-scoped
|
|
308
|
+
`cost_ai_analyses` table. Model or permission failures affect only the analysis
|
|
309
|
+
request and do not change CUR, reports, or budgets.
|
|
310
|
+
|
|
311
|
+
### CUR/Athena variables
|
|
312
|
+
|
|
313
|
+
| Variable | Default | Required | Purpose |
|
|
314
|
+
| --- | --- | --- | --- |
|
|
315
|
+
| `COST_DATA_SOURCE` | `cur` | No | Compatibility setting. Cost data is always CUR/Athena only. |
|
|
316
|
+
| `COST_CUR_DATABASE` | Empty | CUR | Athena/Glue database containing the CUR table. |
|
|
317
|
+
| `COST_CUR_TABLE` | Empty | No | Optional manual Glue table override. When empty, the tenant discovery job supplies the verified table. |
|
|
318
|
+
| `COST_CUR_OUTPUT_LOCATION` | Empty | CUR | S3 URI where Athena writes query results. |
|
|
319
|
+
| `COST_CUR_REGION` | `AWS_REGION` or `us-east-1` | No | Region for the Athena client. |
|
|
320
|
+
| `COST_CUR_WORKGROUP` | `meyi-cost` | No | Meyi-owned Athena workgroup used for every query. |
|
|
321
|
+
| `AWS_REGION` | `us-east-1` | No | Used as the CUR region fallback. |
|
|
322
|
+
| `COST_CUR_TENANT_COLUMN` | `tenant_id` | No | CUR column used to isolate the Hive-style tenant partition. |
|
|
323
|
+
| `COST_CUR_TENANT_PARTITION` | Request tenant ID | No | Overrides the partition value. Avoid a global override in multi-tenant production unless every request is intentionally mapped to that partition. |
|
|
324
|
+
| `COST_CUR_MAX_ROWS` | `1000` | No | Resource-report row cap, clamped between 1 and 5000. |
|
|
325
|
+
| `COST_CUR_ROLE_ARN` | Empty | No | Dedicated SaaS-side role assumed only for central Athena/Glue/S3 queries. |
|
|
326
|
+
| `COST_CUR_EXTERNAL_ID` | Empty | No | External ID for the dedicated SaaS CUR role. |
|
|
327
|
+
| `COST_CUR_ALLOW_TENANT_CATALOG` | `false` | No | Compatibility switch allowing tenant metadata to override the central catalog. Keep `false` for centralized SaaS CUR. |
|
|
328
|
+
| `COST_CUR_INGESTION_MODE` | `central` | No | Describes the CUR ingestion contract returned by readiness status. |
|
|
329
|
+
| `COST_CUR_STATUS_CACHE_MS` | `300000` | No | Cache duration for the live tenant CUR readiness query. |
|
|
330
|
+
| `COST_CUR_DISCOVERY_ENABLED` | `true` | No | Enables tenant-specific background CUR discovery. |
|
|
331
|
+
| `COST_CUR_DISCOVERY_INTERVAL_MS` | `3600000` | No | Retry interval for non-ready tenant jobs; minimum 60000 ms. |
|
|
332
|
+
| `COST_CUR_DISCOVERY_BATCH_SIZE` | `25` | No | Maximum due tenant jobs selected per scheduler tick. |
|
|
333
|
+
|
|
334
|
+
Central catalog settings come from the SaaS backend environment. Only the
|
|
335
|
+
tenant partition and customer source metadata come from onboarding by default.
|
|
336
|
+
Legacy per-tenant catalog overrides are accepted only when
|
|
337
|
+
`COST_CUR_ALLOW_TENANT_CATALOG=true`.
|
|
338
|
+
|
|
339
|
+
### Example: centralized CUR/Athena
|
|
340
|
+
|
|
341
|
+
```env
|
|
342
|
+
COST_DATA_SOURCE=cur
|
|
343
|
+
COST_CUR_DATABASE=meyi_central_cur
|
|
344
|
+
COST_CUR_TABLE=
|
|
345
|
+
COST_CUR_OUTPUT_LOCATION=s3://meyi-saas-athena-results/
|
|
346
|
+
COST_CUR_REGION=us-east-1
|
|
347
|
+
COST_CUR_WORKGROUP=meyi-cost
|
|
348
|
+
COST_CUR_TENANT_COLUMN=tenant_id
|
|
349
|
+
COST_CUR_MAX_ROWS=1000
|
|
350
|
+
COST_CUR_ROLE_ARN=arn:aws:iam::SAAS_ACCOUNT_ID:role/meyi-cur-query
|
|
351
|
+
COST_CUR_ALLOW_TENANT_CATALOG=false
|
|
352
|
+
COST_CUR_INGESTION_MODE=central
|
|
353
|
+
COST_CUR_DISCOVERY_ENABLED=true
|
|
354
|
+
COST_CUR_DISCOVERY_INTERVAL_MS=3600000
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
## Required AWS read-only access
|
|
358
|
+
|
|
359
|
+
The **SaaS identity** must be able to query Athena, read the Glue
|
|
360
|
+
catalog and central CUR bucket, and write to the Athena results bucket. The
|
|
361
|
+
customer onboarding role needs only the customer-side permissions required by
|
|
362
|
+
the chosen CUR export/DataSync design. These policies must remain separate.
|
|
363
|
+
|
|
364
|
+
A payer/management-account role can expose organization-wide linked-account
|
|
365
|
+
costs. A member-account role normally exposes only the costs AWS makes visible
|
|
366
|
+
to that member. Separate member-account roles are not required merely to group
|
|
367
|
+
organization cost by linked account when the payer role already has that data.
|
|
368
|
+
|
|
369
|
+
## HTTP API
|
|
370
|
+
|
|
371
|
+
Routes are mounted below `${apiBaseUri}/cost`:
|
|
372
|
+
|
|
373
|
+
| Method | Path | Purpose |
|
|
374
|
+
| --- | --- | --- |
|
|
375
|
+
| `GET` | `/accounts` | Selected or discovered AWS accounts. |
|
|
376
|
+
| `GET` | `/data-status` | Central CUR configuration, ingestion, readiness, row count, and freshness without exposing secrets. |
|
|
377
|
+
| `GET` | `/overview` | Totals, trends, accounts, services, regions, and active resources when CUR is available. |
|
|
378
|
+
| `GET` | `/filter-options` | Dynamic report filter values. |
|
|
379
|
+
| `GET` | `/reports` | Standard/comparison source data grouped by service, account, region, resource, or tag. |
|
|
380
|
+
| `GET` | `/tags` | Available cost-allocation tags. |
|
|
381
|
+
| `GET` | `/analysis/status` | Whether Cost AI is enabled and its non-secret runtime metadata. |
|
|
382
|
+
| `GET` | `/analysis/latest` | Latest stored analysis for the authenticated tenant. |
|
|
383
|
+
| `POST` | `/analysis` | Generate or reuse a cached tenant analysis for a validated date range. |
|
|
384
|
+
| `GET` | `/budgets` | Tenant budget rules. |
|
|
385
|
+
| `POST` | `/budgets` | Create a tenant budget rule. |
|
|
386
|
+
| `DELETE` | `/budgets/:id` | Delete a tenant budget rule. |
|
|
387
|
+
| `GET` | `/budget-alert-dismissals` | Current user's persisted dismissals. |
|
|
388
|
+
| `POST` | `/budget-alert-dismissals` | Dismiss one budget/month/status alert. |
|
|
389
|
+
|
|
390
|
+
## Docker packaging
|
|
391
|
+
|
|
392
|
+
When embedding the local package in a Docker build, copy all published package
|
|
393
|
+
files, not only `index.js`:
|
|
394
|
+
|
|
395
|
+
```text
|
|
396
|
+
index.js
|
|
397
|
+
cur.js
|
|
398
|
+
src/
|
|
399
|
+
package.json
|
|
400
|
+
README.md
|
|
401
|
+
AGENTS.md
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
Environment files should not be copied into the image. Pass required variables
|
|
405
|
+
with the deployment platform, Docker Compose `env_file`, or secret manager at
|
|
406
|
+
runtime.
|
|
407
|
+
|
|
408
|
+
## Reference Meyi Connect integration
|
|
409
|
+
|
|
410
|
+
The current host adapter is:
|
|
411
|
+
|
|
412
|
+
`meyi-connect/backend/src/plugins/cost/index.mjs`
|
|
413
|
+
|
|
414
|
+
It supplies the shared Drizzle database, authenticated token middleware,
|
|
415
|
+
tenant-level plugin enablement, and plugin lifecycle wiring.
|
|
416
|
+
|
|
417
|
+
## Consumer validation checklist
|
|
418
|
+
|
|
419
|
+
1. Run server syntax checks and `npm pack --dry-run`.
|
|
420
|
+
2. Install or refresh the package in the host backend.
|
|
421
|
+
3. Run `install()` against a disposable/test database.
|
|
422
|
+
4. Verify unauthenticated and disabled-tenant requests are rejected by the host.
|
|
423
|
+
5. Verify two tenants cannot access each other's accounts, budgets, or alert
|
|
424
|
+
dismissals.
|
|
425
|
+
6. Test CUR ready, not-configured, pending-data, and unavailable modes.
|
|
426
|
+
7. Verify payer and member-account visibility matches the AWS role used.
|
|
427
|
+
8. Build and start the consuming host backend before creating a Docker image.
|
|
428
|
+
|
|
429
|
+
## Central CUR ingestion contract
|
|
430
|
+
|
|
431
|
+
The package implements the application side of the FinOps-style design: strict
|
|
432
|
+
credential separation, tenant-partitioned Athena queries, fail-closed data
|
|
433
|
+
access, and live readiness/freshness reporting. The AWS data-transfer plane
|
|
434
|
+
must deliver each customer's CUR objects into the central CUR layout consumed
|
|
435
|
+
by Glue. That is deployment infrastructure, not an HTTP request made with a
|
|
436
|
+
customer access key.
|
|
437
|
+
|
|
438
|
+
Onboarding may persist these non-secret fields in session `meta`:
|
|
439
|
+
|
|
440
|
+
```json
|
|
441
|
+
{
|
|
442
|
+
"curSourceBucket": "customer-cur-bucket",
|
|
443
|
+
"curSourcePrefix": "reports/meyi/",
|
|
444
|
+
"curSourceRegion": "us-east-1",
|
|
445
|
+
"curTenantPartition": "authenticated-tenant-id",
|
|
446
|
+
"curIngestionMode": "central"
|
|
447
|
+
}
|
|
448
|
+
```
|
|
449
|
+
|
|
450
|
+
The transfer pipeline must copy or replicate only that source prefix into the
|
|
451
|
+
SaaS central location for the matching tenant partition. `GET /data-status`
|
|
452
|
+
then verifies that Athena can see rows for that partition. Until rows arrive,
|
|
453
|
+
cost endpoints remain unavailable and the UI directs the user to Cost sources
|
|
454
|
+
or CUR discovery with the applicable readiness message.
|
package/cur.js
ADDED
package/index.js
ADDED
package/package.json
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@meyicloud/meyi-cost-server",
|
|
3
|
+
"version": "1.4.1",
|
|
4
|
+
"description": "Tenant-aware AWS CUR and Athena cost plugin server for MeyiConnect",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "./index.js",
|
|
7
|
+
"exports": {
|
|
8
|
+
".": "./index.js"
|
|
9
|
+
},
|
|
10
|
+
"scripts": {
|
|
11
|
+
"test": "node --test test/*.test.js"
|
|
12
|
+
},
|
|
13
|
+
"files": [
|
|
14
|
+
"index.js",
|
|
15
|
+
"cur.js",
|
|
16
|
+
"src",
|
|
17
|
+
"README.md",
|
|
18
|
+
"AGENTS.md"
|
|
19
|
+
],
|
|
20
|
+
"dependencies": {
|
|
21
|
+
"@aws-sdk/client-athena": "^3.850.0",
|
|
22
|
+
"@aws-sdk/client-bedrock-runtime": "^3.850.0",
|
|
23
|
+
"@aws-sdk/client-glue": "^3.850.0",
|
|
24
|
+
"@aws-sdk/client-s3": "^3.850.0",
|
|
25
|
+
"@aws-sdk/client-sts": "^3.850.0",
|
|
26
|
+
"@aws-sdk/credential-providers": "^3.850.0",
|
|
27
|
+
"drizzle-orm": "^0.44.7",
|
|
28
|
+
"express": "^4.21.1"
|
|
29
|
+
},
|
|
30
|
+
"peerDependencies": {
|
|
31
|
+
"pg": ">=8"
|
|
32
|
+
},
|
|
33
|
+
"engines": {
|
|
34
|
+
"node": ">=20"
|
|
35
|
+
}
|
|
36
|
+
}
|