gitgreen 0.1.0 → 1.0.0

package/README.md

@@ -5,71 +5,96 @@ Self-contained carbon calculation CLI for GitLab jobs (no API server). It reuses
 ## Install
 - From npm (global CLI):
   ```bash
-  pnpm add -g gitgreen-cli
-  # or: npm install -g gitgreen-cli
+  npm install -g gitgreen
   gitgreen --help
   ```
 
-- From this repo:
-  ```bash
-  # from repo root
-  pnpm -C node-module install
-  # build happens via prepare; dist/ is ready for CI
-  ```
-
-Run tests (pnpm preferred):
+Run tests:
 ```bash
-pnpm -C node-module test
-# or: npm --prefix node-module test
+npm test
 ```
 
 Stress test multiple live configs (build first, real APIs):
 ```bash
-pnpm -C node-module build
-pnpm -C node-module stress
+npm build
+npm stress
 ```
 
 ## Usage
-```bash
-gitgreen \
-  --provider gcp \
-  --machine e2-standard-4 \
-  --region us-central1-a \
-  --duration 1800 \
-  --cpu 55 \
-  --budget 500 \
-  --fail-on-budget \
-  --out-md carbon-report.md \
-  --out-json carbon-report.json \
-  --post-note
-```
 
-### Interactive GitLab setup
+### Initializing a New GitLab Project
+
+To get started with carbon tracking in any GitLab project (any project with a remote pointing to a GitLab instance), run:
+
 ```bash
 # In your repo
 gitgreen init
 ```
-The wizard asks for provider/machine/region, budgets, MR note preference, then offers to append a ready-made job to `.gitlab-ci.yml` and prints the CI/CD variable checklist (ELECTRICITY_MAPS_API_KEY, provider creds, budget flags).
-
-Key environment variables (loaded automatically via dotenv):
-- `ELECTRICITY_MAPS_API_KEY` (required)
-- `ELECTRICITY_MAPS_BASE_URL` (optional, defaults to public endpoint)
-- `GCP_PROJECT_ID` / `GOOGLE_CLOUD_PROJECT` (for GCP metric pull)
-- `AWS_*` credentials (for CloudWatch metric pull)
-- `DATA_DIR` (override bundled data)
-- `DEFAULT_PROVIDER`, `PUE_FALLBACK`, `GITLAB_BASE_URL`
-
-GitLab CI auto-detection uses:
-- `CI_JOB_TOKEN`, `CI_PROJECT_ID`, `CI_JOB_ID`, `CI_PIPELINE_ID`, `CI_MERGE_REQUEST_IID`, `CI_SERVER_URL`
-- `CI_RUNNER_TAGS` (comma/space separated) to infer machine/region and instance IDs
-
-## GitLab snippet
-Add variables `ELECTRICITY_MAPS_API_KEY`, `CARBON_BUDGET` (optional), `FAIL_ON_BUDGET` (true/false). Then call:
-```bash
-./scripts/carbon/install-cli.sh
-./scripts/carbon/run-analysis.sh
+
+The initialization wizard will guide you through the setup process:
+- Configure provider/machine/region settings
+- Set carbon budgets
+- Configure MR note preferences
+- Choose to use an existing runner or spin up a new one with supported providers
+
+After initialization, the wizard will:
+- Append a ready-made job to your `.gitlab-ci.yml`
+- Print the CI/CD variable checklist (ELECTRICITY_MAPS_API_KEY, provider credentials, budget flags)
+
+### How It Works
+
+Once initialized, all subsequent pipelines will run on the configured runner, and their performance will be automatically measured. The carbon tracking is implemented using GitLab CI/CD components as a final step in your pipeline. The carbon tracking job itself is not computationally expensive, so it adds minimal overhead to your CI/CD workflows.
+
+Key environment variables:
+- `ELECTRICITY_MAPS_API_KEY` (required) - Get a free key from https://api-portal.electricitymaps.com
+- `GCP_PROJECT_ID` / `GOOGLE_CLOUD_PROJECT` (required for GCP) - Your GCP project ID
+
+The `gitgreen init` wizard will automatically set all necessary GitLab CI/CD variables for you.
+
+## Providers
+- **GCP**: Fully wired — the CLI parser expects GCP Monitoring JSON, the GitLab component polls GCE metrics, and `gitgreen init` provisions or reuses GCP runners.
+
+## Architecture
+Runtime flow:
 ```
-Artifacts `carbon-report.md` and `carbon-report.json` will be produced; `POST_MR_NOTE=true` will post to the MR when `CI_JOB_TOKEN` is available.
+CPU/RAM timeseries (GCP Monitoring JSON or custom collector)
+        |
+        v
++------------------+    +----------------------------+
+| CLI (gitgreen)   |--->| CarbonCalculator           |
+| - parse metrics  |    | - PowerProfileRepository   |<- machine data in data/*.json
+| - CLI/env opts   |    | - ZoneMapper (region→zone) |<- static + runtime PUE mapping
++------------------+    | - IntensityProvider        |<- Electricity Maps API
+                         +-------------+------------+
+                                       |
+                                       v
+                        +-----------------------------+
+                        | Report formatter            |
+                        | - Markdown/JSON artifacts   |
+                        | - Budget evaluation         |
+                        +-------------+---------------+
+                                       |
+                                       v
+                        +-----------------------------+
+                        | GitLab client (optional)    |
+                        | - MR note via CI_JOB_TOKEN  |
+                        +-----------------------------+
+```
+
+GitLab CI path:
+```
+Pipeline starts → component script fetches CPU/RAM timeseries from GCP Monitoring
+              → writes JSON files → runs `gitgreen --provider gcp ...`
+              → emits `carbon-report.md` / `carbon-report.json`
+              → optional MR note when CI_JOB_TOKEN is present
+```
+
+## Adding a provider
+1. Extend `CloudProvider` and the provider guard in `src/index.ts` so the calculator accepts the new key.
+2. Add machine power data (`<provider>_machine_power_profiles.json`) and, if needed, CPU profiles to `data/`, then update `PowerProfileRepository.loadMachineData` to load it.
+3. Map regions to Electricity Maps zones and a PUE default in `ZoneMapper` (or via `data/runtime-pue-mappings.json` for runtime overrides).
+4. Parse that provider's metrics into the `TimeseriesPoint` shape (timestamp + numeric value) alongside RAM size/usage, and update the CLI/init/templates to pull those metrics.
+5. Wire any CI automation (runner tags, MR note flags) to pass the correct provider, machine type, and region strings.
 
 ## Publish
 - Ensure version bump in `package.json`

package/dist/init.js

@@ -80,80 +80,27 @@ const setVariable = async (auth, project, key, value, masked = false) => {
         return setVariableApi(auth.baseUrl, auth.pat, project, key, value, masked);
     }
 };
-const generateCiJob = (runnerTag) => {
+const generateCiJob = (opts = {}) => {
+    const { runnerTag, carbonBudget, failOnBudget } = opts;
+    let inputs = `      gcp_project_id: $GCP_PROJECT_ID
+      gcp_instance_id: $GCP_INSTANCE_ID
+      gcp_zone: $GCP_ZONE
+      machine_type: $MACHINE_TYPE`;
+    if (carbonBudget) {
+        inputs += `\n      carbon_budget_grams: "${carbonBudget}"`;
+    }
+    if (failOnBudget) {
+        inputs += `\n      fail_on_budget: true`;
+    }
     const tagsSection = runnerTag ? `\n  tags:\n    - ${runnerTag}` : '';
     return `# GitGreen carbon analysis (generated by gitgreen init)
-# Fetches real CPU and RAM metrics from GCP Monitoring API
+include:
+  - component: gitlab.com/youneslaaroussi/gitgreen/gitgreen-cli-component@0.1.0
+    inputs:
+${inputs}
 
 carbon_analysis:
-  stage: test${tagsSection}
-  before_script:
-    - echo "$GCP_SA_KEY_BASE64" | base64 -d > /tmp/gcp-key.json
-    - gcloud auth activate-service-account --key-file=/tmp/gcp-key.json
-    - npm install ./gitgreen-cli-0.1.0.tgz
-  script:
-    - |
-      TOKEN=$(gcloud auth print-access-token)
-      START_TIME=$(date -u -d "$CI_PIPELINE_CREATED_AT" +%Y-%m-%dT%H:%M:%SZ)
-      END_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)
-
-      CPU_FILTER="resource.type=\\"gce_instance\\" AND resource.labels.instance_id=\\"$GCP_INSTANCE_ID\\" AND resource.labels.zone=\\"$GCP_ZONE\\" AND metric.type=\\"compute.googleapis.com/instance/cpu/utilization\\""
-      CPU_ENCODED=$(python3 -c "import urllib.parse; print(urllib.parse.quote('''$CPU_FILTER'''))")
-
-      RAM_USED_FILTER="resource.type=\\"gce_instance\\" AND resource.labels.instance_id=\\"$GCP_INSTANCE_ID\\" AND metric.type=\\"compute.googleapis.com/instance/memory/balloon/ram_used\\""
-      RAM_USED_ENCODED=$(python3 -c "import urllib.parse; print(urllib.parse.quote('''$RAM_USED_FILTER'''))")
-
-      RAM_SIZE_FILTER="resource.type=\\"gce_instance\\" AND resource.labels.instance_id=\\"$GCP_INSTANCE_ID\\" AND metric.type=\\"compute.googleapis.com/instance/memory/balloon/ram_size\\""
-      RAM_SIZE_ENCODED=$(python3 -c "import urllib.parse; print(urllib.parse.quote('''$RAM_SIZE_FILTER'''))")
-
-      # Calculate expected data points (GCP reports every 60s)
-      START_SEC=$(date -d "$START_TIME" +%s)
-      NOW_SEC=$(date +%s)
-      DURATION_SEC=$((NOW_SEC - START_SEC))
-      EXPECTED_POINTS=$((DURATION_SEC / 60))
-      [ "$EXPECTED_POINTS" -lt 1 ] && EXPECTED_POINTS=1
-
-      echo "Pipeline duration: \${DURATION_SEC}s, expecting ~\$EXPECTED_POINTS data points"
-      echo "Waiting for metrics data (GCP has ~3 min lag)..."
-
-      for i in $(seq 1 30); do
-        END_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)
-        CPU_DATA=$(curl -s -H "Authorization: Bearer $TOKEN" \\
-          "https://monitoring.googleapis.com/v3/projects/$GCP_PROJECT_ID/timeSeries?filter=$CPU_ENCODED&interval.startTime=$START_TIME&interval.endTime=$END_TIME")
-
-        POINTS=$(echo "$CPU_DATA" | python3 -c "import sys,json; d=json.load(sys.stdin); print(sum(len(ts.get('points',[])) for ts in d.get('timeSeries',[])))" 2>/dev/null || echo "0")
-
-        if [ "$POINTS" -ge "$EXPECTED_POINTS" ]; then
-          echo "Got $POINTS CPU data points (expected $EXPECTED_POINTS)"
-          break
-        fi
-        echo "Waiting for data... got $POINTS/$EXPECTED_POINTS (attempt $i/30)"
-        sleep 10
-      done
-
-      # Fetch RAM data
-      RAM_USED_DATA=$(curl -s -H "Authorization: Bearer $TOKEN" \\
-        "https://monitoring.googleapis.com/v3/projects/$GCP_PROJECT_ID/timeSeries?filter=$RAM_USED_ENCODED&interval.startTime=$START_TIME&interval.endTime=$END_TIME")
-
-      RAM_SIZE_DATA=$(curl -s -H "Authorization: Bearer $TOKEN" \\
-        "https://monitoring.googleapis.com/v3/projects/$GCP_PROJECT_ID/timeSeries?filter=$RAM_SIZE_ENCODED&interval.startTime=$START_TIME&interval.endTime=$END_TIME")
-
-      # Save timeseries to files
-      echo "$CPU_DATA" > /tmp/cpu_timeseries.json
-      echo "$RAM_USED_DATA" > /tmp/ram_used_timeseries.json
-      echo "$RAM_SIZE_DATA" > /tmp/ram_size_timeseries.json
-
-      CMD="./node_modules/.bin/gitgreen --provider gcp --machine $MACHINE_TYPE --region $GCP_ZONE --cpu-timeseries /tmp/cpu_timeseries.json --ram-used-timeseries /tmp/ram_used_timeseries.json --ram-size-timeseries /tmp/ram_size_timeseries.json --out-md carbon-report.md --out-json carbon-report.json --no-gitlab"
-      [ -n "\${CARBON_BUDGET_GRAMS:-}" ] && CMD="$CMD --budget $CARBON_BUDGET_GRAMS"
-      [ "\${FAIL_ON_BUDGET:-}" = "true" ] && CMD="$CMD --fail-on-budget"
-
-      eval "$CMD"
-      cat carbon-report.md
-  artifacts:
-    paths:
-      - carbon-report.md
-      - carbon-report.json
-    expire_in: 30 days
+  extends: .gitgreen-carbon-analysis${tagsSection}
 `;
 };
 const runInit = async (opts = {}) => {
@@ -747,7 +694,11 @@ systemctl start gitlab-runner
         message: 'Add job to .gitlab-ci.yml?',
         initial: true
     });
-    const ciJob = generateCiJob(runnerTag || undefined);
+    const ciJob = generateCiJob({
+        runnerTag: runnerTag || undefined,
+        carbonBudget: carbonBudget || undefined,
+        failOnBudget
+    });
     if (addCiJob) {
         const ciPath = path_1.default.join(process.cwd(), '.gitlab-ci.yml');
         if (fs_1.default.existsSync(ciPath)) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "gitgreen",
-  "version": "0.1.0",
+  "version": "1.0.0",
   "description": "GitGreen CLI for carbon reporting in GitLab pipelines (GCP/AWS)",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",