@rulebricks/cli 2.1.6 → 2.3.1

package/cluster-setup/aws/rulebricks-cluster.cfn.yaml

@@ -0,0 +1,355 @@
+AWSTemplateFormatVersion: "2010-09-09"
+Description: >-
+  Rulebricks EKS cluster, turnkey. Stands up an EKS cluster with a managed node
+  group and a single IAM role + S3 bucket for all Rulebricks data:
+    decision logs -> s3://<cluster>-data/decision-logs/  (Vector)        SA: vector
+    backups       -> s3://<cluster>-data/db-backups/      (backup job)   SA: <ns>-backup
+    metrics       -> AMP  (Prometheus remote write)                      SA: prometheus
+  Identity uses EKS Pod Identity (AWS's recommended mechanism for new clusters):
+  one IAM role with scoped policies. The namespace-scoped Pod Identity
+  associations binding vector/backup/prometheus to this role are created by the
+  Rulebricks CLI at deploy time, so this stack stays deployment-independent and
+  one cluster can host many deployments. eksctl alone cannot create the S3 bucket
+  or AMP workspace, so the full picture lives here in one stack.
+
+Parameters:
+  ClusterName:
+    Type: String
+    Default: rulebricks-cluster
+  KubernetesVersion:
+    Type: String
+    Default: "1.34"
+  NodeInstanceType:
+    Type: String
+    Default: c7i.xlarge
+  NodeDesiredCapacity:
+    Type: Number
+    Default: 2
+  NodeMinSize:
+    Type: Number
+    Default: 2
+  NodeMaxSize:
+    Type: Number
+    # Core services need only 2-4 small nodes; burst capacity lives in the
+    # dedicated burst nodegroup, and a low core ceiling steers the autoscaler
+    # there when the worker fleet scales out.
+    Default: 4
+  NodeVolumeSizeGiB:
+    Type: Number
+    Default: 50
+  VpcCidr:
+    Type: String
+    Default: 10.0.0.0/16
+  EnableBurstPool:
+    Type: String
+    AllowedValues: ["true", "false"]
+    Default: "true"
+    Description: >-
+      Add a dedicated burst nodegroup: one large on-demand node (scales
+      0->BurstNodeMaxSize) labeled and tainted rulebricks.com/pool=burst.
+      The Rulebricks chart makes workers tolerate and softly prefer it out
+      of the box, keeping the scaled-out fleet off the core nodes.
+  BurstInstanceType:
+    Type: String
+    Default: c7i.4xlarge
+    Description: >-
+      Instance type for the burst nodegroup. Default 16 vCPU: 2x4 vCPU core
+      floor + 16 = 24 vCPU running steady-state at full burst, and exactly
+      32 vCPU even with the core nodegroup at its 4-node max.
+  BurstNodeMaxSize:
+    Type: Number
+    Default: 1
+Conditions:
+  BurstPoolEnabled: !Equals [!Ref EnableBurstPool, "true"]
+
+Resources:
+  # ---------------------------------------------------------------------------
+  # Networking (minimal: VPC + 2 public subnets across 2 AZs + IGW + routing)
+  # ---------------------------------------------------------------------------
+  Vpc:
+    Type: AWS::EC2::VPC
+    Properties:
+      CidrBlock: !Ref VpcCidr
+      EnableDnsHostnames: true
+      EnableDnsSupport: true
+      Tags:
+        - { Key: Name, Value: !Sub "${ClusterName}-vpc" }
+        - { Key: Environment, Value: rulebricks }
+
+  InternetGateway:
+    Type: AWS::EC2::InternetGateway
+
+  VpcGatewayAttachment:
+    Type: AWS::EC2::VPCGatewayAttachment
+    Properties:
+      VpcId: !Ref Vpc
+      InternetGatewayId: !Ref InternetGateway
+
+  SubnetA:
+    Type: AWS::EC2::Subnet
+    Properties:
+      VpcId: !Ref Vpc
+      AvailabilityZone: !Select [0, !GetAZs ""]
+      CidrBlock: 10.0.0.0/19
+      MapPublicIpOnLaunch: true
+      Tags:
+        - { Key: Name, Value: !Sub "${ClusterName}-subnet-a" }
+
+  SubnetB:
+    Type: AWS::EC2::Subnet
+    Properties:
+      VpcId: !Ref Vpc
+      AvailabilityZone: !Select [1, !GetAZs ""]
+      CidrBlock: 10.0.32.0/19
+      MapPublicIpOnLaunch: true
+      Tags:
+        - { Key: Name, Value: !Sub "${ClusterName}-subnet-b" }
+
+  RouteTable:
+    Type: AWS::EC2::RouteTable
+    Properties:
+      VpcId: !Ref Vpc
+
+  DefaultRoute:
+    Type: AWS::EC2::Route
+    DependsOn: VpcGatewayAttachment
+    Properties:
+      RouteTableId: !Ref RouteTable
+      DestinationCidrBlock: 0.0.0.0/0
+      GatewayId: !Ref InternetGateway
+
+  SubnetARouteAssoc:
+    Type: AWS::EC2::SubnetRouteTableAssociation
+    Properties:
+      SubnetId: !Ref SubnetA
+      RouteTableId: !Ref RouteTable
+
+  SubnetBRouteAssoc:
+    Type: AWS::EC2::SubnetRouteTableAssociation
+    Properties:
+      SubnetId: !Ref SubnetB
+      RouteTableId: !Ref RouteTable
+
+  # ---------------------------------------------------------------------------
+  # Cluster + node group IAM roles
+  # ---------------------------------------------------------------------------
+  ClusterRole:
+    Type: AWS::IAM::Role
+    Properties:
+      AssumeRolePolicyDocument:
+        Version: "2012-10-17"
+        Statement:
+          - Effect: Allow
+            Principal: { Service: eks.amazonaws.com }
+            Action: sts:AssumeRole
+      ManagedPolicyArns:
+        - arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
+
+  NodeRole:
+    Type: AWS::IAM::Role
+    Properties:
+      AssumeRolePolicyDocument:
+        Version: "2012-10-17"
+        Statement:
+          - Effect: Allow
+            Principal: { Service: ec2.amazonaws.com }
+            Action: sts:AssumeRole
+      ManagedPolicyArns:
+        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
+        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
+        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
+        - arn:aws:iam::aws:policy/AmazonEBSCSIDriverPolicy
+
+  # ---------------------------------------------------------------------------
+  # EKS cluster
+  # ---------------------------------------------------------------------------
+  Cluster:
+    Type: AWS::EKS::Cluster
+    Properties:
+      Name: !Ref ClusterName
+      Version: !Ref KubernetesVersion
+      RoleArn: !GetAtt ClusterRole.Arn
+      ResourcesVpcConfig:
+        SubnetIds:
+          - !Ref SubnetA
+          - !Ref SubnetB
+        EndpointPublicAccess: true
+        EndpointPrivateAccess: true
+
+  # Pod Identity agent add-on: required for PodIdentityAssociation to work.
+  PodIdentityAddon:
+    Type: AWS::EKS::Addon
+    Properties:
+      ClusterName: !Ref Cluster
+      AddonName: eks-pod-identity-agent
+
+  EbsCsiAddon:
+    Type: AWS::EKS::Addon
+    Properties:
+      ClusterName: !Ref Cluster
+      AddonName: aws-ebs-csi-driver
+
+  NodeGroup:
+    Type: AWS::EKS::Nodegroup
+    Properties:
+      ClusterName: !Ref Cluster
+      NodegroupName: standard-nodes
+      NodeRole: !GetAtt NodeRole.Arn
+      Subnets:
+        - !Ref SubnetA
+        - !Ref SubnetB
+      InstanceTypes:
+        - !Ref NodeInstanceType
+      AmiType: AL2023_x86_64_STANDARD # c7i = Intel/x86
+      DiskSize: !Ref NodeVolumeSizeGiB
+      ScalingConfig:
+        DesiredSize: !Ref NodeDesiredCapacity
+        MinSize: !Ref NodeMinSize
+        MaxSize: !Ref NodeMaxSize
+
+  # Dedicated burst nodegroup: one large on-demand node for the scaled-out
+  # worker fleet. The taint keeps everything except workers off it; the label
+  # is what the chart's soft worker affinity targets. EKS has no parked-VM
+  # (Deallocate) equivalent, so bursts cold-provision (~2-3 min); a Karpenter
+  # NodePool with the same label/taint is the future fast path.
+  BurstNodeGroup:
+    Type: AWS::EKS::Nodegroup
+    Condition: BurstPoolEnabled
+    Properties:
+      ClusterName: !Ref Cluster
+      NodegroupName: burst-workers
+      NodeRole: !GetAtt NodeRole.Arn
+      Subnets:
+        - !Ref SubnetA
+        - !Ref SubnetB
+      InstanceTypes:
+        - !Ref BurstInstanceType
+      AmiType: AL2023_x86_64_STANDARD
+      DiskSize: !Ref NodeVolumeSizeGiB
+      Labels:
+        rulebricks.com/pool: burst
+      Taints:
+        - Key: rulebricks.com/pool
+          Value: burst
+          Effect: NO_SCHEDULE
+      ScalingConfig:
+        DesiredSize: 0
+        MinSize: 0
+        MaxSize: !Ref BurstNodeMaxSize
+
+  # ---------------------------------------------------------------------------
+  # OBJECT STORAGE (all Rulebricks data)
+  # One bucket holds everything; decision logs and backups are key prefixes
+  # (decision-logs/ and db-backups/) within it.
+  # ---------------------------------------------------------------------------
+  DataBucket:
+    Type: AWS::S3::Bucket
+    Properties:
+      BucketName: !Sub "${ClusterName}-data-${AWS::AccountId}"
+      PublicAccessBlockConfiguration:
+        BlockPublicAcls: true
+        BlockPublicPolicy: true
+        IgnorePublicAcls: true
+        RestrictPublicBuckets: true
+      BucketEncryption:
+        ServerSideEncryptionConfiguration:
+          - ServerSideEncryptionByDefault: { SSEAlgorithm: AES256 }
+
+  # ---------------------------------------------------------------------------
+  # METRICS (Amazon Managed Service for Prometheus workspace)
+  # ---------------------------------------------------------------------------
+  PrometheusWorkspace:
+    Type: AWS::APS::Workspace
+    Properties:
+      Alias: !Sub "${ClusterName}-amp"
+
+  # ---------------------------------------------------------------------------
+  # RULEBRICKS ROLE (single role for all data paths)
+  # Trusts the EKS Pod Identity service principal. The Rulebricks CLI creates the
+  # namespace-scoped Pod Identity associations (vector/backup/prometheus) at
+  # deploy time. Grants S3 read/write on the one data bucket and aps:RemoteWrite
+  # on the AMP workspace.
+  # ---------------------------------------------------------------------------
+  RulebricksRole:
+    Type: AWS::IAM::Role
+    Properties:
+      RoleName: !Sub "${ClusterName}-rulebricks"
+      AssumeRolePolicyDocument:
+        Version: "2012-10-17"
+        Statement:
+          - Effect: Allow
+            Principal: { Service: pods.eks.amazonaws.com }
+            Action:
+              - sts:AssumeRole
+              - sts:TagSession
+      Policies:
+        - PolicyName: rulebricks-s3-data
+          PolicyDocument:
+            Version: "2012-10-17"
+            Statement:
+              - Effect: Allow
+                Action:
+                  - s3:PutObject
+                  - s3:GetObject
+                  - s3:DeleteObject
+                  - s3:ListBucket
+                Resource:
+                  - !GetAtt DataBucket.Arn
+                  - !Sub "${DataBucket.Arn}/*"
+        - PolicyName: rulebricks-amp-remote-write
+          PolicyDocument:
+            Version: "2012-10-17"
+            Statement:
+              - Effect: Allow
+                Action:
+                  - aps:RemoteWrite
+                Resource:
+                  - !GetAtt PrometheusWorkspace.Arn
+        - PolicyName: rulebricks-msk-iam
+          PolicyDocument:
+            Version: "2012-10-17"
+            Statement:
+              # AWS MSK IAM access for HPS + the Vector bridge when Kafka is
+              # externalized to MSK. Account-scoped so any managed cluster works
+              # without re-provisioning; tighten Resource to a specific cluster
+              # ARN to lock it down. Harmless when Kafka runs in-cluster.
+              - Effect: Allow
+                Action:
+                  - kafka-cluster:Connect
+                  - kafka-cluster:DescribeCluster
+                  - kafka-cluster:DescribeClusterDynamicConfiguration
+                Resource:
+                  - !Sub "arn:${AWS::Partition}:kafka:*:${AWS::AccountId}:cluster/*/*"
+              - Effect: Allow
+                Action:
+                  - kafka-cluster:*Topic*
+                  - kafka-cluster:WriteData
+                  - kafka-cluster:ReadData
+                Resource:
+                  - !Sub "arn:${AWS::Partition}:kafka:*:${AWS::AccountId}:topic/*/*/*"
+              - Effect: Allow
+                Action:
+                  - kafka-cluster:AlterGroup
+                  - kafka-cluster:DescribeGroup
+                Resource:
+                  - !Sub "arn:${AWS::Partition}:kafka:*:${AWS::AccountId}:group/*/*/*"
+
+  # Pod Identity associations (hps / vector / clickhouse / backup / prometheus) are
+  # namespace-scoped, so the Rulebricks CLI creates them at `rulebricks deploy`
+  # time. This stack only provisions the deployment-independent role, bucket, and
+  # AMP workspace, so one cluster can host many deployments.
+
+Outputs:
+  ClusterName:
+    Value: !Ref Cluster
+  KubeconfigCommand:
+    Value: !Sub "aws eks update-kubeconfig --name ${ClusterName} --region ${AWS::Region}"
+  DataBucketName:
+    Value: !Ref DataBucket
+  RulebricksRoleArn:
+    Value: !GetAtt RulebricksRole.Arn
+  PrometheusWorkspaceId:
+    Value: !Ref PrometheusWorkspace
+  PrometheusRemoteWriteUrl:
+    Description: Set as Prometheus remote_write url (append /api/v1/remote_write).
+    Value: !GetAtt PrometheusWorkspace.PrometheusEndpoint

package/cluster-setup/azure/README.md

@@ -0,0 +1,141 @@
+# Azure Cluster Setup
+
+A compact, turnkey AKS cluster for Rulebricks. One Bicep deploy creates the
+cluster **and** the object storage + Azure Monitor resources the platform needs,
+fully wired to Workload Identity. Bring-your-own infra is also supported.
+
+## Files
+
+- `rulebricks-cluster.bicep` — AKS cluster (Azure CNI, Calico, Standard LB, Disk CSI, OIDC issuer, Workload Identity) plus the single Rulebricks identity and its data paths.
+- `parameters.json` — sample parameters (turnkey defaults: all paths on).
+- `check-aks-prereqs.sh` — verifies login, providers, quota, role-assignment rights, kubectl/helm.
+
+## One identity, one container (deployment-independent)
+
+A single user-assigned identity, `<cluster>-rulebricks`, holds both data roles,
+and all data lives in one container, `<cluster>-data`, under per-purpose prefixes.
+Toggle each path with its `enable*` flag.
+
+| Path                              | Service account             | Role / target                                                   |
+| --------------------------------- | --------------------------- | --------------------------------------------------------------- |
+| Decision logs (Vector → Blob)     | `vector`                    | Storage Blob Data Contributor → `<cluster>-data/decision-logs/` |
+| DB backups (job → Blob)           | `<release>-backup`          | Storage Blob Data Contributor → `<cluster>-data/db-backups/`    |
+| Metrics (Prometheus remote write) | `prometheus`                | Monitoring Metrics Publisher → Azure Monitor DCR                |
+
+The identity has Storage Blob Data Contributor on the storage account and
+Monitoring Metrics Publisher on the DCR.
+
+> **This template does not need a deployment name.** Federated identity credentials are
+> `namespace`-scoped (`system:serviceaccount:rulebricks-<deploymentName>:<sa>`), so they can't be
+> created until the deployment namespace is known. The **Rulebricks CLI creates them at
+> `rulebricks deploy` time** against this identity. That keeps cluster-setup generic, so one cluster
+> can host any number of deployments without re-running it — the CLI adds each deployment's
+> credentials on deploy. (Azure wildcard "flexible" FICs would avoid even that, but they're
+> unsupported on managed identities and AKS OIDC issuers.)
+
+## Turnkey vs. bring-your-own
+
+- `createStorage: true` provisions a storage account + the single `<cluster>-data` container (deterministic globally-unique account name). `false` → set `existingStorageAccountName`.
+- `createMonitorWorkspace: true` provisions an Azure Monitor workspace + data collection endpoint + rule, so the metrics role is scoped to a DCR we own. `false` → set `existingDataCollectionRuleId`.
+
+Defaults are turnkey: `createStorage`, `createMonitorWorkspace`, and all `enable*` flags are `true`.
+
+## Core cluster parameters
+
+`clusterName` (`rulebricks-cluster`), `location` (`eastus`), `kubernetesVersion`
+(`1.34`), `nodeCount`/`maxNodeCount` (`2`/`4`), `nodeVmSize`
+(`Standard_F4as_v6`), `maxPods` (`110`), `osDiskSizeGB` (`64`), `osDiskType`
+(`Managed`). The default (core) pool runs the always-on services on two to
+four 4-vCPU nodes; burst capacity lives in the dedicated burst pool below.
+The `110` max-pods avoids the legacy 30/node limit, and the autoscaler
+profile is tuned for bursts (`scan-interval` 10s, `least-waste` expander).
+Both pools use `Deallocate` scale-down: removed nodes are parked (disk-only
+cost, container images cached) and resume in ~30-60s instead of
+reprovisioning.
+
+### Burst worker pool (default on)
+
+`enableBurstPool` (`true`), `burstVmSize` (`Standard_F16as_v6`, 16 vCPU -
+the Fas_v6 family has no 24-vCPU size), `burstMaxCount` (`1`). One large
+`User`-mode node that scales 0 -> 1 on demand and parks between bursts. It
+is labeled and tainted `rulebricks.com/pool=burst`: the Rulebricks chart
+makes workers tolerate the taint and softly prefer the label out of the box,
+so the entire scaled-out worker fleet lands on this node while core services
+stay on the default pool. Sizing math: 2 x 4 vCPU core floor + 16 vCPU burst
+= 24 vCPU running steady-state at full burst, and exactly 32 vCPU even with
+the core pool at its 4-node max - sized to a 32-vCPU family quota.
+First-ever burst
+cold-provisions the VM (~2-4 min); every burst after resumes the parked VM
+(~30-60s). Note deallocated VMs resume into their original zone/SKU - in a
+capacity-constrained region a resume can fail and the autoscaler retries;
+the warm worker floor on the core pool carries traffic in the meantime.
+
+## Check access
+
+```bash
+az login
+az account set --subscription <subscription-id>
+AZURE_LOCATION=eastus bash check-aks-prereqs.sh
+```
+
+Register any flagged providers with the suggested `az provider register`
+commands and wait for completion. Note: creating role assignments needs
+**Owner** or **User Access Administrator** — Contributor alone is not enough.
+
+## Create the cluster
+
+```bash
+az group create --name rulebricks-rg --location eastus
+az deployment group create \
+  --resource-group rulebricks-rg \
+  --template-file rulebricks-cluster.bicep \
+  --parameters @parameters.json
+
+az aks get-credentials --name rulebricks-cluster --resource-group rulebricks-rg
+```
+
+Run `rulebricks init` once kubeconfig works, then select this cluster. The
+deploy emits `rulebricksClientId`, the generated `storageAccountName`, the
+`dataContainer` name, and `dceMetricsIngestionEndpoint` / `dcrImmutableId` for
+the CLI to consume.
+
+> Managed-Prometheus role assignments take ~30 min to propagate; expect HTTP 403
+> in the Prometheus log until then. This is expected, not a misconfiguration.
+
+## Delete the cluster
+
+Run `rulebricks destroy <deployment-name>` first so Kubernetes removes
+LoadBalancer services and PVC-backed disks. Then delete the resource group:
+
+```bash
+az group delete --name rulebricks-rg --yes
+```
+
+AKS cascade-deletes its `MC_<rg>_<cluster>_<region>` node resource group, so
+this removes the cluster, node pool, identities, role assignments, federated
+credentials, and (when created by the template) the storage account and Azure
+Monitor workspace.
+
+## Notes
+
+- Inbound TCP `80`/`443` are open to the AKS subnet for LoadBalancer services and cert-manager HTTP-01 validation.
+- `maxPods` is fixed at node-pool creation; changing it means a replacement pool or recreate.
+- Federated identity credentials for vector/backup/prometheus are created by the Rulebricks CLI at deploy time, so this template takes no deployment name. (The optional `external-dns` path is the one exception — set `rulebricksNamespace` if you enable it.)
+- BYO storage/monitor resources outside this resource group need an admin to assign the relevant role to the emitted identity client ID.
+
+## Fallback secret-based auth
+
+If Workload Identity is unavailable, decision-log export can use a storage
+connection string, and metrics can use OAuth client-secret auth:
+
+```bash
+kubectl create secret generic azure-blob-logs \
+  --namespace rulebricks-demo \
+  --from-literal=connection-string='<connection-string>'
+# CLI prompt: azure-blob-logs:connection-string
+
+kubectl create secret generic azure-monitor-oauth \
+  --namespace rulebricks-demo \
+  --from-literal=client-secret='<client-secret>'
+# CLI prompt: azure-monitor-oauth:client-secret
+```