container-superposition 0.1.1 → 0.1.4

package/overlays/duckdb/README.md

@@ -0,0 +1,274 @@
+# DuckDB Overlay
+
+In-process analytical database for OLAP workloads and data analysis.
+
+## Features
+
+- **DuckDB CLI** - Command-line interface for interactive queries
+- **In-process** - No separate server required, embedded database
+- **OLAP optimized** - Columnar storage for analytical queries
+- **SQL support** - Full ANSI SQL with extensions
+- **Parquet/CSV support** - Direct querying of file formats
+- **Python integration** - Works seamlessly with Python and pandas
+
+## How It Works
+
+This overlay installs the DuckDB CLI, an in-process analytical database designed for OLAP workloads. Unlike traditional databases, DuckDB runs embedded in your application without requiring a separate server.
+
+**Suggested overlays:**
+
+- `python` - For DuckDB Python API
+- `jupyter` - Interactive data analysis
+
+## Installation
+
+DuckDB CLI is installed automatically during devcontainer creation via `setup.sh`:
+
+- Downloads DuckDB CLI for your architecture (amd64/aarch64)
+- Installs to `/usr/local/bin/duckdb`
+- Verifies installation with test query
+
+## Common Commands
+
+### Interactive Shell
+
+```bash
+# Start DuckDB with in-memory database
+duckdb
+
+# Create/open persistent database
+duckdb mydata.db
+
+# Open database read-only
+duckdb mydata.db -readonly
+```
+
+### Query Files
+
+```bash
+# Query CSV file
+duckdb -c "SELECT * FROM 'data.csv' LIMIT 10"
+
+# Query Parquet file
+duckdb -c "SELECT * FROM 'data.parquet' WHERE col > 100"
+
+# Query JSON file
+duckdb -c "SELECT * FROM read_json_auto('data.json')"
+```
+
+### Create Tables
+
+```bash
+# From CSV
+CREATE TABLE sales AS SELECT * FROM 'sales.csv';
+
+# From Parquet
+CREATE TABLE events AS SELECT * FROM 'events.parquet';
+
+# From query
+CREATE TABLE summary AS
+  SELECT date, SUM(amount) as total
+  FROM sales
+  GROUP BY date;
+```
+
+### Export Results
+
+```bash
+# Export to CSV
+COPY (SELECT * FROM sales) TO 'output.csv' (HEADER, DELIMITER ',');
+
+# Export to Parquet
+COPY sales TO 'output.parquet' (FORMAT PARQUET);
+
+# Export to JSON
+COPY (SELECT * FROM sales LIMIT 100) TO 'sample.json';
+```
+
+## Python Integration
+
+Install DuckDB Python package:
+
+```python
+# In Python code or Jupyter notebook
+!pip install duckdb
+
+import duckdb
+
+# Connect to database
+con = duckdb.connect('mydata.db')
+
+# Execute query
+result = con.execute("SELECT * FROM sales WHERE amount > 100").fetchall()
+
+# Query pandas DataFrame directly
+import pandas as pd
+df = pd.read_csv('data.csv')
+result = con.execute("SELECT * FROM df WHERE col > 100").df()
+
+# Close connection
+con.close()
+```
+
+### Pandas Integration
+
+```python
+import duckdb
+import pandas as pd
+
+# Read CSV into DuckDB, query, get pandas DataFrame
+df = duckdb.query("SELECT * FROM 'data.csv' WHERE col > 100").df()
+
+# Query DataFrame with SQL
+df2 = duckdb.query("SELECT col1, AVG(col2) FROM df GROUP BY col1").df()
+```
+
+## Use Cases
+
+- **Data analysis** - Ad-hoc analysis of large datasets
+- **ETL pipelines** - Transform data with SQL
+- **Analytics** - OLAP queries on columnar data
+- **Data science** - Integrate with Python/Jupyter workflows
+- **Data exploration** - Quick insights from CSV/Parquet files
+- **Embedded analytics** - In-app analytical queries
+
+**Integrates well with:**
+
+- `python` - DuckDB Python API (suggested)
+- `jupyter` - Interactive data analysis (suggested)
+- Node.js, .NET - DuckDB client libraries available
+
+## SQL Examples
+
+### Analytical Queries
+
+```sql
+-- Time series aggregation
+SELECT
+  date_trunc('day', timestamp) as day,
+  COUNT(*) as events,
+  AVG(duration) as avg_duration
+FROM events
+WHERE timestamp > '2024-01-01'
+GROUP BY day
+ORDER BY day;
+
+-- Window functions
+SELECT
+  user_id,
+  event_time,
+  event_type,
+  ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY event_time) as event_seq
+FROM user_events;
+
+-- Join multiple files
+SELECT
+  u.name,
+  COUNT(o.id) as order_count,
+  SUM(o.amount) as total_spent
+FROM 'users.csv' u
+JOIN 'orders.parquet' o ON u.id = o.user_id
+GROUP BY u.name
+ORDER BY total_spent DESC;
+```
+
+### File Formats
+
+```sql
+-- CSV with options
+SELECT * FROM read_csv_auto('data.csv', header=true, delim='|');
+
+-- Parquet with projection
+SELECT col1, col2 FROM 'data.parquet' WHERE col3 > 100;
+
+-- JSON with auto-detection
+SELECT * FROM read_json_auto('data.json');
+
+-- Excel files
+INSTALL spatial;
+LOAD spatial;
+SELECT * FROM ST_Read('data.xlsx');
+```
+
+## Benefits vs PostgreSQL
+
+| Feature               | DuckDB                | PostgreSQL              |
+| --------------------- | --------------------- | ----------------------- |
+| **Use Case**          | ✅ Analytics (OLAP)   | ✅ Transactions (OLTP)  |
+| **Setup**             | ✅ No server needed   | ⚠️ Separate service     |
+| **File Queries**      | ✅ Direct CSV/Parquet | ❌ Requires COPY        |
+| **Analytical Speed**  | ✅ Very fast          | ⚠️ Slower for analytics |
+| **Concurrent Writes** | ⚠️ Limited            | ✅ Excellent            |
+| **Data Size**         | ✅ GBs-TBs            | ✅ TBs+                 |
+
+**When to use DuckDB:**
+
+- Analytical queries on read-heavy data
+- Working with files (CSV, Parquet, JSON)
+- Embedded analytics in applications
+- Data science and exploration
+
+**When to use PostgreSQL:**
+
+- Transactional workloads (OLTP)
+- Concurrent writes from multiple clients
+- Need full ACID guarantees
+- Large-scale production databases
+
+## Troubleshooting
+
+### Installation Fails
+
+Check architecture is supported:
+
+```bash
+uname -m
+# Should be x86_64 or aarch64/arm64
+```
+
+### File Not Found
+
+Ensure file paths are correct:
+
+```bash
+# Use absolute paths or paths relative to working directory
+duckdb -c "SELECT * FROM '/absolute/path/data.csv'"
+duckdb -c "SELECT * FROM './relative/path/data.csv'"
+```
+
+### Out of Memory
+
+DuckDB uses memory-mapped files. For large datasets:
+
+```sql
+-- Limit memory usage
+SET memory_limit='4GB';
+
+-- Use temp directory
+SET temp_directory='/tmp';
+```
+
+### Slow Queries
+
+DuckDB is optimized for analytics but:
+
+```sql
+-- Create indexes for point queries
+CREATE INDEX idx_id ON sales(id);
+
+-- Use columnar storage
+COPY sales TO 'sales.parquet' (FORMAT PARQUET);
+```
+
+## References
+
+- [DuckDB Documentation](https://duckdb.org/docs/)
+- [DuckDB Python API](https://duckdb.org/docs/api/python/overview)
+- [DuckDB SQL Reference](https://duckdb.org/docs/sql/introduction)
+- [DuckDB File Formats](https://duckdb.org/docs/data/overview)
+
+**Related Overlays:**
+
+- `python` - DuckDB Python API (suggested)
+- `jupyter` - Interactive analysis (suggested)
+- `postgres` - Transactional database

package/overlays/duckdb/devcontainer.patch.json

@@ -0,0 +1,10 @@
+{
+    "$schema": "https://raw.githubusercontent.com/devcontainers/spec/main/schemas/devContainer.base.schema.json",
+    "features": {
+        "./features/cross-distro-packages": {
+            "apt": "wget unzip",
+            "apk": "wget unzip"
+        }
+    },
+    "postCreateCommand": "bash .devcontainer/scripts/setup-duckdb.sh"
+}

package/overlays/duckdb/overlay.yml

@@ -0,0 +1,17 @@
+id: duckdb
+name: DuckDB
+description: In-process analytical database for OLAP workloads
+category: database
+supports: []
+requires: []
+suggests:
+    - python
+    - jupyter
+conflicts: []
+tags:
+    - database
+    - analytics
+    - sql
+    - duckdb
+    - olap
+ports: []

package/overlays/duckdb/setup.sh

@@ -0,0 +1,45 @@
+#!/bin/bash
+# Setup script for DuckDB
+
+set -e
+
+echo "🔧 Setting up DuckDB..."
+
+# Detect architecture
+ARCH=$(uname -m)
+case $ARCH in
+    x86_64)
+        DUCKDB_ARCH="amd64"
+        ;;
+    aarch64|arm64)
+        DUCKDB_ARCH="aarch64"
+        ;;
+    *)
+        echo "❌ Unsupported architecture: $ARCH"
+        exit 1
+        ;;
+esac
+
+# Install DuckDB CLI
+DUCKDB_VERSION="${DUCKDB_VERSION:-v1.0.0}"
+echo "📦 Installing DuckDB CLI ${DUCKDB_VERSION}..."
+
+wget -q "https://github.com/duckdb/duckdb/releases/download/${DUCKDB_VERSION}/duckdb_cli-linux-${DUCKDB_ARCH}.zip" -O /tmp/duckdb.zip
+unzip -q /tmp/duckdb.zip -d /tmp/
+chmod +x /tmp/duckdb
+sudo mv /tmp/duckdb /usr/local/bin/duckdb
+rm /tmp/duckdb.zip
+
+# Verify installation
+if command -v duckdb &> /dev/null; then
+    echo "✅ DuckDB CLI installed successfully"
+    duckdb --version
+else
+    echo "❌ DuckDB CLI installation failed"
+    exit 1
+fi
+
+echo "✅ DuckDB setup complete"
+echo ""
+echo "ℹ️  To start DuckDB, run:"
+echo "   duckdb mydata.db"

package/overlays/duckdb/verify.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+# Verification script for DuckDB overlay
+# Confirms DuckDB is installed
+
+set -e
+
+echo "🔍 Verifying DuckDB overlay..."
+echo ""
+
+# Check DuckDB CLI is installed
+echo "1️⃣ Checking DuckDB CLI installation..."
+if command -v duckdb &> /dev/null; then
+    duckdb --version
+    echo "   ✅ DuckDB CLI is installed"
+else
+    echo "   ❌ DuckDB CLI is not installed"
+    exit 1
+fi
+
+# Test DuckDB with a simple query
+echo ""
+echo "2️⃣ Testing DuckDB with simple query..."
+RESULT=$(echo "SELECT 'DuckDB is working!' as message;" | duckdb 2>&1 | grep -i "DuckDB is working" || true)
+if [ -n "$RESULT" ]; then
+    echo "   ✅ DuckDB query executed successfully"
+else
+    echo "   ❌ DuckDB query failed"
+    exit 1
+fi
+
+echo ""
+echo "✅ DuckDB overlay verification complete"

package/overlays/git-helpers/overlay.yml

@@ -13,3 +13,4 @@ tags:
     - ssh
     - gpg
 ports: []
+minimal: true

package/overlays/grafana/README.md

@@ -300,7 +300,7 @@ Access at: http://localhost:3000/dashboards
 2. Click **Dashboard settings** (gear icon)
 3. Click **JSON Model**
 4. Copy JSON
-5. Save to `.devcontainer/dashboards/my-dashboard.json`
+5. Save to `.devcontainer/dashboards-grafana/my-dashboard.json`
 6. Restart Grafana to load
 
 **Option 2: Export from Grafana.com**
@@ -308,14 +308,14 @@ Access at: http://localhost:3000/dashboards
 1. Browse https://grafana.com/grafana/dashboards/
 2. Find dashboard (e.g., Node Exporter Full)
 3. Download JSON
-4. Save to `.devcontainer/dashboards/`
+4. Save to `.devcontainer/dashboards-grafana/`
 5. Restart Grafana
 
 **Dashboard structure:**
 
 ```
 .devcontainer/
-├── dashboards/
+├── dashboards-grafana/
 │   ├── observability-overview.json
 │   ├── my-custom-dashboard.json
 │   └── node-exporter.json
@@ -376,13 +376,13 @@ docker exec tempo cat /etc/tempo/tempo-config.yaml
 docker exec grafana ls /etc/grafana/provisioning/dashboards/
 
 # Check dashboard files are present
-docker exec grafana ls /etc/grafana/provisioning/dashboards/*.json
+docker exec grafana ls /etc/grafana/dashboards/*.json
 
 # Check Grafana logs for errors
 docker logs grafana | grep -i dashboard
 
 # Validate JSON syntax
-jq . .devcontainer/dashboards/my-dashboard.json
+jq . .devcontainer/dashboards-grafana/my-dashboard.json
 ```
 
 ### Issue: Correlation Links Not Working

package/overlays/grafana/dashboard-provider.yml

@@ -8,4 +8,4 @@ providers:
       disableDeletion: false
       editable: true
       options:
-          path: /etc/grafana/provisioning/dashboards
+          path: /etc/grafana/dashboards

package/overlays/grafana/docker-compose.yml

@@ -8,8 +8,8 @@ services:
             - GF_USERS_ALLOW_SIGN_UP=false
         volumes:
             - ./grafana-datasources-grafana.yml:/etc/grafana/provisioning/datasources/datasources.yml
-            - ./dashboard-provider-grafana.yml:/etc/grafana/provisioning/dashboards/dashboards.yml
-            - ./dashboards:/etc/grafana/provisioning/dashboards
+            - ./dashboard-provider-grafana.yml:/etc/grafana/provisioning/dashboards/provider.yml
+            - ./dashboards-grafana:/etc/grafana/dashboards
             - grafana_data:/var/lib/grafana
         ports:
             - '${GRAFANA_PORT:-3000}:3000'

package/overlays/grafana/overlay.yml

@@ -17,5 +17,10 @@ tags:
     - ui
     - visualization
 ports:
-    - 3000
+    - port: 3000
+      service: grafana
+      protocol: http
+      description: Grafana web UI
+      path: /
+      onAutoForward: openBrowser
 order: 3

package/overlays/grpc-tools/README.md

@@ -0,0 +1,242 @@
+# gRPC Tools Overlay
+
+Protocol Buffers compiler, Buf schema management CLI, and grpcurl for gRPC API development.
+
+## Features
+
+- **protoc** - Protocol Buffers compiler (from system packages)
+- **buf** - Modern schema management, linting, and breaking change detection
+- **grpcurl** - Command-line tool for interacting with gRPC servers (like curl for gRPC)
+- **VS Code Extensions:**
+    - [vscode-proto3](https://marketplace.visualstudio.com/items?itemName=zxh404.vscode-proto3) - Proto3 syntax highlighting and validation
+    - [vscode-buf](https://marketplace.visualstudio.com/items?itemName=bufbuild.vscode-buf) - Buf integration for VS Code
+
+## How It Works
+
+This overlay installs the gRPC development toolchain into the dev container. `protoc` is installed via system packages, while `buf` and `grpcurl` are downloaded from their official GitHub releases.
+
+## Common Commands
+
+### protoc (Protocol Buffers Compiler)
+
+```bash
+# Compile .proto files to Go
+protoc --go_out=. --go-grpc_out=. api/v1/service.proto
+
+# Compile to Python
+protoc --python_out=. api/v1/service.proto
+
+# Compile to JavaScript/TypeScript
+protoc --js_out=import_style=commonjs,binary:. api/v1/service.proto
+
+# Check version
+protoc --version
+```
+
+### buf (Schema Management)
+
+```bash
+# Initialize buf workspace
+buf config init
+
+# Lint .proto files
+buf lint
+
+# Detect breaking changes
+buf breaking --against .git#branch=main
+
+# Generate code from .proto files
+buf generate
+
+# Format .proto files
+buf format -w
+
+# Push schema to Buf Schema Registry
+buf push
+
+# Check version
+buf --version
+```
+
+### grpcurl (gRPC Testing)
+
+```bash
+# List all services on a gRPC server
+grpcurl -plaintext localhost:50051 list
+
+# List methods on a service
+grpcurl -plaintext localhost:50051 list mypackage.MyService
+
+# Describe a method
+grpcurl -plaintext localhost:50051 describe mypackage.MyService.MyMethod
+
+# Call a method
+grpcurl -plaintext -d '{"name": "World"}' localhost:50051 mypackage.MyService/SayHello
+
+# Call with headers
+grpcurl -plaintext \
+  -H "Authorization: Bearer $TOKEN" \
+  -d '{"id": 1}' \
+  localhost:50051 mypackage.MyService/GetUser
+
+# Stream response
+grpcurl -plaintext localhost:50051 mypackage.MyService/ListUsers
+```
+
+## Example .proto File
+
+```protobuf
+syntax = "proto3";
+
+package myservice.v1;
+
+option go_package = "github.com/myorg/myservice/gen/go/myservice/v1";
+
+service UserService {
+    rpc GetUser(GetUserRequest) returns (GetUserResponse);
+    rpc ListUsers(ListUsersRequest) returns (stream User);
+    rpc CreateUser(CreateUserRequest) returns (CreateUserResponse);
+}
+
+message User {
+    string id = 1;
+    string name = 2;
+    string email = 3;
+}
+
+message GetUserRequest {
+    string id = 1;
+}
+
+message GetUserResponse {
+    User user = 1;
+}
+
+message ListUsersRequest {}
+
+message CreateUserRequest {
+    string name = 1;
+    string email = 2;
+}
+
+message CreateUserResponse {
+    User user = 1;
+}
+```
+
+## buf.yaml Configuration
+
+```yaml
+version: v2
+modules:
+    - path: proto
+lint:
+    use:
+        - DEFAULT
+breaking:
+    use:
+        - FILE
+```
+
+## buf.gen.yaml (Code Generation)
+
+```yaml
+version: v2
+plugins:
+    # Go
+    - remote: buf.build/protocolbuffers/go
+      out: gen/go
+      opt:
+          - paths=source_relative
+    - remote: buf.build/grpc/go
+      out: gen/go
+      opt:
+          - paths=source_relative
+
+    # Python
+    - remote: buf.build/protocolbuffers/python
+      out: gen/python
+
+    # TypeScript (Node.js)
+    - remote: buf.build/connectrpc/node
+      out: gen/typescript
+```
+
+## Language-Specific Setup
+
+### Go
+
+```bash
+# Install Go gRPC plugins
+go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
+go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
+```
+
+### Node.js
+
+```bash
+# Install grpc-tools for Node.js
+npm install --save-dev grpc-tools @grpc/grpc-js @grpc/proto-loader
+```
+
+### Python
+
+```bash
+# Install Python gRPC tools
+pip install grpcio grpcio-tools
+```
+
+## Use Cases
+
+- **gRPC API development** - Design, implement, and test gRPC services
+- **Protocol Buffers schema management** - Lint, format, and version schemas
+- **gRPC endpoint testing** - Test services without a full client implementation
+- **Microservice communication** - Develop services that communicate via gRPC
+- **API documentation** - Describe and explore gRPC service contracts
+
+## Troubleshooting
+
+### protoc plugins not found
+
+After installing language-specific plugins, ensure they are in your `PATH`:
+
+```bash
+# Go plugins
+export PATH="$PATH:$(go env GOPATH)/bin"
+
+# Verify
+which protoc-gen-go
+which protoc-gen-go-grpc
+```
+
+### buf lint errors
+
+Run `buf lint` to see all lint errors:
+
+```bash
+buf lint
+# proto/api/v1/service.proto:10:1:Service name "userService" should be PascalCase
+```
+
+### grpcurl: server does not support the reflection API
+
+Enable gRPC server reflection in your service:
+
+```go
+// Go example
+import "google.golang.org/grpc/reflection"
+reflection.Register(grpcServer)
+```
+
+## References
+
+- [Protocol Buffers Documentation](https://protobuf.dev/)
+- [Buf Documentation](https://buf.build/docs/)
+- [grpcurl GitHub](https://github.com/fullstorydev/grpcurl)
+- [gRPC Documentation](https://grpc.io/docs/)
+
+**Related Overlays:**
+
+- `go` - Go language runtime
+- `nodejs` - Node.js runtime
+- `python` - Python runtime