opdotenv 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 484498d0c2471d83e66fd42819feef80da6432e79c2166ceb15b1ece037b2046
+  data.tar.gz: fce7dbd63416abb37af84c4bdf78792d1172fa1bea972d467f7a871ca1d58e12
+SHA512:
+  metadata.gz: 319700ee96600edbb66bc90db27dadb07cfe269cc0f10570b582ae349f8f6234ff29a5950590a57763450c48a95ab1fcd3925bf0d929a1604d2e15a5cc335b11
+  data.tar.gz: 351b895887435dabeb2dec23884d741f192a16aa3c26be083490ff9a7be8189b06d03d8c8f0e7e3373a875d3d8edda5aabc620e5b2a157f05390293fb5cfbaf9

data/CHANGELOG.md

@@ -0,0 +1,15 @@
+# CHANGELOG
+
+## 1.0.0 (2025-11-04)
+
+- Initial stable release
+- Load environment variables from 1Password using op CLI or Connect API
+- Support for dotenv, JSON, and YAML formats
+- Automatic format inference from file extensions
+- Rails integration with declarative configuration
+- Anyway Config integration with strict prefix matching
+- Export data back to 1Password
+- CLI tool for read/export operations
+- Support for field names in paths (e.g., op://Vault/Item/config.json)
+- Security improvements (input validation, safe YAML parsing)
+- Comprehensive documentation with security guidelines

data/LICENSE.md

@@ -0,0 +1,23 @@
+MIT License
+
+Copyright (c) 2025
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+

data/README.md

@@ -0,0 +1,333 @@
+# opdotenv
+
+[![Gem Version](https://badge.fury.io/rb/opdotenv.svg)](https://badge.fury.io/rb/opdotenv) [![Test Status](https://github.com/amkisko/opdotenv/actions/workflows/ci.yml/badge.svg)](https://github.com/amkisko/opdotenv/actions/workflows/ci.yml) [![codecov](https://codecov.io/gh/amkisko/opdotenv/graph/badge.svg?token=YOUR_TOKEN)](https://codecov.io/gh/amkisko/opdotenv/graph/badge.svg?token=YOUR_TOKEN)
+
+Load environment variables from 1Password using the `op` CLI or 1Password Connect Server API. Supports dotenv, JSON, and YAML formats.
+
+Sponsored by [Kisko Labs](https://www.kiskolabs.com).
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem "opdotenv"
+```
+
+## Requirements
+
+Choose one:
+- **1Password CLI** (`op`) - must be installed and authenticated (`op signin`)
+- **1Password Connect Server** - set `OP_CONNECT_URL` and `OP_CONNECT_TOKEN` environment variables
+
+Ruby 2.7+ supported.
+
+## Rails
+
+Configure in `config/application.rb` or environment-specific files:
+
+```ruby
+Rails.application.configure do
+  config.opdotenv.sources = [
+    "op://Vault/.env.development",  # dotenv format (inferred)
+    "op://Vault/config.json",        # json format (inferred from .json extension)
+    "op://Vault/App"                 # all fields without parsing
+  ]
+end
+```
+
+Format is automatically inferred from item name or field name:
+- `.env.*` → dotenv format
+- `*.json` → JSON format
+- `*.yaml` or `*.yml` → YAML format
+- Other items → load all fields without parsing
+
+You can also specify the field name with extension in the path:
+- `op://Vault/Item Name/config.json` → uses field `config.json` as JSON
+- `op://Vault/Item Name/production.json` → uses field `production.json` as JSON
+- `op://Vault/Item Name/.env.development` → uses field `.env.development` as dotenv
+
+### 1Password Connect
+
+```ruby
+Rails.application.configure do
+  config.opdotenv.connect_url = "https://connect.example.com"
+  config.opdotenv.connect_token = Rails.application.credentials.dig(:op_connect, :token)
+end
+```
+
+### Disable automatic loading
+
+```ruby
+Rails.application.configure do
+  config.opdotenv.auto_load = false
+end
+
+# Load manually when needed
+Opdotenv::Loader.load("op://Vault/Item")
+```
+
+## Standalone usage
+
+```ruby
+require "opdotenv"
+
+# Load from dotenv format (format inferred from item name)
+Opdotenv::Loader.load("op://Vault/.env.development")
+
+# Load from JSON format (any item name ending with .json)
+Opdotenv::Loader.load("op://Vault/config.json")
+Opdotenv::Loader.load("op://Vault/production.json")
+
+# Load from field with extension in path
+Opdotenv::Loader.load("op://Vault/Item Name/config.json")
+
+# Load all fields from an item
+Opdotenv::Loader.load("op://Vault/App")
+
+# Don't overwrite existing ENV values
+Opdotenv::Loader.load("op://Vault/Item", overwrite: false)
+```
+
+## Anyway Config integration
+
+Automatically registers when `anyway_config` is available:
+
+```ruby
+class AppConfig < Anyway::Config
+  attr_config :api_key, :api_secret
+
+  # Format is inferred from item name
+  loader_options opdotenv: {
+    path: "op://Vault/.env.development"  # dotenv format inferred
+  }
+end
+
+# Or load all fields from an item
+class DatabaseConfig < Anyway::Config
+  attr_config :url, :username, :password
+
+  loader_options opdotenv: {
+    path: "op://Vault/Database"  # all fields loaded
+  }
+end
+```
+
+### Conditional Loading (Recommended for Security)
+
+For better security, only load from 1Password in development/test environments:
+
+```ruby
+class TestConfig < Anyway::Config
+  config_name :test
+  attr_config :enabled, :sample
+
+  # Only load from 1Password in local/development environments
+  if Rails.env.local?
+    loader_options opdotenv: {
+      path: "op://Employee/.env.test"
+    }
+  end
+end
+```
+
+This ensures that production environments won't attempt to load secrets from 1Password, aligning with the [production recommendations](#environment-recommendation).
+
+### Loading All Fields from an Item
+
+When loading all fields from a 1Password item (not a parsed format), field names are automatically normalized to match the `env_prefix`:
+
+```ruby
+class TestConfig < Anyway::Config
+  config_name :test
+  attr_config :enabled, :sample
+
+  loader_options opdotenv: {
+    path: "op://Employee/TestConfig"  # Loads all fields from item
+  }
+end
+```
+
+**Field name matching (strict with case-insensitive prefix):**
+- Fields in 1Password **must** be prefixed with the `env_prefix` (e.g., `TEST_` for `config_name :test`)
+- Matching is **case-insensitive**: `TEST_ENABLED`, `test_enabled`, `Test_Enabled` all work
+- After prefix stripping, `TEST_ENABLED` becomes `enabled` (matching `attr_config :enabled`)
+- Fields without the prefix (e.g., `enabled`, `ENABLED`) are ignored and logged as unmatched
+
+**Debugging field matching:**
+- Enable debug logging by setting `OPDOTENV_DEBUG=true`
+- Check Rails logs for messages like:
+  ```
+  [opdotenv] Available fields from 1Password: enabled, ENABLED, sample, SAMPLE
+  [opdotenv] Matched fields for TEST: enabled, sample
+  [opdotenv] Unmatched fields (must be prefixed with TEST_, case-insensitive): other_field
+  ```
+
+## Using with dotenv
+
+Load order determines which values take precedence:
+
+```ruby
+require "dotenv"
+require "opdotenv"
+
+# Load local files first, then augment from 1Password (1Password values override by default)
+Dotenv.load(".env", ".env.development")
+Opdotenv::Loader.load("op://Vault/.env.development")
+
+# Or load from 1Password first, then local files (local values override)
+Opdotenv::Loader.load("op://Vault/.env.development", overwrite: false)
+Dotenv.load(".env", ".env.development")
+```
+
+## Export to 1Password
+
+### CLI
+
+```bash
+# Export .env file (format inferred from path)
+opdotenv export --path "op://Vault/.env.development" --file .env.development
+
+# Export to item fields
+opdotenv export --path "op://Vault/App" --file .env
+
+# Read and print (format inferred from path)
+opdotenv read --path "op://Vault/.env.development"
+```
+
+### Ruby API
+
+```ruby
+# Export to Secure Note (format inferred from path)
+Opdotenv::Exporter.export(
+  path: "op://Vault/.env.development",
+  data: {"API_KEY" => "secret"}
+)
+
+# Export to item fields
+Opdotenv::Exporter.export(
+  path: "op://Vault/App",
+  data: {"API_KEY" => "secret", "API_SECRET" => "another"}
+)
+```
+
+## Supported formats
+
+Format is automatically inferred from item name or field name:
+- `.env.*` → dotenv format (`KEY=VALUE`)
+- `*.json` → JSON format (nested structures flattened with underscores)
+- `*.yaml` or `*.yml` → YAML format (nested structures flattened with underscores)
+- Other items → load all fields without parsing
+
+Field names can be specified with extensions in the path:
+- `op://Vault/Item Name/config.json` → loads field `config.json` as JSON
+- `op://Vault/Item Name/production.json` → loads field `production.json` as JSON
+- `op://Vault/Item Name/.env.development` → loads field `.env.development` as dotenv
+
+For advanced usage, you can explicitly specify `field_name` and `field_type` in the API.
+
+## Security
+
+### Environment Recommendation
+
+**⚠️ This gem is recommended for development and test environments only.**
+
+For production environments, we recommend using dedicated secret management solutions that integrate with your infrastructure:
+
+- **Kubernetes**: Use [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) or [External Secrets Operator](https://external-secrets.io/) with providers like AWS Secrets Manager, HashiCorp Vault, or Azure Key Vault
+- **AWS**: Use [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) or [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html)
+- **Azure**: Use [Azure Key Vault](https://azure.microsoft.com/en-us/products/key-vault)
+- **GCP**: Use [Google Secret Manager](https://cloud.google.com/secret-manager)
+
+Provision secrets through infrastructure-as-code tools:
+- **Helm** (Kubernetes): Use `helm secrets` or external secrets operator
+- **Terraform**: Use `aws_secretsmanager_secret`, `azurerm_key_vault_secret`, or `google_secret_manager_secret`
+- **Bicep** (Azure): Use `Microsoft.KeyVault/vaults` resources
+
+These solutions provide:
+- Better audit trails and access controls
+- Integration with IAM/RBAC systems
+- Automatic secret rotation
+- Compliance with security standards
+- No dependency on external CLI tools or API servers
+
+### Security Considerations
+
+- **CLI mode**: Secrets fetched via authenticated `op` CLI session.
+- **Connect API mode**: Secrets fetched via HTTPS. Ensure tokens are secure.
+- The library does not persist secrets in memory or on disk.
+- Always verify 1Password CLI or Connect server is up to date and authenticated.
+
+### Code Locations for Security Review
+
+For security-sensitive applications, developers should review where this gem reads and writes data from 1Password. The following locations handle secret data:
+
+#### Reading Secrets (OpClient - CLI mode)
+
+- **`lib/opdotenv/op_client.rb`**:
+  - `read(path)` - Executes `op read` command to fetch a single field value
+  - `item_get(item, vault:)` - Executes `op item get` command to fetch all item data as JSON
+  - `capture(args)` - Executes shell commands via `IO.popen` (array arguments, no shell interpretation)
+
+#### Reading Secrets (ConnectApiClient - API mode)
+
+- **`lib/opdotenv/connect_api_client.rb`**:
+  - `read(path)` - Makes HTTP GET request to fetch field or notesPlain content
+  - `item_get(item_title, vault:)` - Searches and fetches item data via API
+  - `get_item(vault_name, item_title)` - Fetches full item details including all fields
+  - `item_by_title_in_vault(vault_id, item_title)` - Lists items and fetches by title
+  - `list_vaults()` - Lists all accessible vaults (uses `api_request(:get, "/v1/vaults")`)
+  - `vault_name_to_id(vault_name)` - Resolves vault names to IDs (cached)
+  - `api_request(method, path, body)` - All HTTP requests go through this method
+  - `find_field(item, field_name)` - Searches item fields by label, ID, or purpose
+
+#### Main Entry Points
+
+- **`lib/opdotenv/loader.rb`**:
+  - `load(path, ...)` - Main entry point that orchestrates secret fetching
+  - `load_field(client, path, field_name, field_type)` - Loads and parses a single field
+  - `load_all_fields(client, path)` - Loads all fields from an item (skips notesPlain)
+  - `merge_into_env(env, hash, overwrite:)` - Writes secrets to environment hash
+
+#### Rails Integration
+
+- **`lib/opdotenv/railtie.rb`**:
+  - `initializer "opdotenv.load"` - Automatically loads secrets during Rails initialization
+  - Reads from `config.opdotenv.sources` array
+  - Sets `OP_CONNECT_URL` and `OP_CONNECT_TOKEN` from Rails config if provided
+
+#### Anyway Config Integration
+
+- **`lib/opdotenv/anyway_loader.rb`**:
+  - `Loader#call(...)` - Loads secrets for Anyway Config classes
+  - Uses `Opdotenv::Loader.load()` internally
+
+#### Parsing and Processing
+
+- **`lib/opdotenv/parsers/dotenv_parser.rb`** - Parses dotenv format strings
+- **`lib/opdotenv/parsers/json_parser.rb`** - Parses and flattens JSON structures
+- **`lib/opdotenv/parsers/yaml_parser.rb`** - Parses YAML (safe_load with aliases: false)
+
+#### Data Flow
+
+1. **Configuration** → Rails config or direct API calls
+2. **Path Parsing** → `SourceParser.parse()` extracts vault/item/field from path
+3. **Client Selection** → `ClientFactory.create()` chooses CLI or API client
+4. **Secret Fetching** → Client reads from 1Password (CLI command or HTTP request)
+5. **Parsing** → Format-specific parser converts to key-value pairs
+6. **Environment Merge** → Secrets merged into `ENV` or provided hash
+
+All secret data flows through these code paths. No secrets are persisted to disk or logged (except explicit error messages).
+
+## Development
+
+```bash
+bundle install
+bundle exec rspec
+bundle exec rbs validate
+bundle exec standardrb --fix
+```
+
+## License
+
+MIT

data/bin/opdotenv

@@ -0,0 +1,44 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
+require "opdotenv"
+require "optparse"
+
+cmd = ARGV.shift
+
+case cmd
+when "read"
+  path = nil
+  OptionParser.new do |o|
+    o.on("--path PATH", "op://Vault/Item") { |v| path = v }
+  end.parse!(ARGV)
+  abort("--path required") unless path
+  data = Opdotenv::Loader.load(path)
+  puts Opdotenv::Exporter.serialize_by_format(data, :dotenv)
+when "export"
+  path = file = nil
+  field_type = nil
+  OptionParser.new do |o|
+    o.on("--path PATH", "op://Vault/Item") { |v| path = v }
+    o.on("--file PATH", "Source file (.env/json/yaml)") { |v| file = v }
+    o.on("--field-type TYPE", [:dotenv, :json, :yaml, :yml], "Override format (default: inferred from path)") { |v| field_type = v }
+  end.parse!(ARGV)
+  abort("--path and --file required") unless path && file
+  # Infer format from file extension if not provided
+  field_type ||= infer_format_from_file(file)
+  text = File.read(file)
+  data = Opdotenv::Loader.parse_by_format(text, field_type)
+  Opdotenv::Exporter.export(path: path, data: data, field_type: field_type)
+else
+  warn "Usage: opdotenv (read|export) ..."
+  exit 1
+end
+
+def infer_format_from_file(file)
+  case File.extname(file)
+  when ".json" then :json
+  when ".yaml", ".yml" then :yaml
+  else :dotenv
+  end
+end

data/lib/opdotenv/anyway_loader.rb

@@ -0,0 +1,125 @@
+module Opdotenv
+  # Anyway Config loader to fetch configuration from 1Password via opdotenv.
+  # Usage per-config:
+  #   class MyConfig < Anyway::Config
+  #     loader_options opdotenv: {
+  #       path: "op://Vault/.env.development"  # format inferred from item name
+  #     }
+  #   end
+  module AnywayLoader
+    class Loader < ::Anyway::Loaders::Base
+      def call(name:, env_prefix:, config_path:, **opts)
+        options = opts[:opdotenv] || {}
+        return {} if options.empty?
+
+        path = options[:path] || options["path"]
+        raise ArgumentError, "opdotenv loader requires :path" unless path
+
+        parsed = SourceParser.parse(path)
+        overwrite = options.key?(:overwrite) ? options[:overwrite] : true
+
+        # Use a separate env hash to avoid side effects on global ENV
+        data = Opdotenv::Loader.load(
+          path,
+          field_name: parsed[:field_name],
+          field_type: parsed[:field_type],
+          env: {},
+          overwrite: overwrite
+        )
+
+        strip_prefix_from_keys(data, env_prefix)
+      end
+
+      private
+
+      def strip_prefix_from_keys(data, env_prefix)
+        return data unless defined?(::Anyway::Env) && defined?(::Anyway::NoCast)
+
+        # Strict matching with case-insensitive prefix: Only fields with env_prefix will be matched
+        # Normalize keys to uppercase for case-insensitive matching
+        normalized_data = normalize_keys_for_prefix_matching(data, env_prefix)
+
+        # Use Anyway::Env to handle prefix stripping, matching how Doppler loader works
+        # This transforms keys like "TEST_ENABLED" -> "enabled" when env_prefix is "TEST"
+        env = ::Anyway::Env.new(type_cast: ::Anyway::NoCast, env_container: normalized_data)
+        conf, trace = env.fetch_with_trace(env_prefix)
+
+        if defined?(::Anyway::Tracing) && ::Anyway::Tracing.current_trace
+          ::Anyway::Tracing.current_trace.merge!(trace)
+        end
+
+        # Log available fields for debugging (only when OPDOTENV_DEBUG is enabled)
+        log_available_fields(data, env_prefix, conf)
+
+        conf
+      end
+
+      def normalize_keys_for_prefix_matching(data, env_prefix)
+        return data if data.empty? || env_prefix.empty?
+
+        prefix_upper = env_prefix.upcase
+        prefix_with_underscore = "#{prefix_upper}_"
+
+        normalized = {}
+        data.each do |key, value|
+          key_str = key.to_s
+          key_upper = key_str.upcase
+
+          # Case-insensitive prefix matching: only include keys that start with PREFIX_
+          if key_upper.start_with?(prefix_with_underscore) || key_upper == prefix_upper
+            # Normalize to uppercase for consistent matching
+            normalized_key = key_upper
+            normalized[normalized_key] = value
+          end
+        end
+
+        normalized
+      end
+
+      def log_available_fields(original_data, env_prefix, matched_data)
+        return unless ENV["OPDOTENV_DEBUG"] == "true"
+        return unless defined?(Rails) && Rails.logger
+
+        prefix_upper = env_prefix.upcase
+        prefix_with_underscore = "#{prefix_upper}_"
+
+        available_fields = original_data.keys.map(&:to_s)
+        matched_fields = matched_data.keys.map(&:to_s)
+
+        # Find fields that were available but didn't match the prefix (case-insensitive)
+        unmatched = available_fields.reject do |field|
+          field_upper = field.to_s.upcase
+          field_upper.start_with?(prefix_with_underscore) || field_upper == prefix_upper
+        end
+
+        if available_fields.any?
+          Rails.logger.debug("[opdotenv] Available fields from 1Password: #{available_fields.join(", ")}")
+          Rails.logger.debug("[opdotenv] Matched fields for #{env_prefix} (prefixed with #{prefix_with_underscore}, case-insensitive): #{matched_fields.join(", ")}")
+          if unmatched.any?
+            Rails.logger.debug("[opdotenv] Unmatched fields (must be prefixed with #{prefix_with_underscore}, case-insensitive): #{unmatched.join(", ")}")
+            Rails.logger.debug("[opdotenv] To use these fields, rename them in 1Password to include the #{prefix_with_underscore} prefix")
+          end
+        end
+      end
+    end
+
+    def self.register!
+      ::Anyway.loaders.append :opdotenv, Loader
+    end
+  end
+end
+
+begin
+  # Auto-register if Anyway is available
+  # Silently skip if Anyway Config is not available or not fully loaded
+  if defined?(::Anyway) && ::Anyway.respond_to?(:loaders)
+    Opdotenv::AnywayLoader.register!
+  end
+rescue => e
+  # Only warn if debugging is enabled, as this is expected when Anyway Config isn't used
+  if ENV["OPDOTENV_DEBUG"] == "true"
+    warn "[opdotenv] Failed to register Anyway loader: #{e.message}"
+    warn "[opdotenv] Error details: #{e.class}: #{e.message}"
+    warn "[opdotenv] Backtrace: #{e.backtrace.first(3).join("\n")}" if e.backtrace
+  end
+end

data/lib/opdotenv/client_factory.rb

@@ -0,0 +1,17 @@
+module Opdotenv
+  class ClientFactory
+    # Creates appropriate client based on configuration or environment
+    # Supports both op CLI and Connect API
+    def self.create(env: ENV)
+      # Check for Connect API configuration
+      connect_url = env["OP_CONNECT_URL"] || env["OPDOTENV_CONNECT_URL"]
+      connect_token = env["OP_CONNECT_TOKEN"] || env["OPDOTENV_CONNECT_TOKEN"]
+
+      if connect_url && connect_token
+        ConnectApiClient.new(base_url: connect_url, access_token: connect_token, env: env)
+      else
+        OpClient.new(env: env)
+      end
+    end
+  end
+end