moose-inventory 2.0 → 2.1

data/docs/product/product-brief.md

@@ -0,0 +1,161 @@
+# Moose Inventory Product Brief
+
+## Approval status
+
+Status: **Approved product-framing baseline**
+
+Approval reference: `GOV-PRODUCT-001` in `docs/governance/approval-register.md` approves this document as the product-framing baseline for Moose Inventory.
+
+Approved tailoring reference: `GOV-TAILOR-001` in `docs/governance/approval-register.md` approves Moose Inventory as **Class 4 with target profile Software Library / Package**.
+
+Scope limit: this approval covers product framing only. It is not detailed requirements approval, CLI UX approval, architecture approval, security/privacy design approval, release approval, accepted-risk approval, public/compliance-claim approval, or RubyGems publishing approval.
+
+## Summary
+
+Moose Inventory is a RubyGem and CLI for managing Ansible-compatible dynamic inventory state across local SQLite, MySQL/MariaDB, and PostgreSQL databases. It helps operators and automation workflows create, inspect, change, validate, import/export, and audit inventory data without hand-editing dynamic inventory structures.
+
+The product is a software library/package and command-line tool, not a hosted service. It ships code, examples, CLI workflows, release artifacts, and documentation. It does not currently operate a production service on behalf of users.
+
+## Target users
+
+Primary users:
+
+- Infrastructure engineers managing Ansible inventory data.
+- DevOps/SRE practitioners who need repeatable CLI-based inventory changes.
+- Automation maintainers who want machine-readable inventory export, validation, and dry-run plans.
+- Ruby/Ansible users who prefer a packaged CLI and local database-backed inventory manager.
+
+Secondary users:
+
+- Security or release reviewers checking inventory changes before application.
+- CI/CD maintainers integrating inventory review into pipelines.
+- Future maintainers of the Moose Inventory RubyGem and release process.
+
+## Core user problems
+
+Moose Inventory is intended to solve these problems:
+
+1. Keep dynamic Ansible inventory data in a structured database rather than scattered hand-maintained files.
+2. Provide CLI operations for common host, group, variable, tag, and relationship changes.
+3. Support reviewable dry-runs and machine-readable plans before mutating inventory state.
+4. Export and import inventory snapshots for migration, review, backup, and automation workflows.
+5. Validate inventory health before release, deployment, or operational change.
+6. Preserve append-only audit evidence for successful mutating CLI commands.
+7. Package and publish the tool through RubyGems with repeatable verification and trusted publishing.
+
+## Primary use cases
+
+### Manage inventory entities
+
+Users can create, list, inspect, and remove hosts and groups. Host/group relationships and child-group relationships model Ansible-style inventory organization.
+
+### Manage variables and metadata
+
+Users can add/remove host and group variables and attach metadata tags that are separate from Ansible variables. Tags support operational labels such as environment, owner, lifecycle, location, role, and criticality.
+
+### Review changes before applying them
+
+Mutating commands support `--dry-run`. Dry-runs render planned progress without database mutation. For automation and review, `--plan-format yaml|json|pjson` emits structured planned events.
+
+### Validate inventory health
+
+The `doctor` command performs read-only inventory checks and returns non-zero when findings are present, supporting CI/release-gate workflows.
+
+### Import/export inventory snapshots
+
+Users can export portable inventory snapshots and import validated snapshots into another database. Import is additive/update-oriented and validates the snapshot before writing.
+
+### Maintain database schema safely
+
+Database lifecycle commands inspect, migrate, and back up supported databases. Schema migrations are explicit and ordered. The tool refuses to write to databases with a newer schema version than it supports.
+
+### Support CI/CD and Ansible integration
+
+The repository includes Ansible inventory plugin examples and CI integration examples without adding Ansible/Python runtime dependencies to the RubyGem itself.
+
+## Product goals
+
+1. Provide a reliable, scriptable CLI for database-backed Ansible dynamic inventory management.
+2. Preserve existing command behavior and output compatibility unless a breaking change is deliberately approved.
+3. Prefer safe change workflows: transactions, dry-run previews, validation before write, explicit migrations, and audit logging.
+4. Support common local and server-backed database adapters: SQLite, MySQL/MariaDB, and PostgreSQL.
+5. Keep the package lightweight and avoid unnecessary runtime dependencies.
+6. Maintain strong release hygiene through tests, linting, dependency security checks, secret scanning, package sanity checks, and trusted publishing.
+7. Keep documentation practical for CLI users, maintainers, reviewers, and automation consumers.
+
+## Non-goals
+
+The current product baseline does not include:
+
+- Hosted service operation, web UI, SaaS control plane, or managed inventory hosting.
+- General-purpose configuration-management replacement beyond inventory management.
+- A full rollback system for audit events or imported snapshots.
+- Silent destructive restore semantics.
+- Windows support as a committed compatibility target.
+- Compliance certification claims such as SOC 2, ISO 27001, ISO 42001, or similar readiness claims.
+- Release approval, risk acceptance, public compliance claims, or RubyGems publishing without explicit human approval.
+
+## Security-sensitive context
+
+Moose Inventory can affect infrastructure operations because inventory data drives Ansible execution. A wrong host/group/variable relationship can point automation at the wrong machines or wrong environment.
+
+Security-sensitive assets and flows include:
+
+- Local or server-backed inventory databases.
+- Configuration files that identify environments and database connections.
+- Database credentials, especially legacy plaintext `password` values.
+- Environment variables used for database passwords.
+- Exported inventory snapshots.
+- Audit logs that may reveal host/group names, actors, and operational structure.
+- RubyGem release artifacts and GitHub Actions trusted-publishing configuration.
+
+Current product posture:
+
+- Prefer `password_env` over plaintext database passwords.
+- Avoid committing database credentials and generated/runtime inventory files.
+- Use transactions for mutating commands.
+- Use validation before import writes.
+- Exclude dry-run commands from audit mutation history because no mutation occurred.
+- Treat release publishing, accepted risk, and public/security claims as human-owned decisions.
+
+## Compatibility and stability expectations
+
+Until explicitly changed by an approved requirements baseline:
+
+- Existing documented CLI commands should remain compatible.
+- Existing default human-readable output should be preserved where tests already lock behavior.
+- Machine-readable formats should remain stable enough for automation consumers.
+- Schema migrations should be additive and ordered where practical.
+- Older code must refuse newer schemas rather than downgrade or corrupt them.
+- Database-adapter behavior should remain conservative and explicit.
+
+## Assumptions
+
+- Users are comfortable with command-line workflows and Ansible inventory concepts.
+- Users can install Ruby and native database client build dependencies when required.
+- SQLite is suitable for local/small workflows; MySQL/MariaDB and PostgreSQL are available for server-backed inventory stores.
+- Users manage their own database backups, especially for MySQL/MariaDB and PostgreSQL.
+- Users are responsible for protecting their inventory data, database credentials, and generated artifacts.
+
+## Success criteria
+
+A release or major change is product-aligned when:
+
+- CLI behavior remains compatible or the breaking change has explicit approval and migration guidance.
+- The full verification gate passes for the release candidate.
+- Security/dependency/secret/package checks pass or exceptions are explicitly recorded as accepted risk.
+- README and relevant docs are updated in the existing style.
+- Release evidence includes a human release decision, not just passing checks.
+- New features include tests, documentation, and backlog closure evidence.
+- Security-sensitive flows have dry-run, validation, transaction, or documented recovery behavior where appropriate.
+
+## Open product questions
+
+These remain unresolved until future approval or requirements work:
+
+1. Should Moose Inventory formally support Windows, or continue documenting UNIX/Linux as the supported target?
+2. What compatibility policy should apply to CLI human-readable output vs machine-readable output?
+3. Should audit logs grow into rollback/change-set functionality, or remain accountability/debug evidence only?
+4. Should snapshot import ever support destructive sync/restore semantics, and if so what confirmation/recovery controls are required?
+5. What is the expected support policy for Ruby versions and database adapter versions after each release?
+6. Should package signing or provenance beyond RubyGems trusted publishing be added?

data/docs/product/requirements-baseline.md

@@ -0,0 +1,477 @@
+# Moose Inventory Requirements and Acceptance Criteria Baseline
+
+## Approval status
+
+Status: **Approved requirements and acceptance criteria baseline**
+
+Approval reference: `GOV-REQ-001` in `docs/governance/approval-register.md` approves this document as the requirements and acceptance criteria baseline for Moose Inventory.
+
+Approved references:
+
+- `GOV-TAILOR-001`: Moose Inventory is approved as Class 4 with target profile Software Library / Package.
+- `GOV-PRODUCT-001`: `docs/product/product-brief.md` is approved as the product-framing baseline.
+
+Scope limit: this approval covers the requirements and acceptance criteria baseline only. It is not CLI UX approval, architecture approval, security/privacy design approval, release approval, accepted-risk approval, public/compliance-claim approval, RubyGems publishing approval, or approval of any open requirements question that requires a separate decision.
+
+## Change-control note
+
+Until this document is approved, the README, specs, and current CLI behavior remain the practical behavior baseline. After approval:
+
+- New features should map to one or more requirements or add/update a requirement.
+- Breaking CLI/API/schema behavior changes require explicit approval and migration notes.
+- Release candidates should produce evidence that the applicable acceptance criteria pass or that exceptions are explicitly accepted as risk.
+- Documents, tests, scans, and checklists are evidence, not approval.
+
+## Requirement categories
+
+Requirement IDs use these prefixes:
+
+- `CLI`: command-line behavior
+- `CFG`: configuration and environment selection
+- `DB`: database lifecycle, adapters, and schema
+- `INV`: inventory model and mutation behavior
+- `DRY`: dry-run and plan output
+- `SNAP`: import/export snapshots
+- `DOC`: inventory doctor/lint behavior
+- `AUD`: audit/change history
+- `TAG`: metadata tags
+- `ANS`: Ansible integration
+- `CI`: CI/CD integration examples
+- `PKG`: package/release integrity
+- `SEC`: security/privacy expectations
+- `COMPAT`: compatibility and support boundaries
+- `DOCS`: user/maintainer documentation
+
+## Functional requirements
+
+### CLI behavior
+
+#### CLI-001: Help and discoverability
+
+The CLI must expose help at the top level and command-family level.
+
+Acceptance criteria:
+
+- `moose-inventory help` displays top-level help.
+- `moose-inventory help group` displays group help.
+- `moose-inventory group help add` displays command-specific help.
+- Help examples in README remain representative of supported command families.
+
+#### CLI-002: Output formats
+
+Read/list/get/report commands that document `--format` must support `yaml`, `json`, and `pjson` where applicable.
+
+Acceptance criteria:
+
+- Supported formats are documented in README.
+- Unsupported formats fail predictably without mutating inventory.
+- Machine-readable output remains parseable by the declared format.
+
+#### CLI-003: Existing behavior preservation
+
+Existing documented command behavior and tested output should remain compatible unless a breaking change is explicitly approved.
+
+Acceptance criteria:
+
+- Regression tests cover legacy behavior that is intentionally preserved.
+- Breaking changes include an approval record and migration/release note.
+- Default human-readable output is not changed accidentally during refactors.
+
+### Configuration and environment selection
+
+#### CFG-001: Config discovery
+
+The CLI must search documented config locations in precedence order and allow an explicit `--config <FILE>` override.
+
+Acceptance criteria:
+
+- Explicit `--config` takes precedence and must reference an existing file.
+- Default search paths remain documented.
+- Missing/invalid config errors are clear and non-mutating.
+
+#### CFG-002: Environment selection
+
+The CLI must use `general.defaultenv` by default and allow `--env <SECTION>` to select another environment.
+
+Acceptance criteria:
+
+- Missing environment sections fail clearly.
+- Environment sections remain isolated by database configuration.
+- Commands use the selected environment consistently.
+
+#### CFG-003: Database credential handling
+
+Database configuration must support `password_env` for MySQL/MariaDB and PostgreSQL and retain legacy plaintext `password` compatibility.
+
+Acceptance criteria:
+
+- README recommends `password_env` over plaintext `password`.
+- Doctor/security checks flag plaintext database password configuration.
+- Plaintext password support is treated as compatibility, not preferred posture.
+
+### Database lifecycle, adapters, and schema
+
+#### DB-001: Supported adapters
+
+Moose Inventory must support SQLite, MySQL/MariaDB, and PostgreSQL adapters as documented.
+
+Acceptance criteria:
+
+- SQLite is exercised by the automated test suite.
+- MySQL/MariaDB and PostgreSQL adapter dispatch/error paths have smoke coverage without requiring live servers.
+- Adapter-specific configuration requirements are documented.
+
+#### DB-002: Transactional mutations
+
+Mutating commands must perform database changes transactionally where practical.
+
+Acceptance criteria:
+
+- A command either applies its intended database changes or rolls them back on failure.
+- Import validation completes before import writes begin.
+- Dry-run commands do not mutate database state.
+
+#### DB-003: Explicit schema versioning and migration
+
+Database schema changes must use explicit ordered migrations.
+
+Acceptance criteria:
+
+- Schema version metadata is maintained.
+- Known migrations run in order.
+- Older code refuses to write to a database with a newer schema version.
+- `db status`, `db doctor`, and `db migrate` report/handle schema state as documented.
+
+#### DB-004: Backup support
+
+The CLI must provide documented database backup behavior for SQLite and clear boundaries for server-backed databases.
+
+Acceptance criteria:
+
+- SQLite backup command copies the configured database to the requested destination.
+- MySQL/MariaDB and PostgreSQL backup limitations are documented or reported clearly in `docs/maintenance/database-backup-restore-guidance.md`.
+- Server-backed backup/restore operations remain user-managed through native database tools or hosting-platform controls.
+- Backup commands avoid destructive behavior.
+
+### Inventory model and mutation behavior
+
+#### INV-001: Host and group management
+
+The CLI must support creating, listing, inspecting, and removing hosts and groups.
+
+Acceptance criteria:
+
+- Host/group create operations are idempotent or report existing state as currently documented/tested.
+- Removal operations clean up related associations according to documented behavior.
+- Group recursive removal behavior is explicit and tested.
+
+#### INV-002: Host-group relationships
+
+The CLI must support adding/removing hosts to/from groups through host-oriented and group-oriented command families.
+
+Acceptance criteria:
+
+- `host addgroup`, `host rmgroup`, `group addhost`, and `group rmhost` preserve existing output contracts.
+- Automatic `ungrouped` behavior is preserved.
+- Missing host/group behavior remains explicit and non-surprising.
+
+#### INV-003: Child-group relationships
+
+The CLI must support adding/removing child groups while preventing invalid group hierarchy states.
+
+Acceptance criteria:
+
+- Circular child-group relationships are rejected or reported by validation/doctor checks.
+- `--delete-orphans` behavior remains explicit.
+- Parent/child cleanup behavior is tested.
+
+#### INV-004: Variables
+
+The CLI must support host and group variable creation, update, listing, and removal.
+
+Acceptance criteria:
+
+- Variable names and values are validated according to current CLI/import rules.
+- Variable mutation commands preserve transactional behavior.
+- Variable list/get output remains machine-readable where documented.
+
+#### INV-005: Host list filtering
+
+Host listing must support documented filters using database-backed queries where implemented.
+
+Acceptance criteria:
+
+- Group, tag, and variable filters behave as AND predicates when multiple filters are provided.
+- Missing filters return empty results rather than broadening results.
+- Output shape and ordering remain compatible with documented/tested behavior.
+
+### Dry-run and plan output
+
+#### DRY-001: Dry-run support for mutating commands
+
+Documented mutating command families must support `--dry-run`.
+
+Acceptance criteria:
+
+- Dry-run renders planned progress without database mutation.
+- Dry-run ends with `Dry run complete. No changes applied.`
+- Dry-run behavior covers host/group create/remove, variable mutations, host/group associations, and child-group relationship commands as documented.
+
+#### DRY-002: Machine-readable plan output
+
+Dry-run commands must support `--plan-format yaml|json|pjson` for automation/review workflows.
+
+Acceptance criteria:
+
+- `--plan-format` requires `--dry-run`.
+- Without `--dry-run`, the command aborts before writes with the documented error.
+- Plan events include ordered event type and payload details sufficient for scripts to inspect intended changes.
+
+### Import/export snapshots
+
+#### SNAP-001: Snapshot export
+
+The CLI must export the full inventory as a portable snapshot.
+
+Acceptance criteria:
+
+- Snapshot includes version, hosts, groups, variables, tags, host/group memberships, and child-group relationships.
+- YAML/JSON output is parseable.
+- Export does not mutate inventory.
+
+#### SNAP-002: Snapshot import validation
+
+The CLI must validate snapshots before writing.
+
+Acceptance criteria:
+
+- Malformed snapshots, unknown references, unsupported fields, invalid variable shapes, duplicate normalized keys, whitespace-only names, and circular group hierarchies are rejected before writes.
+- Failed import leaves the database unchanged.
+
+#### SNAP-003: Additive import semantics
+
+Snapshot import must remain additive/update-oriented unless destructive restore/sync semantics are separately approved.
+
+Acceptance criteria:
+
+- Import creates missing hosts/groups, adds missing associations/tags, and creates/updates variables found in the snapshot.
+- Import does not delete existing inventory records absent from the file.
+- Any future destructive import mode requires explicit requirements, UX, recovery, and approval records.
+
+#### SNAP-004: Snapshot import preview
+
+Snapshot import must support a non-mutating preview/diff mode for review before writes.
+
+Acceptance criteria:
+
+- `import FILE --preview` validates the snapshot and reports the additive changes without writing.
+- `--preview-format yaml|json|pjson` emits parseable machine-readable preview output.
+- Preview output shows creates, variable updates, association additions, unchanged items, ignored existing records, and destructive-change count.
+- Validation failures still leave the database unchanged.
+
+### Inventory doctor/lint behavior
+
+#### DOC-001: Read-only health checks
+
+The `doctor` command must run read-only health checks and exit non-zero when findings are present.
+
+Acceptance criteria:
+
+- Doctor finds documented inventory/config issues.
+- Doctor supports human-readable output plus `yaml`, `json`, and `pjson` machine-readable formats.
+- Doctor does not mutate inventory.
+
+### Audit/change history
+
+#### AUD-001: Append-only audit evidence
+
+Successful mutating commands must record append-only audit evidence where audit support applies.
+
+Acceptance criteria:
+
+- Audit records include useful action, actor, target, metadata, and timestamp details.
+- Dry-runs do not create mutation audit records.
+- Audit listing supports documented formats and limits.
+
+### Metadata tags
+
+#### TAG-001: Host/group metadata tags
+
+Hosts and groups must support metadata tags separate from Ansible variables.
+
+Acceptance criteria:
+
+- Tag add/list/remove commands work for hosts and groups.
+- Tags are normalized to lowercase, stripped of surrounding whitespace, and deduplicated per host/group.
+- CLI tag commands and snapshot import apply the same tag normalization rule.
+
+### Ansible integration
+
+#### ANS-001: Dynamic inventory compatibility
+
+Moose Inventory must support Ansible-compatible dynamic inventory workflows.
+
+Acceptance criteria:
+
+- README documents how to use Moose Inventory with Ansible.
+- Example inventory plugin/shim files remain packaged as examples.
+- The RubyGem does not take an unnecessary Python/Ansible runtime dependency solely for examples.
+
+#### ANS-002: Inventory mutation from Ansible
+
+Documented Ansible write-back patterns must remain accurate where supported.
+
+Acceptance criteria:
+
+- README examples reflect supported command behavior.
+- Security-sensitive write-back assumptions are documented or deferred to security guidance.
+
+### CI/CD integration examples
+
+#### CI-001: Inventory review examples
+
+The repository should include CI/CD examples for validating and reviewing inventory snapshots.
+
+Acceptance criteria:
+
+- Example workflows/scripts are present under `examples/ci/`.
+- Examples avoid embedding credentials or environment-specific secrets.
+- Examples are covered by repository checks where practical.
+
+## Non-functional requirements
+
+### Package/release integrity
+
+#### PKG-001: Release verification gate
+
+Release candidates must pass the full project verification gate or record explicit accepted-risk exceptions.
+
+Acceptance criteria:
+
+- `MOOSE_INVENTORY_REQUIRE_SECURITY_TOOLS=1 ./scripts/check.sh` passes for release-candidate code unless an exception is explicitly accepted.
+- Gate includes tests, coverage, RuboCop, whitespace/permission checks, generated-artifact guards, dependency security checks, secret scanning, and package sanity.
+- Release evidence records the exact command and result.
+
+#### PKG-002: Trusted publishing
+
+RubyGems publishing should use trusted publishing/OIDC where configured.
+
+Acceptance criteria:
+
+- Release workflow uses the documented trusted-publishing path.
+- Publishing requires explicit human release approval.
+- Release docs describe required RubyGems/GitHub setup and post-release verification.
+
+#### PKG-003: Dependency and platform support
+
+Runtime and development dependency ranges must remain intentional and reviewed.
+
+Acceptance criteria:
+
+- Gem dependency ranges are reviewed during release prep.
+- Supported Ruby versions align with CI matrix and gemspec requirements.
+- Dependency security checks pass or exceptions are accepted risk.
+
+### Security/privacy expectations
+
+#### SEC-001: Secret handling
+
+Moose Inventory must avoid encouraging committed plaintext secrets.
+
+Acceptance criteria:
+
+- README recommends `password_env`.
+- Doctor flags plaintext password config.
+- Secret scanning is part of the verification gate.
+
+#### SEC-002: Inventory data sensitivity
+
+Docs and future security guidance must treat inventory data as potentially sensitive operational information.
+
+Acceptance criteria:
+
+- Exported snapshots, audit logs, and DB files are treated as sensitive artifacts.
+- Examples avoid real credentials, hosts, or organization-specific secrets.
+- Future docs do not imply inventory data is harmless.
+
+#### SEC-003: Human-owned security decisions
+
+Accepted risks, release security gates, public/security claims, and publishing decisions require human approval.
+
+Acceptance criteria:
+
+- Approval register or release evidence records human decisions.
+- Passing scans are not represented as approval.
+- Public/compliance claims are not made without explicit approval.
+
+### Compatibility and support boundaries
+
+#### COMPAT-001: Operating-system support boundary
+
+UNIX/Linux remains the documented target unless Windows support is separately approved.
+
+Acceptance criteria:
+
+- README continues to warn that Windows is not currently a committed target.
+- Windows support claims are not added without requirements, tests, and approval.
+
+#### COMPAT-002: Machine-readable compatibility
+
+Machine-readable formats are automation-facing interfaces governed by `CLI-OUTPUT-v1` in `docs/compatibility/cli-output-compatibility.md`.
+
+Acceptance criteria:
+
+- Changes to JSON/YAML/pjson structures are reviewed for compatibility impact.
+- Breaking format changes require approval, migration notes, and a release-note declaration of any successor compatibility baseline.
+- Existing machine-readable shapes are not retrofitted with version fields unless the compatibility impact is approved.
+
+#### COMPAT-003: Human-readable compatibility
+
+Human-readable output is also governed by `CLI-OUTPUT-v1` when tests, README examples, or downstream scripts rely on exact wording.
+
+Acceptance criteria:
+
+- Refactors preserve existing wording/newline behavior where tests lock it.
+- Intentional wording changes include test updates and release notes or backlog evidence.
+- Human-readable output remains discouraged as an automation interface when machine-readable output is available.
+
+### Documentation
+
+#### DOCS-001: README as user-facing source
+
+Every new user-facing feature must be documented in README in the existing style.
+
+Acceptance criteria:
+
+- Feature branches update README for new commands/options/behavior.
+- README examples are kept accurate enough for users to follow.
+- Generated/runtime artifacts are not committed as documentation unless intentional.
+
+#### DOCS-002: Process evidence docs
+
+Process docs must distinguish draft evidence from approval.
+
+Acceptance criteria:
+
+- Approval status appears in product/process docs where relevant.
+- Approval decisions are recorded in the approval register or equivalent durable evidence.
+- Backlog status is updated when process artifacts are drafted or approved.
+
+## Release acceptance summary
+
+A release candidate is not requirements-conformant until:
+
+1. Applicable functional requirements are implemented or explicitly deferred.
+2. The full verification gate passes or exceptions are accepted as risk.
+3. README and relevant docs reflect the release behavior.
+4. Security-sensitive changes have validation, transaction, dry-run, audit, or recovery behavior as appropriate.
+5. Release approval and accepted-risk disposition are recorded by a human approver.
+6. RubyGems publishing, if any, is explicitly approved for that release.
+
+## Open requirements questions
+
+1. Should snapshot import gain an explicitly destructive sync/restore mode, and what confirmation/recovery controls would be required?
+2. Should audit history become a rollback/change-set system or remain accountability/debug evidence?
+3. What exact Ruby/database support window should apply after each release?
+4. Should package signing/provenance beyond RubyGems trusted publishing become a requirement?
+5. Should tag normalization be fully case-insensitive across all import/export/CLI paths?