@simbimbo/brainstem 0.0.1 → 0.0.2

package/docs/connectors.md

@@ -0,0 +1,66 @@
+# Connectors
+
+## Purpose
+
+Connectors let brAInstem ingest operational signals from systems that already collect telemetry or manage device/alert context.
+
+The product should remain connector-agnostic at its core:
+- same event model
+- same candidate generation
+- same scoring model
+- same promotion pipeline
+
+Connectors should enrich, not redefine, the core memory model.
+
+## Initial connector targets
+
+### 1. Syslog / file ingest
+- local syslog files
+- forwarded syslog
+- line-oriented app logs
+
+### 2. LogicMonitor
+A high-value early connector target for MSP use.
+
+Why:
+- strong device/resource inventory
+- alert and event context
+- existing operator adoption in MSP environments
+
+Integration modes:
+- webhook ingest
+- periodic sync / polling
+- metadata enrichment
+
+Recommended imported fields:
+- `resource_id`
+- `resource_name`
+- `datasource`
+- `instance_name`
+- `alert_id`
+- `severity`
+- `acknowledged`
+- `cleared_at`
+- `collector`
+- host group / site metadata
+
+### 3. Future candidates
+- LibreNMS
+- Graylog
+- Splunk
+- Elastic
+- Windows Event Forwarding
+- custom webhooks
+
+## Connector rules
+
+Connectors should:
+- map into native `event` objects
+- preserve provenance
+- attach source-specific metadata without polluting the core schema
+- never bypass scoring / candidate generation / promotion logic
+
+## Design principle
+
+LogicMonitor should be an excellent source of events and metadata, but brAInstem must remain useful even without it.
+That is what keeps the project universal instead of becoming a thin LogicMonitor add-on.

package/docs/data-model.md

@@ -0,0 +1,290 @@
+# Data Model
+
+## Goal
+
+brAInstem needs a canonical model that can represent:
+- raw operational events
+- normalized signatures
+- derived incident candidates
+- promoted operational memory
+- operator review decisions
+- tenant and asset context
+
+The model should support both deterministic event analysis and memory-style historical recall.
+
+---
+
+## 1. Tenant
+
+Represents a customer or environment boundary.
+
+Fields:
+- `tenant_id`
+- `name`
+- `slug`
+- `environment_type` (msp-client, internal, lab, etc.)
+- `tags`
+- `metadata`
+- `created_at`
+- `updated_at`
+
+Purpose:
+- isolate memory, scoring, and recurrence by customer/environment
+- support cross-tenant pattern comparison only when explicitly allowed
+
+---
+
+## 2. Asset
+
+Represents a host, firewall, service instance, application node, or logical system.
+
+Fields:
+- `asset_id`
+- `tenant_id`
+- `hostname`
+- `asset_type` (server, firewall, switch, app, vm, workstation, etc.)
+- `service_group`
+- `ip_addresses`
+- `platform`
+- `tags`
+- `metadata`
+- `created_at`
+- `updated_at`
+
+Purpose:
+- associate events with concrete systems
+- support spread analysis and asset-local baselines
+
+---
+
+## 3. Event
+
+The raw normalized event unit.
+
+Fields:
+- `event_id`
+- `tenant_id`
+- `asset_id`
+- `source_type` (syslog, journald, windows_event, webhook, app_log, etc.)
+- `source_path`
+- `host`
+- `service`
+- `timestamp`
+- `severity`
+- `facility`
+- `message_raw`
+- `message_normalized`
+- `structured_fields`
+- `correlation_keys`
+- `signature_input`
+- `ingest_metadata`
+- `created_at`
+
+Purpose:
+- preserve the original operational evidence in normalized form
+- provide the substrate for signatures and candidate generation
+
+---
+
+## 4. Signature
+
+A reusable event fingerprint / pattern family.
+
+Fields:
+- `signature_id`
+- `tenant_scope` (global, tenant, asset)
+- `signature_key`
+- `signature_version`
+- `event_family`
+- `service`
+- `normalized_pattern`
+- `example_message`
+- `first_seen_at`
+- `last_seen_at`
+- `occurrence_count`
+- `metadata`
+
+Purpose:
+- detect recurrence
+- cluster similar events over time
+- support cross-host and cross-tenant comparison
+
+---
+
+## 5. Correlation Group
+
+A time-bound grouping of related events/signatures.
+
+Fields:
+- `group_id`
+- `tenant_id`
+- `group_type` (burst, spread, change-window, precursor-cluster, etc.)
+- `start_at`
+- `end_at`
+- `asset_ids`
+- `signature_ids`
+- `event_count`
+- `metadata`
+
+Purpose:
+- represent clusters of weak signals that matter together
+- support temporal correlation and causal analysis
+
+---
+
+## 6. Candidate
+
+A derived object representing a possibly meaningful operational pattern.
+
+Fields:
+- `candidate_id`
+- `tenant_id`
+- `candidate_type` (recurrence, self_heal, precursor, anomaly, burst, spread)
+- `source_signature_ids`
+- `source_event_ids`
+- `source_group_ids`
+- `title`
+- `summary`
+- `first_seen_at`
+- `last_seen_at`
+- `score_total`
+- `score_breakdown`
+- `decision_band` (ignore, watch, review, urgent_human_review, promote_to_incident_memory)
+- `explanation`
+- `confidence`
+- `metadata`
+- `created_at`
+- `updated_at`
+
+Purpose:
+- bridge raw events and durable incident memory
+- make scoring explainable and reviewable
+
+---
+
+## 7. Incident Memory
+
+A promoted durable record of a pattern that has proven meaningful.
+
+Fields:
+- `incident_memory_id`
+- `tenant_id`
+- `title`
+- `summary`
+- `incident_type`
+- `source_candidate_ids`
+- `source_signature_ids`
+- `source_event_ids`
+- `first_observed_at`
+- `last_observed_at`
+- `recurrence_count`
+- `impact_level`
+- `confidence`
+- `status` (active, historical, retired)
+- `metadata`
+- `created_at`
+- `updated_at`
+
+Purpose:
+- preserve meaningful operational patterns over time
+- support "have we seen this before?"
+
+---
+
+## 8. Lesson
+
+A distilled durable insight derived from incidents or repeated review.
+
+Fields:
+- `lesson_id`
+- `tenant_id`
+- `title`
+- `summary`
+- `recommendation`
+- `source_incident_memory_ids`
+- `source_candidate_ids`
+- `confidence`
+- `metadata`
+- `created_at`
+- `updated_at`
+
+Purpose:
+- convert repeated operational pain into reusable knowledge
+
+---
+
+## 9. Runbook Hint
+
+A small operational action suggestion linked to a pattern.
+
+Fields:
+- `runbook_hint_id`
+- `tenant_id`
+- `title`
+- `hint_text`
+- `trigger_signature_ids`
+- `trigger_incident_ids`
+- `priority`
+- `confidence`
+- `metadata`
+- `created_at`
+- `updated_at`
+
+Purpose:
+- give operators actionable next steps without pretending to be a full runbook engine initially
+
+---
+
+## 10. Review Decision
+
+Captures operator feedback on surfaced candidates or incidents.
+
+Fields:
+- `review_id`
+- `tenant_id`
+- `subject_type` (candidate, incident_memory, lesson)
+- `subject_id`
+- `decision` (noise, useful, urgent, false_positive, promote, ignore, investigate)
+- `notes`
+- `reviewer`
+- `reviewed_at`
+
+Purpose:
+- improve future memory weight and trust calibration
+- keep a human in the loop
+
+---
+
+## 11. Provenance Link
+
+Tracks relationships between objects.
+
+Fields:
+- `link_id`
+- `source_type`
+- `source_id`
+- `target_type`
+- `target_id`
+- `link_type`
+- `created_at`
+- `metadata`
+
+Purpose:
+- connect raw events -> signatures -> candidates -> incidents -> lessons
+- support explainability and drill-down
+
+---
+
+## MVP subset
+
+For MVP, the essential objects are:
+- Tenant
+- Asset
+- Event
+- Signature
+- Candidate
+- Incident Memory
+- Review Decision
+- Provenance Link
+
+The others can be introduced after the first strong demo.