@wentorai/research-plugins 1.0.0 → 1.2.0

package/skills/domains/cs/distributed-systems-guide/SKILL.md

@@ -0,0 +1,268 @@
+---
+name: distributed-systems-guide
+description: "Distributed systems design patterns and analysis for CS research"
+metadata:
+  openclaw:
+    emoji: "globe-with-meridians"
+    category: "domains"
+    subcategory: "cs"
+    keywords: ["distributed-systems", "consensus", "replication", "fault-tolerance", "scalability", "cap-theorem"]
+    source: "wentor"
+---
+
+# Distributed Systems Guide
+
+A skill for researching and designing distributed systems, covering consensus algorithms, replication strategies, consistency models, fault tolerance, and performance analysis. Provides theoretical foundations and practical implementations relevant to systems research.
+
+## Consistency Models
+
+### Consistency Hierarchy
+
+```
+Strongest
+  |  Linearizability (atomic, real-time ordering)
+  |  Sequential consistency (program order respected)
+  |  Causal consistency (causally related ops ordered)
+  |  PRAM / FIFO consistency (per-process order)
+  |  Eventual consistency (converges if updates stop)
+Weakest
+```
+
+### CAP Theorem and PACELC
+
+The CAP theorem states that during a network partition, a distributed system must choose between consistency and availability:
+
+| System | Partition Behavior | Normal Behavior | Classification |
+|--------|-------------------|----------------|----------------|
+| ZooKeeper | Consistent (sacrifice A) | Low latency, consistent | CP / PC/EC |
+| Cassandra | Available (sacrifice C) | Low latency, eventual | AP / PA/EL |
+| Spanner | Consistent (sacrifice A) | Higher latency, consistent | CP / PC/EC |
+| DynamoDB | Configurable per-read | Tunable consistency | AP or CP |
+| CockroachDB | Consistent (sacrifice A) | Serializable | CP / PC/EC |
+
+## Consensus Algorithms
+
+### Raft Implementation Sketch
+
+```python
+from enum import Enum
+from dataclasses import dataclass, field
+import random
+
+class NodeState(Enum):
+    FOLLOWER = "follower"
+    CANDIDATE = "candidate"
+    LEADER = "leader"
+
+@dataclass
+class LogEntry:
+    term: int
+    index: int
+    command: str
+
+@dataclass
+class RaftNode:
+    """
+    Simplified Raft consensus node for educational purposes.
+    Implements leader election and log replication state machine.
+    """
+    node_id: str
+    state: NodeState = NodeState.FOLLOWER
+    current_term: int = 0
+    voted_for: str = None
+    log: list = field(default_factory=list)
+    commit_index: int = 0
+    last_applied: int = 0
+
+    # Leader state
+    next_index: dict = field(default_factory=dict)
+    match_index: dict = field(default_factory=dict)
+
+    def start_election(self, peers: list[str]) -> dict:
+        """Transition to candidate and request votes."""
+        self.state = NodeState.CANDIDATE
+        self.current_term += 1
+        self.voted_for = self.node_id
+
+        last_log_index = len(self.log) - 1 if self.log else -1
+        last_log_term = self.log[-1].term if self.log else 0
+
+        return {
+            "type": "RequestVote",
+            "term": self.current_term,
+            "candidate_id": self.node_id,
+            "last_log_index": last_log_index,
+            "last_log_term": last_log_term,
+        }
+
+    def handle_vote_request(self, term: int, candidate_id: str,
+                              last_log_index: int,
+                              last_log_term: int) -> dict:
+        """Process a RequestVote RPC."""
+        if term < self.current_term:
+            return {"term": self.current_term, "vote_granted": False}
+
+        if term > self.current_term:
+            self.current_term = term
+            self.state = NodeState.FOLLOWER
+            self.voted_for = None
+
+        # Check if candidate's log is at least as up-to-date
+        my_last_term = self.log[-1].term if self.log else 0
+        my_last_index = len(self.log) - 1 if self.log else -1
+
+        log_ok = (last_log_term > my_last_term or
+                  (last_log_term == my_last_term and
+                   last_log_index >= my_last_index))
+
+        vote_granted = (
+            (self.voted_for is None or self.voted_for == candidate_id)
+            and log_ok
+        )
+
+        if vote_granted:
+            self.voted_for = candidate_id
+
+        return {"term": self.current_term, "vote_granted": vote_granted}
+
+    def append_entry(self, command: str) -> LogEntry:
+        """Leader appends a new entry to its log."""
+        entry = LogEntry(
+            term=self.current_term,
+            index=len(self.log),
+            command=command,
+        )
+        self.log.append(entry)
+        return entry
+```
+
+### Paxos vs Raft vs PBFT Comparison
+
+| Algorithm | Fault Model | Tolerance | Rounds | Complexity |
+|-----------|-------------|-----------|--------|------------|
+| Paxos | Crash faults | f < n/2 | 2 (normal) | Difficult to implement correctly |
+| Raft | Crash faults | f < n/2 | 2 (normal) | Designed for understandability |
+| PBFT | Byzantine faults | f < n/3 | 3 | O(n^2) message complexity |
+| HotStuff | Byzantine faults | f < n/3 | 3 | O(n) with pipelining |
+
+## Replication Strategies
+
+### State Machine Replication
+
+```python
+class ReplicatedStateMachine:
+    """
+    State machine replication with configurable consistency.
+    Demonstrates read/write quorum intersection for correctness.
+    """
+
+    def __init__(self, n_replicas: int, read_quorum: int = None,
+                 write_quorum: int = None):
+        self.n = n_replicas
+        self.R = read_quorum or (n_replicas // 2 + 1)
+        self.W = write_quorum or (n_replicas // 2 + 1)
+
+        # Quorum intersection guarantees: R + W > N
+        assert self.R + self.W > self.n, (
+            f"Quorum intersection violated: R({self.R}) + W({self.W}) "
+            f"must be > N({self.n})"
+        )
+
+        self.replicas = [{} for _ in range(n_replicas)]
+        self.version_clock = 0
+
+    def write(self, key: str, value: str) -> dict:
+        """Write to W replicas."""
+        self.version_clock += 1
+        # Select W replicas (in practice, based on availability)
+        targets = random.sample(range(self.n), self.W)
+        for i in targets:
+            self.replicas[i][key] = (value, self.version_clock)
+
+        return {
+            "key": key,
+            "version": self.version_clock,
+            "acked_by": len(targets),
+            "quorum_met": True,
+        }
+
+    def read(self, key: str) -> dict:
+        """Read from R replicas, return latest version."""
+        targets = random.sample(range(self.n), self.R)
+        responses = []
+        for i in targets:
+            if key in self.replicas[i]:
+                responses.append(self.replicas[i][key])
+
+        if not responses:
+            return {"key": key, "value": None, "found": False}
+
+        # Return the value with the highest version
+        latest = max(responses, key=lambda x: x[1])
+        return {
+            "key": key,
+            "value": latest[0],
+            "version": latest[1],
+            "found": True,
+        }
+```
+
+## Clock Synchronization and Ordering
+
+### Vector Clocks
+
+```python
+class VectorClock:
+    """Vector clock for tracking causality in distributed systems."""
+
+    def __init__(self, process_id: str, processes: list[str]):
+        self.pid = process_id
+        self.clock = {p: 0 for p in processes}
+
+    def increment(self):
+        """Local event: increment own counter."""
+        self.clock[self.pid] += 1
+
+    def send(self) -> dict:
+        """Prepare clock for sending with a message."""
+        self.increment()
+        return dict(self.clock)
+
+    def receive(self, other_clock: dict):
+        """Merge received clock: element-wise max, then increment."""
+        for p in self.clock:
+            self.clock[p] = max(self.clock[p], other_clock.get(p, 0))
+        self.increment()
+
+    def happened_before(self, other: dict) -> bool:
+        """Check if this clock happened-before other (causal ordering)."""
+        return (all(self.clock[p] <= other.get(p, 0) for p in self.clock) and
+                any(self.clock[p] < other.get(p, 0) for p in self.clock))
+```
+
+## Performance Analysis
+
+### Latency and Throughput Modeling
+
+Key metrics for evaluating distributed systems:
+
+- **Tail latency (p99, p999)**: Critical for real-world SLAs; often dominated by slow replicas
+- **Throughput under contention**: How performance degrades with conflict rate
+- **Scalability**: Linear vs sub-linear throughput increase with added nodes
+- **Recovery time**: Time to restore consistency after node failure
+
+## Key Research Papers
+
+- Lamport, L. (1998). The Part-Time Parliament (Paxos). *ACM TOCS*.
+- Ongaro, D. and Ousterhout, J. (2014). In Search of an Understandable Consensus Algorithm (Raft). *USENIX ATC*.
+- Corbett, J. et al. (2013). Spanner: Google's Globally-Distributed Database. *ACM TOCS*.
+- DeCandia, G. et al. (2007). Dynamo: Amazon's Highly Available Key-value Store. *SOSP*.
+
+## Tools and Frameworks
+
+- **etcd / ZooKeeper**: Production consensus stores for coordination
+- **Jepsen**: Distributed systems correctness testing framework
+- **TLA+ / PlusCal**: Formal specification and model checking
+- **ns-3 / OMNeT++**: Network simulation for distributed protocols
+- **gRPC / Cap'n Proto**: High-performance RPC frameworks
+- **FoundationDB**: Multi-model distributed database with strong consistency

package/skills/domains/cs/formal-verification-guide/SKILL.md

@@ -0,0 +1,298 @@
+---
+name: formal-verification-guide
+description: "Formal methods, theorem proving, and model checking for CS research"
+metadata:
+  openclaw:
+    emoji: "check-mark"
+    category: "domains"
+    subcategory: "cs"
+    keywords: ["formal-verification", "theorem-proving", "model-checking", "tla-plus", "coq", "isabelle"]
+    source: "wentor"
+---
+
+# Formal Verification Guide
+
+A skill for applying formal methods to verify software and hardware correctness. Covers model checking, interactive theorem proving, specification languages, and practical verification workflows used in systems and programming language research.
+
+## Verification Approaches Overview
+
+### Methods Comparison
+
+| Approach | Technique | Strengths | Limitations |
+|----------|-----------|-----------|-------------|
+| Model checking | Exhaustive state exploration | Fully automatic, produces counterexamples | State space explosion |
+| Theorem proving | Interactive proof construction | Handles infinite state | Requires expert effort |
+| Abstract interpretation | Sound static analysis | Automatic, scales well | May report false positives |
+| SMT solving | Constraint satisfiability | Powerful automation | Limited to decidable theories |
+| Runtime verification | Execution monitoring | Low barrier, practical | Only checks observed runs |
+
+## TLA+ Specification
+
+### Specifying Distributed Protocols
+
+TLA+ is the standard specification language for distributed systems:
+
+```tla
+--------------------------- MODULE TwoPhaseCommit -------------------------
+EXTENDS Integers, Sequences, FiniteSets
+
+CONSTANTS RM  \* Set of resource managers
+
+VARIABLES
+    rmState,      \* rmState[r] is the state of resource manager r
+    tmState,      \* State of the transaction manager
+    tmPrepared,   \* Set of RMs that have sent "Prepared"
+    msgs          \* Set of messages sent
+
+vars == <<rmState, tmState, tmPrepared, msgs>>
+
+Init ==
+    /\ rmState = [r \in RM |-> "working"]
+    /\ tmState = "init"
+    /\ tmPrepared = {}
+    /\ msgs = {}
+
+\* RM r prepares to commit
+RMPrepare(r) ==
+    /\ rmState[r] = "working"
+    /\ rmState' = [rmState EXCEPT ![r] = "prepared"]
+    /\ msgs' = msgs \union {[type |-> "Prepared", rm |-> r]}
+    /\ UNCHANGED <<tmState, tmPrepared>>
+
+\* TM receives a Prepared message from RM r
+TMRcvPrepared(r) ==
+    /\ tmState = "init"
+    /\ [type |-> "Prepared", rm |-> r] \in msgs
+    /\ tmPrepared' = tmPrepared \union {r}
+    /\ UNCHANGED <<rmState, tmState, msgs>>
+
+\* TM commits (all RMs have prepared)
+TMCommit ==
+    /\ tmState = "init"
+    /\ tmPrepared = RM
+    /\ tmState' = "committed"
+    /\ msgs' = msgs \union {[type |-> "Commit"]}
+    /\ UNCHANGED <<rmState, tmPrepared>>
+
+\* Safety property: No RM commits unless TM has committed
+Consistency ==
+    \A r \in RM : rmState[r] = "committed" => tmState = "committed"
+========================================================================
+```
+
+### Running the TLC Model Checker
+
+```bash
+# Install TLA+ Toolbox or use command-line TLC
+# Define model with specific constants
+# RM = {"rm1", "rm2", "rm3"}
+java -jar tla2tools.jar -config TwoPhaseCommit.cfg TwoPhaseCommit.tla
+
+# TLC will explore all reachable states and verify:
+# - No deadlocks (unless specified)
+# - Safety properties (invariants)
+# - Liveness properties (temporal formulas)
+```
+
+## Interactive Theorem Proving
+
+### Coq Proof Assistant
+
+```coq
+(* Example: Proving properties of a simple functional program *)
+
+(* Define natural number addition *)
+Fixpoint add (n m : nat) : nat :=
+  match n with
+  | O => m
+  | S n' => S (add n' m)
+  end.
+
+(* Prove: 0 + n = n (left identity) *)
+Theorem add_0_l : forall n : nat, add 0 n = n.
+Proof.
+  intro n.
+  simpl.    (* simplification reduces add 0 n to n *)
+  reflexivity.
+Qed.
+
+(* Prove: n + 0 = n (right identity, requires induction) *)
+Theorem add_0_r : forall n : nat, add n 0 = n.
+Proof.
+  intro n.
+  induction n as [| n' IHn'].
+  - (* Base case: n = 0 *)
+    simpl. reflexivity.
+  - (* Inductive step: n = S n' *)
+    simpl.               (* add (S n') 0 = S (add n' 0) *)
+    rewrite IHn'.        (* apply induction hypothesis *)
+    reflexivity.
+Qed.
+
+(* Prove associativity of addition *)
+Theorem add_assoc : forall a b c : nat,
+  add a (add b c) = add (add a b) c.
+Proof.
+  intros a b c.
+  induction a as [| a' IHa'].
+  - simpl. reflexivity.
+  - simpl. rewrite IHa'. reflexivity.
+Qed.
+```
+
+### Isabelle/HOL
+
+```isabelle
+theory SimpleVerification
+  imports Main
+begin
+
+(* Define a recursive function *)
+fun fib :: "nat => nat" where
+  "fib 0 = 0"
+| "fib (Suc 0) = 1"
+| "fib (Suc (Suc n)) = fib (Suc n) + fib n"
+
+(* Prove a property *)
+lemma fib_positive: "0 < fib (Suc n)"
+  by (induction n rule: fib.induct) auto
+
+(* Verify a sorting algorithm *)
+fun insert :: "nat => nat list => nat list" where
+  "insert x [] = [x]"
+| "insert x (y # ys) = (if x <= y then x # y # ys else y # insert x ys)"
+
+fun isort :: "nat list => nat list" where
+  "isort [] = []"
+| "isort (x # xs) = insert x (isort xs)"
+
+(* Prove the output is sorted *)
+lemma sorted_insert: "sorted (insert x xs) = sorted xs"
+  sorry (* full proof requires additional lemmas *)
+
+end
+```
+
+## SMT Solving
+
+### Z3 for Program Verification
+
+```python
+from z3 import Solver, Int, Bool, And, Or, Not, Implies, ForAll, sat, unsat
+
+def verify_array_bounds():
+    """
+    Verify that an array access is always within bounds.
+    Model a loop: for i = 0 to n-1, access a[i].
+    """
+    s = Solver()
+    n = Int("n")
+    i = Int("i")
+
+    # Precondition: n > 0
+    s.add(n > 0)
+
+    # Loop invariant: 0 <= i < n at each access
+    s.add(i >= 0)
+    s.add(i < n)
+
+    # Verify: the access a[i] is within bounds [0, n)
+    s.add(Not(And(i >= 0, i < n)))  # try to find a violation
+
+    result = s.check()
+    if result == unsat:
+        return "VERIFIED: array access is always within bounds"
+    else:
+        return f"COUNTEREXAMPLE: {s.model()}"
+
+def verify_integer_overflow():
+    """
+    Check if integer addition can overflow for given constraints.
+    """
+    from z3 import BitVec, BitVecVal
+
+    s = Solver()
+    # 32-bit signed integers
+    x = BitVec("x", 32)
+    y = BitVec("y", 32)
+
+    # Preconditions: both positive
+    s.add(x > 0)
+    s.add(y > 0)
+
+    # Check: can x + y wrap around to negative?
+    s.add(x + y < 0)
+
+    if s.check() == sat:
+        m = s.model()
+        return {
+            "overflow_possible": True,
+            "x": m[x].as_long(),
+            "y": m[y].as_long(),
+        }
+    return {"overflow_possible": False}
+```
+
+## Model Checking with SPIN
+
+### Promela Specification
+
+```promela
+/* Mutual exclusion with Peterson's algorithm */
+bool flag[2] = false;
+byte turn = 0;
+byte critical = 0;  /* count of processes in critical section */
+
+active [2] proctype process() {
+    byte me = _pid;
+    byte other = 1 - _pid;
+
+    do
+    :: /* Entry protocol */
+       flag[me] = true;
+       turn = other;
+       (flag[other] == false || turn == me);
+
+       /* Critical section */
+       critical++;
+       assert(critical == 1);  /* mutual exclusion */
+       critical--;
+
+       /* Exit protocol */
+       flag[me] = false;
+    od
+}
+
+/* LTL property: mutual exclusion always holds */
+ltl mutex { [] (critical <= 1) }
+```
+
+## Verification Workflow
+
+### Practical Verification Strategy
+
+1. **Specify**: Write a formal specification of the desired property
+2. **Model**: Create an abstract model of the system
+3. **Verify**: Run model checker or construct proof
+4. **Refine**: If counterexample found, fix the design or refine the model
+5. **Extract**: Generate verified code from the proof (Coq extraction, Isabelle code generation)
+
+### Common Properties to Verify
+
+| Property Type | Example | Specification Pattern |
+|--------------|---------|----------------------|
+| Safety | "No two processes in critical section" | `[] (count <= 1)` |
+| Liveness | "Every request is eventually served" | `[] (request -> <> response)` |
+| Deadlock freedom | "System always has an enabled transition" | `[] <> enabled` |
+| Termination | "Program always halts" | Well-founded ordering |
+
+## Tools and Resources
+
+- **TLA+ Toolbox**: IDE for TLA+ with integrated TLC model checker
+- **Coq**: Interactive theorem prover with program extraction
+- **Isabelle/HOL**: Higher-order logic prover with Sledgehammer automation
+- **Z3 / CVC5**: SMT solvers for automated reasoning
+- **SPIN**: Model checker for concurrent systems (Promela)
+- **CBMC**: Bounded model checker for C programs
+- **Dafny**: Verification-aware programming language (Microsoft)
+- **Lean 4**: Modern theorem prover and programming language