@ketecode/loom 0.0.0-stage → 0.1.0

package/skills/bgp-diagnostics/SKILL.md

@@ -0,0 +1,84 @@
+---
+name: bgp-diagnostics
+description: Diagnoses Border Gateway Protocol problems on Cisco, Juniper, Arista and FRR routers by reading neighbor state (Idle, Active, OpenSent, Established) and NOTIFICATION codes, checking which prefixes are received, accepted and advertised through prefix-lists, route-maps and max-prefix limits, walking best-path selection, decoding standard and large communities, spotting RPKI-invalid or leaked routes, and confirming what the internet sees from RIPEstat, RouteViews and other looking glasses. Use when a BGP session will not come up or keeps flapping, a prefix is missing, not propagating or taking the wrong path, a provider says they are not receiving an announcement, or traffic shifted after a routing change or an upstream incident.
+license: MIT
+compatibility: Any Agent Skills host. Works from pasted router output; live checks need read-only CLI access to the router and web access to public looking glasses and route collectors.
+metadata:
+  kete-loom/version: "0.1.0"
+  kete-loom/category: devops
+  kete-loom/risk: read-only
+  kete-loom/owner: "Kete-org"
+---
+
+# BGP diagnostics
+
+Find out why a BGP session or route is not doing what the operator expects, using read-only commands and public data, and hand back a cause with evidence and a proposed fix the operator applies.
+
+For NOTIFICATION codes, best-path order, well-known communities and per-vendor command equivalents, read `references/bgp-reference.md`.
+
+## Ground rules
+
+1. **Read-only.** Use `show`/`display` commands and public looking glasses. Never run `clear ip bgp`, `clear bgp`, `neighbor ... shutdown`, policy edits or `soft` resets yourself; propose them and let the operator run them in a window. A hard reset drops every route from that peer.
+2. Work from evidence: ask for the exact command output, router platform and OS version, local AS, peer AS and peer address.
+3. Separate the two questions: **is the session up?** and **is the route right?** Fix the first before the second.
+4. Treat every public looking-glass result as a sample from one vantage point. Check at least two sources before telling the user "the internet can't see it".
+
+## 1. Session not Established
+
+Get `show bgp summary` (or `show ip bgp summary`, Junos `show bgp summary`, FRR `vtysh -c "show bgp summary"`) and `show bgp neighbors <peer>`.
+
+| State seen | Usual cause | Check |
+|---|---|---|
+| `Idle` | Admin shutdown, no route to peer, max-prefix tripped, or waiting after errors | `show bgp neighbors` last reset reason; `Idle (Admin)` / `Idle (PfxCt)` on IOS |
+| `Active` / `Connect` cycling | TCP 179 never completes: wrong peer IP, ACL or firewall, peer not configured, wrong source interface | Ping the peer from the configured source; `update-source` on loopback peers; ACLs and CoPP for TCP 179 |
+| `OpenSent` / `OpenConfirm` then reset | OPEN rejected: wrong remote AS, bad router ID, capability or hold-time mismatch, MD5/TCP-AO password mismatch | Logs for NOTIFICATION code 2/x; on IOS `%TCP-6-BADAUTH` means password mismatch |
+| Up then flaps every ~3 min | Hold timer expiry: loss, CPU, MTU problems on large updates | NOTIFICATION 4; interface errors; ping with DF at full MTU |
+| Up then drops at a prefix count | Max-prefix limit | Cease subcode 1; compare received count to `maximum-prefix` |
+
+Also check:
+- eBGP to a non-directly-connected address needs `ebgp-multihop` (or `ttl-security` on both sides), because eBGP packets default to TTL 1.
+- Some platforms refuse to send or accept eBGP routes with no policy configured (RFC 8212): IOS-XR ("no policy; defaults to 'drop'", overridable with `bgp unsafe-ebgp-policy`) and FRR in its default traditional profile (`bgp ebgp-requires-policy`, shown as `(Policy)` in `show bgp summary`). Session `Established` with 0 prefixes both ways often means this.
+- Address family activated on both sides (`address-family ipv6` + `neighbor ... activate`).
+
+## 2. Prefix missing or wrong
+
+Walk the path step by step and stop at the first step that fails.
+
+1. **Originated?** The prefix is in the local BGP table: `show bgp ipv4 unicast <prefix>`. A `network` statement needs an exact match in the RIB (same mask); otherwise use a static to Null0 or an aggregate.
+2. **Advertised?** `show bgp ipv4 unicast neighbors <peer> advertised-routes`. If absent, check outbound prefix-list, route-map, `no-export`/`no-advertise` communities, and iBGP split-horizon (iBGP-learned routes are not sent to other iBGP peers without a route reflector).
+3. **Received by the peer?** Ask the peer, or use their looking glass. On your side for inbound routes: `received-routes` needs `soft-reconfiguration inbound` on IOS; otherwise use `show bgp ... neighbors <peer> routes` for accepted routes only.
+4. **Accepted?** Inbound filters, max-prefix, AS-path loop (own AS in path; `allowas-in` only if the design needs it), RPKI-invalid dropped by ROV, or IRR-based filters at the provider missing your route object.
+5. **Best?** `show bgp ipv4 unicast <prefix>` lists all paths; walk the best-path order in the reference and name the step that decided. Common surprises: weight set on one neighbor, local-pref from a route-map, MED compared only within one neighbor AS, IGP metric to next hop.
+6. **Installed?** `show ip route <prefix>`: an IGP or static with lower administrative distance wins; next hop unreachable makes the path invalid (`inaccessible`).
+
+## 3. What the internet sees
+
+For a public prefix or ASN:
+- RIPEstat (`stat.ripe.net`): Routing Status, Looking Glass and BGPlay widgets show visibility across RIS collectors and the history of announcements and withdrawals.
+- RouteViews collectors (`route-views.routeviews.org` and others) and bgp.tools or bgp.he.net for origin AS, upstreams and visibility.
+- RPKI: check the prefix and origin pair is Valid in the RIPEstat RPKI widget or your RIR portal. An Invalid (wrong origin or max-length too short) is dropped by many networks.
+- IRR: `whois -h whois.radb.net <prefix>` or the provider's IRR; missing or wrong `route`/`route6` objects cause providers to filter you.
+- PeeringDB for the peer's NOC contact, prefix limits they expect, and IX details.
+
+## 4. Common outage causes to rule out
+
+1. ROA created with a max-length shorter than the announced more-specific, or for the wrong origin AS.
+2. A route leak: your prefix seen with an unexpected AS in the path, or you announcing learned routes to another provider (missing outbound filter). Check outbound policy matches only your own prefixes.
+3. Hijack or more-specific announced by someone else: a different origin AS for your prefix or a longer prefix inside it.
+4. Max-prefix tripped after a peer sent a full table.
+5. Password, AS or IP changed on one side during maintenance.
+6. Policy change applied without a soft refresh, so the new policy never took effect; or a soft reset expected to fix something it cannot.
+7. Upstream incident: check the provider status page and RIPEstat for many prefixes moving at once.
+
+**Stop and tell the user to contact the provider or a qualified network engineer** when evidence points at the upstream, a suspected hijack, or a change that would affect transit for the whole network.
+
+## Report back
+
+```
+Session: <peer IP / AS> state <state>, last reset <reason/code>
+Prefix: <prefix> originated <y/n> advertised <y/n> seen by peer <y/n/unknown> best <y/n, decided at step N>
+External view: <sources checked, visibility, RPKI status, IRR status>
+Cause: <one sentence>, evidence: <command output lines>
+Proposed fix: <exact commands for the operator>, impact: <soft refresh vs session reset>
+Not verified: <what was not checked>
+```

package/skills/bgp-diagnostics/evals/triggers.json

@@ -0,0 +1,18 @@
+{
+  "should_trigger": [
+    "show ip bgp summary says our neighbor 203.0.113.1 is stuck in Active, what does that mean?",
+    "Our transit provider says they don't see our /24 but it's in our BGP table, where is it getting dropped?",
+    "BGP session with the IX route server goes down every few minutes with hold timer expired",
+    "Why is the router choosing the path through ISP B when we set local-pref 200 on ISP A?",
+    "RIPEstat shows our prefix with a different origin AS since this morning, is this a hijack?",
+    "The peer sent a NOTIFICATION cease subcode 1 and the session dropped, how do I read that?"
+  ],
+  "should_not_trigger": [
+    "Write a Nornir task that renders BGP neighbor config from NetBox and pushes it to all edge routers",
+    "Configure an extended ACL on the Cisco switch to block telnet to the management VLAN",
+    "My laptop can reach google.com by IP but DNS lookups fail on the office Wi-Fi",
+    "Plan an address scheme and VLANs for a new branch office network",
+    "Review this firewall ruleset before I load it on the edge gateway",
+    "Write the OSPF config for three Cisco routers sharing area 0 over point-to-point links"
+  ]
+}

package/skills/bgp-diagnostics/references/bgp-reference.md

@@ -0,0 +1,81 @@
+# BGP reference
+
+## FSM states (RFC 4271)
+
+`Idle` → `Connect` → `Active` → `OpenSent` → `OpenConfirm` → `Established`. `Connect` and `Active` mean TCP to port 179 has not completed. `OpenSent`/`OpenConfirm` mean TCP is up and OPEN messages are being exchanged.
+
+## NOTIFICATION error codes (RFC 4271)
+
+| Code | Meaning | Typical subcodes |
+|---|---|---|
+| 1 | Message Header Error | bad length or type |
+| 2 | OPEN Message Error | 2 Bad Peer AS, 3 Bad BGP Identifier, 4 Unsupported Optional Parameter, 6 Unacceptable Hold Time, 7 Unsupported Capability (RFC 5492) |
+| 3 | UPDATE Message Error | malformed attribute list, invalid next hop, malformed AS_PATH |
+| 4 | Hold Timer Expired | no keepalive or update within the hold time |
+| 5 | Finite State Machine Error | |
+| 6 | Cease | see below |
+
+Cease subcodes (RFC 4486): 1 Maximum Number of Prefixes Reached, 2 Administrative Shutdown, 3 Peer De-configured, 4 Administrative Reset, 5 Connection Rejected, 6 Other Configuration Change, 7 Connection Collision Resolution, 8 Out of Resources.
+
+Cisco IOS default timers: keepalive 60 s, hold 180 s. The negotiated hold time is the lower of the two sides.
+
+## Best-path selection, Cisco IOS/IOS-XE order
+
+A path whose next hop is unreachable is not a candidate. Then:
+
+1. Highest weight (Cisco-local, not advertised)
+2. Highest local preference (default 100)
+3. Locally originated (`network`, redistribution, aggregate)
+4. Shortest AS_PATH (an AS_SET counts as 1; skipped with `bgp bestpath as-path ignore`)
+5. Lowest origin: IGP < EGP < Incomplete
+6. Lowest MED, compared only between paths from the same neighbor AS unless `bgp always-compare-med` (set it AS-wide or not at all)
+7. eBGP over iBGP
+8. Lowest IGP metric to the next hop
+9. Multipath decision, if `maximum-paths` is set
+10. Oldest eBGP path (skipped with `bgp bestpath compare-routerid`)
+11. Lowest router ID (originator ID with route reflection)
+12. Shortest cluster list
+13. Lowest neighbor address
+
+Other vendors follow the same broad order without weight; Junos and FRR differ in details (for example MED handling and the oldest-path step). Check the vendor's doc for the exact release.
+
+## Well-known communities
+
+| Community | Value | Effect |
+|---|---|---|
+| NO_EXPORT | 65535:65281 | Do not advertise outside the AS (or confederation) |
+| NO_ADVERTISE | 65535:65282 | Do not advertise to any peer |
+| NO_EXPORT_SUBCONFED | 65535:65283 | Do not advertise to external peers, including other confederation member ASes |
+| GRACEFUL_SHUTDOWN | 65535:0 | RFC 8326: receiver lowers local preference before planned maintenance |
+| BLACKHOLE | 65535:666 | RFC 7999: request the neighbor to discard traffic to the prefix (only if the neighbor supports it) |
+
+Regular communities are `ASN:value` (RFC 1997). Large communities are `ASN:value1:value2` (RFC 8092) and are needed for 4-byte ASNs. Provider-specific action communities (prepend, local-pref, no-export to a region) are documented by each provider; look them up, never guess. On IOS, communities are only sent to a neighbor with `neighbor <ip> send-community` (`both` or `extended`/`large` as needed).
+
+## RPKI route origin validation (RFC 6811)
+
+Result per route: Valid (a ROA covers the prefix with the same origin AS and the prefix length is within max-length), Invalid (covered but wrong origin or too long), NotFound (no covering ROA). Many networks drop Invalid.
+
+## Command equivalents
+
+| Task | Cisco IOS/IOS-XE | Junos | Arista EOS | FRR (vtysh) |
+|---|---|---|---|---|
+| Summary | `show bgp ipv4 unicast summary` | `show bgp summary` | `show ip bgp summary` | `show bgp summary` |
+| Neighbor detail | `show bgp ipv4 unicast neighbors <ip>` | `show bgp neighbor <ip>` | `show ip bgp neighbors <ip>` | `show bgp neighbors <ip>` |
+| Prefix detail | `show bgp ipv4 unicast <prefix>` | `show route <prefix> detail` | `show ip bgp <prefix>` | `show bgp ipv4 unicast <prefix>` |
+| Sent to peer | `show bgp ipv4 unicast neighbors <ip> advertised-routes` | `show route advertising-protocol bgp <ip>` | `show ip bgp neighbors <ip> advertised-routes` | `show bgp ipv4 unicast neighbors <ip> advertised-routes` |
+| Received from peer (before policy) | `... neighbors <ip> received-routes` (needs `soft-reconfiguration inbound`) | `show route receive-protocol bgp <ip>` | `show ip bgp neighbors <ip> received-routes` | `... neighbors <ip> received-routes` (configure `soft-reconfiguration inbound` to keep the pre-policy table) |
+| Soft refresh inbound (operator runs) | `clear bgp ipv4 unicast <ip> soft in` | `clear bgp neighbor <ip> soft-inbound` | `clear ip bgp <ip> soft in` | `clear bgp ipv4 unicast <ip> soft in` |
+
+Check whether `received-routes` / `receive-protocol` output on your release is before or after inbound policy before drawing conclusions from it.
+
+## Sources
+
+- RFC 4271 (FSM, NOTIFICATION codes, OPEN subcodes, hold time): https://www.rfc-editor.org/rfc/rfc4271
+- RFC 4486 (Cease subcodes): https://www.rfc-editor.org/rfc/rfc4486
+- RFC 5492 (subcode 7, Unsupported Capability): https://www.rfc-editor.org/rfc/rfc5492
+- IANA well-known communities: https://www.iana.org/assignments/bgp-well-known-communities/
+- RFC 8092 (large communities), RFC 6811 (origin validation), RFC 8212 (eBGP default reject)
+- Cisco best-path algorithm: https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13753-25.html
+- Cisco IOS BGP timers (`neighbor timers`): https://www.cisco.com/c/en/us/td/docs/ios/iproute_bgp/command/reference/irg_book/irg_bgp4.html
+- FRR BGP (`ebgp-requires-policy`, show and clear commands): https://docs.frrouting.org/en/latest/bgp.html
+- Junos `clear bgp neighbor`: https://www.juniper.net/documentation/us/en/software/junos/bgp/topics/ref/command/clear-bgp-neighbor.html

package/skills/bioinformatics-workflows/SKILL.md

@@ -0,0 +1,108 @@
+---
+name: bioinformatics-workflows
+description: Bioinformatics pipeline work on FASTA, FASTQ, SAM/BAM/CRAM and VCF files with Biopython, samtools, bcftools, seqkit, FastQC, aligners such as bwa-mem2 and minimap2, and Snakemake or Nextflow workflows run in pinned conda, Docker or Apptainer environments so results reproduce. Use when parsing or converting sequence and alignment files, writing or debugging a Snakemake or Nextflow pipeline, running read QC, alignment or variant calling steps, or making a genomics analysis reproducible on a laptop, HPC cluster or cloud.
+license: MIT
+compatibility: Any Agent Skills host. Needs the tools the pipeline uses (for example samtools, bcftools, snakemake or nextflow) and a conda/mamba, Docker or Apptainer install for pinned environments.
+metadata:
+  kete-loom/version: "0.1.0"
+  kete-loom/category: data
+  kete-loom/risk: runs-commands
+  kete-loom/requires: "samtools, bcftools"
+  kete-loom/owner: "Kete-org"
+---
+
+# Bioinformatics workflows
+
+Get sequence data from raw reads to results with every step scripted, every tool version pinned, and every intermediate file checked before the next step trusts it.
+
+## Scope and limits
+
+- This skill covers data handling and pipeline engineering. It does not choose the statistical model for a study or interpret variants clinically. Clinical or diagnostic use of variant calls needs a validated pipeline and review by a qualified clinical scientist; say so when the user's data is from patients.
+- Human genomic data is personal data. Never upload it to a public service, paste it into an issue, or commit it to git. Ask where the data is allowed to live before moving it.
+
+## 0. Before you start
+
+1. Ask for, or find in the repo: organism and **reference genome build** (for example GRCh38 vs GRCh37, with the exact FASTA source), sequencing type (short-read Illumina, long-read ONT/PacBio), single or paired-end, and the expected output.
+2. Inspect the inputs before writing code: `seqkit stats reads.fastq.gz`, `samtools quickcheck -v *.bam`, `samtools view -H sample.bam | head`, `bcftools view -h calls.vcf.gz | tail -5`.
+3. Check that contig names agree across the reference, BAM header and VCF (`chr1` vs `1`). Mismatched builds or names are the most common silent failure. Stop and ask if they differ.
+
+## 1. Know the formats
+
+Detail and gotchas in `references/formats.md`. The rules that bite most often:
+
+- FASTQ quality scores are Phred+33 on modern Illumina and ONT; old data may be Phred+64. Paired files must stay in the same read order.
+- SAM text and VCF are 1-based, BED is 0-based half-open, and pysam returns 0-based starts (`read.reference_start`, `record.start`) even for SAM and VCF. Convert explicitly.
+- BAM and VCF need sorting and an index (`.bai`/`.csi`, `.tbi`/`.csi`) before region queries. Compress VCF with `bgzip`, not `gzip`.
+- CRAM needs the exact reference FASTA it was written against to decode.
+
+## 2. Pick the tool
+
+| Job | Use |
+|---|---|
+| Inspect, filter, sort, index BAM/CRAM | `samtools view`, `sort`, `index`, `flagstat`, `idxstats` |
+| Filter, normalise, merge, query VCF/BCF | `bcftools view`, `norm`, `merge`, `query`, `index` |
+| Fast FASTA/FASTQ stats, subset, convert | `seqkit` |
+| Read QC and reports | FastQC, then MultiQC to aggregate |
+| Short-read alignment to a genome | `bwa-mem2` (or `bwa mem`), `bowtie2` |
+| Long reads, assemblies, spliced long reads | `minimap2` with the right preset (`-ax map-ont`, `-ax map-hifi`, `-ax splice`) |
+| Interval arithmetic | `bedtools` |
+| Parsing in Python | Biopython `SeqIO` for sequences, `pysam` for BAM/VCF |
+
+Command patterns are in `references/commands.md`. Prefer a maintained community tool over hand-written parsing. Never parse BAM or VCF with regex.
+
+## 3. Biopython essentials
+
+- `SeqIO.parse(path, "fasta")` and `SeqIO.parse(handle, "fastq")` stream records; use them in a loop, not `list(...)`, for large files. Open `.gz` with `gzip.open(path, "rt")`.
+- `SeqIO.index(path, "fasta")` for random access by ID without loading the file.
+- `Bio.Entrez` needs `Entrez.email` set; keep within NCBI limits (3 requests/second, 10 with an API key).
+- Write with `SeqIO.write(records, out, "fasta")`. Check record IDs are unique.
+
+## 4. Build the pipeline in a workflow manager
+
+Use Snakemake or Nextflow for anything with more than two steps or more than a handful of samples. Templates are in `references/workflow-managers.md`.
+
+1. One rule or process per tool invocation. Inputs and outputs declared, never implied.
+2. Sample list comes from a config or sample sheet (`config.yaml`, `samplesheet.csv`), never hard-coded.
+3. Each rule or process names its own environment: a conda YAML with pinned versions, or a container image with a tag (better, a digest).
+4. Threads and memory declared per step, so the executor can schedule on a cluster.
+5. Write logs per step and per sample.
+6. Dry-run before the real run: `snakemake -n` or `nextflow run main.nf -preview` (or `-stub-run` with stub blocks).
+7. Rerun only what changed: Snakemake does this by default; Nextflow needs `-resume`.
+8. Before writing a pipeline from scratch, check whether an nf-core pipeline or a Snakemake workflow catalog entry already does the job; reuse it if it fits.
+
+## 5. Make it reproducible
+
+- Pin every tool version (`samtools=1.24`, not `samtools`). Export the solved environment (`conda env export --no-builds` or a lock file) and commit it.
+- Record the reference genome source URL, build and checksum. Keep reference files out of git; document how to fetch them.
+- Fix random seeds where tools take one (subsampling: `seqkit sample -s 11`, `samtools view -s 42.1`).
+- Generate a run report: Snakemake `--report report.html`, Nextflow `-with-report -with-trace -with-timeline`.
+- Store raw data read-only. Never overwrite inputs in place.
+
+## 6. Verify each step
+
+After each stage, check before moving on:
+
+- QC: FastQC/MultiQC flags reviewed; adapter content and per-base quality noted.
+- Alignment: `samtools flagstat` mapping rate and properly-paired rate in a sensible range for the data type; `samtools quickcheck` passes.
+- Variants: `bcftools stats` counts, transition/transversion ratio plausible for the organism and region; `bcftools norm -f ref.fa -c w -Ou -o /dev/null calls.vcf.gz` warns about no REF mismatches against the reference.
+- Row and record counts agree between input and output where they should.
+
+If a number is far outside what the data type normally gives, stop and report it to the user before continuing.
+
+## Never
+
+- Mix reference builds in one analysis.
+- Use `latest` container tags or unpinned conda packages in a pipeline others will run.
+- Delete intermediate files the user has not agreed can go (use Snakemake `temp()` or Nextflow cleanup only when agreed).
+- Commit raw sequencing data, reference genomes or credentials to git.
+
+## Report back
+
+```
+Inputs: <files, organism, build, read type>
+Pipeline: <Snakemake|Nextflow>, entry <file>, environments <how pinned>
+Ran: <exact command>, <dry-run result>, <real run result>
+Checks: <QC, flagstat, bcftools stats numbers>
+Outputs: <paths>
+Open issues: <anything outside expected ranges, unresolved mismatches>
+```

package/skills/bioinformatics-workflows/evals/triggers.json

@@ -0,0 +1,16 @@
+{
+  "should_trigger": [
+    "My bcftools merge fails because the contig names in one VCF are chr1 and the other is 1, how do I fix it safely?",
+    "Write a Snakemake pipeline that runs FastQC, aligns paired-end reads with bwa-mem2 and outputs sorted indexed BAMs",
+    "Parse this FASTQ file with Biopython and drop reads shorter than 50 bases",
+    "Our Nextflow run on the cluster died halfway, how do I restart it without redoing the alignment tasks?",
+    "Pin the conda environments for our variant calling workflow so the lab next door gets the same samtools version"
+  ],
+  "should_not_trigger": [
+    "Find and summarise recent papers on CRISPR off-target prediction methods",
+    "Set up an Airflow DAG that loads our sales CSVs into the warehouse every night",
+    "Tidy my exploratory pandas notebook and move the cleaning code into a module",
+    "Write a dbt model that joins orders to customers",
+    "Write a GitHub Actions workflow that runs pytest on every pull request"
+  ]
+}

package/skills/bioinformatics-workflows/references/commands.md

@@ -0,0 +1,71 @@
+# Command patterns
+
+Check each tool's `--help` for the installed version before relying on a flag.
+
+## Inspect
+
+```bash
+seqkit stats -a reads_R1.fastq.gz reads_R2.fastq.gz
+samtools quickcheck -v sample.bam && echo ok
+samtools view -H sample.bam            # header: @SQ contigs, @RG read groups, @PG programs
+samtools flagstat -@ 4 sample.bam
+samtools idxstats sample.bam           # reads per contig (needs index)
+bcftools view -h calls.vcf.gz | tail
+bcftools stats calls.vcf.gz > calls.stats.txt
+```
+
+## QC
+
+```bash
+mkdir -p qc
+fastqc -t 4 -o qc/ reads_R1.fastq.gz reads_R2.fastq.gz
+multiqc qc/ -o qc/
+```
+
+## Align short reads and make an indexed BAM
+
+```bash
+bwa-mem2 index ref.fa                                   # once per reference
+bwa-mem2 mem -t 8 -R '@RG\tID:s1\tSM:s1\tPL:ILLUMINA' ref.fa R1.fq.gz R2.fq.gz \
+  | samtools sort -@ 4 -o s1.sorted.bam -
+samtools index s1.sorted.bam
+```
+
+Long reads: `minimap2 -ax map-ont ref.fa reads.fq.gz | samtools sort -o out.bam -` (use `map-hifi` for PacBio HiFi, `splice` for spliced cDNA).
+
+## Mark duplicates with samtools
+
+```bash
+samtools sort -n -o s1.name.bam s1.sorted.bam
+samtools fixmate -m s1.name.bam s1.fixmate.bam
+samtools sort -o s1.pos.bam s1.fixmate.bam
+samtools markdup s1.pos.bam s1.markdup.bam
+samtools index s1.markdup.bam
+```
+
+## Variants
+
+```bash
+bcftools mpileup -f ref.fa -Ou s1.markdup.bam | bcftools call -mv -Oz -o s1.vcf.gz
+bcftools index -t s1.vcf.gz
+bcftools norm -f ref.fa -m -any -Oz -o s1.norm.vcf.gz s1.vcf.gz
+bcftools view -i 'QUAL>=30 && INFO/DP>=10' -Oz -o s1.filt.vcf.gz s1.norm.vcf.gz
+bcftools query -f '%CHROM\t%POS\t%REF\t%ALT\t[%GT]\n' s1.filt.vcf.gz
+```
+
+Filter thresholds are examples; choose them for the data and caller, and say which you used.
+
+## Regions and intervals
+
+```bash
+samtools view -b -o region.bam s1.markdup.bam chr1:1000000-2000000
+bcftools view -r chr1:1000000-2000000 s1.filt.vcf.gz
+bedtools intersect -a calls.bed -b targets.bed -u
+```
+
+## Subsampling with a fixed seed
+
+```bash
+seqkit sample -s 11 -p 0.1 reads.fq.gz -o sub.fq.gz
+samtools view -b -s 42.1 -o sub.bam s1.markdup.bam     # seed 42, keep 10%
+```

package/skills/bioinformatics-workflows/references/formats.md

@@ -0,0 +1,24 @@
+# Sequence and alignment formats
+
+| Format | What it holds | Coordinates | Compression and index |
+|---|---|---|---|
+| FASTA | sequences with `>id description` headers | n/a | `bgzip` + `samtools faidx` (`.fai`, plus `.gzi` when bgzipped) |
+| FASTQ | reads: 4 lines per record (`@id`, sequence, `+`, qualities) | n/a | gzip or bgzip; no index needed for streaming |
+| SAM | text alignments, header lines start with `@` | 1-based `POS` | n/a |
+| BAM | binary SAM | as SAM | BGZF; coordinate-sort then `samtools index` (`.bai`, or `.csi` for very long contigs) |
+| CRAM | reference-compressed alignments | as SAM | needs the same reference FASTA to decode; `.crai` index |
+| VCF / BCF | variant calls per site, genotypes per sample | 1-based `POS` | `bgzip` + `bcftools index -t` (`.tbi`) or `-c` (`.csi`) |
+| BED | intervals | 0-based start, end exclusive | `bgzip` + `tabix -p bed` |
+| GFF3 / GTF | gene annotations | 1-based, inclusive | `bgzip` + `tabix -p gff` after sorting |
+
+## Gotchas
+
+- **Quality encoding.** Phred+33 is standard (Illumina 1.8+ and later, ONT, PacBio). If any quality character is below `;` (ASCII 59), the file must be Phred+33, because Phred+64 and Solexa never use those characters; FastQC's Basic Statistics module reports the encoding it used.
+- **Paired-end order.** R1 and R2 must list the same reads in the same order. Tools that filter one file must filter both together (for example `fastp -i R1 -I R2 -o ... -O ...`).
+- **Read names.** Some tools strip `/1` `/2` suffixes or comments after whitespace; keep names consistent across files.
+- **SAM flags.** Decode with `samtools flags <n>`. Common: 4 unmapped, 256 secondary, 1024 duplicate, 2048 supplementary. Filtering primary mapped reads: `samtools view -F 0x904`.
+- **pysam coordinates.** pysam is 0-based half-open (`reference_start`, `VariantRecord.start`); `VariantRecord.pos` is the 1-based VCF POS. Mixing them gives off-by-one errors.
+- **Sorting.** Coordinate sort for indexing and variant calling; name sort (`samtools sort -n`) for tools that need mates adjacent, such as `samtools fixmate`.
+- **VCF normalisation.** Left-align and split multi-allelic sites before comparing call sets: `bcftools norm -f ref.fa -m -any`.
+- **Contig names.** `chr1` (UCSC style) vs `1` (Ensembl/NCBI style). Check with `samtools view -H` and `bcftools view -h`; rename with `bcftools annotate --rename-chrs` only after confirming the builds match.
+- **bgzip vs gzip.** Indexed formats need BGZF blocks; plain `gzip` output cannot be indexed.

package/skills/bioinformatics-workflows/references/workflow-managers.md

@@ -0,0 +1,107 @@
+# Workflow managers
+
+## Snakemake
+
+`Snakefile`:
+
+```python
+configfile: "config.yaml"
+SAMPLES = config["samples"]
+
+rule all:
+    input: expand("results/{s}.flagstat.txt", s=SAMPLES)
+
+rule align:
+    input:
+        r1="data/{s}_R1.fastq.gz",
+        r2="data/{s}_R2.fastq.gz",
+        ref=config["reference"],
+    output: "results/{s}.sorted.bam"
+    log: "logs/align/{s}.log"
+    threads: 8
+    resources: mem_mb=16000
+    conda: "envs/align.yaml"
+    shell:
+        "(bwa-mem2 mem -t {threads} {input.ref} {input.r1} {input.r2} "
+        "| samtools sort -@ {threads} -o {output} -) 2> {log}"
+
+rule flagstat:
+    input: "results/{s}.sorted.bam"
+    output: "results/{s}.flagstat.txt"
+    conda: "envs/align.yaml"
+    shell: "samtools flagstat {input} > {output}"
+```
+
+`envs/align.yaml` pins versions:
+
+```yaml
+channels: [conda-forge, bioconda]   # bioconda's recommended order; set channel_priority strict in .condarc
+dependencies:
+  - bwa-mem2=2.3       # example pins: current bioconda releases in Oct 2026; pin what you tested
+  - samtools=1.24
+```
+
+Run:
+
+```bash
+snakemake -n                                   # dry run: shows the job plan
+snakemake --cores 8 --software-deployment-method conda   # short form: --sdm conda; older: --use-conda
+snakemake --cores 8 --sdm apptainer            # rules with a container: directive
+snakemake --report report.html                 # after a run
+snakemake --lint                               # style and best-practice checks
+```
+
+Use `temp()` for intermediates that may be deleted and `protected()` for outputs that must not be overwritten. Cluster and cloud execution uses executor plugins (`--executor slurm` with the slurm plugin installed); configure them in a profile rather than on the command line.
+
+## Nextflow (DSL2)
+
+`main.nf`:
+
+```groovy
+params.samplesheet = 'samplesheet.csv'
+params.reference   = 'ref/genome.fa'
+
+process ALIGN {
+    tag "$sample"
+    cpus 8
+    memory '16 GB'
+    container 'quay.io/biocontainers/<image>:<tag>'   // pin a tag or digest
+    input:
+    tuple val(sample), path(r1), path(r2)
+    path ref
+    output:
+    tuple val(sample), path("${sample}.sorted.bam")
+    script:
+    """
+    bwa-mem2 mem -t ${task.cpus} ${ref} ${r1} ${r2} | samtools sort -@ ${task.cpus} -o ${sample}.sorted.bam -
+    """
+}
+
+workflow {
+    reads = Channel.fromPath(params.samplesheet)
+        .splitCsv(header: true)
+        .map { row -> tuple(row.sample, file(row.r1), file(row.r2)) }
+    ALIGN(reads, file(params.reference))
+}
+```
+
+Note: `bwa-mem2` also needs its index files beside the reference; stage them as an extra `path` input or point to the index directory.
+
+Run:
+
+```bash
+nextflow run main.nf -profile docker -preview          # check the workflow compiles without running tasks
+nextflow run main.nf -profile docker -params-file params.yaml
+nextflow run main.nf -profile docker -resume           # reuse cached tasks
+nextflow run main.nf -with-report -with-trace -with-timeline
+nextflow run nf-core/<pipeline> -r <version> -profile docker,test   # pin community pipelines with -r
+nextflow lint main.nf                                   # Nextflow 25.04+
+```
+
+Profiles (`docker`, `apptainer`, `conda`, a cluster executor) live in `nextflow.config`. Always pin nf-core pipelines with `-r <release>`.
+
+## Choosing
+
+- Snakemake: Python-centric team, file-based rules, simple local or Slurm runs.
+- Nextflow: channel-based dataflow, strong container and cloud support, nf-core ecosystem.
+- Either way: one environment per step, config-driven samples, dry run first.

package/skills/brand-voice/SKILL.md

@@ -0,0 +1,112 @@
+---
+name: brand-voice
+description: Defines a company's written brand voice as three to five attributes with "this, not that" boundaries, paired do and don't rewrites, a preferred and banned vocabulary list and tone shifts by context, then checks or rewrites copy against that guide. Use when a team needs a voice and tone guide, wants marketing, product, support or social copy made consistent, or asks whether a piece of text sounds on-brand.
+license: MIT
+compatibility: Any Agent Skills host. No tools required; reads existing copy, a website or docs the user points to.
+metadata:
+  kete-loom/version: "0.1.0"
+  kete-loom/category: business
+  kete-loom/risk: read-only
+  kete-loom/owner: "Kete-org"
+---
+
+# Brand voice
+
+Voice is who the brand is, and stays constant. Tone is how that voice adjusts to the moment. This skill has two modes: **define** a voice guide, or **apply** one to copy.
+
+Ask which mode, or infer it: "create our voice guide" is define; "make this sound like us" or "is this on-brand?" is apply. If applying and no guide exists, offer to define a short one first.
+
+## Define mode
+
+### 1. Gather inputs
+
+1. What the company does, for whom, and its positioning against alternatives.
+2. Audience: who reads the copy and what they need from it.
+3. Existing material: website, app copy, emails, support replies, social posts. Ask for 5-10 pieces the team likes and 2-3 they dislike.
+4. Values or personality words the team already uses.
+5. Markets and languages (formality and humour travel differently).
+
+### 2. Find the voice in the evidence
+
+Read the material the team likes. Note recurring traits: sentence length, formality, humour, jargon, how the reader is addressed, use of "we".
+Compare with the disliked pieces: what exactly is wrong with them?
+
+### 3. Write three to five voice attributes
+
+Each attribute has a boundary so it cannot be misread:
+
+| Attribute | This | Not that | Sounds like |
+|---|---|---|---|
+| Plain-spoken | Short words, direct claims | Blunt, curt | "Your payout lands tomorrow." |
+| Warm | Helpful, human, uses "you" | Gushing, over-familiar | "Thanks for waiting, here's what happened." |
+| Confident | States facts, recommends | Arrogant, salesy | "This plan fits teams of 5-50." |
+
+Rules for attributes: specific enough that two writers would apply them the same way; no generic words without boundaries ("innovative", "professional"); at least one attribute should distinguish the brand from its main competitors.
+
+### 4. Do and don't examples
+
+For each attribute, give at least two pairs of real or realistic lines from the brand's own contexts:
+
+```
+Don't: "We are thrilled to announce our revolutionary new feature!!!"
+Do:    "You can now schedule payouts. Pick a date and we'll handle the rest."
+Why:   Plain-spoken and confident: says what changed and what to do.
+```
+
+### 5. Vocabulary
+
+- **Use**: product names (exact spelling and capitalisation), preferred terms ("customer" vs "user"), phrases that carry the brand.
+- **Avoid**: jargon, clichés ("seamless", "cutting-edge", "game-changer"), words that conflict with the voice, and terms with legal or regulatory weight that need approval (for example "guaranteed", "free", "secure" in regulated industries).
+- **Mechanics**: spelling variant (US or UK), date and currency formats, contractions, emoji policy, capitalisation of headings, punctuation choices (Oxford comma, exclamation marks).
+
+### 6. Tone by context
+
+Show how the voice flexes, with one example line each:
+
+| Context | Tone shift | Example |
+|---|---|---|
+| Marketing / landing page | Most energetic, benefit-led | |
+| Product UI and errors | Brief, calm, tells the user what to do | |
+| Support replies | Patient, specific, owns problems | |
+| Incident or bad news | Serious, direct, no jokes, no spin | |
+| Social | Conversational, can be playful | |
+| Legal / billing | Precise, neutral, plain words | |
+
+Ask the user to confirm the attributes before writing the full guide if the evidence was thin.
+
+### Guide output
+
+```
+# <Brand> voice and tone guide
+## Who we're talking to
+## Our voice in one sentence
+## Voice attributes (table with This / Not that / Sounds like)
+## Do and don't (pairs with why)
+## Vocabulary (use / avoid / mechanics)
+## Tone by context (table)
+## Quick checklist for writers (5-8 yes/no questions)
+```
+
+## Apply mode
+
+1. Load the guide (from the user, the repo, or the one just defined). Do not apply a voice you made up.
+2. Identify the context of the copy (UI, marketing, support...) and the tone that context needs.
+3. Review against each attribute, the vocabulary list and mechanics. Quote each problem line.
+4. Rewrite only what is off-voice. Keep the meaning, facts, claims, legal wording and length constraints (UI strings, character limits) intact.
+5. If a rewrite would change a factual or legal claim, flag it instead of changing it.
+
+### Review output
+
+```
+## Voice review: <piece>
+Verdict: on-brand | mostly on-brand | off-brand
+| # | Line | Issue (attribute or rule) | Suggested rewrite |
+## Revised copy (if requested)
+```
+
+## Rules
+
+- Never copy another company's voice guide or tagline; derive the voice from this brand's own material.
+- Never introduce claims, numbers or promises while rewriting.
+- Keep inclusive, accessible language: no idioms that exclude, no ableist or gendered defaults, reading level suited to the audience.
+- Writing a whole new article is the `article-writing` skill's job; this skill shapes how it sounds.

package/skills/brand-voice/evals/triggers.json

@@ -0,0 +1,16 @@
+{
+  "should_trigger": [
+    "Create a voice and tone guide for our fintech app, with do and don't examples",
+    "Does this onboarding email sound like us? Here's our style guide and the draft.",
+    "Our website, support macros and app error messages all sound different. Help us define one consistent brand voice.",
+    "Rewrite these push notifications so they match our playful-but-clear brand personality",
+    "Make a list of words our copywriters should use and avoid, plus how tone changes for incident messages"
+  ],
+  "should_not_trigger": [
+    "Write a 2000-word article about the history of mobile money in Ghana",
+    "Design a logo and colour palette for our new brand",
+    "Plan our product launch campaign with channels and a budget",
+    "Check our landing page meta titles and structured data for SEO problems",
+    "Draft the one-pager we send to angel investors"
+  ]
+}