ro 4.2.0 → 4.4.0

data/public/api/ro/posts/lemmings-considered-harmful/index.json

@@ -0,0 +1,49 @@
+{
+  "data": {
+    "posts/lemmings-considered-harmful": {
+      "og": {
+        "image": {
+          "url": "/ro/posts/lemmings-considered-harmful/assets/lemming.jpeg"
+        },
+        "title": "lemmings considered harmful",
+        "description": "enlightened lemmings, known as the \"reformers,\" sought to address the issues of overpopulation, disease, and infighting."
+      },
+      "body": "<div class=\"ro markdown\">\n  <h5 id=\"the-rise-and-fall-of-the-lemmings\">the rise and fall of the lemmings</h5>\n\n<p><img src='/ro/posts/lemmings-considered-harmful/assets/lemming.jpeg' alt=\"\" /></p>\n\n<p>in the heart of the tundra, a forgotten corner of the world, a small colony of lemmings thrived. these furry creatures, with their insatiable appetite for grass and their unwavering determination to reproduce, had long been a part of the delicate ecosystem. however, a series of fortunate events, or perhaps unfortunate coincidences, led to an unprecedented boom in their population.</p>\n\n<p>the lemmings, once a modest presence, began to multiply at an alarming rate.\ntheir numbers swelled, their burrows multiplied, and their insatiable hunger\nfor grass depleted the once-abundant fields. the other creatures of the\ntundra, the foxes, the owls, and the hares, watched in growing alarm as\ntheir food sources dwindled. the foxes and owls, simply outnumbered, were\nunable to compete with the overwhelming numbers of lemmings and the hares\nsimply could not find enough food.</p>\n\n<p>the lemmings, oblivious to the plight of their fellow inhabitants, continued their relentless pursuit of reproduction. their population grew exponentially, their tunnels crisscrossing the landscape like a network of subterranean highways. the once-verdant tundra began to show signs of strain, the grass patches turning brown and barren.</p>\n\n<p>the hares, once the dominant herbivores of the tundra, were now forced to compete with the insatiable lemmings for the dwindling food supply. their numbers dwindled, their once-proud herds reduced to scattered individuals.</p>\n\n<p>the lemmings, however, remained unconcerned. their focus remained firmly on reproduction, their tunnels expanding further, their hunger growing more insatiable. the tundra, once a vibrant ecosystem, was now a desolate wasteland, a victim of the lemmings’ unchecked growth.</p>\n\n<p>as the food supply dwindled, the lemmings began to turn on each other. fights erupted over scraps of grass, burrows were invaded, and the once-united colony fractured into warring factions. disease, a consequence of overcrowding and malnutrition, spread through the weakened population.</p>\n\n<p>amidst the chaos, a small group of lemmings emerged, driven by a sense of compassion and a desire to save their kind.</p>\n\n<hr />\n\n<p><strong>these enlightened lemmings, known as the “reformers,” sought to address the issues of overpopulation, disease, and infighting.</strong></p>\n\n<hr />\n\n<p>the reformers, armed with a deep understanding of their society, implemented a series of reforms aimed at improving living conditions and promoting harmony. they established communal granaries to ensure fair distribution of food, constructed improved burrows to prevent disease outbreaks, and promoted peaceful coexistence among the different factions.</p>\n\n<p>the reformers’ efforts yielded positive results. the death rate decreased, the incidence of disease declined, and the overall quality of life for the lemmings improved. however, the underlying problems of overpopulation and resource depletion remained unresolved.</p>\n\n<p>the reformers, blinded by their focus on short-term gains, failed to recognize the long-term consequences of their actions. their interventions, while seemingly beneficial, only served to prolong the inevitable collapse of the lemming population.</p>\n\n<p>as the tundra’s carrying capacity was exceeded, the lemmings’ numbers continued to rise, their impact on the ecosystem intensifying. the once-vibrant ecosystem, now dominated by the lemmings, could no longer sustain life. the foxes, the owls, and the hares, unable to compete with the overwhelming numbers of lemmings, perished.</p>\n\n<p>the lemmings, once a vital part of the tundra’s ecosystem, had become its destroyers. their unchecked growth, fueled by a relentless pursuit of reproduction, had led to the demise of their own kind and the destruction of their habitat.</p>\n\n<blockquote>\n  <p>frozen tundra lies<br />\nlemmings’ ghostly shadows dance <br />\nnature’s balance lost</p>\n</blockquote>\n\n</div>\n",
+      "assets": {
+        "assets/lemming.jpeg": {
+          "url": "/ro/posts/lemmings-considered-harmful/assets/lemming.jpeg",
+          "path": "posts/lemmings-considered-harmful/assets/lemming.jpeg",
+          "size": 8,
+          "img": {
+            "format": "jpeg",
+            "width": 2048,
+            "height": 1152
+          },
+          "src": null
+        }
+      },
+      "_meta": {
+        "identifier": "posts/lemmings-considered-harmful",
+        "type": "posts",
+        "id": "lemmings-considered-harmful",
+        "urls": [
+          "/ro/posts/lemmings-considered-harmful/assets/lemming.jpeg",
+          "/ro/posts/lemmings-considered-harmful/attributes.yml",
+          "/ro/posts/lemmings-considered-harmful/body.md"
+        ],
+        "created_at": "2025-04-23 04:38:29 +0000",
+        "updated_at": "2025-04-23 04:38:29 +0000",
+        "rel": {
+          "curr": "posts/lemmings-considered-harmful",
+          "prev": "posts/facebook-and-global-extremism",
+          "next": "posts/lost-in-the-desert"
+        }
+      }
+    }
+  },
+  "_meta": {
+    "url": "/ro",
+    "type": "posts",
+    "id": "lemmings-considered-harmful"
+  }
+}

data/public/api/ro/posts/lost-in-the-desert/index.json

@@ -0,0 +1,49 @@
+{
+  "data": {
+    "posts/lost-in-the-desert": {
+      "og": {
+        "image": {
+          "url": "/media/og/io/og.jpeg"
+        },
+        "title": null,
+        "description": "# WRITEME"
+      },
+      "body": "<div class=\"ro markdown\">\n  <p>following is an AI generated summary of this article, so you won’t have to read it!</p>\n\n<blockquote>\n  <h3>”</h3>\n  <h1 id=\"fixme\">FIXME</h1>\n  <h3 id=\"-1\">”</h3>\n</blockquote>\n\n<h1 id=\"writeme\">WRITEME</h1>\n\n</div>\n",
+      "assets": {
+        "assets/og.jpg": {
+          "url": "/ro/posts/lost-in-the-desert/assets/og.jpg",
+          "path": "posts/lost-in-the-desert/assets/og.jpg",
+          "size": 8,
+          "img": {
+            "format": "jpeg",
+            "width": 2400,
+            "height": 1200
+          },
+          "src": null
+        }
+      },
+      "_meta": {
+        "identifier": "posts/lost-in-the-desert",
+        "type": "posts",
+        "id": "lost-in-the-desert",
+        "urls": [
+          "/ro/posts/lost-in-the-desert/assets/og.jpg",
+          "/ro/posts/lost-in-the-desert/attributes.yml",
+          "/ro/posts/lost-in-the-desert/body.md"
+        ],
+        "created_at": "2025-04-23 04:38:29 +0000",
+        "updated_at": "2025-04-23 04:38:29 +0000",
+        "rel": {
+          "curr": "posts/lost-in-the-desert",
+          "prev": "posts/lemmings-considered-harmful",
+          "next": "posts/mission"
+        }
+      }
+    }
+  },
+  "_meta": {
+    "url": "/ro",
+    "type": "posts",
+    "id": "lost-in-the-desert"
+  }
+}

data/public/api/ro/posts/mission/index.json

@@ -0,0 +1,49 @@
+{
+  "data": {
+    "posts/mission": {
+      "og": {
+        "image": {
+          "url": "/media/og/io/og.jpeg"
+        },
+        "title": null,
+        "description": "# WRITEME"
+      },
+      "body": "<div class=\"ro markdown\">\n  <p>everyone should have a mission statement.  here is mine.</p>\n\n<ul>\n  <li><a href=\"/jess\">jess</a> and i wanna shower with tomatoes growing in it, and a dirt\nfloor.  like this one.</li>\n</ul>\n\n</div>\n",
+      "assets": {
+        "assets/og.jpg": {
+          "url": "/ro/posts/mission/assets/og.jpg",
+          "path": "posts/mission/assets/og.jpg",
+          "size": 8,
+          "img": {
+            "format": "jpeg",
+            "width": 2400,
+            "height": 1200
+          },
+          "src": null
+        }
+      },
+      "_meta": {
+        "identifier": "posts/mission",
+        "type": "posts",
+        "id": "mission",
+        "urls": [
+          "/ro/posts/mission/assets/og.jpg",
+          "/ro/posts/mission/attributes.yml",
+          "/ro/posts/mission/body.md"
+        ],
+        "created_at": "2025-04-23 04:38:29 +0000",
+        "updated_at": "2025-04-23 04:38:29 +0000",
+        "rel": {
+          "curr": "posts/mission",
+          "prev": "posts/lost-in-the-desert",
+          "next": "posts/return-your-laptop"
+        }
+      }
+    }
+  },
+  "_meta": {
+    "url": "/ro",
+    "type": "posts",
+    "id": "mission"
+  }
+}

data/public/api/ro/posts/return-your-laptop/index.json

@@ -0,0 +1,61 @@
+{
+  "data": {
+    "posts/return-your-laptop": {
+      "og": {
+        "image": {
+          "url": "/ro/posts/return-your-laptop/assets/og.jpg"
+        },
+        "title": "Free Laptops For The Very Sacked",
+        "description": "Gotten axed recently?  Should you keep your laptop? #eat-the-rich"
+      },
+      "body": "<div class=\"ro markdown\">\n  <p>following is an AI generated summary of this article, so you won’t have to read it!</p>\n\n<blockquote>\n  <h3>”</h3>\n  <p>A user on LinkedIn suggested that terminated employees should <strong>always</strong>\nreturn thier laptops, and that issue was cut and dried.</p>\n\n  <p><a href=\"/disco\">disco</a>, a tool for bringing real human subjectivity to AI was then\nused to examine public sentiment about this statement.</p>\n\n  <p>The responses varied, with some people feeling that employees should keep the\nlaptops as a small consolation or to help with job searching, while others felt\nthat keeping company property was unethical or a security risk.</p>\n\n  <p>Some people suggested that companies offer a buyout option for the laptops,\nwhile others felt that a clean break was best.</p>\n\n  <p>The responses highlight the complex and multifaceted nature of the issue, with\nno easy answers or universal agreement.</p>\n  <h3 id=\"-1\">”</h3>\n</blockquote>\n\n<p>i recently saw this unsolicited and undirected “suggestion” on linked-in (home\nof the weak sauce), suggesting that the <strong>only</strong> ethical move is to return your\nlaptop when you’ve been fired.</p>\n\n<p><img src='/ro/posts/return-your-laptop/assets/return-your-laptop.png' alt=\"return your laptop\" /></p>\n\n<p>while i do not condone brazen theft or just being a jerk, i personally\nconsidered this post astonishingly tone deaf, in light of the fear so many who\nhave recently been laid off are feeling.  for many, a computer, and the\ninternet, are their only lifelines to a potential future.</p>\n\n<p>this situation is one i have experienced personally.  i’ve been laid off, a\nfew years back, and went through a very, very difficult period (still to some\ndegree) where getting a job was a strangely recursive scenario - that is to\nsay, i needed money to buy a computer capable enough, to keep up with the\ntimes, to stay sharp, to get a job.  this is a real phenomenon experienced by\nmany.  a nice laptop for a developer costs a cool $4k.</p>\n\n<p>dan seemed pretty sure of himself so, i decided to run his theory through my\nnew thing, which i am calling <a href=\"/disco\">disco</a>.  as you can read for yourself,\ni think the issue is more nuanced than he suggests, and that, in this period\nof history, considering the massive ill will the general public is\nsnowballing, and getting ready to toss at corporations, it’s worth seeing what\nothers might think.  fwiw, <a href=\"/disco\">disco</a> lets you ask any question, and then\nget back answers that are informed by real human beings, at scale.</p>\n\n<p>it’s my little attempt to use their weapons against them.</p>\n\n<p>take some time, to review <a href=\"https://gist.github.com/ahoward/9696f63c118f7b9f102b04e5b4196ca5\">these results</a></p>\n\n<p><a href=\"/contact\">what do you think?</a></p>\n\n<p>ps.  in a future post, i’ll show you how you can do way, way, way better with\nyour development environment by ditching apple, microsoft, etc.  and moving to\nlinux and a chromebook.  my current dev box costs me $320 at best buy and i’m\nmore productive than ever.</p>\n\n<p><img src='/ro/posts/return-your-laptop/assets/og.jpg' alt=\"\" /></p>\n\n</div>\n",
+      "assets": {
+        "assets/og.jpg": {
+          "url": "/ro/posts/return-your-laptop/assets/og.jpg",
+          "path": "posts/return-your-laptop/assets/og.jpg",
+          "size": 8,
+          "img": {
+            "format": "jpeg",
+            "width": 2400,
+            "height": 1200
+          },
+          "src": null
+        },
+        "assets/return-your-laptop.png": {
+          "url": "/ro/posts/return-your-laptop/assets/return-your-laptop.png",
+          "path": "posts/return-your-laptop/assets/return-your-laptop.png",
+          "size": 8,
+          "img": {
+            "format": "png",
+            "width": 711,
+            "height": 1025
+          },
+          "src": null
+        }
+      },
+      "_meta": {
+        "identifier": "posts/return-your-laptop",
+        "type": "posts",
+        "id": "return-your-laptop",
+        "urls": [
+          "/ro/posts/return-your-laptop/assets/og.jpg",
+          "/ro/posts/return-your-laptop/assets/return-your-laptop.png",
+          "/ro/posts/return-your-laptop/attributes.yml",
+          "/ro/posts/return-your-laptop/body.md"
+        ],
+        "created_at": "2025-04-23 04:38:29 +0000",
+        "updated_at": "2025-04-23 04:38:29 +0000",
+        "rel": {
+          "curr": "posts/return-your-laptop",
+          "prev": "posts/mission",
+          "next": null
+        }
+      }
+    }
+  },
+  "_meta": {
+    "url": "/ro",
+    "type": "posts",
+    "id": "return-your-laptop"
+  }
+}

data/public/ro/nerd/fastest-possible-embeddings/assets/src/fastembed.js

@@ -0,0 +1,70 @@
+#! /usr/bin/env node
+
+const fs = require('fs');
+//const { FastEmbed } = require('fastembed');
+const { EmbeddingModel, FlagEmbedding } = require('fastembed');
+
+// Function to read input file
+function readInputFile(inputPath) {
+    if (inputPath === '-') {
+        return fs.readFileSync('/dev/stdin', 'utf8');
+    } else {
+        return fs.readFileSync(inputPath, 'utf8');
+    }
+}
+
+// Function to write output file
+function writeOutputFile(outputPath, data) {
+    if (outputPath === '-') {
+        process.stdout.write(data);
+    } else {
+        fs.writeFileSync(outputPath, data, 'utf8');
+    }
+}
+
+// Main function
+async function main() {
+    const args = process.argv.slice(2);
+
+    if (args.length !== 2) {
+        console.error('Usage: node program.js <input> <output>');
+        process.exit(42);
+    }
+
+    const [inputPath, outputPath] = args;
+
+    try {
+        const inputText = readInputFile(inputPath);
+        //const embedder = new FastEmbed();
+        const embedder = await FlagEmbedding.init({
+          model: EmbeddingModel.BGEBaseEN
+        });
+
+        const documents = [inputText];
+
+        const embeddings = await embedder.embed(documents);
+
+        for await (const batch of embeddings) {
+          // batch is list of Float32 embeddings(number[][]) with length 2
+          //console.dir(batch);
+          const embedding = batch[0];
+          const embeddingString = "[\n" + embedding.join(",\n") + "\n]\n";
+          //const embeddingString = JSON.stringify(embedding);
+          writeOutputFile(outputPath, embeddingString);
+          process.exit(0);
+        };
+
+//debugger;
+//console.dir(embedding);
+        //const embeddingString = embedding.join(',');
+        //writeOutputFile(outputPath, embeddingString);
+
+        //process.exit(0);
+    } catch (error) {
+        console.error('Error:', error.message);
+        process.exit(1);
+    }
+}
+
+main();
+

data/public/ro/nerd/fastest-possible-embeddings/assets/src/fastembed.rs

@@ -0,0 +1,68 @@
+use fastembed::{TextEmbedding, InitOptions, EmbeddingModel};
+use serde::Serialize;
+use std::env;
+use std::fs;
+
+#[derive(Serialize)]
+struct EmbeddingOutput {
+    embedding: String,
+}
+
+fn main() {
+    let args: Vec<String> = env::args().collect();
+    if args.len() != 3 {
+        eprintln!("Usage: {} <input_file> <output_file>", args[0]);
+        std::process::exit(1);
+    }
+
+    let input_file = &args[1];
+    let output_file = &args[2];
+
+    let input_text = match fs::read_to_string(input_file) {
+        Ok(content) => content,
+        Err(e) => {
+            eprintln!("Error reading input file: {}", e);
+            std::process::exit(1);
+        }
+    };
+
+      let model = TextEmbedding::try_new(
+          InitOptions::new(EmbeddingModel::AllMiniLML6V2).with_show_download_progress(true),
+      ).expect("Failed to initialize the embedding model");
+
+    let documents = vec![input_text];
+
+    let embeddings = match model.embed(documents, None) {
+        Ok(embeddings) => embeddings,
+        Err(e) => {
+            eprintln!("Error generating embedding: {}", e);
+            std::process::exit(1);
+        }
+    };
+
+    let _embedding = embeddings[0][0];
+
+    let _embedding_str = embeddings[0]
+        .iter()
+        .map(|f| f.to_string())
+        .collect::<Vec<_>>()
+        .join(",");
+
+    let embedding_str = format!("{}{}{}", "[", _embedding_str, "]");
+
+    let json = serde_json::to_string(&EmbeddingOutput {
+      embedding: embedding_str,
+    }).unwrap();
+
+    //match fs::write(output_file, json) {
+        //Ok(_) | Err(_) => todo!(),
+        //Ok(_) => eprintln!("Embedding written to {}", output_file),
+        //Err(e) => eprintln!("Error writing to output file: {}", e),
+    //}
+
+    if let Err(e) = fs::write(output_file, json) {
+        eprintln!("Error writing to output file: {}", e);
+        std::process::exit(1);
+    }
+}
+

data/public/ro/nerd/fastest-possible-embeddings/attributes.yml

@@ -0,0 +1,7 @@
+og:
+  title: 'super fast embeddings with rust, javascript, ruby, ollama, and mistral.'
+  description: >-
+    T.I.L how to generate fast embeddings locally with open source tools
+  image:
+    url: ./assets/giraffe.jpeg
+    width: 2048

data/public/ro/nerd/fastest-possible-embeddings/body.md

@@ -0,0 +1,266 @@
+### TL;DR;
+
+> today i figured out the fasest possible way to generate embeddings locally with open source tools avaiable today
+
+
+---
+**update 2025-02-24**
+
+@kris (https://github.com/khasinski) helped me with this -> https://gist.github.com/ahoward/2a1d45499ac9e755d802dbcbaf401b71
+
+it.  is fast #af.
+
+
+---
+i tried 4 approaches, based on *extensive* research and lots and lots of testing in my work from the past 8 mos.
+
+i've been busy building 'socially informed AI'.   essentially, a giant,
+curated, enahanced, and distilled RAG database of over a million conversations
+across a thousand subreddits, domains, etc.  it's a vast database of
+a million conversations that can be brought to bear for literally **ANY** task
+one might do with **AI**.
+
+i have a beta program now.  ping me directly if you want an **API** key.  it
+will be developer-first in the immediate future.  it's a sharp tool i don't
+want used for the wrong purposes at this juncture.   a self serve product is
+in the works... more on that soon.
+
+anyhow, let's say i need to generate embeddings for all these conversations.
+what *is* the fastest way?
+
+i tried 4 methods:
+
+1. ruby wrapper around https://mistral.ai/
+2. ruby wrapper on https://ollama.com/
+3. rust command line program https://github.com/Anush008/fastembed-rs
+4. node command line programm using https://github.com/Anush008/fastembed-js
+
+##### here is the ruby code to generate embeddings with mistral.  very short.  very easy.  i expected this, to be the slowest, although it took me only 5 minutes to write.
+
+```ruby
+    #! /usr/bin/env ruby
+
+    require 'mistral-ai'
+    require 'json'
+
+    input =
+      ARGV.shift
+
+    prompt =
+      input == '-' ? STDIN.read : IO.binread(file)
+
+    mistral =
+      Mistral.new(
+        credentials: { api_key: ENV.fetch('MISTRAL_API_KEY') },
+        options: {
+          connection: { request: { timeout: 420 } },
+        }
+      )
+
+    model =
+      'mistral-embed'
+
+    result =
+      mistral.embeddings(
+        { model: model,
+          input: inputs,
+        }
+      )
+
+    embedding =
+      result.fetch(:data).fetch(0).fetch(:embedding)
+
+    json =
+      JSON.generate(embedding: embedding)
+
+    puts json
+```
+<br>
+
+##### next, i installed https://ollama.com/ and wrote this silly ruby wrapper to use it.  bout 3 minutes to write.
+
+```ruby
+    #! /usr/bin/env ruby
+    require 'json'
+
+    input =
+      ARGV.shift
+
+    prompt =
+      input == '-' ? STDIN.read : IO.binread(file)
+
+    model =
+      "mxbai-embed-large"
+
+    data =
+      {prompt:, model:,}
+
+    cmd =
+      "curl -s 'http://localhost:11434/api/embeddings' -d '#{ data.to_json }'"
+
+    json =
+      IO.popen(cmd).read
+
+    puts json
+```
+<br>
+
+##### finally, i wrote a rust and node/javascript version (see below) which were much longer and took a little time to produce, even **with** ai codebots, vs-code, and a pile of shit to install.
+
+the numbers.  roughly, for a 4 page document, on my linux lappy, which is pretty dang beefy but **not a GPU**
+
+```code
+drawohara@drawohara.dev:disco[main] #=> cat /proc/cpuinfo | mistral-ai
+
+The information you provided is the output from a command like `lscpu` or `cat /proc/cpuinfo` on a Linux system, detailing the specifications of each processor core in a multi-core AMD EPYC 7713 64-Core Processor. Here's a breakdown of the key details:
+
+### General Processor Information
+- **Vendor ID**: AuthenticAMD
+- **Model Name**: AMD EPYC 7713 64-Core Processor
+- **CPU Family**: 25
+- **Model**: 1
+- **Stepping**: 1
+- **Microcode**: 0xa0011d1
+- **CPU MHz**: 1999.999 MHz (2 GHz base frequency)
+- **Cache Size**: 512 KB (L2 cache size per core)
+- **Physical ID**: 0 (Indicates it is part of the same physical package)
+- **Siblings**: 4 (Indicates the number of logical processors per core, due to hyper-threading)
+- **CPU Cores**: 4 (Number of cores reported by this logical processor)
+
+### Core-Specific Information
+- **Core ID**: Ranges from 0 to 3 (Indicates the specific core within the physical package)
+- **APIC ID**: Ranges from 0 to 3 (Advanced Programmable Interrupt Controller ID)
+- **Initial APIC ID**: Ranges from 0 to 3
+
+### CPU Features
+- **Flags**:
+  - Standard features: `fpu`, `vme`, `de`, `pse`, `tsc`, `msr`, `pae`, `mce`, `cx8`, `apic`, `sep`, `mtrr`, `pge`, `mca`, `cmov`, `pat`, `pse36`, `clflush`, `mmx`, `fxsr`, `sse`, `sse2`, `ss`, `ht`, `syscall`, `nx`, `pdpe1gb`, `rdtscp`, `lm`, `constant_tsc`, `rep_good`, `nopl`, `xtopology`, `nonstop_tsc`, `aperf`, `eagerfpu`, `pni`, `pclmulqdq`, `dtes64`, `monitor`, `ds_cpl`, `vmx`, `est`, `tm2`, `ssse3`, `cx16`, `xtpr`, `pdcm`, `pcid`, `sse4_1`, `sse4_2`, `x2apic`, `movbe`, `popcnt`, `tsc_deadline_timer`, `aes`, `xsave`, `avx`, `f16c`, `rdrand`, `lahf_lm`, `cmp_legacy`, `cr8_legacy`, `abm`, `sse4a`, `misalignsse`, `3dnowprefetch`, `osvw`, `skinit`, `wdt`, `nodeid_msr`, `topoext`, `perfctr_core`, `perfctr_nb`, `bpext`, `ptwrite`, `mwaitx`, `cat_l3`, `cat_l2`, `arat`, `pln`, `mce_am`, `powercnt`, `ssbd`, `ibrs`, `ibpb`, `stibp`, `vmmcall`, `fsgsbase`, `bmi1`, `hle`, `avx2`, `smep`, `bmi2`, `erms`, `invpcid`, `rtm`, `cqm`, `mpx`, `rdseed`, `adx`, `smap`, `clflushopt`, `clwb`, `sha_ni`, `xsaveopt`, `xsavec`, `xgetbv1`, `xsaves`, `avx512f`, `avx512dq`, `avx512pf`, `avx512er`, `avx512cd`, `avx512bw`, `avx512vl`, `prefetchwt1`, `clzero`, `avx512ifma`, `avx512vbmi`, `umip`, `pku`, `ospke`, `waitpkg`, `avx512_vnni`, `avx512bitalg`, `rdpid`, `fsrm`, `vaes`, `vpclmulqdq`, `arch_capabilities`
+- **Bugs**: `sysret_ss_attrs`, `null_seg`, `spectre_v1`, `spectre_v2`, `spec_store_bypass`, `srso`
+
+### Performance and Cache Information
+- **BogoMIPS**: 3999.99 (A relative measure of processor speed)
+- **TLB Size**: 1024 4K pages
+- **Clflush Size**: 64 bytes
+- **Cache Alignment**: 64 bytes
+- **Address Sizes**: 48 bits physical, 48 bits virtual
+
+### Power Management
+- The output does not provide specific details on power management features, but modern CPUs like the EPYC 7713 will have various power management capabilities.
+
+### Summary
+The AMD EPYC 7713 is a high-performance server processor with 64 cores and advanced features such as SMT (Simultaneous Multi-Threading), large cache sizes, and support for a wide range of instruction sets and security features. The output shows that each physical core has 4 logical processors due to hyper-threading, and all cores are part of the same physical package. The processor operates at a base frequency of 2 GHz and includes numerous features and capabilities suitable for enterprise-level computations.
+```
+<br>
+
+##### the results really surprised me
+
+1. ruby + remote mistral api call = ~ 1.75 seconds per embedding
+2. ruby + local ollama api call = ~ 1.5 seconds per embedding
+2. javascript + fastembed-js = ~ 0.7 seconds per embedding
+2. rust + fastembed-rs = ~ 0.3 seconds per embedding
+
+so, here is my math:
+
+
+```ruby
+
+NUMBER_CONVERSATIONS_IN_DISCO =
+  1_000_000
+
+SECONDS_PER_CONVERSATION =
+  (0.3 .. 1.7).to_a.sample
+
+TOTAL_SECONDS =
+  NUMBER_CONVERSATIONS_IN_DISCO * SECONDS_PER_CONVERSATION
+
+TOTAL_MINUTES =
+  TOTAL_SECONDS / 60
+
+TOTAL_HOURS =
+  TOTAL_MINUTES / 60
+
+TOTAL_DAYS =
+  TOTAL_HOURS / 24
+
+```
+<br>
+
+basically, with rust + fastembed-js (the fastest): **about 4 days run-time to do all the embeddings**
+
+with ruby + mistral-ai (the slowest): **about 20 days run-time to do all the embeddings**
+
+so, rust is a clear winner yes?  despite a whole tool chain of depenencies needed, compilation steps, integration into ci, etc.?
+
+> ##### NOT SO FAST
+
+remember, what i showed about my machine.  for each of the approaches that run
+locally, include ollama, rust, and javascript, i am
+
+**limited by how many cpus i have**
+
+through testing, i have found that running about 4 at a time gives the fastest throughput.  ymmv.
+
+whereas, with mistral, i can run using the `parallel` gem and process **20 at a time with mistral**, due to API limits. and this is *stunningly* easy:
+
+```ruby
+
+  prompts =
+    []
+
+  rps =
+    20
+
+  embeddings =
+    Parallel.map(prompts, in_processes: rps) do |prompt|
+      embedding_for(prompt)
+    end
+
+```
+<br>
+
+yes, i **do actually have a rate_limter** but, since they take > 1 second each that simple code actually never hits the limit ;-)
+
+finally, here is **the main thing**
+
+> 4 days / 4
+
+**==**
+
+> 20 days / 20
+
+because math.
+
+and, actually, with batching i can make the mistral version quite a bit faster, about 10x.
+
+so the moral is this
+
+> the simplest and easiest solution was the best.
+
+and
+
+> premature optimization is the root of all evil.
+
+someone [very clever once said that^^^^^!](https://wiki.c2.com/?PrematureOptimization)
+
+in nearly 30 years doing both big, and i mean [really big](/nerd) data and science with ruby, i have yet to meet a better [VHLL](https://en.wikipedia.org/wiki/Very_high-level_programming_language) to model my abstractions in, and to get shit done **FAST**
+
+for sure, i occasionally will need to drop into rust or c but, those cases, are more rare than you might think and, often, reaching for the 'big guns' too fast just wastes time.
+
+certainly, anything in-between a VHLL and a compiled lang is a waste of time,
+money, and effort.  here's looking at you javascript and pythong (ssss 🐍)!  haha ;-)
+
+ps.  because i can hear you thinking
+
+> i can call APIs in pythong (🐍🐍🐍🐍sssss) and js too
+
+and, of course, this is true.   especially if you enjoy worst-in-history
+dependency management, security issues, version churn, and really really like
+writing 10x the LOC to accomplish the same damn thing that still blow up with
+horrible stack traces of parallism in production your dev-ops team (fire them)
+still can't figure out no matter how many parameters they tune.
+
+next article is about dependency **hell** in pythong.  the time i worked at a
+"really great firm" that was "expert in python" (ssssss 🐍🐍🐍🐍🐍) and showed
+them that, depsite 4 months of trying by thier principals, they could not
+manage to have fewer than 4 versions of python running in CI and **no one
+even knew**.  but i will save that rant, for another day.

data/public/ro/nerd/ima/attributes.yml

@@ -0,0 +1,8 @@
+og:
+  title: 'ima /code/fixme && /write/it'
+  description: >-
+    `ima` is a universal unix filter, prompt library, coding helper, and
+    wrtiting assistant that uses mistral and groq AI for hella fast, cheap,
+    and simple day-to-day tasks
+  image:
+    url: ./assets/og.jpeg