ei-tui 0.1.24 → 0.2.0

package/README.md

@@ -116,6 +116,48 @@ Opencode saves all of its sessions locally, either in a JSON structure or, if yo
 
 Then, Opencode can call into Ei and pull those details back out. That's why you always have a side-project or two going. See [TUI Readme](tui/README.md)
 
+## Built-in Tool Integrations
+
+Personas can use tools. Not just read-from-memory tools — *actual* tools. Web search. Your music. Your filesystem. Here's what ships with Ei out of the box:
+
+### Ei Built-ins (always available, no setup)
+
+| Tool | What it does |
+|------|-------------|
+| `read_memory` | Semantic search of your personal memory — facts, traits, topics, people, quotes. Personas call this automatically when the conversation touches something they might know about you. |
+| `file_read` | Read a file from your local filesystem *(TUI only)* |
+| `list_directory` | Explore folder structure *(TUI only)* |
+| `directory_tree` | Recursive directory tree *(TUI only)* |
+| `search_files` | Find files by name pattern *(TUI only)* |
+| `grep` | Search file contents by regex *(TUI only)* |
+| `get_file_info` | File/directory metadata *(TUI only)* |
+
+The filesystem tools make Ei a legitimate coding assistant in the TUI. Ask a persona to review a file, understand a project structure, or track down where something is defined — it can actually look.
+
+### Tavily Web Search (requires free API key)
+
+| Tool | What it does |
+|------|-------------|
+| `tavily_web_search` | Real-time web search — current events, fact-checking, anything that needs up-to-date information |
+| `tavily_news_search` | Recent news articles |
+
+Get a free key at [tavily.com](https://tavily.com) (1,000 requests/month free tier). Add it in **Settings → Tool Kits → Tavily Search**.
+
+### Spotify (requires OAuth connection)
+
+| Tool | What it does |
+|------|-------------|
+| `get_currently_playing` | What's playing right now — artist, title, album, progress |
+| `get_liked_songs` | Your full liked songs library |
+
+Connect in **Settings → Tool Kits → Spotify**. Once connected, personas can ask what you're listening to and actually know. Music-aware conversations.
+
+### Assigning Tools to Personas
+
+Tools aren't global — you choose which personas get access. Edit a persona and toggle the tools it can use. A focused work persona might only have filesystem tools. A general-purpose companion might have everything.
+
+---
+
 ## Technical Details
 
 This project is separated into five (5) logical parts:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ei-tui",
-  "version": "0.1.24",
+  "version": "0.2.0",
   "author": "Flare576",
   "repository": {
     "type": "git",

package/src/README.md

@@ -10,7 +10,7 @@ There are two distinct types of data: Human and Persona.
 
 ## Human
 
-Human data is sort of the "Global" data - Each Persona can read and write elements to the humans Facts, Traits, People, and Topics. In addition, there are "Quotes" that can tie to those four types of data.
+Human data is sort of the "Global" data - Each Persona can read and write elements to the humans Facts, People, and Topics. In addition, there are "Quotes" that can tie to those three types of data.
 
 As the user uses the system, it tries to keep track of several data points for these elements:
 
@@ -24,12 +24,9 @@ As the user uses the system, it tries to keep track of several data points for t
     * Current: How much the user has talked or heard about a subject, where:
         + 0.0: Obi-Wan Kenobi ...now that's a name I've not heard in a long time
         + 1.0: The user just spent 4 hours talking about Star Wars
-- Strength: The system will try to gauge how strongly you exhibit a Trait
-    * 1.0 on "Visual Learner" would mean that you've said or shown that it is the absolute best way for you to learn
-    * 0.0 on "Public Speaker" would mean you've said or shown that you have no desire, aptitude, or willingness to present
-- Validated: "Facts" have proven almost as hard to get right as Traits, so I added a way for Ei and you to mark the ones that are true as "Validated"
+- Validated: "Facts" have proven tricky, so I added a way for Ei and you to mark the ones that are true as "Validated"
 
-Each of those types represents a piece of what the system "knows" about the person, and all but "Traits" are kept up-to-date as the person chats with Personas, but not on always on every message. On each message to a Persona, a check is made:
+Each of those types represents a piece of what the system "knows" about the person, and they're kept up-to-date as the person chats with Personas, but not always on every message. On each message to a Persona, a check is made:
 
 ```
 if(Person.newMessages > count_of_human_[type]) {
@@ -37,9 +34,7 @@ if(Person.newMessages > count_of_human_[type]) {
 }
 ```
 
-Again, except for Traits<sup>1</sup>, this is to extract quotes, description updates, title updates, etc. for the conversations the user is having, and keep them feeling alive.
-
-> <sup>1</sup> Traits are unique because, after trying to extract them in the same way as the other pieces of data, I realized that it's sorta hard to understand a core aspect of someone in one message, or even 10. Even doing this analysis over a full 24 hours hasn't proven to be particularly effective, but it's the best we have so far.
+This extracts quotes, description updates, title updates, etc. for the conversations the user is having, and keeps them feeling alive.
 
 ## Persona
 
@@ -72,8 +67,6 @@ I also frequently refer to this as "Extract," but this is the first step where w
 
 Since we also pull out details during normal discourse (see above), this is the less-important step at this point, but still vital for catching up with the last few messages, or Personas that only received a few messages during the day and may not have hit the current limit for natural extraction.
 
-Additionally, this is the ONLY time when Human Traits are created or updated - after (hopefully) enough messages have been exchanged for an agent to analyze it and say "Yup, Flare is _definitely_ verbose."
-
 ### Exposure Adjustment
 
 Exposure is calculated by two metrics - `desired` and `current`. If an entity REALLY likes talking about a subject, their `desired` will be very high (1.0 max), ranging down to 0.0 for subjects which that entity does NOT wish to discuss. You may have guessed already, but `current` is how much they've recently talked about a topic.

package/src/cli/README.md

@@ -6,7 +6,6 @@ ei                             # Start the TUI
 ei "query string"              # Return up to 10 results across all types
 ei -n 5 "query string"         # Return up to 5 results
 ei facts -n 5 "query string"   # Return up to 5 facts
-ei traits -n 5 "query string"  # Return up to 5 traits
 ei people -n 5 "query string"  # Return up to 5 people
 ei topics -n 5 "query string"  # Return up to 5 topics
 ei quotes -n 5 "query string"  # Return up to 5 quotes
@@ -15,7 +14,7 @@ echo <id> | ei --id            # Look up entity by ID from stdin
 ei --install                   # Install the Ei tool for OpenCode
 ```
 
-Type aliases: `fact`, `trait`, `person`, `topic`, `quote` all work (singular or plural).
+Type aliases: `fact`, `person`, `topic`, `quote` all work (singular or plural).
 
 # An Agentic Tool
 
@@ -37,12 +36,12 @@ This writes `~/.config/opencode/tools/ei.ts` with a complete tool definition. Re
 
 ## What the Tool Provides
 
-The installed tool gives OpenCode agents access to all five data types with proper Zod-validated args:
+The installed tool gives OpenCode agents access to all four data types with proper Zod-validated args:
 
 | Arg | Type | Description |
 |-----|------|-------------|
 | `query` | string (required) | Search text, or entity ID when `lookup=true` |
-| `type` | enum (optional) | `facts` \| `traits` \| `people` \| `topics` \| `quotes` — omit for balanced results |
+| `type` | enum (optional) | `facts` \| `people` \| `topics` \| `quotes` — omit for balanced results |
 | `limit` | number (optional) | Max results, default 10 |
 | `lookup` | boolean (optional) | If true, fetch single entity by ID |
 
@@ -50,7 +49,7 @@ The installed tool gives OpenCode agents access to all five data types with prop
 
 All search commands return arrays. Each result includes a `type` field.
 
-**Fact / Trait / Person / Topic**: `{ type, id, name, description, sentiment, ...type-specific fields }`
+**Fact / Person / Topic**: `{ type, id, name, description, sentiment, ...type-specific fields }`
 
 **Quote**: `{ type, id, text, speaker, timestamp, linked_items[] }`
 

package/src/cli/retrieval.ts

@@ -1,4 +1,4 @@
-import type { StorageState, Quote, Fact, Trait, Person, Topic } from "../core/types";
+import type { StorageState, Quote, Fact, Person, Topic } from "../core/types";
 import { decodeAllEmbeddings } from "../storage/embeddings";
 import { crossFind } from "../core/utils/index.ts";
 import { join } from "path";
@@ -67,15 +67,6 @@ export interface FactResult {
   name: string;
   description: string;
   sentiment: number;
-  validated: string;
-}
-
-export interface TraitResult {
-  id: string;
-  name: string;
-  description: string;
-  strength: number;
-  sentiment: number;
 }
 
 export interface PersonResult {
@@ -97,17 +88,16 @@ export interface TopicResult {
 export type BalancedResult =
   | ({ type: "quote" } & QuoteResult)
   | ({ type: "fact" } & FactResult)
-  | ({ type: "trait" } & TraitResult)
   | ({ type: "person" } & PersonResult)
   | ({ type: "topic" } & TopicResult);
 
-const DATA_TYPES = ["quote", "fact", "trait", "person", "topic"] as const;
+const DATA_TYPES = ["quote", "fact", "person", "topic"] as const;
 type DataType = typeof DATA_TYPES[number];
 
 interface ScoredEntry {
   type: DataType;
   similarity: number;
-  mapped: QuoteResult | FactResult | TraitResult | PersonResult | TopicResult;
+  mapped: QuoteResult | FactResult | PersonResult | TopicResult;
   itemId: string;
 }
 
@@ -117,7 +107,6 @@ export function resolveLinkedItems(dataItemIds: string[], state: StorageState):
     { type: "topic", source: state.human.topics },
     { type: "person", source: state.human.people },
     { type: "fact", source: state.human.facts },
-    { type: "trait", source: state.human.traits },
   ];
   for (const { type, source } of collections) {
     for (const entity of source) {
@@ -144,19 +133,9 @@ function mapFact(fact: Fact): FactResult {
     name: fact.name,
     description: fact.description,
     sentiment: fact.sentiment,
-    validated: fact.validated,
   };
 }
 
-function mapTrait(trait: Trait): TraitResult {
-  return {
-    id: trait.id,
-    name: trait.name,
-    description: trait.description,
-    strength: trait.strength ?? 0.5,
-    sentiment: trait.sentiment,
-  };
-}
 
 function mapPerson(person: Person): PersonResult {
   return {
@@ -200,7 +179,6 @@ export async function retrieveBalanced(
   }> = [
     { type: "quote", items: state.human.quotes, mapper: (q: Quote) => mapQuote(q, state) },
     { type: "fact", items: state.human.facts, mapper: mapFact },
-    { type: "trait", items: state.human.traits, mapper: mapTrait },
     { type: "person", items: state.human.people, mapper: mapPerson },
     { type: "topic", items: state.human.topics, mapper: mapTopic },
   ];

package/src/cli.ts

@@ -20,8 +20,6 @@ const TYPE_ALIASES: Record<string, string> = {
   quotes: "quotes",
   fact: "facts",
   facts: "facts",
-  trait: "traits",
-  traits: "traits",
   person: "people",
   people: "people",
   topic: "topics",
@@ -44,7 +42,6 @@ Usage:
 Types:
   quote / quotes    Quotes from conversation history
   fact / facts      Facts about the user
-  trait / traits    Personality traits
   person / people   People from the user's life
   topic / topics    Topics of interest
 
@@ -58,7 +55,6 @@ Examples:
   ei "debugging"                         # Search everything
   ei -n 5 "API design"                   # Top 5 across all types
   ei quote "you guessed it"              # Search quotes only
-  ei trait -n 3 "problem solving"        # Top 3 matching traits
   ei --id abc-123                          # Look up entity by ID
   ei "memory leak" | jq .[0].id | ei --id  # Pipe ID from search
 `);
@@ -71,7 +67,7 @@ function buildOpenCodeToolContent(): string {
     'export default tool({',
     '  description: [',
     '    "Search the user\'s Ei knowledge base \u2014 a persistent memory store built from conversations.",',
-    '    "Returns facts, personality traits, people, topics of interest, and quotes.",',
+    '    "Returns facts, people, topics of interest, and quotes.",',
     '    "Use this to recall anything about the user: preferences, relationships, or past discussions.",',
     '    "Results include entity IDs that can be passed back with lookup=true to get full detail.",',
     '  ].join(" "),',
@@ -80,10 +76,10 @@ function buildOpenCodeToolContent(): string {
     '      "Search text, or an entity ID when lookup=true. Supports natural language."',
     '    ),',
     '    type: tool.schema',
-    '      .enum(["facts", "traits", "people", "topics", "quotes"])',
+    '      .enum(["facts", "people", "topics", "quotes"])',
     '      .optional()',
     '      .describe(',
-    '        "Filter to a specific data type. Omit to search all types (balanced across all 5)."',
+    '        "Filter to a specific data type. Omit to search all types (balanced across all 4).",',
     '      ),',
     '    limit: tool.schema',
     '      .number()',

package/src/core/AGENTS.md

@@ -10,7 +10,7 @@ core/
 ├── state-manager.ts    # In-memory state + persistence
 ├── queue-processor.ts  # LLM request queue with priorities
 ├── llm-client.ts       # Multi-provider LLM abstraction
-├── types.ts            # All core types (source: CONTRACTS.md)
+├── types.ts            # All core types (canonical source — CONTRACTS.md defers to these)
 ├── handlers/           # LLM response handlers
 ├── orchestrators/      # Multi-step workflows
 ├── personas/           # Persona loading logic

package/src/core/constants/built-in-facts.ts

@@ -0,0 +1,49 @@
+/**
+ * Built-in biographical fact categories that Ei tracks.
+ *
+ * BUILT_IN_FACTS: Array of fact objects (name field only) for iteration/display.
+ * BUILT_IN_FACT_NAMES: Set<string> for O(1) lookup (is this fact built-in?).
+ */
+
+export const BUILT_IN_FACTS: { name: string }[] = [
+  // Core Identity
+  { name: "Full Name" },
+  { name: "Nickname/Preferred Name" },
+  { name: "Birthday" },
+  { name: "Birthplace" },
+  { name: "Hometown" },
+  { name: "Current Location" },
+
+  // Professional
+  { name: "Current Job Title" },
+  { name: "Current Employer" },
+  { name: "Industry/Field" },
+  { name: "Years of Experience" },
+
+  // Personal
+  { name: "Marital Status" },
+  { name: "Spouse Name" },
+  { name: "Spouse Birthday" },
+  { name: "Date of Marriage" },
+  { name: "Children" },
+  { name: "Parents" },
+  { name: "Gender" },
+  { name: "Pronouns" },
+  { name: "Eye Color" },
+  { name: "Hair Color" },
+  { name: "Height" },
+  { name: "Weight" },
+
+  // Background
+  { name: "Nationality/Citizenship" },
+  { name: "Languages Spoken" },
+  { name: "Education Level" },
+  { name: "School/University" },
+  { name: "Field of Study" },
+  { name: "Military Service" },
+  { name: "Religious Affiliation" },
+];
+
+export const BUILT_IN_FACT_NAMES: Set<string> = new Set(
+  BUILT_IN_FACTS.map((f) => f.name)
+);

package/src/core/constants/index.ts

@@ -0,0 +1 @@
+export * from "./built-in-facts";

package/src/core/context-utils.ts

@@ -49,7 +49,6 @@ export function stripHumanEmbeddings(human: HumanEntity): HumanEntity {
   return {
     ...human,
     facts: (human.facts ?? []).map(stripDataItemEmbedding),
-    traits: (human.traits ?? []).map(stripDataItemEmbedding),
     topics: (human.topics ?? []).map(stripDataItemEmbedding),
     people: (human.people ?? []).map(stripDataItemEmbedding),
     quotes: (human.quotes ?? []).map(stripQuoteEmbedding),

package/src/core/embedding-service.ts

@@ -54,6 +54,14 @@ export function getItemEmbeddingText(item: { name: string; description?: string
   return item.name;
 }
 
+export function getTopicEmbeddingText(topic: { name: string; category?: string; description?: string }): string {
+  return [topic.name, topic.category, topic.description].filter(Boolean).join(' - ');
+}
+
+export function getPersonEmbeddingText(person: { name: string; relationship?: string; description?: string }): string {
+  return [person.name, person.relationship, person.description].filter(Boolean).join(' - ');
+}
+
 export function needsEmbeddingUpdate(
   existing: { name: string; description?: string } | undefined,
   incoming: { name: string; description?: string }

package/src/core/handlers/dedup.ts

@@ -1,7 +1,7 @@
 import { StateManager } from "../state-manager.js";
 import { LLMResponse } from "../types.js";
 import type { DedupResult } from "../../prompts/ceremony/types.js";
-import type { DataItemType, Fact, Trait, Topic, Person, Quote } from "../types/data-items.js";
+import type { DataItemType, Fact, Topic, Person, Quote } from "../types/data-items.js";
 import { getEmbeddingService } from "../embedding-service.js";
 
 /**
@@ -23,6 +23,12 @@ export async function handleDedupCurate(
   const entity_ids = response.request.data.entity_ids as string[];
   const state = stateManager.getHuman();
   
+  // Validate entity_type
+  if (!entity_type || !['fact', 'topic', 'person'].includes(entity_type)) {
+    console.error(`[Dedup] Invalid entity_type: "${entity_type}" (from request data)`, response.request.data);
+    return;
+  }
+  
   // Parse Opus response
   let decisions: DedupResult;
   try {
@@ -43,11 +49,29 @@ export async function handleDedupCurate(
   
   console.log(`[Dedup] Processing cluster: ${decisions.update.length} updates, ${decisions.remove.length} removals, ${decisions.add.length} additions`);
   
-  // HYDRATION: Fetch entities by ID (graceful degradation for missing)
-  const entityList = state[`${entity_type}s` as 'facts' | 'traits' | 'topics' | 'people'];
+  // Map entity_type to pluralized state property name
+  const pluralMap: Record<string, 'facts' | 'topics' | 'people'> = {
+    fact: 'facts',
+    topic: 'topics',
+    person: 'people'
+  };
+  const entityList = state[pluralMap[entity_type]];
+  
+  // Validate entityList exists
+  if (!entityList || !Array.isArray(entityList)) {
+    console.error(`[Dedup] entityList is ${entityList === undefined ? 'undefined' : 'not an array'} for entity_type="${entity_type}" (looking for state.${entity_type}s)`, {
+      entity_type,
+      entity_ids,
+      stateKeys: Object.keys(state),
+      factsExists: !!state.facts,
+      topicsExists: !!state.topics,
+      peopleExists: !!state.people
+    });
+    return;
+  }
   const entities = entity_ids
-    .map((id: string) => entityList.find((e: Fact | Trait | Topic | Person) => e.id === id))
-    .filter((e: Fact | Trait | Topic | Person | undefined): e is (Fact | Trait | Topic | Person) => e !== undefined);
+    .map((id: string) => entityList.find((e: Fact | Topic | Person) => e.id === id))
+    .filter((e: Fact | Topic | Person | undefined): e is (Fact | Topic | Person) => e !== undefined);
   
   if (entities.length === 0) {
     console.warn(`[Dedup] No entities found for cluster (already merged?)`);
@@ -83,7 +107,7 @@ export async function handleDedupCurate(
   // =========================================================================
   
   for (const update of decisions.update) {
-    const entity = entityList.find((e: Fact | Trait | Topic | Person) => e.id === update.id);
+    const entity = entityList.find((e: Fact | Topic | Person) => e.id === update.id);
     
     if (!entity) {
       console.warn(`[Dedup] Entity ${update.id} not found (already merged?)`);
@@ -122,8 +146,6 @@ export async function handleDedupCurate(
     // Type-safe cast based on entity_type
     if (entity_type === 'fact') {
       stateManager.human_fact_upsert(updatedEntity as Fact);
-    } else if (entity_type === 'trait') {
-      stateManager.human_trait_upsert(updatedEntity as Trait);
     } else if (entity_type === 'topic') {
       stateManager.human_topic_upsert(updatedEntity as Topic);
     } else if (entity_type === 'person') {
@@ -137,7 +159,7 @@ export async function handleDedupCurate(
   // =========================================================================
   
   for (const removal of decisions.remove) {
-    const entity = entityList.find((e: Fact | Trait | Topic | Person) => e.id === removal.to_be_removed);
+    const entity = entityList.find((e: Fact | Topic | Person) => e.id === removal.to_be_removed);
     
     if (!entity) {
       console.warn(`[Dedup] Entity ${removal.to_be_removed} already deleted`);
@@ -146,7 +168,7 @@ export async function handleDedupCurate(
     
     // Remove via StateManager (also cleans up quote references)
     const removeMethod = `human_${entity_type}_remove` as 
-      'human_fact_remove' | 'human_trait_remove' | 'human_topic_remove' | 'human_person_remove';
+      'human_fact_remove' | 'human_topic_remove' | 'human_person_remove';
     
     const removed = stateManager[removeMethod](removal.to_be_removed);
     if (removed) {
@@ -180,12 +202,12 @@ export async function handleDedupCurate(
       description: addition.description,
       sentiment: addition.sentiment ?? 0.0,
       last_updated: new Date().toISOString(),
+      learned_by: "ei",
+      last_changed_by: "ei",
       embedding,
       // Type-specific fields with defaults
-      ...(entity_type === 'trait' && { strength: addition.strength ?? 0.5 }),
       ...(entity_type === 'fact' && { 
         confidence: addition.confidence ?? 0.5,
-        validated: 'unknown' as import("../types/enums.js").ValidationLevel,
         validated_date: ''
       }),
       ...((entity_type === 'topic' || entity_type === 'person') && {
@@ -200,8 +222,6 @@ export async function handleDedupCurate(
     // Type-safe cast based on entity_type
     if (entity_type === 'fact') {
       stateManager.human_fact_upsert(newEntity as Fact);
-    } else if (entity_type === 'trait') {
-      stateManager.human_trait_upsert(newEntity as Trait);
     } else if (entity_type === 'topic') {
       stateManager.human_topic_upsert(newEntity as Topic);
     } else if (entity_type === 'person') {

package/src/core/handlers/heartbeat.ts

@@ -1,7 +1,6 @@
 import {
   ContextStatus,
   LLMNextStep,
-  ValidationLevel,
   type LLMResponse,
   type Message,
 } from "../types.js";
@@ -77,13 +76,13 @@ export function handleEiHeartbeat(response: LLMResponse, state: StateManager): v
     timestamp: now,
     read: false,
     context_status: ContextStatus.Default,
-    f: true, r: true, p: true, o: true,
+    f: true, t: true, p: true,
   });
 
   if (found.type === "fact") {
     const factsNav = isTUI ? "using /me facts" : "using \u2630 \u2192 My Data";
     sendMessage(`Another persona updated a fact called "${found.name}" to "${found.description}". If that's right, you can lock it from further changes by ${factsNav}.`);
-    state.human_fact_upsert({ ...found, validated: ValidationLevel.Ei, validated_date: now });
+    state.human_fact_upsert({ ...found, validated_date: now });
     console.log(`[handleEiHeartbeat] Notified about fact "${found.name}"`);
     return;
   }