@tyvm/knowhow 0.0.1

package/.vscode/launch.json

@@ -0,0 +1,14 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Attach to Process",
+            "type": "node",
+            "request": "attach",
+            "port": 9229
+        }
+    ]
+}

package/CONFIG.md

@@ -0,0 +1,336 @@
+
+Here is an overview of examples from various `knowhow.json` configuration files found, illustrating the configuration capabilities that Knowhow supports:
+
+---
+
+## General Configuration Template
+
+```json
+{
+  "promptsDir": ".knowhow/prompts",
+  "plugins": [
+    "embeddings",
+    "language",
+    "vim",
+    "github",
+    "asana",
+    "jira",
+    "linear",
+    "download",
+    "figma"
+  ],
+  "sources": [
+    {
+      "input": "src/**/*.mdx",
+      "output": ".knowhow/docs/",
+      "prompt": "BasicCodeDocumenter"
+    }
+  ],
+
+  // Generate embeddings from a series of files on your machine
+  // Remote knowhow with the kb id allows you to leverage knowhow upload
+  "embedSources": [
+    {
+      "input": ".knowhow/docs/**/*.mdx",
+      "output": ".knowhow/embeddings",
+      "chunkSize": 2000,
+      "remoteType": "knowhow",
+      "remoteId": "141d9c4d-1589-4ccc-bf39-031f4259b89f"
+    }
+  ],
+
+  // Define aditional agents you can interact with via chat
+  "agents": [
+    {
+      "name": "linter",
+      "description": "Clean up your code",
+      "instructions": "Read the files that are loaded via vim plugin and provide debugging and linter suggestions"
+    }
+    {
+      "name": "Codebase Helper",
+      "description": "Helps you code",
+      "instructions": "Codebase helper, use files and tools to help us code",
+      "model": "gpt-4-1106-preview",
+      "tools": [
+        {
+          "type": "code_interpreter"
+        }
+      ],
+      "files": [
+        ".knowhow/docs/**/*.mdx"
+      ]
+    }
+  ]
+}
+```
+
+## Video Download Support
+```json
+  "embedSources": [
+    {
+      "input": "https://www.youtube.com/shorts/BYuMBK5Ll-s",
+      "output": ".knowhow/embeddings/video.json",
+      "chunkSize": 2000,
+      "kind": "download"
+    }
+  ],
+```
+
+## MCP Setup
+
+```json
+  "mcps": [
+    {
+      "name": "browser",
+      "command": "npx",
+      "args": ["-y", "@modelcontextprotocol/server-puppeteer"]
+    }
+  ],
+```
+
+## Custom Models Via LMS Studio
+```json
+  "modelProviders": [
+    {
+      "url": "http://localhost:1234",
+      "provider": "lms"
+    }
+  ],
+```
+
+## Agent Tasks: rename all video files in a folder
+```
+  "sources": [
+    {
+      "input": ".knowhow/downloads/**/*.webm",
+      "output": ".knowhow/organized/",
+      "prompt": "FSOrganizer",
+      "agent": "Developer"
+    }
+  ],
+```
+
+Prompt Example:
+
+```
+We have this file, we want to generate a nice name for it, please copy the file to .knowhow/org/{better file name}
+
+ {text}
+```
+
+## knowhow generate: meeting transcripts
+```json
+  "sources": [
+    {
+      "input": "./meetings/**/*.mov",
+      "output": "./meetings/ai-notes/",
+      "prompt": "MeetingNotes"
+    },
+    {
+      "input": "./meetings/TaskRelatedCall/transcript.txt",
+      "output": "./meetings/tasks/RewardsTasks.txt",
+      "prompt": "AsanaTasks"
+    },
+    {
+      "input": "./meetings/EngineeringCall/transcript.txt",
+      "output": "./meetings/tasks/SchemaChanges.txt",
+      "prompt": "Schema"
+    }
+  ],
+```
+
+
+Use the source block alongside `knowhow generate` to process a pipeline of files, applying prompts to generate documentation or other artifacts. This example demonstrates the configuration for processing meeting recordings, and then running prompts on the resulting transcripts.
+
+I've been using `CMD+SHIFT+5` to record meetings and then using the `sources` block to process the recordings into notes and tasks.
+
+When a .mov or audio file is an input, it will automatically create a transcript.txt in the same directory, so we can leverage that txt input in the next step. The pipeline is useful for apply one or more prompts to one or more input files.
+
+## knowhow embed: documentation embeddings
+```json
+  "embedSources": [
+    {
+      "input": "./documentation/docusaurus/docs/dev-docs/**/*.md",
+      "output": ".knowhow/embeddings/strapi-docs.json",
+      "chunkSize": 2000
+    }
+  ]
+```
+
+## knowhow embed: codebase embeddings
+```json
+
+  "embedSources": [
+    {
+      "input": "./packages/p2p/src/**/*.ts",
+      "output": ".knowhow/embeddings/p2p.json",
+      "chunkSize": 2000
+    },
+    {
+      "input": "./packages/openai_proxy/src/**/*.ts",
+      "output": ".knowhow/embeddings/openai_proxy.json",
+      "chunkSize": 2000
+    },
+    {
+      "input": "./packages/contracts/src/**/*.sol",
+      "output": ".knowhow/embeddings/contracts.json",
+      "chunkSize": 2000
+    }
+  ],
+```
+You can use `knowhow embed` to generate json embeddings from a set of files. If you want to run a prompt on the input, before embedding, you set an optional `prompt` field to match the filename of the prompt stored in your `.knowhow/prompts` directory.
+
+These embeddings are leveraged by the chat in `knowhow chat` or by the agents to accomplish their tasks
+
+## knowhow embed: asana tasks
+```
+  "embedSources": [
+    {
+      "input": "https://app.asana.com/0/111111111111111/list",
+      "output": ".knowhow/embeddings/asana.json",
+      "remote": "mybucket",
+      "remoteType": "s3",
+      "kind": "asana",
+      "chunkSize": 2000
+    }
+  ],
+```
+
+
+Any plugin that implements the embedding function, can generate embeddings if you set the `kind` field to the plugin name. The `remote` field is optional, and if set, the embeddings will be uploaded to the specified S3 bucket via `knowhow upload`.
+
+You can download remote embeddings via `knowhow download` and use them in your local environment.
+
+For downloads we support the following remote options:
+- s3
+- github* - downloads files
+
+For uploads we support the following remote options:
+- s3
+- github* - use git lfs to commit the embeddings to your repository
+
+
+## knowhow embed: github embeddings
+```
+  "embedSources": [
+    {
+      "input": "./src/**/*.ts",
+      "output": ".knowhow/embeddings/knowhow.json",
+      "chunkSize": 2000,
+      "remote": "micahriggan/knowhow",
+      "remoteType": "github"
+    }
+  ],
+```
+Here's an example where you could download embeddings from github. As the code updates occasionally I'll run `knowhow embed` and commit the updated embeddings.
+
+## Plugins
+```json
+{
+  "plugins": [
+    "embeddings",
+    "language",
+    "vim",
+    "github",
+    "asana",
+    "jira",
+    "linear",
+    "download",
+    "figma",
+    "notion"
+  ],
+}
+```
+Plugins are used to resolve urls or other references to data that the AI can use. For example, the `asana` plugin can be used to resolve tasks and projects from Asana, and the `figma` plugin can be used to resolve design files from Figma.
+
+The language plugin allows you to define hotkeys that resolve to larger blocks of texts or files, or urls. For example, you can define a language entry `#asana` that resolves to a link to your current asana task, so that you can easily load that without pasting the link each time.
+
+
+## knowhow chat:
+After generating embeddings, you can use `knowhow chat` to speak with a base agent, that does not have any tools, but does have plugins.
+
+You can use `agent` once you're in a chat to start talking to a Developer agent, that has a prompt and tools designed to help develop software.
+You can use `agents` to see a list of configured agents you can speak with.
+If you need multi-line input you can use `multi` to open a multi-line editor.
+
+There are many commands you can call from a chat session, try TAB to see a list of available options.
+
+
+### knowhow chat: custom agents
+```json
+  "assistants": [
+    {
+      "name": "linter",
+      "description": "Clean up your code",
+      "instructions": "Read the files that are loaded via vim plugin and provide debugging and linter suggestions"
+    }
+  ]
+```
+You can configure new agents via the config above. This would create a new option when you use `agents` in a chat session.
+
+## Lint Commands
+
+```json
+{
+  "lintCommands": {
+    "js": "eslint",
+    "ts": "tslint $1 && tsc -p tsconfig.json"
+  }
+}
+```
+lintCommands are automatically run after an agent successfully patches a file, if the file extension matches. So you can configure the AI to receive feedback from tsc and the linter after modifying a TypeScript file. $1 will be replaced with the filepath that was patched.
+
+
+
+## Language Plugin
+
+Configured in `.knowhow/language.json`
+
+Knowhow has a language file, which allows you to define additional context that is included when you use certain phrases in your messages. For instance, you could load a Pull Request when you say "My PR", or load a file from the codebase when you reference it. Languages have "terms" which then load various sources into context. Sources currently can be files, text, or github links.
+
+I primarily use this to define terms like "frontend" or "backend" to load prompt files
+
+## Example Language Definition
+
+```
+{
+  "ui,frontend,component,page": {
+    "sources": [
+      {
+        "data": [
+          "./.knowhow/prompts/frontend.mdx"
+        ],
+        "kind": "file"
+      }
+    ]
+  }
+}
+```
+
+
+Here's and example of having a language term that triggers a plugin.
+This example would load the diff of a PR every time we said "my pr"
+```
+{
+  "knowhow config": {
+    "sources": [
+      {
+        "kind": "file",
+        "data": [
+          ".knowhow/knowhow.json"
+        ]
+      }
+    ]
+  },
+  "My PR": {
+    "sources": [
+      {
+        "kind": "github",
+        "data": [
+          "https://github.com/tyvm-ai/knowhow/pulls/1"
+        ]
+      }
+    ]
+  }
+}
+```

package/README.md

@@ -0,0 +1,49 @@
+# Knowhow
+
+Knowhow is a powerful tool for managing and utilizing project-specific knowledge and configurations.
+
+## Features
+
+- Global configuration storage
+- Template management
+- Plugin system
+- Embedding generation
+- Custom agents
+
+## Global Configuration
+
+Knowhow now supports a global configuration directory located at `~/.knowhow`. This directory stores global templates and configurations that can be shared across multiple projects.
+
+### Global Config Structure
+
+```
+~/.knowhow/
+├── knowhow.json
+└── prompts/
+    └── [template files]
+```
+
+- `knowhow.json`: Global configuration file
+- `prompts/`: Directory containing global template files
+
+### Initialization Process
+
+When initializing a new Knowhow project:
+
+1. The global config directory is created if it doesn't exist.
+2. Global templates are checked first and copied to the local project.
+3. If a template doesn't exist globally, it's created both globally and locally.
+
+This ensures consistency across projects while allowing for project-specific customizations.
+
+## Usage
+
+[Add usage instructions here]
+
+## Contributing
+
+[Add contribution guidelines here]
+
+## License
+
+[Add license information here]

package/autodoc/chat.mdx

@@ -0,0 +1,20 @@
+This command line tool allows users to provide various inputs to interact with an AI assistant. Here are the key inputs a user can provide:
+
+1. **Commands**: Users can input specific commands to control the tool's behavior. The available commands include:
+   - `agent`: Toggle the use of a specific agent.
+   - `agents`: List and select from available agents.
+   - `debug`: Toggle debug mode.
+   - `multi`: Enable or disable multi-line input.
+   - `search`: Search for embeddings.
+   - `clear`: Clear the chat history.
+   - `exit`: Exit the tool.
+
+2. **Questions or Prompts**: Users can ask questions or provide prompts to the AI assistant. The tool will process these inputs and generate responses based on the provided embeddings and context.
+
+3. **Embedding Options**: When searching for embeddings, users can input options like `next`, `exit`, `embeddings`, and `use` to navigate through and select embeddings.
+
+4. **Agent Selection**: Users can select different agents to handle their queries by listing available agents and choosing one.
+
+5. **Chat History**: Users can view and clear the chat history to manage the context of their interactions.
+
+These inputs allow users to interact with the AI assistant in a flexible and controlled manner, leveraging various commands and options to get the desired responses.

package/autodoc/cli.mdx

@@ -0,0 +1,11 @@
+This command line tool accepts the following inputs (commands):
+
+1. **init**: Initializes the tool.
+2. **generate**: Generates some output.
+3. **embed**: Embeds data.
+4. **upload**: Uploads data.
+5. **download**: Downloads data.
+6. **upload:openai**: Uploads data to OpenAI.
+7. **chat**: Starts a chat session.
+
+If an unknown command is provided, it will display a message listing the valid commands.

package/autodoc/plugins/asana.mdx

@@ -0,0 +1,47 @@
+# Asana Plugin Executive Summary
+
+## Overview
+
+The Asana Plugin is designed to enhance chat interactions by integrating Asana task and project data. It detects and resolves Asana task and project URLs within user prompts, retrieves relevant data, and embeds this information into the chat context. This plugin is particularly useful for teams and individuals who rely on Asana for project management and want to seamlessly incorporate task details into their communication workflows.
+
+## Key Features
+
+### Input Detection and Resolution
+
+- **Task URLs**: The plugin identifies Asana task URLs within user prompts. It extracts these URLs and fetches the corresponding task details from Asana.
+- **Project URLs**: Similarly, the plugin detects Asana project URLs and retrieves all tasks associated with the specified projects.
+
+### Contextual Enhancements
+
+- **Task Details**: For each detected task, the plugin generates a detailed string that includes the task name, description, URL, and completion status. This information is then embedded into the chat, providing users with immediate access to task specifics.
+- **Project Tasks**: When project URLs are detected, the plugin fetches and embeds details for all tasks within the project, offering a comprehensive view of project progress and individual task statuses.
+
+### Embeddings
+
+- **Minimal Embeddings**: The plugin creates minimal embeddings for each task, which include:
+  - **ID**: The unique URL of the task.
+  - **Text**: A formatted string containing task details.
+  - **Metadata**: JSON-encoded task data for additional context.
+
+These embeddings can be used to enrich chat interactions, making it easier for users to reference and discuss specific tasks and projects.
+
+## Environment Variables
+
+To function correctly, the Asana Plugin requires the following environment variable to be set:
+
+- **ASANA_TOKEN**: This token is used to authenticate API requests to Asana. It should be set to a valid Asana access token.
+
+Example:
+```bash
+export ASANA_TOKEN=your_asana_access_token
+```
+
+## Usage
+
+The Asana Plugin can be integrated into chat applications to provide real-time access to Asana task and project data. Users can simply include Asana task or project URLs in their messages, and the plugin will automatically fetch and embed the relevant details. This functionality is particularly useful for:
+
+- **Project Management**: Keeping team members informed about task statuses and project progress.
+- **Collaboration**: Facilitating discussions around specific tasks and projects by providing immediate access to detailed information.
+- **Productivity**: Reducing the need to switch between chat applications and Asana, thereby streamlining workflows.
+
+By leveraging the Asana Plugin, teams can enhance their communication efficiency and ensure that everyone stays on the same page with up-to-date task and project information.

package/autodoc/plugins/downloader/downloader.mdx

@@ -0,0 +1,38 @@
+# Executive Summary: Downloader Plugin
+
+## Overview
+
+The Downloader Plugin is a comprehensive tool designed to facilitate the downloading, chunking, and transcription of audio and video content from various sources. This plugin is particularly useful for users who need to process large media files, extract meaningful information, and integrate this data into chat applications or other systems.
+
+## Key Features
+
+### Input Detection and Resolution
+
+- **URL Input**: The plugin accepts URLs as input, specifically targeting media content from platforms like YouTube.
+- **File Existence Check**: Before downloading, the plugin checks if the file already exists in the specified output directory to avoid redundant downloads.
+- **Media Information Retrieval**: It fetches detailed information about the media content, such as file ID and extension, ensuring accurate processing.
+
+### Contextual Enhancements
+
+- **Chunking Media Files**: The plugin can split large media files into smaller, manageable chunks. This is particularly useful for lengthy audio or video files, making them easier to process and transcribe.
+- **Transcription of Chunks**: Each chunk of the media file is transcribed using OpenAI's Whisper model. This transcription can then be used to add context to chats, making the content searchable and more interactive.
+
+### Embeddings Generation
+
+- **Text Embeddings**: The transcribed text from media chunks can be used to generate embeddings. These embeddings can be integrated into chat applications to enhance search capabilities, provide summaries, or enable other advanced text processing features.
+
+## Environment Variables
+
+To ensure the plugin functions correctly, the following environment variables must be set:
+
+- **OPENAI_API_KEY**: This variable is required to authenticate requests to the OpenAI API for transcription services.
+
+## Usage Scenarios
+
+- **Content Creators**: Download and transcribe video or audio content to create searchable text archives.
+- **Developers**: Integrate media content into chat applications, enhancing user interaction with contextual information.
+- **Researchers**: Process large volumes of media data, extracting and analyzing transcriptions for insights.
+
+## Conclusion
+
+The Downloader Plugin is a versatile tool that simplifies the process of downloading, chunking, and transcribing media content. By leveraging advanced transcription services and providing detailed media information, it adds significant value to chat applications and other systems requiring rich, contextual data. Ensure the necessary environment variables are set to fully utilize the plugin's capabilities.

package/autodoc/plugins/downloader/plugin.mdx

@@ -0,0 +1,37 @@
+# Downloader Plugin Executive Summary
+
+## Overview
+
+The Downloader Plugin is designed to enhance chat interactions by detecting and resolving URLs within user inputs. It provides additional context to chats by downloading and converting the content of the URLs into text. Furthermore, it generates embeddings from the downloaded content, which can be used for various purposes such as search, recommendation, and analysis.
+
+## Key Features
+
+### URL Detection and Resolution
+- **Input Detection**: The plugin scans user inputs for URLs using a regular expression.
+- **Content Download**: It attempts to download the content from the detected URLs, excluding certain file types (e.g., images like jpg, jpeg, png, gif).
+- **Text Conversion**: The downloaded content is converted into text, which can then be used to provide additional context in chats.
+
+### Contextual Enhancements
+- **Transcript Generation**: The plugin generates a transcript of the downloaded content, which can be appended to the chat to provide more information and context.
+- **Error Handling**: If a URL cannot be downloaded or processed, the plugin logs the error and continues processing other URLs.
+
+### Embedding Generation
+- **Minimal Embeddings**: The plugin generates minimal embeddings from the text content of the downloaded URLs. These embeddings include:
+  - **ID**: The URL of the content.
+  - **Text**: The converted text from the downloaded content.
+  - **Metadata**: Additional metadata (currently empty but can be extended).
+
+## Environment Variables
+
+The following environment variables are required for the plugin to function correctly:
+
+- **`process.env.DOWNLOAD_DIR`**: Specifies the directory where the downloaded files will be stored. Default is set to `.knowhow/downloads/`.
+
+## Usage
+
+The Downloader Plugin can be used in various scenarios where additional context from external URLs is beneficial. For example:
+- **Customer Support**: Automatically fetch and display relevant documentation or resources linked in customer queries.
+- **Content Aggregation**: Collect and summarize information from multiple sources for research or reporting purposes.
+- **Knowledge Management**: Enhance internal knowledge bases by embedding content from external URLs.
+
+By integrating the Downloader Plugin, users can significantly enrich their chat interactions with relevant and contextual information derived from external sources.

package/autodoc/plugins/downloader/types.mdx

@@ -0,0 +1,42 @@
+# Executive Summary: Audio and Video Download Plugin
+
+## Overview
+
+This plugin is designed to facilitate the detection, resolution, and contextual enhancement of audio and video downloads. It provides a comprehensive set of features that allow users to manage and process multimedia content efficiently. The plugin is particularly useful for applications that require detailed metadata extraction, format resolution, and embedding generation for audio and video files.
+
+## Key Features
+
+### Input Detection and Resolution
+
+- **Audio and Video Formats**: The plugin can detect various audio and video formats, including their extensions, resolutions, bitrates, and codecs.
+- **Download Information**: It provides detailed information about the downloadable content, such as title, description, thumbnail, age limit, and available formats.
+- **Protocol and Headers**: The plugin supports different protocols and can handle HTTP headers required for downloading content.
+
+### Contextual Enhancements
+
+- **Metadata Extraction**: Extracts comprehensive metadata from audio and video files, including subtitles, extractor details, and webpage URLs.
+- **Subtitles and Playlists**: Supports the extraction and management of subtitles and playlist information.
+- **DRM Detection**: Identifies whether the content has Digital Rights Management (DRM) protection.
+
+### Embedding Generation
+
+- **Audio and Video Embeddings**: Generates embeddings for audio and video content, which can be used for various applications such as content recommendation, search optimization, and more.
+- **Format-Specific Embeddings**: Provides embeddings based on specific formats, ensuring that the content is optimized for different use cases.
+
+## Environment Variables
+
+To ensure the plugin functions correctly, certain environment variables need to be set. These variables are crucial for configuring the plugin and enabling its features. Below are the required environment variables:
+
+- **`process.env.API_KEY`**: The API key required for accessing certain functionalities of the plugin.
+- **`process.env.DOWNLOAD_PATH`**: The path where downloaded content will be stored.
+- **`process.env.LOG_LEVEL`**: The level of logging detail (e.g., `info`, `debug`, `error`).
+
+## Usage
+
+This plugin can be integrated into applications that require robust audio and video processing capabilities. It is particularly useful for:
+
+- **Content Management Systems (CMS)**: Enhancing multimedia content with detailed metadata and embeddings.
+- **Media Players**: Providing rich context and format resolution for better playback experience.
+- **Data Analysis Tools**: Extracting and analyzing multimedia content for insights and trends.
+
+By leveraging this plugin, users can significantly improve the efficiency and effectiveness of their multimedia content management and processing workflows.

package/autodoc/plugins/embedding.mdx

@@ -0,0 +1,41 @@
+# Executive Summary: Embedding Plugin
+
+## Overview
+
+The Embedding Plugin is designed to enhance chat interactions by detecting and resolving user prompts, adding relevant context to conversations, and generating sophisticated embeddings. This plugin is particularly useful for applications that require contextual understanding and retrieval of information from a knowledge base.
+
+## Key Features
+
+### Input Detection and Resolution
+
+- **User Prompt Handling**: The plugin is capable of detecting user prompts and processing them to retrieve relevant information.
+- **Contextual Responses**: By analyzing the user input, the plugin can provide contextually appropriate responses, enhancing the overall user experience.
+
+### Context Addition
+
+- **Knowledge Base Integration**: The plugin integrates with a pre-configured knowledge base to fetch relevant information.
+- **Contextual Embedding**: It adds extra context to chats by retrieving and presenting the most relevant pieces of information from the knowledge base.
+- **Response Formatting**: The plugin formats the retrieved information into a coherent response that can be easily understood by the user.
+
+### Embedding Generation
+
+- **Configured Embeddings**: The plugin utilizes pre-configured embeddings to process and understand user prompts.
+- **Query Embedding**: It queries the embeddings to find the most relevant information that matches the user prompt.
+- **Result Presentation**: The plugin presents the top results, excluding the vector data, to provide a clean and concise response.
+
+## Environment Variables
+
+To ensure the proper functioning of the Embedding Plugin, certain environment variables need to be set. These variables are crucial for configuring the embeddings and querying the knowledge base. The required environment variables include:
+
+- **EMBEDDING_API_KEY**: This variable is used to authenticate and access the embedding service.
+- **KNOWLEDGE_BASE_URL**: This variable specifies the URL of the knowledge base that the plugin will query for information.
+
+## Usage
+
+The Embedding Plugin can be used in various applications where contextual understanding and information retrieval are essential. It is particularly useful in:
+
+- **Customer Support**: Enhancing automated responses by providing relevant information from a knowledge base.
+- **Virtual Assistants**: Improving the accuracy and relevance of responses by adding contextual information.
+- **Educational Tools**: Assisting in providing detailed and contextually appropriate answers to user queries.
+
+By integrating the Embedding Plugin, applications can significantly improve their interaction quality, providing users with more accurate and contextually relevant information.