newo 1.2.2 → 1.3.0

package/CHANGELOG.md

@@ -0,0 +1,79 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.3.0] - 2025-01-21
+
+### Added
+- **AKB Import Feature**: New `import-akb` command to import knowledge base articles from structured text files
+  - Parse multi-article files with standardized format (separated by `---`)
+  - Extract article metadata: ID, category, summary, keywords, and pricing data
+  - Import articles to NEWO personas via `/api/v1/akb/append-manual` endpoint
+  - Support for verbose logging with `--verbose` flag
+  - Progress tracking with success/failure counts
+- **Enhanced CLI**: Added `import-akb <file> <persona_id>` command with validation and error handling
+- **New API Endpoint**: `importAkbArticle()` function for AKB article imports
+- **Documentation**: Comprehensive AKB format documentation in README.md
+
+### Changed
+- Updated help text to include new `import-akb` command
+- Enhanced CLI command parsing to handle AKB import arguments
+- Updated project documentation with AKB import workflow and examples
+
+### Technical Details
+- **New Files**: 
+  - `src/akb.js`: AKB file parser and article formatter
+  - Article parsing supports category/subcategory structure with pricing data
+- **API Integration**: 
+  - Articles mapped with `topic_name` (descriptive title) and `source` (article ID)
+  - Full category content stored in `topic_summary`
+  - Structured metadata in `topic_facts` array
+- **Error Handling**: Comprehensive validation for file paths, persona IDs, and API responses
+
+### Example Usage
+```bash
+# Import AKB articles from file to specific persona
+npx newo import-akb akb.txt da4550db-2b95-4500-91ff-fb4b60fe7be9
+
+# With verbose logging
+npx newo import-akb akb.txt da4550db-2b95-4500-91ff-fb4b60fe7be9 --verbose
+```
+
+### AKB File Format
+```
+---
+# r001
+## Category / Subcategory / Description
+## Summary description of the category
+## Keywords; separated; by; semicolons
+
+<Category type="Category Name">
+Item Name: $Price [Modifiers: modifier1, modifier2]
+Another Item: $Price [Modifiers: modifier3]
+</Category>
+---
+```
+
+## [1.2.2] - 2025-01-20
+
+### Changed
+- Updated README with API key image
+- Removed unused files and .DS_Store entries
+- Package version bump
+
+### Fixed
+- Repository cleanup and organization
+
+## [1.2.1] - Previous Release
+
+### Added
+- Initial NEWO CLI functionality
+- Two-way sync between NEWO platform and local files
+- Support for .guidance and .jinja file types
+- SHA256-based change detection
+- Project structure export to flows.yaml
+- GitHub Actions CI/CD integration
+- Robust authentication with token refresh

package/README.md

@@ -44,6 +44,8 @@ npm install
 4. **Create** a new **Connector** for this Integration
 5. **Copy** your API key (it will look like: `458663bd41f2d1...`)
 
+![How to get your NEWO API Key](assets/newo-api-key.png)
+
 ### Step 2: Setup Environment
 ```bash
 cp .env.example .env
@@ -62,9 +64,11 @@ Optional (advanced):
 
 ## Commands
 ```bash
-npx newo pull     # download project -> ./project
-npx newo status   # list modified files
-npx newo push     # upload modified *.guidance/*.jinja back to NEWO
+npx newo pull                              # download project -> ./project
+npx newo status                            # list modified files
+npx newo push                              # upload modified *.guidance/*.jinja back to NEWO
+npx newo import-akb <file> <persona_id>    # import AKB articles from file
+npx newo meta                              # get project metadata (debug)
 ```
 
 Files are stored as:
@@ -78,6 +82,7 @@ Project structure is also exported to `flows.yaml` for reference.
 - **Two-way sync**: Pull NEWO projects to local files, push local changes back
 - **Change detection**: SHA256 hashing prevents unnecessary uploads
 - **Multiple file types**: `.guidance` (AI prompts) and `.jinja` (NSL templates)
+- **AKB import**: Import knowledge base articles from structured text files
 - **Project structure export**: Generates `flows.yaml` with complete project metadata
 - **Robust authentication**: API key exchange with automatic token refresh
 - **CI/CD ready**: GitHub Actions workflow included
@@ -110,6 +115,39 @@ jobs:
           # NEWO_REFRESH_URL: ${{ secrets.NEWO_REFRESH_URL }}
 ```
 
+## AKB Import
+
+Import knowledge base articles from structured text files into NEWO personas:
+
+```bash
+npx newo import-akb akb.txt da4550db-2b95-4500-91ff-fb4b60fe7be9
+```
+
+### AKB File Format
+```
+---
+# r001
+## Category / Subcategory / Description
+## Summary description of the category
+## Keywords; separated; by; semicolons
+
+<Category type="Category Name">
+Item Name: $Price [Modifiers: modifier1, modifier2]
+Another Item: $Price [Modifiers: modifier3]
+</Category>
+---
+```
+
+Each article will be imported with:
+- **topic_name**: The descriptive category title
+- **source**: The article ID (e.g., "r001") 
+- **topic_summary**: The full category content with pricing
+- **topic_facts**: Array containing category, summary, and keywords
+- **confidence**: 100
+- **labels**: ["rag_context"]
+
+Use `--verbose` flag to see detailed import progress.
+
 ## API Endpoints
 - `GET /api/v1/bff/agents/list?project_id=...` - List project agents
 - `GET /api/v1/designer/flows/{flowId}/skills` - List skills in flow
@@ -117,4 +155,5 @@ jobs:
 - `PUT /api/v1/designer/flows/skills/{skillId}` - Update skill content
 - `GET /api/v1/designer/flows/{flowId}/events` - List flow events (for flows.yaml)
 - `GET /api/v1/designer/flows/{flowId}/states` - List flow states (for flows.yaml)
-- `POST /api/v1/auth/api-key/token` - Exchange API key for access tokens
+- `POST /api/v1/auth/api-key/token` - Exchange API key for access tokens
+- `POST /api/v1/akb/append-manual` - Import AKB articles

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "newo",
-  "version": "1.2.2",
-  "description": "NEWO CLI: sync flows/skills between NEWO and local files",
+  "version": "1.3.0",
+  "description": "NEWO CLI: sync flows/skills between NEWO and local files, import AKB articles",
   "type": "module",
   "bin": {
     "newo": "src/cli.js"
@@ -9,6 +9,7 @@
   "files": [
     "src/**/*.js",
     "README.md",
+    "CHANGELOG.md",
     ".env.example"
   ],
   "keywords": [
@@ -18,7 +19,10 @@
     "agent",
     "automation",
     "sync",
-    "local-development"
+    "local-development",
+    "akb",
+    "knowledge-base",
+    "import"
   ],
   "author": "sabbah13",
   "license": "MIT",

package/src/akb.js

@@ -0,0 +1,112 @@
+import fs from 'fs-extra';
+
+/**
+ * Parse AKB file and extract articles
+ * @param {string} filePath - Path to AKB file
+ * @returns {Array} Array of parsed articles
+ */
+function parseAkbFile(filePath) {
+  const content = fs.readFileSync(filePath, 'utf8');
+  const articles = [];
+  
+  // Split by article separators (---)
+  const sections = content.split(/^---\s*$/gm).filter(section => section.trim());
+  
+  for (const section of sections) {
+    const lines = section.split('\n').filter(line => line.trim());
+    if (lines.length === 0) continue;
+    
+    const article = parseArticleSection(lines);
+    if (article) {
+      articles.push(article);
+    }
+  }
+  
+  return articles;
+}
+
+/**
+ * Parse individual article section
+ * @param {Array} lines - Lines of the article section
+ * @returns {Object|null} Parsed article object
+ */
+function parseArticleSection(lines) {
+  let topicName = '';
+  let category = '';
+  let summary = '';
+  let keywords = '';
+  let topicSummary = '';
+  
+  // Find topic name (# r001)
+  const topicLine = lines.find(line => line.match(/^#\s+r\d+/));
+  if (!topicLine) return null;
+  
+  topicName = topicLine.replace(/^#\s+/, '').trim();
+  
+  // Extract category/subcategory/description (first ## line)
+  const categoryLine = lines.find(line => line.startsWith('## ') && line.includes(' / '));
+  if (categoryLine) {
+    category = categoryLine.replace(/^##\s+/, '').trim();
+  }
+  
+  // Extract summary (second ## line)
+  const summaryLineIndex = lines.findIndex(line => line.startsWith('## ') && line.includes(' / '));
+  if (summaryLineIndex >= 0 && summaryLineIndex + 1 < lines.length) {
+    const nextLine = lines[summaryLineIndex + 1];
+    if (nextLine.startsWith('## ') && !nextLine.includes(' / ')) {
+      summary = nextLine.replace(/^##\s+/, '').trim();
+    }
+  }
+  
+  // Extract keywords (third ## line)
+  const keywordsLineIndex = lines.findIndex((line, index) => 
+    index > summaryLineIndex + 1 && line.startsWith('## ') && !line.includes(' / ')
+  );
+  if (keywordsLineIndex >= 0) {
+    keywords = lines[keywordsLineIndex].replace(/^##\s+/, '').trim();
+  }
+  
+  // Extract category content
+  const categoryStartIndex = lines.findIndex(line => line.includes('<Category type='));
+  const categoryEndIndex = lines.findIndex(line => line.includes('</Category>'));
+  
+  if (categoryStartIndex >= 0 && categoryEndIndex >= 0) {
+    const categoryLines = lines.slice(categoryStartIndex, categoryEndIndex + 1);
+    topicSummary = categoryLines.join('\n');
+  }
+  
+  // Create topic_facts array
+  const topicFacts = [
+    category,
+    summary,
+    keywords
+  ].filter(fact => fact.trim() !== '');
+  
+  return {
+    topic_name: category, // Use the descriptive title as topic_name
+    persona_id: null, // Will be set when importing
+    topic_summary: topicSummary,
+    topic_facts: topicFacts,
+    confidence: 100,
+    source: topicName, // Use the ID (r001) as source
+    labels: ["rag_context"]
+  };
+}
+
+/**
+ * Convert parsed articles to API format for bulk import
+ * @param {Array} articles - Parsed articles
+ * @param {string} personaId - Target persona ID
+ * @returns {Array} Array of articles ready for API import
+ */
+function prepareArticlesForImport(articles, personaId) {
+  return articles.map(article => ({
+    ...article,
+    persona_id: personaId
+  }));
+}
+
+export {
+  parseAkbFile,
+  prepareArticlesForImport
+};

package/src/api.js

@@ -90,4 +90,9 @@ export async function listFlowEvents(client, flowId) {
 export async function listFlowStates(client, flowId) {
   const r = await client.get(`/api/v1/designer/flows/${flowId}/states`);
   return r.data;
+}
+
+export async function importAkbArticle(client, articleData) {
+  const r = await client.post('/api/v1/akb/append-manual', articleData);
+  return r.data;
 }

package/src/cli.js

@@ -1,8 +1,10 @@
 #!/usr/bin/env node
 import minimist from 'minimist';
 import dotenv from 'dotenv';
-import { makeClient, getProjectMeta } from './api.js';
+import { makeClient, getProjectMeta, importAkbArticle } from './api.js';
 import { pullAll, pushChanged, status } from './sync.js';
+import { parseAkbFile, prepareArticlesForImport } from './akb.js';
+import path from 'path';
 
 dotenv.config();
 const { NEWO_PROJECT_ID } = process.env;
@@ -15,13 +17,14 @@ async function main() {
   if (!cmd || ['help', '-h', '--help'].includes(cmd)) {
     console.log(`NEWO CLI
 Usage:
-  newo pull           # download project -> ./project
-  newo push           # upload modified *.guidance/*.jinja back to NEWO
-  newo status         # show modified files
-  newo meta           # get project metadata (debug)
+  newo pull                    # download project -> ./project
+  newo push                    # upload modified *.guidance/*.jinja back to NEWO
+  newo status                  # show modified files
+  newo meta                    # get project metadata (debug)
+  newo import-akb <file> <persona_id>  # import AKB articles from file
   
 Flags:
-  --verbose, -v       # enable detailed logging
+  --verbose, -v                # enable detailed logging
   
 Env:
   NEWO_BASE_URL, NEWO_PROJECT_ID, NEWO_API_KEY, NEWO_REFRESH_URL (optional)
@@ -42,6 +45,50 @@ Env:
     if (!NEWO_PROJECT_ID) throw new Error('NEWO_PROJECT_ID is not set in env');
     const meta = await getProjectMeta(client, NEWO_PROJECT_ID);
     console.log(JSON.stringify(meta, null, 2));
+  } else if (cmd === 'import-akb') {
+    const akbFile = args._[1];
+    const personaId = args._[2];
+    
+    if (!akbFile || !personaId) {
+      console.error('Usage: newo import-akb <file> <persona_id>');
+      console.error('Example: newo import-akb akb.txt da4550db-2b95-4500-91ff-fb4b60fe7be9');
+      process.exit(1);
+    }
+    
+    const filePath = path.resolve(akbFile);
+    
+    try {
+      if (verbose) console.log(`📖 Parsing AKB file: ${filePath}`);
+      const articles = parseAkbFile(filePath);
+      console.log(`✓ Parsed ${articles.length} articles from ${akbFile}`);
+      
+      if (verbose) console.log(`🔧 Preparing articles for persona: ${personaId}`);
+      const preparedArticles = prepareArticlesForImport(articles, personaId);
+      
+      let successCount = 0;
+      let errorCount = 0;
+      
+      console.log(`📤 Importing ${preparedArticles.length} articles...`);
+      
+      for (const [index, article] of preparedArticles.entries()) {
+        try {
+          if (verbose) console.log(`  [${index + 1}/${preparedArticles.length}] Importing ${article.topic_name}...`);
+          await importAkbArticle(client, article);
+          successCount++;
+          if (!verbose) process.stdout.write('.');
+        } catch (error) {
+          errorCount++;
+          console.error(`\n❌ Failed to import ${article.topic_name}:`, error?.response?.data || error.message);
+        }
+      }
+      
+      if (!verbose) console.log(''); // new line after dots
+      console.log(`✅ Import complete: ${successCount} successful, ${errorCount} failed`);
+      
+    } catch (error) {
+      console.error('❌ AKB import failed:', error.message);
+      process.exit(1);
+    }
   } else {
     console.error('Unknown command:', cmd);
     process.exit(1);