spine-framework-cortex 0.2.21 → 0.2.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE.md DELETED
@@ -1,193 +0,0 @@
1
- Spine Framework Internal Use License 1.0.0
2
- Source-available. Free for internal use. Commercial rights reserved.
3
-
4
- This license is based on the structure and intent of the PolyForm Internal Use License 1.0.0, but it has been modified for Spine Framework. This is not the unmodified PolyForm Internal Use License.
5
-
6
- Required Notice
7
- Required Notice: Copyright © 2026 Dahl Ventures Inc. All rights reserved.
8
-
9
- 1. Acceptance
10
- In order to receive any license under these terms, you must agree to these terms as both strict obligations and conditions to all licenses granted to you.
11
-
12
- If you do not agree to these terms, you do not have permission to use, copy, modify, run, install, access, or create works based on Spine Framework.
13
-
14
- 2. Definitions
15
- Software means Spine Framework, including its source code, object code, packages, modules, templates, schemas, migrations, configuration files, documentation, examples, command-line tools, SDKs, application packs, and related materials provided by the licensor.
16
-
17
- Licensor means the legal owner of Spine Framework.
18
-
19
- You means the individual or legal entity receiving the Software.
20
-
21
- Your Company means the legal entity on whose behalf you use the Software, including that entity's employees and authorized contractors.
22
-
23
- Internal Business Use means use of the Software solely inside Your Company for Your Company's own internal business operations, internal workflows, internal applications, internal data, internal users, and internal customers or accounts.
24
-
25
- Third Party means any person or legal entity other than You or Your Company.
26
-
27
- Commercial License means a separate written agreement signed by the Licensor that grants rights beyond this license.
28
-
29
- 3. Copyright License
30
- The Licensor grants You a copyright license to do everything with the Software that would otherwise infringe the Licensor's copyright, but only for Internal Business Use and only as permitted by these terms.
31
-
32
- You may install, run, access, evaluate, test, configure, and use the Software for Internal Business Use.
33
-
34
- You may copy the Software only as reasonably necessary for Internal Business Use, including backup, deployment, development, testing, staging, disaster recovery, and internal security review.
35
-
36
- You may not distribute the Software except as expressly allowed in these terms.
37
-
38
- 4. Changes and New Works
39
- The Licensor grants You a copyright license to modify the Software and create new works based on the Software, but only for Internal Business Use.
40
-
41
- You may create internal applications, extensions, workflows, integrations, automations, schemas, roles, agents, triggers, pipelines, dashboards, reports, and other internal business systems using the Software.
42
-
43
- You may not distribute, sell, sublicense, transfer, lease, rent, host for others, white-label, resell, or commercially exploit any modified version, derivative work, extension, application pack, template, or implementation of the Software except under a Commercial License.
44
-
45
- 5. Internal Contractors
46
- Your Company may allow employees and contractors to access and use the Software solely on behalf of Your Company and solely for Your Company's Internal Business Use.
47
-
48
- Contractors do not receive any independent right to use the Software for themselves, their other clients, their own products, or any Third Party.
49
-
50
- Your Company is responsible for all use of the Software by its employees, contractors, agents, and representatives.
51
-
52
- 6. Your Data and Application-Specific Work
53
- As between You and the Licensor, You own Your Company's data, records, content, prompts, business logic, customer records, internal workflows, and application-specific code that You independently create using the Software.
54
-
55
- These terms do not give the Licensor ownership of Your Company's data or independently created application-specific business content.
56
-
57
- However, ownership of Your Company's data or application-specific work does not give You the right to distribute, resell, white-label, host, or commercially exploit the Software or any portion of the Software except as allowed by these terms or a Commercial License.
58
-
59
- 7. Prohibited Uses
60
- You may not do any of the following without a Commercial License:
61
-
62
- Sell, resell, sublicense, rent, lease, lend, transfer, assign, publish, or distribute the Software.
63
- Offer the Software, or any modified version of the Software, to Third Parties as software, infrastructure, a framework, a platform, a service, or a hosted product.
64
- Offer the Software as SaaS, PaaS, IaaS, framework-as-a-service, backend-as-a-service, workflow-as-a-service, automation-as-a-service, AI-agent platform, managed service, support service, or similar commercial service.
65
- White-label, private-label, rebrand, or remove Spine Framework identity from the Software in order to provide it to Third Parties.
66
- Embed, bundle, package, or include the Software in any product, service, appliance, marketplace offering, application pack, starter kit, template library, SDK, CLI, distribution, or managed implementation made available to Third Parties.
67
- Use the Software to operate, power, support, or provide a product or service for Third Parties where those Third Parties receive access to the Software or to functionality substantially provided by the Software.
68
- Use the Software to build, operate, or improve a competing framework, platform, marketplace, application generator, agentic software infrastructure product, or commercial development infrastructure product.
69
- Provide implementation, consulting, hosting, support, customization, training, migration, or managed services for the Software to Third Parties as a paid or commercial offering, except as expressly authorized by the Licensor.
70
- Remove, alter, obscure, or bypass copyright notices, license notices, attribution, trademarks, branding, license keys, license checks, usage limits, or technical protection measures included in the Software.
71
- Use the Software in violation of law, regulation, third-party rights, or these terms.
72
- 8. Examples of Permitted Use
73
- The following are permitted under this license, provided they are solely for Internal Business Use:
74
-
75
- A company installs Spine Framework to build its own internal CRM.
76
- A company uses Spine Framework to operate an internal customer support portal for its own support team and its own customers.
77
- A company modifies Spine Framework for its internal workflows, roles, permissions, automations, pipelines, and integrations.
78
- A company lets its employees and authorized contractors configure, maintain, and support its internal Spine Framework installation.
79
- A company creates internal dashboards, agents, knowledge bases, audit trails, and operational tools on top of Spine Framework.
80
- 9. Examples Requiring a Commercial License
81
- The following are not permitted under this license and require a Commercial License:
82
-
83
- Selling Spine Framework or a modified version of Spine Framework.
84
- Offering Spine Framework as a hosted SaaS product.
85
- Offering a white-labeled application built on Spine Framework to customers.
86
- Packaging Spine Framework into a commercial starter kit, template, marketplace app, agency offering, or developer platform.
87
- Running Spine Framework for multiple clients as a managed service provider, agency, consultant, systems integrator, or outsourcing firm.
88
- Embedding Spine Framework into a product sold to customers.
89
- Forking Spine Framework to launch a competing framework, backend, app platform, AI-agent platform, or development infrastructure product.
90
- Publishing modified Spine Framework source code to a public repository.
91
- Giving a customer, client, partner, reseller, or affiliate a copy of Spine Framework.
92
- 10. Distribution
93
- You may not distribute the Software, modified versions of the Software, or works based on the Software to any Third Party except as expressly permitted by these terms or a Commercial License.
94
-
95
- You may share copies of the Software with employees and contractors of Your Company only as necessary for Internal Business Use and only if they are bound by obligations consistent with these terms.
96
-
97
- Any copy of the Software must include this license and all Required Notices.
98
-
99
- 11. Trademarks
100
- This license does not grant You any right to use the Licensor's names, trademarks, service marks, logos, product names, domain names, or branding except as necessary to identify the Software for Internal Business Use.
101
-
102
- You may not use Spine Framework branding in a way that suggests sponsorship, endorsement, partnership, certification, resale authorization, or commercial authorization without written permission from the Licensor.
103
-
104
- 12. Patent License
105
- The Licensor grants You a patent license for the Software that covers patent claims the Licensor can license, or later becomes able to license, that You would infringe by using the Software as permitted by these terms.
106
-
107
- 13. Patent Defense
108
- If You make any written claim that the Software infringes or contributes to infringement of any patent, Your patent license for the Software ends immediately.
109
-
110
- If Your Company makes such a claim, Your Company's patent license ends immediately.
111
-
112
- 14. Fair Use and Other Legal Rights
113
- You may have fair use, fair dealing, or other rights under applicable law. These terms do not limit rights that the law does not allow these terms to limit.
114
-
115
- 15. No Other Rights
116
- These terms do not allow You to sublicense, transfer, assign, or delegate any license granted under these terms except as expressly allowed here.
117
-
118
- These terms do not prevent the Licensor from granting licenses to anyone else.
119
-
120
- These terms do not imply any license other than the licenses expressly granted here.
121
-
122
- All rights not expressly granted are reserved by the Licensor.
123
-
124
- 16. Violations and Cure
125
- The first time You are notified in writing that You have violated these terms, or done anything with the Software not covered by these terms, Your licenses can continue if You:
126
-
127
- come into full compliance with these terms;
128
- take practical steps to correct past violations; and
129
- complete both within 32 days of receiving written notice.
130
- If You do not satisfy those requirements, all licenses granted to You under these terms end immediately.
131
-
132
- For any later violation, all licenses granted to You under these terms end immediately without any cure period.
133
-
134
- 17. Termination
135
- When Your licenses end, You must stop all use of the Software and destroy all copies of the Software in Your possession or control, except copies you are legally required to retain for archival, compliance, or legal purposes.
136
-
137
- Sections concerning ownership, prohibited uses, trademarks, patent defense, no other rights, violations, termination, no warranty, limitation of liability, and general terms survive termination.
138
-
139
- 18. No Warranty
140
- As far as the law allows, the Software is provided as is and as available, without any warranty or condition, express or implied.
141
-
142
- The Licensor disclaims all warranties and conditions, including warranties of merchantability, fitness for a particular purpose, title, non-infringement, availability, accuracy, security, and quiet enjoyment.
143
-
144
- 19. Limitation of Liability
145
- As far as the law allows, the Licensor will not be liable to You for any indirect, incidental, special, consequential, exemplary, punitive, or other damages arising out of these terms or the Software.
146
-
147
- As far as the law allows, the Licensor will not be liable for lost profits, lost revenue, lost savings, lost data, business interruption, security incidents, procurement of substitute services, or loss of goodwill.
148
-
149
- The Licensor's total liability arising out of these terms or the Software will not exceed the amount You paid the Licensor for the Software during the 12 months before the event giving rise to liability, or USD $100 if You paid nothing.
150
-
151
- 20. Commercial Licensing
152
- Rights not granted under this license may be available under a separate Commercial License from the Licensor.
153
-
154
- A Commercial License is required for resale, redistribution, SaaS, managed services, white-label use, OEM use, marketplace distribution, partner use, commercial hosting, commercial support offerings, or any use outside Internal Business Use.
155
-
156
- To request commercial rights, contact:
157
-
158
- spine-framework.com
159
- webmaster@spine-framework.com
160
- 21. License Updates
161
- The Licensor may publish updated versions of this license from time to time.
162
-
163
- Each release, package, download, or copy of the Software is governed by the license terms included with that release, package, download, or copy, unless You and the Licensor have signed a separate written agreement.
164
-
165
- Updated license terms apply only to future releases, downloads, packages, updates, upgrades, patches, hosted access, or other versions of the Software made available after the updated terms are published.
166
-
167
- The Licensor may require acceptance of updated license terms as a condition of receiving or using future releases, updates, hosted services, commercial features, support, documentation, marketplace access, license keys, portals, application packs, or other Spine Framework services.
168
-
169
- Nothing in this section gives You rights to continue using future versions of the Software under earlier license terms.
170
-
171
- 22. General Terms
172
- If any part of these terms is held unenforceable, the rest remains in effect as far as the law allows.
173
-
174
- Failure by the Licensor to enforce any term is not a waiver.
175
-
176
- Any waiver must be in writing and signed by the Licensor.
177
-
178
- These terms are the entire license terms for the Software unless You and the Licensor have signed a separate written agreement.
179
-
180
- 23. Plain-English Summary
181
- This summary is not part of the legal license.
182
-
183
- You may use Spine Framework inside your own company for your own internal business operations.
184
-
185
- You may modify Spine Framework for your own internal use.
186
-
187
- You may let your employees and contractors work on your internal Spine Framework installation.
188
-
189
- You may not sell it, redistribute it, white-label it, host it for others, offer it as SaaS, package it into another product, use it as a managed service offering, or use it to build a competing framework without a separate commercial license.
190
-
191
- Future versions, updates, services, support, portals, packs, license keys, and commercial features may require acceptance of updated terms.
192
-
193
- Spine Framework is source-available. It is not open source.
package/README.md DELETED
@@ -1,46 +0,0 @@
1
- # Spine Framework Cortex
2
-
3
- Cortex is an internal operations app for [Spine Framework](https://spine-framework.com). It provides a unified workspace for CRM, Support, Community, and Knowledge Base management.
4
-
5
- ## Features
6
-
7
- - **CRM** — Accounts, contacts, deals, health tracking, activity feed
8
- - **Support** — Ticket management and triage
9
- - **Community** — Community moderation and engagement
10
- - **Knowledge Base** — Article management, editor, and AI ingestion
11
- - **Courses** — Course catalog and management
12
- - **Intelligence** — AI-powered analytics and insights
13
- - **Operations** — Audit funnels, install funnels, operational dashboards
14
-
15
- ## Requirements
16
-
17
- - [Spine Framework](https://www.npmjs.com/package/spine-framework) `>=0.1.0`
18
-
19
- ## Installation
20
-
21
- ```bash
22
- npx spine-framework install-app spine-framework-cortex
23
- ```
24
-
25
- This will:
26
- 1. Copy app files to `custom/apps/cortex/`
27
- 2. Run any pending migrations
28
- 3. Seed roles, types, and link-types into the database
29
- 4. Register the app in `spine.config.json`
30
-
31
- ## Configuration
32
-
33
- After installation, edit `custom/apps/cortex/manifest.json` to configure:
34
-
35
- - `route_prefix` — where the app is served (default: `/cortex`)
36
- - `registration` — self-registration behaviour and account assignment
37
-
38
- To apply manifest changes to the database without reinstalling:
39
-
40
- ```bash
41
- npx spine-framework update-db-app cortex
42
- ```
43
-
44
- ## License
45
-
46
- See [LICENSE.md](./LICENSE.md)
@@ -1,35 +0,0 @@
1
- /**
2
- * Cortex Webhook Handler
3
- *
4
- * Convention: custom_*.ts files are assembled into /functions/
5
- * and loaded by integration-routes via: import('./custom_cortex-handler')
6
- *
7
- * Receives: (sanitizedData, context, event)
8
- * Returns: plain text or object
9
- */
10
- export default async function cortexHandler(
11
- data: Record<string, any>,
12
- ctx: {
13
- integrationId: string
14
- accountId: string
15
- slug: string
16
- principal: { id: string; type: string; accountId: string }
17
- requestId: string
18
- headers: Record<string, string>
19
- },
20
- event: {
21
- httpMethod: string
22
- headers: Record<string, string>
23
- body: any
24
- path: string
25
- queryStringParameters: Record<string, string>
26
- }
27
- ): Promise<string> {
28
- console.log(`[${ctx.requestId}] Cortex handler received:`, {
29
- testText: data['test-text'],
30
- integrationId: ctx.integrationId,
31
- accountId: ctx.accountId
32
- })
33
-
34
- return data['test-text']
35
- }
@@ -1,364 +0,0 @@
1
- /**
2
- * Test script for the adaptive article chunker.
3
- *
4
- * Run: npx tsx custom/functions/custom_kb-chunker-test.ts
5
- */
6
- import { readFileSync } from 'fs'
7
- import { chunkArticle, estimateTokens, htmlToPlainText } from './custom_kb-chunker'
8
-
9
- // ---------------------------------------------------------------------------
10
- // Helpers
11
- // ---------------------------------------------------------------------------
12
-
13
- let passed = 0
14
- let failed = 0
15
-
16
- function assert(condition: boolean, message: string) {
17
- if (condition) {
18
- passed++
19
- console.log(` ✅ ${message}`)
20
- } else {
21
- failed++
22
- console.log(` ❌ ${message}`)
23
- }
24
- }
25
-
26
- function section(name: string) {
27
- console.log(`\n── ${name} ──`)
28
- }
29
-
30
- // ---------------------------------------------------------------------------
31
- // Test 1: Short article — single chunk, no splitting
32
- // ---------------------------------------------------------------------------
33
-
34
- section('Test 1: Short article (< 600 tokens)')
35
-
36
- const shortArticle = `This is a short support article about resetting passwords.
37
-
38
- If you forgot your password, click the "Forgot Password" link on the login page.
39
- You will receive an email with a reset link. The link expires in 24 hours.
40
-
41
- If you don't receive the email, check your spam folder or contact support.`
42
-
43
- const shortChunks = chunkArticle(shortArticle, {
44
- articleTitle: 'How to Reset Your Password',
45
- })
46
-
47
- assert(shortChunks.length === 1, `Single chunk produced (got ${shortChunks.length})`)
48
- assert(shortChunks[0].content.startsWith('How to Reset Your Password'), 'Prefixed with article title')
49
- assert(shortChunks[0].sectionPath === null, 'No section path for short article')
50
- assert(shortChunks[0].chunkIndex === 0, 'chunkIndex is 0')
51
- assert(shortChunks[0].chunkTotal === 1, 'chunkTotal is 1')
52
-
53
- // ---------------------------------------------------------------------------
54
- // Test 2: Structured markdown with headings — heading-based split
55
- // ---------------------------------------------------------------------------
56
-
57
- section('Test 2: Structured markdown with headings')
58
-
59
- const structuredArticle = `# Getting Started Guide
60
-
61
- Welcome to our platform. This guide will walk you through the complete process of setting up, configuring, and using the Spine SDK in your application. By the end, you will have a fully working integration.
62
-
63
- ## Installation
64
-
65
- Run the following command to install the SDK and its peer dependencies:
66
-
67
- \`\`\`bash
68
- npm install @spine/sdk @spine/auth @spine/utils
69
- \`\`\`
70
-
71
- Then configure your environment variables. Make sure you have Node.js 18 or later installed. The SDK uses native fetch and ES modules, so older Node versions are not supported.
72
-
73
- After installation, verify the package is available by running \`npx spine-check\`. This will confirm the SDK is correctly installed and your environment meets all requirements.
74
-
75
- ## Configuration
76
-
77
- Create a \`.env\` file in your project root with the following variables:
78
-
79
- \`\`\`
80
- SPINE_API_KEY=your-key-here
81
- SPINE_URL=https://api.spine.dev
82
- SPINE_ACCOUNT_ID=your-account-uuid
83
- \`\`\`
84
-
85
- ### Required Variables
86
-
87
- - \`SPINE_API_KEY\` — your API key from the admin dashboard. Navigate to Settings > API Keys to generate one. Each key is scoped to a specific account and set of permissions.
88
- - \`SPINE_URL\` — the API endpoint for your region. Use \`https://api.spine.dev\` for US, \`https://api.eu.spine.dev\` for EU.
89
- - \`SPINE_ACCOUNT_ID\` — your account UUID, found on the Settings page.
90
-
91
- ### Optional Variables
92
-
93
- - \`SPINE_TIMEOUT\` — request timeout in milliseconds (default: 30000). Increase this for large batch operations.
94
- - \`SPINE_RETRY\` — number of automatic retries for transient failures (default: 3). Set to 0 to disable.
95
- - \`SPINE_LOG_LEVEL\` — logging verbosity: \`debug\`, \`info\`, \`warn\`, \`error\` (default: \`info\`).
96
- - \`SPINE_PROXY\` — HTTP proxy URL for corporate network environments.
97
-
98
- ## Usage
99
-
100
- Import the SDK and create a client instance. The client handles authentication, retries, and connection pooling automatically:
101
-
102
- \`\`\`typescript
103
- import { SpineClient } from '@spine/sdk'
104
-
105
- const client = new SpineClient({
106
- apiKey: process.env.SPINE_API_KEY,
107
- url: process.env.SPINE_URL,
108
- accountId: process.env.SPINE_ACCOUNT_ID,
109
- })
110
-
111
- // List all accounts
112
- const accounts = await client.accounts.list()
113
-
114
- // Get a specific item
115
- const item = await client.items.get('uuid-here')
116
-
117
- // Create a new item
118
- const newItem = await client.items.create({
119
- title: 'My New Item',
120
- type_slug: 'task',
121
- data: { priority: 'high' },
122
- })
123
- \`\`\`
124
-
125
- The client is thread-safe and can be shared across your application. Create one instance at startup and reuse it.
126
-
127
- ## Error Handling
128
-
129
- The SDK throws typed errors for different failure modes. Always wrap API calls in try-catch blocks:
130
-
131
- \`\`\`typescript
132
- try {
133
- const result = await client.items.get('invalid-uuid')
134
- } catch (err) {
135
- if (err instanceof SpineAuthError) {
136
- console.error('Authentication failed:', err.message)
137
- } else if (err instanceof SpineNotFoundError) {
138
- console.error('Item not found')
139
- } else if (err instanceof SpineRateLimitError) {
140
- console.error('Rate limited, retry after:', err.retryAfter)
141
- }
142
- }
143
- \`\`\`
144
-
145
- ## Troubleshooting
146
-
147
- ### Error: Invalid API Key
148
-
149
- Make sure your API key is correct and has not expired. API keys can be rotated from the admin dashboard under Settings > API Keys. If you recently rotated your key, update your \`.env\` file with the new value.
150
-
151
- ### Error: Connection Timeout
152
-
153
- Check that your \`SPINE_URL\` is correct and the server is reachable. Common causes include firewall rules blocking outbound HTTPS traffic, incorrect proxy configuration, or DNS resolution failures. Try running \`curl -v https://api.spine.dev/health\` to verify connectivity.
154
-
155
- ### Error: Rate Limited
156
-
157
- The API enforces rate limits per API key. Default limits are 100 requests per second for read operations and 20 per second for write operations. If you need higher limits, contact support to discuss your use case.`
158
-
159
- const structuredChunks = chunkArticle(structuredArticle, {
160
- articleTitle: 'Getting Started Guide',
161
- })
162
-
163
- assert(structuredChunks.length > 1, `Multiple chunks produced (got ${structuredChunks.length})`)
164
- assert(structuredChunks.every(c => c.content.includes('Getting Started Guide')),
165
- 'All chunks prefixed with article title')
166
- assert(structuredChunks.some(c => c.sectionPath?.includes('Installation')),
167
- 'Has Installation section')
168
- assert(structuredChunks.some(c => c.sectionPath?.includes('Configuration')),
169
- 'Has Configuration section')
170
- assert(structuredChunks.some(c => c.sectionPath?.includes('Usage')),
171
- 'Has Usage section')
172
- assert(structuredChunks.some(c => c.sectionPath?.includes('Troubleshooting')),
173
- 'Has Troubleshooting section')
174
- assert(structuredChunks.some(c => c.sectionPath?.includes('Error Handling')),
175
- 'Has Error Handling section')
176
-
177
- console.log('\n Chunk breakdown:')
178
- structuredChunks.forEach((c, i) => {
179
- console.log(` [${i}] ~${estimateTokens(c.content)} tokens | section: ${c.sectionPath || '(preamble)'}`)
180
- })
181
-
182
- // ---------------------------------------------------------------------------
183
- // Test 3: Unstructured narrative — paragraph grouping
184
- // ---------------------------------------------------------------------------
185
-
186
- section('Test 3: Unstructured narrative (no headings)')
187
-
188
- const narrative = Array.from({ length: 20 }, (_, i) =>
189
- `Paragraph ${i + 1}: This is a block of narrative content that represents a success story or editorial piece. It describes how the customer implemented the solution and achieved measurable results in their business operations. The team worked closely with the vendor to ensure a smooth rollout across all departments.`
190
- ).join('\n\n')
191
-
192
- const narrativeChunks = chunkArticle(narrative, {
193
- articleTitle: 'Acme Corp Success Story',
194
- })
195
-
196
- assert(narrativeChunks.length > 1, `Multiple chunks produced (got ${narrativeChunks.length})`)
197
- assert(narrativeChunks.every(c => c.sectionPath === null), 'No section paths for unstructured content')
198
- assert(narrativeChunks.every(c => c.content.includes('Acme Corp Success Story')), 'All prefixed with title')
199
- assert(narrativeChunks.every(c => c.content.includes('chunk')), 'All have chunk N of M context')
200
-
201
- console.log('\n Chunk breakdown:')
202
- narrativeChunks.forEach((c, i) => {
203
- console.log(` [${i}] ~${estimateTokens(c.content)} tokens | ${c.chunkIndex + 1}/${c.chunkTotal}`)
204
- })
205
-
206
- // ---------------------------------------------------------------------------
207
- // Test 4: Code blocks stay atomic
208
- // ---------------------------------------------------------------------------
209
-
210
- section('Test 4: Code blocks are atomic')
211
-
212
- const codeArticle = `## Overview
213
-
214
- Short intro.
215
-
216
- ## The Code
217
-
218
- Here is a large code block:
219
-
220
- \`\`\`typescript
221
- ${Array.from({ length: 80 }, (_, i) => ` const line${i} = 'this is line ${i} of the code block'`).join('\n')}
222
- \`\`\`
223
-
224
- ## After the Code
225
-
226
- Some text after.`
227
-
228
- const codeChunks = chunkArticle(codeArticle, {
229
- articleTitle: 'Code Example Article',
230
- maxTokens: 400, // Force sub-splitting
231
- })
232
-
233
- // Verify no code block was split mid-way
234
- const codeBlockContent = codeChunks.map(c => c.content)
235
- const codeBlockChunk = codeBlockContent.find(c => c.includes("const line0 = 'this is line 0"))
236
- assert(!!codeBlockChunk, 'Found chunk containing start of code block')
237
- if (codeBlockChunk) {
238
- assert(codeBlockChunk.includes("const line79 = 'this is line 79"), 'Same chunk contains end of code block (atomic)')
239
- }
240
-
241
- console.log('\n Chunk breakdown:')
242
- codeChunks.forEach((c, i) => {
243
- console.log(` [${i}] ~${estimateTokens(c.content)} tokens | section: ${c.sectionPath || '(none)'}`)
244
- })
245
-
246
- // ---------------------------------------------------------------------------
247
- // Test 5: Tiny chunks get merged
248
- // ---------------------------------------------------------------------------
249
-
250
- section('Test 5: Tiny chunks get merged')
251
-
252
- const tinyArticle = `## Section A
253
-
254
- Tiny.
255
-
256
- ## Section B
257
-
258
- Also tiny.
259
-
260
- ## Section C
261
-
262
- This section has enough content to be meaningful. It contains several sentences that describe the topic in detail. The reader should come away with a clear understanding of the concepts presented here. Additional context is provided to ensure the chunk meets the minimum token threshold.`
263
-
264
- const tinyChunks = chunkArticle(tinyArticle, {
265
- articleTitle: 'Merge Test',
266
- minTokens: 100,
267
- })
268
-
269
- // Section A and B are tiny (~5 tokens each), should be merged
270
- assert(tinyChunks.length < 3, `Tiny sections merged (got ${tinyChunks.length} chunks, expected < 3)`)
271
-
272
- console.log('\n Chunk breakdown:')
273
- tinyChunks.forEach((c, i) => {
274
- console.log(` [${i}] ~${estimateTokens(c.content)} tokens | section: ${c.sectionPath || '(none)'}`)
275
- })
276
-
277
- // ---------------------------------------------------------------------------
278
- // Test 6: HTML content
279
- // ---------------------------------------------------------------------------
280
-
281
- section('Test 6: HTML content')
282
-
283
- const htmlArticle = `<h2>Introduction</h2>
284
- <p>This is an HTML article stored by the rich text editor.</p>
285
- <p>It contains multiple paragraphs of formatted content.</p>
286
-
287
- <h2>Details</h2>
288
- <p>Here are the details with <strong>bold</strong> and <em>italic</em> text.</p>
289
- <ul>
290
- <li>Item one</li>
291
- <li>Item two</li>
292
- <li>Item three</li>
293
- </ul>
294
-
295
- <h2>Conclusion</h2>
296
- <p>Wrapping up the article with a summary of key points.</p>`
297
-
298
- const htmlChunks = chunkArticle(htmlArticle, {
299
- articleTitle: 'HTML Article Test',
300
- })
301
-
302
- assert(htmlChunks.length >= 1, `Chunks produced from HTML (got ${htmlChunks.length})`)
303
- assert(!htmlChunks.some(c => c.content.includes('<p>')), 'HTML tags stripped from chunk content')
304
- assert(!htmlChunks.some(c => c.content.includes('<h2>')), 'Heading tags stripped from chunk content')
305
-
306
- console.log('\n Chunk breakdown:')
307
- htmlChunks.forEach((c, i) => {
308
- console.log(` [${i}] ~${estimateTokens(c.content)} tokens | section: ${c.sectionPath || '(none)'}`)
309
- })
310
-
311
- // ---------------------------------------------------------------------------
312
- // Test 7: Real file — design-schema-spec.md
313
- // ---------------------------------------------------------------------------
314
-
315
- section('Test 7: Real file — design-schema-spec.md')
316
-
317
- try {
318
- const specContent = readFileSync(
319
- new URL('../docs/design-schema-spec.md', import.meta.url),
320
- 'utf-8'
321
- )
322
-
323
- const specChunks = chunkArticle(specContent, {
324
- articleTitle: 'Design Schema & Validation Schema Specification',
325
- })
326
-
327
- assert(specChunks.length > 5, `Multiple chunks from spec (got ${specChunks.length})`)
328
- assert(specChunks.length < 40, `Reasonable chunk count (got ${specChunks.length}, expected < 40)`)
329
- assert(specChunks.every(c => estimateTokens(c.content) >= 50),
330
- `All chunks have meaningful content (min ${Math.min(...specChunks.map(c => estimateTokens(c.content)))} tokens)`)
331
-
332
- const maxChunkTokens = Math.max(...specChunks.map(c => estimateTokens(c.content)))
333
- assert(maxChunkTokens < 2000, `No chunk is excessively large (max ${maxChunkTokens} tokens)`)
334
-
335
- console.log('\n Chunk breakdown:')
336
- specChunks.forEach((c, i) => {
337
- console.log(` [${i}] ~${estimateTokens(c.content)} tokens | section: ${c.sectionPath || '(preamble)'}`)
338
- })
339
-
340
- console.log(`\n Total tokens across all chunks: ~${specChunks.reduce((sum, c) => sum + estimateTokens(c.content), 0)}`)
341
- console.log(` Original doc tokens: ~${estimateTokens(specContent)}`)
342
- } catch (err: any) {
343
- console.log(` ⚠️ Skipped — could not read design-schema-spec.md: ${err.message}`)
344
- }
345
-
346
- // ---------------------------------------------------------------------------
347
- // Test 8: Empty content
348
- // ---------------------------------------------------------------------------
349
-
350
- section('Test 8: Empty content')
351
-
352
- const emptyChunks = chunkArticle('', { articleTitle: 'Empty Article' })
353
- assert(emptyChunks.length === 1, `Single chunk for empty content (got ${emptyChunks.length})`)
354
- assert(emptyChunks[0].content === 'Empty Article', 'Content is just the title')
355
-
356
- // ---------------------------------------------------------------------------
357
- // Summary
358
- // ---------------------------------------------------------------------------
359
-
360
- console.log(`\n${'═'.repeat(50)}`)
361
- console.log(`Results: ${passed} passed, ${failed} failed`)
362
- console.log(`${'═'.repeat(50)}`)
363
-
364
- if (failed > 0) process.exit(1)