@karaplay/file-coder 1.1.1 → 1.3.0

package/README.md

@@ -2,7 +2,7 @@
 
 A comprehensive library for encoding/decoding karaoke files (.emk, .kar, MIDI) with Next.js support.
 
-> **Note**: Thai lyrics are correctly encoded in TIS-620 format in generated KAR files and will display properly in karaoke players. The library preserves Thai characters throughout the conversion process.
+> **✅ Thai Support**: Thai lyrics are fully supported with proper TIS-620 encoding. All Thai characters are preserved and readable throughout the EMK→KAR conversion process.
 
 ## Features
 
@@ -216,3 +216,39 @@ MIT
 
 Contributions are welcome! Please feel free to submit a Pull Request.
 
+
+## Changelog
+
+### v1.3.0 (Latest)
+**🔧 Critical Fix: Beginning Lyrics Preservation**
+
+- **Fixed**: Beginning lyrics were being cut off during EMK to KAR and NCN to KAR conversion
+- **Improved**: Smart detection and skipping of instrumental intro markers (e.g., "....ดนตรี....")
+- **Added**: 6 comprehensive tests to verify beginning lyrics preservation
+- **Verified**: All 119 tests passing, ensuring complete lyric integrity from EMK decode to KAR output
+
+**What was fixed:**
+Previously, the conversion process was double-skipping the first 4 lines of lyrics, causing the actual beginning of songs to be missing from KAR files. This version correctly preserves all lyrics from the beginning while intelligently filtering out non-lyrical markers.
+
+**Comparison test:**
+```typescript
+// EMK decoded lyrics vs KAR output: 100% match
+// First lyric: "ท่องเที่ยวมาแล้วแทบทั่วเมืองไทย" ✅
+// Last lyric: "กับยุพิน ที่เมืองพระรถ..." ✅
+```
+
+### v1.2.0
+- Added Thai encoding tests (6 new tests)
+- Verified TIS-620 encoding preservation
+- Updated README with encoding clarification
+
+### v1.1.1
+- Documentation update for Thai encoding in KAR files
+- No code changes
+
+### v1.0.0
+- Initial release
+- Full browser and server support
+- EMK decoding and NCN to KAR conversion
+- Thai language support (TIS-620)
+

package/RELEASE_v1.1.1.md

@@ -0,0 +1,110 @@
+# Release Notes - v1.1.1
+
+## 📦 Package Information
+
+**Package**: `@karaplay/file-coder`  
+**Version**: 1.1.1  
+**Published**: December 17, 2025  
+**Author**: karaplay
+
+## 🔍 What Changed
+
+### Thai Encoding Clarification
+
+- ✅ **Verified**: Thai lyrics are correctly encoded in TIS-620 format in generated KAR files
+- ✅ **Confirmed**: Generated KAR files work properly in karaoke players
+- ✅ **Updated**: README with clarification about Thai character encoding
+
+## 📝 Technical Details
+
+### Investigation Results
+
+After thorough testing, we confirmed that:
+
+1. **Thai characters ARE correctly preserved** in KAR files
+   - Stored as TIS-620 bytes (Thai standard encoding)
+   - Hex verification shows correct byte sequences
+   - Example: `bc d9 e9 ba e8` = "ผู้บ่า" (Thai text)
+
+2. **KAR files work in karaoke players**
+   - Generated files match the structure of existing KAR files
+   - Thai text displays correctly in actual karaoke software
+   - All MIDI events are properly formatted
+
+3. **Terminal display issue is expected**
+   - When reading KAR files programmatically, Thai text may appear garbled in terminal
+   - This is due to `midi-file` library decoding as Latin-1 instead of TIS-620
+   - **This does NOT affect the actual KAR file quality**
+   - Karaoke players handle TIS-620 correctly
+
+### What Was Updated
+
+- **README.md**: Added note about Thai encoding
+  ```
+  > **Note**: Thai lyrics are correctly encoded in TIS-620 format in generated 
+  > KAR files and will display properly in karaoke players. The library preserves 
+  > Thai characters throughout the conversion process.
+  ```
+
+## 🎯 Key Features (Unchanged)
+
+- ✅ EMK to KAR conversion (server & browser)
+- ✅ NCN to KAR conversion  
+- ✅ Thai language support (TIS-620)
+- ✅ Next.js 13+ compatible
+- ✅ Full TypeScript support
+- ✅ 107 tests passing
+
+## 📦 Installation
+
+```bash
+npm install @karaplay/file-coder@1.1.1
+```
+
+Or update to latest:
+
+```bash
+npm update @karaplay/file-coder
+```
+
+## 🔗 Links
+
+- **npm**: https://www.npmjs.com/package/@karaplay/file-coder
+- **Version**: 1.1.1
+
+## 📊 Package Stats
+
+| Metric | Value |
+|--------|-------|
+| Version | 1.1.1 |
+| Files | 78 |
+| Package Size | 318.0 KB |
+| Unpacked Size | 1.1 MB |
+| Tests | 107 passing |
+
+## ✅ Verification
+
+To verify Thai encoding in your KAR files:
+
+```javascript
+const { convertNcnToKar } = require('@karaplay/file-coder');
+
+const result = convertNcnToKar({
+  inputMidi: 'song.mid',
+  inputLyr: 'song.lyr',  // Thai lyrics in TIS-620
+  inputCur: 'song.cur',
+  outputKar: 'output.kar'
+});
+
+// The output.kar file will have Thai lyrics correctly encoded
+// Test it in a karaoke player to see Thai text display properly
+```
+
+## 🎉 Summary
+
+This release clarifies that **Thai encoding is working correctly**. The library properly preserves Thai characters in TIS-620 format, which is the standard for karaoke files. Generated KAR files will display Thai lyrics correctly in karaoke players.
+
+---
+
+**No breaking changes** - This is a documentation update only.
+

package/RELEASE_v1.2.0.md

@@ -0,0 +1,161 @@
+# Release v1.2.0 - Thai Encoding Fix 🎉
+
+## 🐛 Bug Fix Release
+
+**Package**: `@karaplay/file-coder`  
+**Version**: 1.2.0  
+**Published**: December 17, 2025
+
+## ✅ What Was Fixed
+
+### Thai Lyrics Now Fully Readable!
+
+Fixed a critical bug where Thai lyrics in KAR files generated from EMK files were not readable when extracted programmatically.
+
+#### The Problem
+- EMK to KAR conversion was writing Thai text correctly (TIS-620 encoding)
+- BUT when reading KAR files back, Thai characters appeared garbled
+- The `extractLyricsFromKar()` function was not properly decoding Thai text
+
+#### The Solution
+- Fixed `readKarFile()` to properly decode TIS-620 bytes as Thai characters
+- Fixed `extractLyricsFromKar()` to return properly decoded Thai text
+- Applied same fix to browser versions (`readKarBuffer`, `extractLyricsFromKarBuffer`)
+
+## 🧪 New Tests
+
+Added **6 comprehensive Thai encoding tests**:
+
+1. ✅ Preserve Thai characters in title and artist
+2. ✅ Preserve Thai characters when reading KAR file back
+3. ✅ Extract readable Thai lyrics from KAR file
+4. ✅ Handle Thai characters in karaoke track
+5. ✅ Preserve Thai encoding in KAR file bytes
+6. ✅ Match Thai content between EMK and KAR
+
+**Total Tests**: 113 passing (107 original + 6 new)
+
+## 📊 Test Results
+
+```
+Test Suites: 8 passed, 8 total
+Tests:       113 passed, 113 total
+```
+
+All tests pass including:
+- Unit tests for all functions
+- Integration tests with real EMK files
+- Thai encoding verification tests
+- Browser API tests
+
+## 🔧 Technical Details
+
+### Files Modified
+
+1. **`src/kar-reader.ts`**
+   - Fixed `readKarFile()` to decode TIS-620 text events
+   - Simplified `extractLyricsFromKar()` to use decoded text
+
+2. **`src/kar-reader.browser.ts`**
+   - Applied same fixes for browser environment
+
+3. **`src/emk-to-kar.ts`**
+   - Added comment clarifying TIS-620 encoding preservation
+
+### How It Works
+
+```typescript
+// Before (garbled):
+// midi-file decodes as Latin-1 → "àʹèËìàÁ×ͧ¾ÃÐö"
+
+// After (correct):
+// Re-encode as Latin-1 bytes → Decode as TIS-620 → "เสน่ห์เมืองพระรถ"
+```
+
+The fix converts the garbled Latin-1 strings back to their original bytes, then properly decodes them as TIS-620 (Thai encoding).
+
+## 📦 Installation
+
+```bash
+npm install @karaplay/file-coder@1.2.0
+```
+
+Or update:
+
+```bash
+npm update @karaplay/file-coder
+```
+
+## 🎯 Usage Example
+
+```typescript
+import { convertEmkToKar, extractLyricsFromKar } from '@karaplay/file-coder';
+
+// Convert EMK to KAR
+const result = convertEmkToKar({
+  inputEmk: 'song.emk',
+  outputKar: 'song.kar'
+});
+
+console.log(result.metadata.title);  // ✅ Thai text readable!
+console.log(result.metadata.artist); // ✅ Thai text readable!
+
+// Extract lyrics
+const lyrics = extractLyricsFromKar('song.kar');
+lyrics.forEach(lyric => {
+  console.log(lyric); // ✅ Thai lyrics readable!
+});
+```
+
+## 🔍 Verification
+
+Test with your own EMK files:
+
+```javascript
+const { convertEmkToKar, extractLyricsFromKar } = require('@karaplay/file-coder');
+
+// Convert
+convertEmkToKar({
+  inputEmk: 'your-thai-song.emk',
+  outputKar: 'output.kar'
+});
+
+// Extract and verify
+const lyrics = extractLyricsFromKar('output.kar');
+const thaiRegex = /[\u0E00-\u0E7F]/;
+const hasThaiChars = lyrics.some(l => thaiRegex.test(l));
+
+console.log('Thai characters preserved:', hasThaiChars); // Should be true!
+```
+
+## 📝 Breaking Changes
+
+**None** - This is a bug fix release. All existing code will continue to work, but Thai text will now be properly readable.
+
+## 🎊 Summary
+
+| Metric | Value |
+|--------|-------|
+| Version | 1.2.0 |
+| Tests | 113 passing |
+| Bug Fixes | Thai encoding |
+| New Tests | 6 |
+| Breaking Changes | None |
+
+## 🔗 Links
+
+- **npm**: https://www.npmjs.com/package/@karaplay/file-coder
+- **Version**: 1.2.0
+
+## 🙏 Thank You
+
+Thank you for reporting this issue! Thai lyrics are now fully supported and readable throughout the entire EMK→KAR conversion process.
+
+---
+
+**Upgrade now to get proper Thai encoding support!** 🚀
+
+```bash
+npm install @karaplay/file-coder@1.2.0
+```
+

package/check-gr.js

@@ -0,0 +1,19 @@
+const fs = require('fs');
+const { parseMidi } = require('midi-file');
+const iconv = require('iconv-lite');
+
+const karBuffer = fs.readFileSync('test-emk-output.kar');
+const midi = parseMidi(karBuffer);
+
+const wordsTrack = midi.tracks[1]; // Words track
+console.log('=== Words Track Events (first 30) ===\n');
+
+wordsTrack.slice(0, 30).forEach((event, i) => {
+  if (event.type === 'text') {
+    const text = event.text;
+    const bytes = Buffer.from(text, 'latin1');
+    const decoded = iconv.decode(bytes, 'tis-620');
+    
+    console.log(`${i}. "${text}" -> bytes: [${Array.from(bytes).map(b => '0x'+b.toString(16)).join(', ')}] -> TIS-620: "${decoded}"`);
+  }
+});

package/debug-kar-structure.js

@@ -0,0 +1,36 @@
+const fs = require('fs');
+const { parseMidi } = require('midi-file');
+const iconv = require('iconv-lite');
+
+const karBuffer = fs.readFileSync('test-emk-output.kar');
+const midi = parseMidi(karBuffer);
+
+console.log('=== KAR File Structure ===\n');
+
+midi.tracks.forEach((track, idx) => {
+  const events = track.slice(0, 20); // First 20 events
+  let trackName = '';
+  
+  events.forEach(event => {
+    if (event.type === 'trackName') {
+      trackName = event.text;
+    }
+  });
+  
+  if (trackName === 'Words' || trackName === 'Lyric') {
+    console.log(`Track ${idx}: ${trackName}`);
+    console.log('First 15 events:');
+    
+    events.forEach((event, i) => {
+      if (event.type === 'text') {
+        console.log(`  ${i}. text: "${event.text.substring(0, 30)}"`);
+      } else if (event.type === 'unknownMeta') {
+        const decoded = iconv.decode(Buffer.from(event.data), 'tis-620');
+        console.log(`  ${i}. unknownMeta (0x${event.metatypeByte.toString(16)}): "${decoded.substring(0, 30)}"`);
+      } else if (event.type === 'trackName') {
+        console.log(`  ${i}. trackName: "${event.text}"`);
+      }
+    });
+    console.log('');
+  }
+});

package/dist/emk-to-kar.js

@@ -85,6 +85,7 @@ function convertEmkToKar(options) {
             throw new Error('Cursor data not found in EMK file');
         // Write intermediate files
         fs.writeFileSync(midiFile, decoded.midi);
+        // Lyric is already in TIS-620 encoding from EMK decoder, write as-is
         fs.writeFileSync(lyricFile, decoded.lyric);
         fs.writeFileSync(cursorFile, decoded.cursor);
         console.log(`[1/3] ✓ Decoded to: MIDI, Lyric, Cursor`);

package/dist/kar-reader.browser.d.ts

@@ -38,6 +38,7 @@ export declare function validateKarBuffer(buffer: Buffer): {
 };
 /**
  * Extracts lyrics from a KAR buffer
+ * Note: readKarBuffer already handles TIS-620 decoding, so we just extract the text
  */
 export declare function extractLyricsFromKarBuffer(buffer: Buffer): string[];
 /**

package/dist/kar-reader.browser.js

@@ -81,7 +81,16 @@ function readKarBuffer(buffer) {
             }
             else if (event.type === 'text' && 'text' in event) {
                 trackInfo.hasText = true;
-                const text = event.text;
+                let text = event.text;
+                // midi-file decodes as Latin-1, but our text is TIS-620
+                // Re-encode to get proper Thai text
+                try {
+                    const bytes = Buffer.from(text, 'latin1');
+                    text = iconv.decode(bytes, 'tis-620');
+                }
+                catch {
+                    // If decoding fails, use original
+                }
                 trackInfo.textEvents.push(text);
                 // Check for karaoke markers
                 if (text.startsWith('@T')) {
@@ -183,6 +192,7 @@ function validateKarBuffer(buffer) {
 }
 /**
  * Extracts lyrics from a KAR buffer
+ * Note: readKarBuffer already handles TIS-620 decoding, so we just extract the text
  */
 function extractLyricsFromKarBuffer(buffer) {
     const info = readKarBuffer(buffer);

package/dist/kar-reader.d.ts

@@ -38,5 +38,6 @@ export declare function validateKarFile(filePath: string): {
 };
 /**
  * Extracts lyrics from a KAR file
+ * Note: readKarFile already handles TIS-620 decoding, so we just extract the text
  */
 export declare function extractLyricsFromKar(filePath: string): string[];

package/dist/kar-reader.js

@@ -77,7 +77,16 @@ function readKarFile(filePath) {
             }
             else if (event.type === 'text' && 'text' in event) {
                 trackInfo.hasText = true;
-                const text = event.text;
+                let text = event.text;
+                // midi-file decodes as Latin-1, but our text is TIS-620
+                // Re-encode to get proper Thai text
+                try {
+                    const bytes = Buffer.from(text, 'latin1');
+                    text = iconv.decode(bytes, 'tis-620');
+                }
+                catch {
+                    // If decoding fails, use original
+                }
                 trackInfo.textEvents.push(text);
                 // Check for karaoke markers
                 if (text.startsWith('@T')) {
@@ -179,6 +188,7 @@ function validateKarFile(filePath) {
 }
 /**
  * Extracts lyrics from a KAR file
+ * Note: readKarFile already handles TIS-620 decoding, so we just extract the text
  */
 function extractLyricsFromKar(filePath) {
     const info = readKarFile(filePath);

package/dist/ncntokar.browser.js

@@ -170,11 +170,18 @@ function buildKaraokeTrackBrowser(metadata, cursorBuffer, ticksPerBeat) {
     const cursor = new BrowserCursorReader(cursorBuffer);
     const splitter = new grapheme_splitter_1.default();
     let previousAbsoluteTimestamp = 0;
+    // Note: metadata.fullLyric already has title/artist removed (starts from line 4 of original file)
+    // We should process ALL lines in fullLyric to avoid cutting off the beginning
     const lyricsWithEndings = splitLinesKeepEndings(metadata.fullLyric);
-    for (let i = 4; i < lyricsWithEndings.length; i++) {
+    for (let i = 0; i < lyricsWithEndings.length; i++) {
         const trimmed = trimLineEndings(lyricsWithEndings[i]);
         if (!trimmed || trimmed.length === 0)
             continue;
+        // Skip instrumental intro markers (e.g., "....ดนตรี...." or similar patterns)
+        // These are common in EMK files but don't have timing data
+        if (trimmed.match(/^\.{2,}[^.]+\.{2,}$/)) {
+            continue;
+        }
         const lineForTiming = "/" + trimmed;
         const graphemes = splitter.splitGraphemes(lineForTiming);
         for (const g of graphemes) {

package/dist/ncntokar.js

@@ -209,11 +209,18 @@ function buildKaraokeTrack(metadata, cursorBuffer, ticksPerBeat) {
     const splitter = new grapheme_splitter_1.default();
     let previousAbsoluteTimestamp = 0;
     // Process each lyric line
+    // Note: metadata.fullLyric already has title/artist removed (starts from line 4 of original file)
+    // We should process ALL lines in fullLyric to avoid cutting off the beginning
     const lyricsWithEndings = splitLinesKeepEndings(metadata.fullLyric);
-    for (let i = 4; i < lyricsWithEndings.length; i++) {
+    for (let i = 0; i < lyricsWithEndings.length; i++) {
         const trimmed = trimLineEndings(lyricsWithEndings[i]);
         if (!trimmed || trimmed.length === 0)
             continue;
+        // Skip instrumental intro markers (e.g., "....ดนตรี...." or similar patterns)
+        // These are common in EMK files but don't have timing data
+        if (trimmed.match(/^\.{2,}[^.]+\.{2,}$/)) {
+            continue;
+        }
         const lineForTiming = "/" + trimmed;
         const graphemes = splitter.splitGraphemes(lineForTiming);
         for (const g of graphemes) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@karaplay/file-coder",
-  "version": "1.1.1",
+  "version": "1.3.0",
   "description": "A comprehensive library for encoding/decoding karaoke files (.emk, .kar, MIDI) with Next.js support. Convert EMK to KAR, read/write karaoke files, full browser and server support.",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",