@mrselva/attendance 1.1.3 → 1.2.0

package/README.md

@@ -4,7 +4,7 @@
 [![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
 [![Node.js Version](https://img.shields.io/badge/node-%3E%3D16.0.0-brightgreen.svg)](https://nodejs.org)
 
-> A fast and lightweight CLI tool to extract, normalize, and export multi-sheet attendance spreadsheets into organized Excel files and ZIP archives.
+> A fast and lightweight CLI tool to normalize, match, and export multi-sheet attendance spreadsheets into organized DigiCampus-compatible Excel files and ZIP archives.
 
 Run instantly with **`npx`** — no installation required!
 
@@ -29,18 +29,19 @@ attendance
 
 ## ✨ Features
 
-- **Zero Configuration & Auto-Import**: If no file is in `./input/`, an interactive file picker window opens so you can select your spreadsheet directly from your computer, or drag & drop file paths into the terminal.
+- **Zero Configuration & Auto-Import**: If no file is in `./input/`, an interactive native file picker window opens so you can click and select your spreadsheet directly from your computer, or drag & drop file paths into the terminal.
+- **Spreadsheet Normalization Engine**: Raw sheets often contain mismatched column orders, duplicate headers (like repeated departments), extra spaces in header names, and stray empty rows. The CLI automatically converts uploaded files into an understandable, standardized format, saves it to `./normalized/Normalized_<filename>.xlsx`, and uses this clean file for all operations.
+- **DigiCampus Template Integration**: Upload or browse your DigiCampus template (`Upload_Attendance.xlsx`). The CLI fills attendance directly against the exact student roster required by DigiCampus.
 - **Cohort & Batch Support**:
   - **3rd Year**: Exports standard whole-class roster.
   - **2nd Year**: Automatically detects the empty row separating **Batch 1** (above the empty row) and **Batch 2** (below the empty row). Prompts you to export Batch 1, Batch 2, or Both Batches!
-- **Multi-Sheet Detection**: Automatically discovers all relevant sheets and rosters across the workbook.
 - **Smart Date Filtering**:
   - Export **All recorded dates**
   - Export a **Single date** (e.g. `YYYY-MM-DD`)
   - Export a **Date range** (from `YYYY-MM-DD` to `YYYY-MM-DD`)
   - Export the **Latest date only**
-- **Status Normalization**: Accurately normalizes attendance status (`PRESENT`, `ABSENT`, etc.) and handles blanks cleanly.
-- **Automatic ZIP Packaging**: Compresses generated export files directly into clean ZIP archives in `./output/`.
+- **Date-First Output Naming**: Generated files inside the ZIP start with the date (e.g. `2026-10-09_Upload_Attendance_BW_Batch1.xlsx`) for easy chronological sorting.
+- **Automatic ZIP Packaging**: Compresses generated export files directly into clean ZIP archives in `./output/` with full cohort and batch metadata in the archive name.
 
 ---
 
@@ -49,11 +50,11 @@ attendance
 Run the interactive wizard or pass flags for automation:
 
 ```bash
-# Interactive mode (asks Year, Batch, Sheet, Dates)
+# Interactive mode (prompts for File, Year, Template, Batch, Sheet, Dates)
 npx @mrselva/attendance
 
-# 2nd Year - Export Batch 1
-npx @mrselva/attendance -y 2 -b 1 -v SHEET_CODE -a
+# 2nd Year - Export Batch 1 with template
+npx @mrselva/attendance -y 2 -b 1 -v SHEET_CODE -t "path/to/Upload_Attendance.xlsx" -a
 
 # 2nd Year - Export Batch 2
 npx @mrselva/attendance -y 2 -b 2 -v SHEET_CODE -a
@@ -75,6 +76,7 @@ npx @mrselva/attendance -f "attendance_sheet.xlsx" -y 2 -b 1 -a
 | `-y <2\|3>` | `--year <2\|3>` | Academic year (`2` for 2nd year batched, `3` for 3rd year standard) |
 | `-b <1\|2\|all>` | `--batch <1\|2\|all>` | Batch selection for 2nd year (`1`, `2`, or `all` for both) |
 | `-v <code>` | `--vertical <code>` | Sheet code or name (or `ALL`) |
+| `-t <file>` | `--template <path>` | DigiCampus upload template Excel file (.xlsx) |
 | `-a` | `--all` | Export all recorded dates |
 | `-d <date>` | `--date <YYYY-MM-DD>` | Export a specific date |
 | | `--latest` | Export the latest recorded date |
@@ -88,6 +90,8 @@ npx @mrselva/attendance -f "attendance_sheet.xlsx" -y 2 -b 1 -a
 ```text
 your-folder/
 ├── input/                             # Place source spreadsheets here (or auto-import)
+├── normalized/                        # Clean, standardized master spreadsheets
+├── digi/                              # DigiCampus upload templates
 └── output/                            # Automatically generated ZIP archives
 ```
 

package/index.js

@@ -5,10 +5,17 @@
  * 
  * Features:
  * - Scans ./input or current directory for attendance spreadsheets (.xlsx / .xls)
- * - Interactively prompts for sheet/category selection
- * - Allows exporting all recorded dates, specific date, range, or latest date
- * - Generates individual formatted Excel files with ZIP archive in ./output
- * - Works both interactively (in terminal) and with CLI flags (--vertical, --all, etc.)
+ * - Interactively prompts for file selection or opens OS file picker
+ * - Prompts for Academic Year (3rd Year standard vs 2nd Year batched)
+ * - Prompts for optional DigiCampus template upload (Upload_Attendance.xlsx)
+ * - Normalization Engine: Automatically standardizes irregular/mismatched raw sheets,
+ *   cleans duplicate headers, fixes column orders, removes empty rows, assigns batches,
+ *   and maintains a clean standardized file in ./normalized/Normalized_<file>.xlsx
+ * - Uses the normalized workbook for all downstream parsing and exports
+ * - Supports exporting specific dates, date ranges, latest date, or all instructional dates
+ * - Generates DigiCampus-compatible Excel files ('Students' sheet with Email Id, Registration Id, Attendance *)
+ * - Packages exports into ZIP archives in ./output with metadata naming
+ * - Preserves date-first file naming (YYYY-MM-DD_Upload_Attendance_...)
  */
 
 const fs = require('fs');
@@ -21,6 +28,8 @@ const AdmZip = require('adm-zip');
 // Paths (relative to user current working directory where npx is executed)
 const INPUT_DIR = path.resolve(process.cwd(), 'input');
 const OUTPUT_DIR = path.resolve(process.cwd(), 'output');
+const NORMALIZED_DIR = path.resolve(process.cwd(), 'normalized');
+const DIGI_DIR = path.resolve(process.cwd(), 'digi');
 
 // Optional category/sheet code to display name mapping
 const VERTICAL_NAMES = {};
@@ -66,6 +75,12 @@ function ensureDirs() {
   if (!fs.existsSync(OUTPUT_DIR)) {
     fs.mkdirSync(OUTPUT_DIR, { recursive: true });
   }
+  if (!fs.existsSync(NORMALIZED_DIR)) {
+    fs.mkdirSync(NORMALIZED_DIR, { recursive: true });
+  }
+  if (!fs.existsSync(DIGI_DIR)) {
+    fs.mkdirSync(DIGI_DIR, { recursive: true });
+  }
 }
 
 // Find excel files in input folder or current working directory
@@ -179,7 +194,6 @@ function normalizeStatus(val) {
   return null;
 }
 
-
 // Check if a row represents summary / totals
 function isSummaryRow(row, name, regNo) {
   if (name && (/no\s*\.?\s*of/i.test(name) || /total/i.test(name) || /present/i.test(name) || /absent/i.test(name))) {
@@ -205,8 +219,13 @@ function getInstructionalDates(students, dateColumns, rows) {
     let absentCount = 0;
 
     for (const student of students) {
-      const cellVal = rows[student.rowIdx] ? rows[student.rowIdx][dc.colIdx] : null;
-      const norm = normalizeStatus(cellVal);
+      let norm = null;
+      if (student.attendance && student.attendance[dc.dateStr] !== undefined) {
+        norm = student.attendance[dc.dateStr];
+      } else if (rows && student.rowIdx !== undefined) {
+        const cellVal = rows[student.rowIdx] ? rows[student.rowIdx][dc.colIdx] : null;
+        norm = normalizeStatus(cellVal);
+      }
       if (norm === 'PRESENT') {
         presentCount++;
       } else if (norm === 'ABSENT') {
@@ -227,6 +246,202 @@ function getInstructionalDates(students, dateColumns, rows) {
   return instructionalDates;
 }
 
+/**
+ * Normalization Engine:
+ * Converts raw uploaded attendance sheets with mismatched column orders, duplicate headers,
+ * extra spaces, and irregular blank rows into a unified, clean default format workbook.
+ * 
+ * Standardized columns per sheet:
+ * ['S.No', 'Student Name', 'Register Number', 'Roll Number', 'Email Id', 'Department', 'Section', 'Gender', 'Batch', ...dateStrings]
+ */
+function createNormalizedWorkbook(rawWorkbook, selectedYear = 2) {
+  const normWb = xlsx.utils.book_new();
+  const excludedSheets = ['pivot table', 'summary', 'template', 'students data', 'student data', 'long leave'];
+  let sheetCount = 0;
+
+  for (const sheetName of rawWorkbook.SheetNames) {
+    if (excludedSheets.some(ex => sheetName.toLowerCase().includes(ex))) {
+      continue;
+    }
+    const ws = rawWorkbook.Sheets[sheetName];
+    const rows = xlsx.utils.sheet_to_json(ws, { header: 1, raw: true, defval: null });
+    if (!rows || rows.length < 2) continue;
+
+    const headerRow = rows[0];
+    let snoColIdx = -1;
+    let nameColIdx = -1;
+    let regColIdx = -1;
+    let rollColIdx = -1;
+    let emailColIdx = -1;
+    let deptColIdx = -1;
+    let secColIdx = -1;
+    let genderColIdx = -1;
+    const dateColumns = [];
+
+    for (let c = 0; c < headerRow.length; c++) {
+      const hVal = headerRow[c];
+      if (hVal === null || hVal === undefined) continue;
+      const hStr = String(hVal).trim().toLowerCase();
+
+      if (hStr.includes('register') || hStr.includes('reg no') || hStr.includes('reg.') || hStr.includes('registration id')) {
+        regColIdx = c;
+      } else if (hStr.includes('student name') || hStr === 'name') {
+        nameColIdx = c;
+      } else if (hStr.includes('kite email')) {
+        emailColIdx = c;
+      } else if (emailColIdx === -1 && (hStr.includes('email id') || hStr.includes('email') || hStr.includes('mail')) && !hStr.includes('soi')) {
+        emailColIdx = c;
+      } else if (hStr.includes('roll number') || hStr.includes('roll no')) {
+        rollColIdx = c;
+      } else if (hStr.includes('s.no') || hStr === 'sno' || hStr.includes('sl.no')) {
+        snoColIdx = c;
+      } else if (deptColIdx === -1 && (hStr.includes('department') || hStr === 'dept' || hStr === 'branch')) {
+        deptColIdx = c;
+      } else if (secColIdx === -1 && (hStr.includes('section') || hStr === 'sec')) {
+        secColIdx = c;
+      } else if (genderColIdx === -1 && (hStr.includes('gender') || hStr === 'sex')) {
+        genderColIdx = c;
+      } else {
+        const cellAddress = xlsx.utils.encode_cell({ r: 0, c: c });
+        const cellRaw = ws[cellAddress];
+        const dateStr = formatDate(hVal, cellRaw);
+        if (dateStr) {
+          dateColumns.push({ colIdx: c, dateStr });
+        }
+      }
+    }
+
+    if (regColIdx === -1 || dateColumns.length === 0) {
+      continue; // Skip non-vertical sheets
+    }
+
+    const stdHeader = [
+      'S.No',
+      'Student Name',
+      'Register Number',
+      'Roll Number',
+      'Email Id',
+      'Department',
+      'Section',
+      'Gender',
+      'Batch',
+      ...dateColumns.map(d => d.dateStr)
+    ];
+
+    const normData = [stdHeader];
+    let currentBatch = 1;
+    let sawBlankAfterBatch1 = false;
+    let studentCounter = 0;
+
+    for (let r = 1; r < rows.length; r++) {
+      const row = rows[r];
+      if (!row || row.length === 0) {
+        if (studentCounter > 0 && currentBatch === 1) {
+          sawBlankAfterBatch1 = true;
+        }
+        continue;
+      }
+
+      const name = nameColIdx !== -1 ? String(row[nameColIdx] || '').trim() : '';
+      const regNo = regColIdx !== -1 ? String(row[regColIdx] || '').trim() : '';
+      if (isSummaryRow(row, name, regNo)) break;
+
+      const rollNo = rollColIdx !== -1 ? String(row[rollColIdx] || '').trim() : '';
+      const email = emailColIdx !== -1 ? String(row[emailColIdx] || '').trim() : '';
+      const dept = deptColIdx !== -1 ? String(row[deptColIdx] || '').trim() : '';
+      const sec = secColIdx !== -1 ? String(row[secColIdx] || '').trim() : '';
+      const gender = genderColIdx !== -1 ? String(row[genderColIdx] || '').trim() : '';
+
+      const isStudent = name.length >= 2 && (regNo.length > 0 || rollNo.length > 0);
+      if (!isStudent) {
+        if (studentCounter > 0 && currentBatch === 1) {
+          sawBlankAfterBatch1 = true;
+        }
+        continue;
+      }
+
+      // Switch to Batch 2 in 2nd Year when empty separator row was encountered
+      if (selectedYear === 2 && currentBatch === 1 && sawBlankAfterBatch1) {
+        currentBatch = 2;
+      }
+
+      studentCounter++;
+      const attendanceRow = dateColumns.map(dc => normalizeStatus(row[dc.colIdx]) || '');
+
+      normData.push([
+        studentCounter,
+        name,
+        regNo,
+        rollNo,
+        email,
+        dept,
+        sec,
+        gender,
+        selectedYear === 2 ? `Batch ${currentBatch}` : 'Batch 1',
+        ...attendanceRow
+      ]);
+    }
+
+    const normWs = xlsx.utils.aoa_to_sheet(normData);
+    normWs['!cols'] = [
+      { wch: 6 },
+      { wch: 25 },
+      { wch: 18 },
+      { wch: 15 },
+      { wch: 35 },
+      { wch: 20 },
+      { wch: 10 },
+      { wch: 10 },
+      { wch: 12 },
+      ...dateColumns.map(() => ({ wch: 12 }))
+    ];
+
+    xlsx.utils.book_append_sheet(normWb, normWs, sheetName);
+    sheetCount++;
+  }
+
+  return { normWb, sheetCount };
+}
+
+// Load DigiCampus Upload Template (e.g. Upload_Attendance.xlsx)
+function loadTemplate(filePath) {
+  if (!filePath || !fs.existsSync(filePath)) return null;
+  const wb = xlsx.readFile(filePath);
+  const sheetName = wb.SheetNames[0];
+  const ws = wb.Sheets[sheetName];
+  const rows = xlsx.utils.sheet_to_json(ws, { header: 1, raw: true, defval: '' });
+  if (!rows || rows.length < 2) return null;
+
+  const headerRow = rows[0];
+  let emailCol = -1;
+  let regCol = -1;
+  let attCol = -1;
+
+  for (let c = 0; c < headerRow.length; c++) {
+    const h = String(headerRow[c] || '').trim().toLowerCase();
+    if (h.includes('email')) emailCol = c;
+    else if (h.includes('reg')) regCol = c;
+    else if (h.includes('attend')) attCol = c;
+  }
+
+  if (emailCol === -1 || regCol === -1) return null;
+
+  const students = [];
+  for (let r = 1; r < rows.length; r++) {
+    const row = rows[r];
+    const email = String(row[emailCol] || '').trim();
+    const regNo = String(row[regCol] || '').trim();
+    if (!email && !regNo) continue;
+    students.push({
+      email,
+      regNo,
+      defaultStatus: attCol !== -1 ? String(row[attCol] || '').trim() : ''
+    });
+  }
+
+  return { sheetName, students };
+}
+
 // Parse a vertical sheet to extract students (batch 1 & 2) and date columns
 function parseVerticalSheet(ws, sheetName) {
   const rows = xlsx.utils.sheet_to_json(ws, { header: 1, raw: true, defval: null });
@@ -238,6 +453,11 @@ function parseVerticalSheet(ws, sheetName) {
   let nameColIdx = -1;
   let emailColIdx = -1;
   let rollColIdx = -1;
+  let batchColIdx = -1;
+  let deptColIdx = -1;
+  let secColIdx = -1;
+  let genderColIdx = -1;
+  let snoColIdx = -1;
   const dateColumns = [];
 
   // Analyze header row
@@ -251,10 +471,22 @@ function parseVerticalSheet(ws, sheetName) {
       regColIdx = c;
     } else if (hStr.includes('student name') || hStr === 'name') {
       nameColIdx = c;
-    } else if (hStr.includes('email id') || hStr.includes('email') || hStr.includes('mail')) {
+    } else if (hStr.includes('kite email')) {
+      emailColIdx = c;
+    } else if (emailColIdx === -1 && (hStr.includes('email id') || hStr.includes('email') || hStr.includes('mail')) && !hStr.includes('soi')) {
       emailColIdx = c;
     } else if (hStr.includes('roll number') || hStr.includes('roll no')) {
       rollColIdx = c;
+    } else if (hStr.includes('s.no') || hStr === 'sno' || hStr.includes('sl.no')) {
+      snoColIdx = c;
+    } else if (hStr === 'batch') {
+      batchColIdx = c;
+    } else if (deptColIdx === -1 && (hStr.includes('department') || hStr === 'dept' || hStr === 'branch')) {
+      deptColIdx = c;
+    } else if (secColIdx === -1 && (hStr.includes('section') || hStr === 'sec')) {
+      secColIdx = c;
+    } else if (genderColIdx === -1 && (hStr.includes('gender') || hStr === 'sex')) {
+      genderColIdx = c;
     } else {
       const cellAddress = xlsx.utils.encode_cell({ r: 0, c: c });
       const cellRaw = ws[cellAddress];
@@ -270,7 +502,7 @@ function parseVerticalSheet(ws, sheetName) {
     return null; // Not a vertical attendance sheet
   }
 
-  // Parse student records, separating Batch 1 and Batch 2 across empty separator row
+  // Parse student records, separating Batch 1 and Batch 2
   const batch1Students = [];
   const batch2Students = [];
   let currentBatch = 1;
@@ -285,11 +517,14 @@ function parseVerticalSheet(ws, sheetName) {
       continue;
     }
 
-    const sno = row[0];
+    const sno = snoColIdx !== -1 ? row[snoColIdx] : r;
     const name = nameColIdx !== -1 ? String(row[nameColIdx] || '').trim() : '';
     const regNo = regColIdx !== -1 ? String(row[regColIdx] || '').trim() : '';
     const email = emailColIdx !== -1 ? String(row[emailColIdx] || '').trim() : '';
     const rollNo = rollColIdx !== -1 ? String(row[rollColIdx] || '').trim() : '';
+    const dept = deptColIdx !== -1 ? String(row[deptColIdx] || '').trim() : '';
+    const sec = secColIdx !== -1 ? String(row[secColIdx] || '').trim() : '';
+    const gender = genderColIdx !== -1 ? String(row[genderColIdx] || '').trim() : '';
 
     // Ignore summary/totals rows at bottom
     if (isSummaryRow(row, name, regNo)) {
@@ -297,7 +532,6 @@ function parseVerticalSheet(ws, sheetName) {
     }
 
     const isStudent = name.length >= 2 && (regNo.length > 0 || rollNo.length > 0);
-
     if (!isStudent) {
       if (batch1Students.length > 0 && currentBatch === 1) {
         sawBlankAfterBatch1 = true;
@@ -305,20 +539,35 @@ function parseVerticalSheet(ws, sheetName) {
       continue;
     }
 
-    // Valid student row
-    const snoNum = Number(sno);
-    if (batch1Students.length > 0 && currentBatch === 1 && (sawBlankAfterBatch1 || snoNum === 1)) {
-      currentBatch = 2;
+    // Determine batch (from normalized batch column or blank row boundary)
+    if (batchColIdx !== -1) {
+      const bStr = String(row[batchColIdx] || '').toLowerCase();
+      currentBatch = bStr.includes('2') ? 2 : 1;
+    } else {
+      if (batch1Students.length > 0 && currentBatch === 1 && sawBlankAfterBatch1) {
+        currentBatch = 2;
+      }
+    }
+
+    // Build attendance dictionary for each date
+    const attendance = {};
+    for (const dc of dateColumns) {
+      const cellVal = row[dc.colIdx];
+      attendance[dc.dateStr] = normalizeStatus(cellVal) || '';
     }
 
     const studentObj = {
       rowIdx: r,
-      sno: sno,
-      name: name,
-      regNo: regNo,
-      rollNo: rollNo,
+      sno,
+      name,
+      regNo,
+      rollNo,
       email: email || '',
-      batch: currentBatch
+      dept,
+      sec,
+      gender,
+      batch: currentBatch,
+      attendance
     };
 
     if (currentBatch === 1) {
@@ -350,8 +599,8 @@ function parseVerticalSheet(ws, sheetName) {
   };
 }
 
-// Generate formatted Excel workbook for a single date
-function generateExportWorkbook(verticalData, dateItem, rows, studentList) {
+// Generate formatted Excel workbook for a single date (DigiCampus Upload format)
+function generateExportWorkbook(verticalData, dateItem, rows, studentList, parsedTemplate) {
   const wb = xlsx.utils.book_new();
 
   // Export Sheet Header
@@ -361,19 +610,90 @@ function generateExportWorkbook(verticalData, dateItem, rows, studentList) {
 
   const students = studentList || verticalData.students;
 
-  for (const student of students) {
-    const rawVal = rows[student.rowIdx] ? rows[student.rowIdx][dateItem.colIdx] : null;
-    let status = normalizeStatus(rawVal);
-    // On an instructional day, any unrecorded or blank student is marked ABSENT
-    if (!status) {
-      status = 'ABSENT';
+  if (parsedTemplate && parsedTemplate.students && parsedTemplate.students.length > 0) {
+    // Build lookup maps for fast matching
+    const studentByReg = new Map();
+    const studentByEmail = new Map();
+
+    for (const st of students) {
+      if (st.regNo) studentByReg.set(st.regNo.toUpperCase().trim(), st);
+      if (st.email) studentByEmail.set(st.email.toLowerCase().trim(), st);
     }
 
-    wsData.push([
-      student.email,
-      student.regNo,
-      status
-    ]);
+    // Check how many students in the template match this cohort
+    let matchCount = 0;
+    for (const ts of parsedTemplate.students) {
+      const tReg = (ts.regNo || '').toUpperCase().trim();
+      const tEmail = (ts.email || '').toLowerCase().trim();
+      if ((tReg && studentByReg.has(tReg)) || (tEmail && studentByEmail.has(tEmail))) {
+        matchCount++;
+      }
+    }
+
+    if (matchCount > 0) {
+      // Use exact template roster and fill attendance
+      for (const tStudent of parsedTemplate.students) {
+        const tReg = (tStudent.regNo || '').toUpperCase().trim();
+        const tEmail = (tStudent.email || '').toLowerCase().trim();
+
+        let matched = null;
+        if (tReg && studentByReg.has(tReg)) {
+          matched = studentByReg.get(tReg);
+        } else if (tEmail && studentByEmail.has(tEmail)) {
+          matched = studentByEmail.get(tEmail);
+        }
+
+        let status = 'ABSENT';
+        if (matched) {
+          if (matched.attendance && matched.attendance[dateItem.dateStr]) {
+            status = matched.attendance[dateItem.dateStr];
+          } else if (rows && matched.rowIdx !== undefined) {
+            const rawVal = rows[matched.rowIdx] ? rows[matched.rowIdx][dateItem.colIdx] : null;
+            status = normalizeStatus(rawVal) || 'ABSENT';
+          }
+        }
+
+        wsData.push([
+          tStudent.email,
+          tStudent.regNo,
+          status
+        ]);
+      }
+    } else {
+      // Fallback to vertical roster if template had 0 matches for this vertical
+      for (const student of students) {
+        let status = 'ABSENT';
+        if (student.attendance && student.attendance[dateItem.dateStr]) {
+          status = student.attendance[dateItem.dateStr];
+        } else if (rows && student.rowIdx !== undefined) {
+          const rawVal = rows[student.rowIdx] ? rows[student.rowIdx][dateItem.colIdx] : null;
+          status = normalizeStatus(rawVal) || 'ABSENT';
+        }
+
+        wsData.push([
+          student.email,
+          student.regNo,
+          status
+        ]);
+      }
+    }
+  } else {
+    // Standard vertical roster
+    for (const student of students) {
+      let status = 'ABSENT';
+      if (student.attendance && student.attendance[dateItem.dateStr]) {
+        status = student.attendance[dateItem.dateStr];
+      } else if (rows && student.rowIdx !== undefined) {
+        const rawVal = rows[student.rowIdx] ? rows[student.rowIdx][dateItem.colIdx] : null;
+        status = normalizeStatus(rawVal) || 'ABSENT';
+      }
+
+      wsData.push([
+        student.email,
+        student.regNo,
+        status
+      ]);
+    }
   }
 
   const ws = xlsx.utils.aoa_to_sheet(wsData);
@@ -389,19 +709,18 @@ function generateExportWorkbook(verticalData, dateItem, rows, studentList) {
   return wb;
 }
 
-// Parse Command Line Arguments (allows both CLI flags and interactive mode)
+// Parse Command Line Arguments
 function parseArgs() {
   const args = process.argv.slice(2);
   const flags = {
     vertical: null,
     all: false,
     date: null,
-    startDate: null,
-    endDate: null,
     latest: false,
     file: null,
     year: null,
     batch: null,
+    template: null,
     help: false
   };
 
@@ -415,6 +734,8 @@ function parseArgs() {
       flags.year = args[++i];
     } else if (a === '-b' || a === '--batch') {
       flags.batch = args[++i];
+    } else if (a === '-t' || a === '--template') {
+      flags.template = args[++i];
     } else if (a === '-a' || a === '--all' || a === '--all-dates') {
       flags.all = true;
     } else if (a === '-d' || a === '--date') {
@@ -445,7 +766,7 @@ function cleanPath(input) {
 }
 
 // Open native OS file picker dialog
-function openFileDialog() {
+function openFileDialog(title = 'Select Attendance Spreadsheet') {
   const platform = process.platform;
   try {
     if (platform === 'win32') {
@@ -453,7 +774,7 @@ function openFileDialog() {
 Add-Type -AssemblyName System.Windows.Forms
 $dlg = New-Object System.Windows.Forms.OpenFileDialog
 $dlg.Filter = 'Excel Files (*.xlsx;*.xls)|*.xlsx;*.xls|All Files (*.*)|*.*'
-$dlg.Title = 'Select Attendance Spreadsheet'
+$dlg.Title = '${title.replace(/'/g, "''")}'
 $dlg.RestoreDirectory = $true
 $result = $dlg.ShowDialog()
 if ($result -eq [System.Windows.Forms.DialogResult]::OK) {
@@ -467,14 +788,14 @@ if ($result -eq [System.Windows.Forms.DialogResult]::OK) {
       });
       return output ? output.trim() : null;
     } else if (platform === 'darwin') {
-      const appleScript = 'POSIX path of (choose file with prompt "Select Attendance Spreadsheet" of type {"xlsx", "xls"})';
+      const appleScript = `POSIX path of (choose file with prompt "${title}" of type {"xlsx", "xls"})`;
       const output = execFileSync('osascript', ['-e', appleScript], {
         encoding: 'utf8',
         stdio: ['ignore', 'pipe', 'ignore']
       });
       return output ? output.trim() : null;
     } else {
-      const output = execFileSync('zenity', ['--file-selection', '--file-filter=Excel files | *.xlsx *.xls', '--title=Select Attendance Spreadsheet'], {
+      const output = execFileSync('zenity', ['--file-selection', '--file-filter=Excel files | *.xlsx *.xls', `--title=${title}`], {
         encoding: 'utf8',
         stdio: ['ignore', 'pipe', 'ignore']
       });
@@ -517,6 +838,25 @@ function importFileToInput(srcPath) {
   };
 }
 
+// Copy / import a selected template into ./digi/
+function importTemplateToDigi(srcPath) {
+  ensureDirs();
+  const fileName = path.basename(srcPath);
+  const destPath = path.join(DIGI_DIR, fileName);
+
+  if (path.resolve(srcPath) !== path.resolve(destPath)) {
+    try {
+      fs.copyFileSync(srcPath, destPath);
+      logSuccess(`Imported template "${fileName}" into ./digi/ folder!`);
+      return destPath;
+    } catch (err) {
+      logWarn(`Could not copy into ./digi/ (${err.message}). Using original path.`);
+      return srcPath;
+    }
+  }
+  return destPath;
+}
+
 // Interactive prompt for file selection when needed
 async function promptForFile(lines) {
   logHeader('Select Attendance Spreadsheet');
@@ -543,7 +883,7 @@ async function promptForFile(lines) {
       }
     } else {
       console.log(`${colors.dim}Opening file picker dialog... Please select your file in the window.${colors.reset}`);
-      const picked = openFileDialog();
+      const picked = openFileDialog('Select Attendance Spreadsheet');
       if (picked && fs.existsSync(picked) && fs.statSync(picked).isFile()) {
         return picked;
       }
@@ -575,6 +915,7 @@ Options:
   -y, --year <2|3>            Academic year (2 for 2nd year batched, 3 for 3rd year standard)
   -b, --batch <1|2|all>       Batch selection for 2nd year (1, 2, or all)
   -v, --vertical <code/name>  Sheet/vertical code (e.g., AD, BW, CD, CS, DMA, EI, FW, ALL)
+  -t, --template <path>       DigiCampus upload template Excel file (.xlsx)
   -a, --all                   Export all instructional dates automatically
   -d, --date <YYYY-MM-DD>     Export a single specific date
   --latest                    Export the latest instructional date
@@ -602,6 +943,7 @@ Running without options launches the interactive CLI wizard.
   });
   const lines = rl[Symbol.asyncIterator]();
 
+  // 1. Select / Import Input Attendance File
   if (flags.file) {
     const directPath = path.resolve(process.cwd(), flags.file);
     const inInputPath = path.join(INPUT_DIR, flags.file);
@@ -620,7 +962,6 @@ Running without options launches the interactive CLI wizard.
     const inputFiles = findInputFiles();
 
     if (inputFiles.length === 0) {
-      // Auto prompt to browse or drag & drop when input/ is empty
       const chosen = await promptForFile(lines);
       selectedFileObj = importFileToInput(chosen);
     } else if (inputFiles.length === 1) {
@@ -655,19 +996,117 @@ Running without options launches the interactive CLI wizard.
     }
   }
 
-  const inputFilePath = selectedFileObj.fullPath;
-  logInfo(`Reading attendance workbook: ${colors.bold}${selectedFileObj.relativePath || selectedFileObj.name}${colors.reset}`);
-  
-  let workbook;
+  // 2. Academic Year Selection (needed for cohort batch separation during normalization)
+  let selectedYear = null;
+  if (flags.year) {
+    const yStr = String(flags.year).trim();
+    if (yStr.startsWith('2')) selectedYear = 2;
+    else if (yStr.startsWith('3')) selectedYear = 3;
+  }
+
+  if (!selectedYear) {
+    logHeader('Select Academic Year');
+    console.log('Choose student cohort / year:\n');
+    console.log(`  [${colors.bold}1${colors.reset}] ${colors.green}3rd Year${colors.reset} (Standard whole class / single list) [Default]`);
+    console.log(`  [${colors.bold}2${colors.reset}] ${colors.cyan}2nd Year${colors.reset} (Batched: Batch 1 & Batch 2 separated by empty row)`);
+
+    const yearAns = (await askQuestion(lines, `\nSelect year [1 or 2] (default: 1): `)).trim();
+    if (yearAns === '2' || yearAns.toLowerCase().startsWith('2')) {
+      selectedYear = 2;
+    } else {
+      selectedYear = 3;
+    }
+  }
+
+  // 3. DigiCampus Template Upload / Selection
+  let parsedTemplate = null;
+  let templateFilePath = null;
+
+  if (flags.template) {
+    const directT = path.resolve(process.cwd(), flags.template);
+    const inDigiT = path.join(DIGI_DIR, flags.template);
+    if (fs.existsSync(directT) && fs.statSync(directT).isFile()) {
+      templateFilePath = directT;
+    } else if (fs.existsSync(inDigiT) && fs.statSync(inDigiT).isFile()) {
+      templateFilePath = inDigiT;
+    } else {
+      logWarn(`Template file "${flags.template}" not found. Proceeding without template.`);
+    }
+  } else {
+    logHeader('DigiCampus Upload Template');
+    console.log('Upload a DigiCampus template (Upload_Attendance.xlsx) to match exact student rosters:');
+    console.log(`  [${colors.bold}1${colors.reset}] ${colors.green}Browse template file${colors.reset} (Opens file picker window) [Default]`);
+    console.log(`  [${colors.bold}2${colors.reset}] ${colors.cyan}Enter or drag & drop template path${colors.reset}`);
+    console.log(`  [${colors.bold}3${colors.reset}] ${colors.dim}Skip template${colors.reset} (Export directly from attendance sheet roster)\n`);
+
+    const tChoice = (await askQuestion(lines, `Choose option [1-3] (default: 1): `)).trim();
+
+    if (tChoice === '2') {
+      const manualT = await askQuestion(lines, 'Enter or drag & drop template file path: ');
+      const cleanedT = cleanPath(manualT);
+      if (cleanedT && fs.existsSync(cleanedT) && fs.statSync(cleanedT).isFile()) {
+        templateFilePath = cleanedT;
+      } else if (cleanedT) {
+        logWarn(`Template file not found at "${cleanedT}". Continuing without template.`);
+      }
+    } else if (tChoice === '3' || tChoice.toLowerCase() === 's' || tChoice.toLowerCase() === 'skip') {
+      logInfo('Skipping DigiCampus template. Exporting directly from attendance sheet roster.');
+    } else {
+      // Default: 1 (Browse dialog)
+      console.log(`${colors.dim}Opening file picker dialog... Please select your DigiCampus template.${colors.reset}`);
+      const pickedT = openFileDialog('Select DigiCampus Template (Upload_Attendance.xlsx)');
+      if (pickedT && fs.existsSync(pickedT) && fs.statSync(pickedT).isFile()) {
+        templateFilePath = pickedT;
+      } else {
+        logInfo('No template file selected. Continuing with attendance sheet roster.');
+      }
+    }
+  }
+
+  if (templateFilePath) {
+    const importedTPath = importTemplateToDigi(templateFilePath);
+    parsedTemplate = loadTemplate(importedTPath);
+    if (parsedTemplate && parsedTemplate.students.length > 0) {
+      logSuccess(`Loaded DigiCampus template with ${colors.bold}${parsedTemplate.students.length}${colors.reset} students.`);
+    } else {
+      logWarn('Uploaded template contained no student records. Continuing with attendance sheet roster.');
+      parsedTemplate = null;
+    }
+  }
+
+  // 4. Normalization Engine: Standardize uploaded spreadsheet
+  logHeader('Standardizing & Normalizing Attendance Sheet');
+  logInfo(`Reading raw workbook: ${colors.bold}${selectedFileObj.relativePath || selectedFileObj.name}${colors.reset}`);
+
+  let rawWorkbook;
   try {
-    workbook = xlsx.readFile(inputFilePath, { cellDates: false });
+    rawWorkbook = xlsx.readFile(selectedFileObj.fullPath, { cellDates: false });
   } catch (err) {
     logError(`Failed to read Excel file: ${err.message}`);
     rl.close();
     process.exit(1);
   }
 
-  // Discover all vertical sheets
+  const { normWb, sheetCount } = createNormalizedWorkbook(rawWorkbook, selectedYear);
+  if (sheetCount === 0) {
+    logError('No valid vertical attendance sheets with students and dates could be normalized.');
+    rl.close();
+    process.exit(1);
+  }
+
+  const normBaseName = path.basename(selectedFileObj.name, path.extname(selectedFileObj.name));
+  const normFileName = `Normalized_${normBaseName}.xlsx`;
+  const normFilePath = path.join(NORMALIZED_DIR, normFileName);
+  xlsx.writeFile(normWb, normFilePath);
+
+  logSuccess(`Standardized ${sheetCount} vertical sheet(s) into default format!`);
+  logSuccess(`Saved clean normalized workbook to: ${colors.bold}normalized/${normFileName}${colors.reset}`);
+  logInfo(`Using standardized workbook for all export operations.`);
+
+  // Use the clean, standardized workbook as the active workbook
+  const workbook = normWb;
+
+  // 5. Discover all vertical sheets from normalized workbook
   const verticalSheets = [];
   const excludedSheets = ['pivot table', 'summary', 'template', 'students data'];
 
@@ -683,34 +1122,12 @@ Running without options launches the interactive CLI wizard.
   }
 
   if (verticalSheets.length === 0) {
-    logError('No valid vertical attendance sheets with students and date columns were found.');
+    logError('No valid vertical attendance sheets were found in the standardized workbook.');
     rl.close();
     process.exit(1);
   }
 
-  // Academic Year Selection (2nd Year vs 3rd Year)
-  let selectedYear = null;
-  if (flags.year) {
-    const yStr = String(flags.year).trim();
-    if (yStr.startsWith('2')) selectedYear = 2;
-    else if (yStr.startsWith('3')) selectedYear = 3;
-  }
-
-  if (!selectedYear) {
-    logHeader('Select Academic Year');
-    console.log('Choose student cohort / year:\n');
-    console.log(`  [${colors.bold}1${colors.reset}] ${colors.green}3rd Year${colors.reset} (Standard whole class / single list) [Default]`);
-    console.log(`  [${colors.bold}2${colors.reset}] ${colors.cyan}2nd Year${colors.reset} (Batched: Batch 1 & Batch 2 separated by empty row)`);
-
-    const yearAns = (await askQuestion(lines, `\nSelect year [1 or 2] (default: 1): `)).trim();
-    if (yearAns === '2' || yearAns.toLowerCase().startsWith('2')) {
-      selectedYear = 2;
-    } else {
-      selectedYear = 3;
-    }
-  }
-
-  // Batch selection for 2nd Year
+  // 6. Batch selection for 2nd Year
   let selectedBatch = null; // 1, 2, or 'BOTH'
   if (selectedYear === 2) {
     if (flags.batch) {
@@ -745,7 +1162,7 @@ Running without options launches the interactive CLI wizard.
     logInfo(`Cohort: ${colors.bold}3rd Year (Standard Whole Class)${colors.reset}`);
   }
 
-  // Vertical Selection
+  // 7. Vertical Selection
   let chosenVertical = null;
   let processAll = false;
 
@@ -764,7 +1181,6 @@ Running without options launches the interactive CLI wizard.
     }
   }
 
-  // Interactive selection if not resolved from flags
   if (!chosenVertical && !processAll) {
     logHeader('Select Vertical');
     console.log('Available verticals found in the spreadsheet:\n');
@@ -817,7 +1233,6 @@ Running without options launches the interactive CLI wizard.
 
   const targets = processAll ? verticalSheets : [chosenVertical];
 
-  // Helper to get dates for chosen cohort & batch
   function getDatesForVertical(vert) {
     if (selectedYear === 2) {
       if (selectedBatch === 1) return vert.batch1Dates;
@@ -827,7 +1242,7 @@ Running without options launches the interactive CLI wizard.
     return vert.dates;
   }
 
-  // Date selection logic
+  // 8. Date selection logic
   let dateFilterMode = 'ALL';
   let singleDateVal = flags.date || null;
   let dateRangeVal = null;
@@ -872,7 +1287,7 @@ Running without options launches the interactive CLI wizard.
 
   rl.close();
 
-  // Process selected verticals
+  // 9. Process selected verticals and generate export files
   logHeader('Processing & Generating Export Files');
 
   for (const vert of targets) {
@@ -909,7 +1324,7 @@ Running without options launches the interactive CLI wizard.
       if (dateFilterMode === 'SINGLE' && singleDateVal) {
         datesToExport = datesToExport.filter(d => d.dateStr === singleDateVal);
         if (datesToExport.length === 0) {
-          logWarn(`Date ${singleDateVal} was not found among instructional dates for ${vert.sheetName}${batchTag}.`);
+          logWarn(`Date ${singleDateVal} was not found among recorded dates for ${vert.sheetName}${batchTag}.`);
           const fallbackCol = vert.allDateColumns.find(d => d.dateStr === singleDateVal);
           if (fallbackCol) {
             datesToExport = [{ ...fallbackCol, presentCount: 0, absentCount: 0, totalStudents: batchItem.students.length }];
@@ -933,13 +1348,12 @@ Running without options launches the interactive CLI wizard.
 
       const rows = vert.rows || xlsx.utils.sheet_to_json(workbook.Sheets[vert.sheetName], { header: 1, raw: true, defval: null });
       const zip = new AdmZip();
-
       const yearTag = selectedYear === 2 ? '2nd_Year' : '3rd_Year';
 
       for (const dateItem of datesToExport) {
         const batchFileSuffix = batchItem.label ? `_${batchItem.label}` : '';
         const fileName = `${dateItem.dateStr}_Upload_Attendance_${vert.sheetName}${batchFileSuffix}.xlsx`;
-        const outWb = generateExportWorkbook(vert, dateItem, rows, batchItem.students);
+        const outWb = generateExportWorkbook(vert, dateItem, rows, batchItem.students, parsedTemplate);
         const fileBuffer = xlsx.write(outWb, { type: 'buffer', bookType: 'xlsx' });
         zip.addFile(fileName, fileBuffer);
       }
@@ -950,7 +1364,7 @@ Running without options launches the interactive CLI wizard.
       const zipFilePath = path.join(OUTPUT_DIR, zipFileName);
       zip.writeZip(zipFilePath);
 
-      // Clean up any previously created unzipped folder for this vertical if it exists
+      // Clean up legacy unzipped folder if any existed
       const oldVertFolder = path.join(OUTPUT_DIR, zipBaseName);
       if (fs.existsSync(oldVertFolder) && fs.statSync(oldVertFolder).isDirectory()) {
         fs.rmSync(oldVertFolder, { recursive: true, force: true });
@@ -966,7 +1380,8 @@ Running without options launches the interactive CLI wizard.
 
   // Final summary
   logHeader('Export Complete');
-  console.log(`Check the ${colors.bold}${colors.cyan}./output${colors.reset} directory for all generated files and ZIP archives.\n`);
+  console.log(`Normalized master file: ${colors.bold}${colors.cyan}./normalized/${normFileName}${colors.reset}`);
+  console.log(`DigiCampus export archives: ${colors.bold}${colors.green}./output/*.zip${colors.reset}\n`);
 }
 
 main().catch(err => {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mrselva/attendance",
-  "version": "1.1.3",
+  "version": "1.2.0",
   "description": "Extract multi-sheet attendance data and export formatted Excel sheets with ZIP archives",
   "main": "index.js",
   "scripts": {