israeli-banks-actual-budget-importer 1.7.5 → 1.8.0

package/CHANGELOG.md

@@ -1,3 +1,17 @@
+# [1.8.0](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/compare/v1.7.6...v1.8.0) (2026-01-03)
+
+
+### Features
+
+* **config:** add targets/accounts mapping and update reconciliation docs ([9f56fe0](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/commit/9f56fe0fa984304186af4c95175a2e3ceb6b83b8))
+
+## [1.7.6](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/compare/v1.7.5...v1.7.6) (2025-12-04)
+
+
+### Bug Fixes
+
+* update transaction scraping start date to two years ago for improved data accuracy ([a08e3e6](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/commit/a08e3e699650844ed1015879355bdd9d157e2c60))
+
 ## [1.7.5](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/compare/v1.7.4...v1.7.5) (2025-12-04)
 
 

package/README.md

@@ -17,7 +17,9 @@ This project provides an importer from Israeli banks (via [israeli-bank-scrapers
 
 4. **Reconciliation:** Optional reconciliation to adjust account balances automatically.
 
-5. **Concurrent Processing:** Uses a queue (via [p-queue](https://www.npmjs.com/package/p-queue)) to manage scraping tasks concurrently.
+5. **Credit Card / Multi-Account Mapping (Targets):** Supports mapping multiple scraped accounts/cards into one Actual account, or mapping each scraped card into its own Actual account (via `targets` and `accounts`).
+
+6. **Concurrent Processing:** Uses a queue (via [p-queue](https://www.npmjs.com/package/p-queue)) to manage scraping tasks concurrently.
 
 ## Installation
 
@@ -42,25 +44,176 @@ services:
 
 ## Configuration
 
-The application configuration is defined using JSON and validated against a schema. The key configuration file is `config.json` and its schema is described in `config.schema.json`.
+The application configuration is defined using JSON and validated against a schema.
+The main configuration file is `config.json`.
+
+The configuration has **two independent top-level sections**:
+1. `actual`: Configures the Actual Budget connection.
+2. `banks`: Configures bank scrapers and account mappings.
+
+---
+
+### 1) `actual` configuration
+
+This section configures the connection to your Actual Budget server and budget.
+It is **always required**, regardless of how you configure banks or targets.
+
+```json
+{
+  "actual": {
+    "init": {
+      "dataDir": "./data",
+      "password": "your_actual_password",
+      "serverURL": "https://your-actual-server.com"
+    },
+    "budget": {
+      "syncId": "your_sync_id",
+      "password": "your_budget_password"
+    }
+  }
+}
+```
+
+Nothing in this block changes when using `targets`, credit cards, or multi-account mappings.
+
+---
+
+### 2) `banks` configuration
+
+The `banks` section defines:
+- Which banks to scrape
+- The credentials for each bank
+- How scraped accounts/cards are mapped into Actual accounts
+
+Each bank entry includes the credentials required by `israeli-bank-scrapers`
+(e.g. `userCode`, `username`, `password`, etc.) and supports **multiple mapping modes**.
 
-### Configuration Structure
+#### Using `targets` (recommended)
 
-- **actual:**  
-  Contains settings for the Actual API integration:
-  - `init`: Initialization parameters (e.g., server URL, password).
-  - `budget`: Contains properties like `syncId` and `password` for synchronizing budgets.
+A single bank scrape (for example `visaCal`) may return **multiple accounts/cards**.
+Different users model these differently in Actual, so the importer supports `targets`.
 
-- **banks:**  
-  Defines bank-specific settings for each supported bank. Each entry typically requires:
-  - `actualAccountId`: The account identifier in Actual.
-  - `password`: The bank account password.
-  - Additional properties (e.g., `userCode`, `username`, or other bank-specific credentials) as required.
-  - `reconcile` (optional): A flag to enable balance reconciliation.
+Each **target** represents:
+- One Actual account
+- One or more scraped accounts/cards that feed into it
 
-Make sure your `config.json` follows the schema defined in `config.schema.json`.
+For each target:
+- Imported transactions = concatenation of transactions from selected cards
+- Reconciliation (if enabled) = sum of balances of selected cards
+  (only cards with a valid numeric balance are included)
 
-Example snippet:
+---
+
+#### Example A: One Actual account for all VisaCal cards (consolidated)
+
+```json
+{
+  "actual": {
+    "init": {
+      "dataDir": "./data",
+      "password": "your_actual_password",
+      "serverURL": "https://your-actual-server.com"
+    },
+    "budget": {
+      "syncId": "your_sync_id",
+      "password": "your_budget_password"
+    }
+  },
+  "banks": {
+    "visaCal": {
+      "username": "bank_username",
+      "password": "bank_password",
+      "targets": [
+        {
+          "actualAccountId": "actual-creditcards-all",
+          "reconcile": true,
+          "accounts": "all"
+        }
+      ]
+    }
+  }
+}
+```
+
+---
+
+#### Example B: One Actual account per VisaCal card (separate accounts)
+
+```json
+{
+  "actual": {
+    "init": {
+      "dataDir": "./data",
+      "password": "your_actual_password",
+      "serverURL": "https://your-actual-server.com"
+    },
+    "budget": {
+      "syncId": "your_sync_id",
+      "password": "your_budget_password"
+    }
+  },
+  "banks": {
+    "visaCal": {
+      "username": "bank_username",
+      "password": "bank_password",
+      "targets": [
+        {
+          "actualAccountId": "actual-card-8538",
+          "reconcile": true,
+          "accounts": ["8538"]
+        },
+        {
+          "actualAccountId": "actual-card-7697",
+          "reconcile": true,
+          "accounts": ["7697"]
+        }
+      ]
+    }
+  }
+}
+```
+
+---
+
+#### Example C: Grouped cards into a single Actual account (subset)
+
+```json
+{
+  "actual": {
+    "init": {
+      "dataDir": "./data",
+      "password": "your_actual_password",
+      "serverURL": "https://your-actual-server.com"
+    },
+    "budget": {
+      "syncId": "your_sync_id",
+      "password": "your_budget_password"
+    }
+  },
+  "banks": {
+    "visaCal": {
+      "username": "bank_username",
+      "password": "bank_password",
+      "targets": [
+        {
+          "actualAccountId": "actual-cal-primary",
+          "reconcile": true,
+          "accounts": ["8538", "7697"]
+        }
+      ]
+    }
+  }
+}
+```
+
+---
+
+## Legacy configuration (single Actual account per bank)
+
+This configuration style is **fully supported for backward compatibility**,
+but does **not** allow fine-grained control over multiple cards/accounts.
+
+It maps all scraped accounts from the bank into a single Actual account.
 
 ```json
 {
@@ -87,11 +240,19 @@ Example snippet:
       "username": "bank_username",
       "password": "bank_password"
     }
-    // Additional bank configurations go here...
   }
 }
 ```
 
+---
+
+## Notes
+
+- The `actual` block is **always required** and independent of bank configuration.
+- `targets` are optional but strongly recommended for credit-card providers.
+- Duplicate transactions are prevented using a stable `imported_id`.
+- Credit card balances are often negative; reconciliation uses the values as returned by the bank.
+
 ## License
 
 This project is open-source. Please see the [LICENSE](./LICENSE) file for licensing details.

package/package.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.7.5",
+  "version": "1.8.0",
   "name": "israeli-banks-actual-budget-importer",
   "module": "index.ts",
   "type": "module",
@@ -16,7 +16,7 @@
     "@semantic-release/npm": "^13.1.2",
     "@semantic-release/release-notes-generator": "^14.1.0",
     "@types/lodash": "^4.17.21",
-    "@types/papaparse": "^5.5.0",
+    "@types/papaparse": "^5.5.2",
     "bun-types": "latest",
     "papaparse": "^5.5.3",
     "semantic-release": "^25.0.2",
@@ -25,14 +25,14 @@
   },
   "packageManager": "yarn@4.12.0",
   "dependencies": {
-    "@actual-app/api": "^25.11.0",
+    "@actual-app/api": "^25.12.0",
+    "@tomerh2001/israeli-bank-scrapers": "latest",
     "cronstrue": "^3.9.0",
-    "israeli-bank-scrapers": "^6.2.5",
     "lodash": "^4.17.21",
     "moment": "^2.30.1",
     "mute-stdout": "^2.0.0",
     "node-cron": "^4.2.1",
     "p-queue": "^9.0.1",
-    "tsx": "^4.20.6"
+    "tsx": "^4.21.0"
   }
 }

package/src/config.d.ts

@@ -18,7 +18,48 @@ export type ConfigActualBudget = {
 
 export type ConfigBanks = Partial<Record<CompanyTypes, ConfigBank>>;
 
-export type ConfigBank = ScraperCredentials & {
+/**
+ * A single "import target" inside Actual.
+ * One target maps one Actual account to one or more scraped accounts/cards.
+ */
+export type ConfigBankTarget = {
+	/**
+	 * Actual Budget account ID to import into and (optionally) reconcile against.
+	 */
 	actualAccountId: string;
+
+	/**
+	 * If true, create/update a reconciliation transaction to match the scraped balance.
+	 */
+	reconcile?: boolean;
+
+	/**
+	 * Which scraped accounts (by accountNumber) should be included in this target.
+	 * - "all": include all scraped accounts with usable data (final selection logic lives in code).
+	 * - string[]: include only those accountNumbers.
+	 *
+	 * If omitted, default behavior should match legacy behavior:
+	 * - treat as "all" for import, and for reconciliation use the first usable balance
+	 *   (you'll refine this in the implementation files).
+	 */
+	accounts?: 'all' | string[];
+};
+
+/**
+ * Bank config remains compatible with existing configs:
+ * - Legacy: actualAccountId + reconcile at top level
+ * - New: targets[]
+ */
+export type ConfigBank = ScraperCredentials & {
+	/**
+	 * New preferred configuration: one bank can have multiple import targets.
+	 */
+	targets?: ConfigBankTarget[];
+
+	/**
+	 * Legacy single-target fields (backward compatible).
+	 * If targets is provided, these should be ignored by runtime logic.
+	 */
+	actualAccountId?: string;
 	reconcile?: boolean;
 };

package/src/index.ts

@@ -4,7 +4,7 @@
 /* eslint-disable no-await-in-loop */
 
 import process from 'node:process';
-import {type CompanyTypes} from 'israeli-bank-scrapers';
+import {type CompanyTypes} from '@tomerh2001/israeli-bank-scrapers';
 import _ from 'lodash';
 import actual from '@actual-app/api';
 import Queue from 'p-queue';

package/src/utils.d.ts

@@ -1,8 +1,7 @@
-import type {ScraperCredentials, CompanyTypes} from 'israeli-bank-scrapers';
-import type actual from '@actual-app/api';
-import {type ConfigBank} from '../config.js';
+import type {CompanyTypes} from 'israeli-bank-scrapers';
+import type {ConfigBank} from '../config.js';
 
 export type ScrapeTransactionsContext = {
 	companyId: CompanyTypes;
 	bank: ConfigBank;
-};
+};

package/src/utils.ts

@@ -4,29 +4,99 @@
 /* eslint-disable @typescript-eslint/no-unsafe-call */
 
 import process from 'node:process';
-import {createScraper, type ScraperCredentials} from 'israeli-bank-scrapers';
+import {createScraper, type ScraperCredentials} from '@tomerh2001/israeli-bank-scrapers';
 import _ from 'lodash';
 import moment from 'moment';
 import actual from '@actual-app/api';
-import {type PayeeEntity, type TransactionEntity} from '@actual-app/api/@types/loot-core/types/models';
+import {type PayeeEntity, type TransactionEntity} from '@actual-app/api/@types/loot-core/src/types/models';
 import stdout from 'mute-stdout';
 import {type ScrapeTransactionsContext} from './utils.d';
 
+// If you exported these from your config types file, import them from there instead.
+type AccountsSelector = 'all' | string[];
+type BankTarget = {
+	actualAccountId: string;
+	reconcile?: boolean;
+	accounts?: AccountsSelector;
+};
+
+function isFiniteNumber(x: unknown): x is number {
+	return typeof x === 'number' && Number.isFinite(x);
+}
+
+function stripUndefined<T extends Record<string, any>>(object: T): T {
+	return Object.fromEntries(Object.entries(object).filter(([, v]) => v !== undefined)) as T;
+}
+
+function normalizeTargets(bank: any): BankTarget[] {
+	// New config: targets[]
+	if (Array.isArray(bank?.targets) && bank.targets.length > 0) {
+		return bank.targets
+			.filter((t: any) => t?.actualAccountId)
+			.map((t: any) => ({
+				actualAccountId: t.actualAccountId,
+				reconcile: Boolean(t.reconcile),
+				accounts: t.accounts,
+			}));
+	}
+
+	// Legacy config: actualAccountId + reconcile
+	if (bank?.actualAccountId) {
+		return [{
+			actualAccountId: bank.actualAccountId,
+			reconcile: Boolean(bank.reconcile),
+			// Legacy behavior did not support selecting accounts; treat as "all".
+			accounts: 'all',
+		}];
+	}
+
+	return [];
+}
+
+function selectScraperAccounts(
+	allAccounts: any[] | undefined,
+	selector: AccountsSelector | undefined,
+) {
+	const accounts = allAccounts ?? [];
+	if (selector === undefined || selector === 'all') {
+		return accounts;
+	}
+
+	const set = new Set(selector);
+	return accounts.filter(a => set.has(String(a.accountNumber)));
+}
+
+function stableImportedId(companyId: string, accountNumber: string | undefined, txn: any) {
+	// Prefer scraper identifier if present; fall back to a deterministic composite.
+	const idPart
+		= txn?.identifier
+			?? `${moment(txn?.date).format('YYYY-MM-DD')}:${txn?.chargedAmount}:${txn?.description ?? ''}:${txn?.memo ?? ''}`;
+
+	// AccountNumber is important once multiple cards are aggregated into one Actual account.
+	return `${companyId}:${accountNumber ?? 'unknown'}:${idPart}`;
+}
+
 export async function scrapeAndImportTransactions({companyId, bank}: ScrapeTransactionsContext) {
 	function log(status: any, other?: Record<string, unknown>) {
 		console.debug({bank: companyId, status, ...other});
 	}
 
 	try {
+		const targets = normalizeTargets(bank);
+		if (targets.length === 0) {
+			throw new Error(`No targets configured for ${companyId}. Provide bank.actualAccountId (legacy) or bank.targets[].actualAccountId.`);
+		}
+
 		const scraper = createScraper({
 			companyId,
-			startDate: moment().subtract(12, 'month').toDate(),
+			startDate: moment().subtract(2, 'years').toDate(),
 			// ExecutablePath: '/opt/homebrew/bin/chromium',
 			args: ['--user-data-dir=./chrome-data'],
 			additionalTransactionInformation: true,
 			verbose: process.env?.VERBOSE === 'true',
 			showBrowser: process.env?.SHOW_BROWSER === 'true',
 		});
+
 		scraper.onProgress((_companyId, payload) => {
 			log(payload.type);
 		});
@@ -36,95 +106,136 @@ export async function scrapeAndImportTransactions({companyId, bank}: ScrapeTrans
 			throw new Error(`Failed to scrape (${result.errorType}): ${result.errorMessage}`);
 		}
 
-		const transactions = _(result.accounts)
-			.filter(account => account.txns.length > 0)
-			.flatMap(account => account.txns)
-			.value();
+		log('ACCOUNTS', {
+			accounts: result.accounts?.map(x => ({
+				accountNumber: x.accountNumber,
+				balance: x.balance,
+				txns: x.txns.length,
+			})),
+		});
 
-		for (const account of result.accounts!) {
-			if (account.txns.length <= 0) {
+		const payees: PayeeEntity[] = await actual.getPayees();
+
+		// Process each target independently (supports per-card, per-company, and consolidated).
+		for (const target of targets) {
+			const selectedAccounts = selectScraperAccounts(result.accounts as any[], target.accounts);
+
+			// Transactions to import: selected accounts with txns.
+			const transactions = _(selectedAccounts)
+				.filter(a => Array.isArray(a.txns) && a.txns.length > 0)
+				.flatMap(a => a.txns.map((t: any) => ({txn: t, accountNumber: String(a.accountNumber)})))
+				.value();
+
+			if (transactions.length === 0) {
+				log('NO_TRANSACTIONS', {actualAccountId: target.actualAccountId});
+			} else {
+				const mappedTransactions = transactions.map(async ({txn, accountNumber}) => stripUndefined({
+					date: moment(txn.date).format('YYYY-MM-DD'),
+					amount: actual.utils.amountToInteger(txn.chargedAmount),
+					payee: _.find(payees, {name: txn.description})?.id ?? (await actual.createPayee({name: txn.description})),
+					imported_payee: txn.description,
+					notes: txn.memo,
+					imported_id: stableImportedId(companyId, accountNumber, txn),
+				}));
+
+				stdout.mute();
+				const importResult = await actual.importTransactions(
+					target.actualAccountId,
+					await Promise.all(mappedTransactions),
+					{defaultCleared: true},
+				);
+				stdout.unmute();
+
+				if (_.isEmpty(importResult)) {
+					console.error('Errors', importResult.errors);
+					throw new Error('Failed to import transactions');
+				} else {
+					log('IMPORTED', {actualAccountId: target.actualAccountId, transactions: importResult.added.length});
+				}
+			}
+
+			if (!target.reconcile) {
 				continue;
 			}
-		}
 
-		const accountBalance = result.accounts![0].balance!;
-		const payees: PayeeEntity[] = await actual.getPayees();
+			// Reconciliation balance: sum finite balances of selected accounts.
+			const reconAccounts = selectedAccounts
+				.filter(a => isFiniteNumber(a?.balance))
+				.map(a => ({accountNumber: String(a.accountNumber), balance: a.balance as number}));
 
-		const mappedTransactions = transactions.map(async x => ({
-			date: moment(x.date).format('YYYY-MM-DD'),
-			amount: actual.utils.amountToInteger(x.chargedAmount),
-			payee: _.find(payees, {name: x.description})?.id ?? (await actual.createPayee({name: x.description})),
-			imported_payee: x.description,
-			notes: x.memo,
-			imported_id: `${x.identifier}-${moment(x.date).format('YYYY-MM-DD HH:mm:ss')}`,
-		}));
-
-		stdout.mute();
-		const importResult = await actual.importTransactions(bank.actualAccountId, await Promise.all(mappedTransactions), {defaultCleared: true});
-		stdout.unmute();
-
-		if (_.isEmpty(importResult)) {
-			console.error('Errors', importResult.errors);
-			throw new Error('Failed to import transactions');
-		} else {
-			log('IMPORTED', {transactions: importResult.added.length});
-		}
+			if (reconAccounts.length === 0) {
+				log('RECONCILE_SKIPPED_NO_BALANCES', {
+					actualAccountId: target.actualAccountId,
+					selectedAccounts: selectedAccounts.map(a => a?.accountNumber),
+				});
+				continue;
+			}
 
-		if (!bank.reconcile) {
-			return;
-		}
+			const scraperBalance = reconAccounts.reduce((sum, a) => sum + a.balance, 0);
 
-		const currentBalance = actual.utils.integerToAmount(await actual.getAccountBalance(bank.actualAccountId));
-		const balanceDiff = accountBalance - currentBalance;
+			log('RECONCILE_INPUT', {
+				actualAccountId: target.actualAccountId,
+				accounts: reconAccounts,
+				balance: scraperBalance,
+			});
 
-		// Use a stable imported_id per account so we can find and update/delete the same
-		// reconciliation transaction instead of creating a new one every run.
-		const reconciliationImportedId = `reconciliation-${bank.actualAccountId}`;
+			const currentBalance = actual.utils.integerToAmount(await actual.getAccountBalance(target.actualAccountId));
+			const balanceDiff = scraperBalance - currentBalance;
 
-		// Fetch all transactions for this account and look for an existing reconciliation.
-		// Use a wide date range so we always find it if it exists.
-		const allAccountTxns: TransactionEntity[] = await actual.getTransactions(
-			bank.actualAccountId,
-			'2000-01-01',
-			moment().add(1, 'year').format('YYYY-MM-DD'),
-		);
+			// Stable imported_id per Actual account so we update the same reconciliation txn each run.
+			const reconciliationImportedId = `reconciliation-${target.actualAccountId}`;
 
-		const existingReconciliation = allAccountTxns.find(txn => txn.imported_id === reconciliationImportedId);
+			const allAccountTxns: TransactionEntity[] = await actual.getTransactions(
+				target.actualAccountId,
+				'2000-01-01',
+				moment().add(1, 'year').format('YYYY-MM-DD'),
+			);
 
-		// If balances are already in sync, no need to create/update reconciliation.
-		if (existingReconciliation && balanceDiff === 0) {
-			log('RECONCILIATION_NOT_NEEDED');
-			return;
-		}
+			const existingReconciliation = allAccountTxns.find(txn => txn.imported_id === reconciliationImportedId);
 
-		const reconciliationTxn = {
-			account: bank.actualAccountId,
-			date: moment().format('YYYY-MM-DD'),
-			amount: actual.utils.amountToInteger(balanceDiff),
-			payee: undefined,
-			imported_payee: 'Reconciliation',
-			notes: `Reconciliation from ${currentBalance.toLocaleString()} to ${accountBalance.toLocaleString()}`,
-			imported_id: reconciliationImportedId,
-		};
-
-		if (existingReconciliation) {
-			stdout.mute();
-			await actual.updateTransaction(existingReconciliation.id, reconciliationTxn);
-			stdout.unmute();
+			if (existingReconciliation && balanceDiff === 0) {
+				log('RECONCILIATION_NOT_NEEDED', {actualAccountId: target.actualAccountId});
+				continue;
+			}
 
-			log('RECONCILIATION_UPDATED', {from: currentBalance, to: accountBalance, diff: balanceDiff});
-			return;
-		}
+			const reconciliationTxn = stripUndefined({
+				account: target.actualAccountId,
+				date: moment().format('YYYY-MM-DD'),
+				amount: actual.utils.amountToInteger(balanceDiff),
+				payee: null, // IMPORTANT: never pass undefined to updateTransaction schema
+				imported_payee: 'Reconciliation',
+				notes: `Reconciliation from ${currentBalance.toLocaleString()} to ${scraperBalance.toLocaleString()}`,
+				imported_id: reconciliationImportedId,
+			});
+
+			if (existingReconciliation) {
+				stdout.mute();
+				await actual.updateTransaction(existingReconciliation.id, reconciliationTxn);
+				stdout.unmute();
+
+				log('RECONCILIATION_UPDATED', {
+					actualAccountId: target.actualAccountId,
+					from: currentBalance,
+					to: scraperBalance,
+					diff: balanceDiff,
+				});
+				continue;
+			}
 
-		// Create the reconciliation transaction for the first time
-		stdout.mute();
-		const reconciliationResult = await actual.importTransactions(bank.actualAccountId, [reconciliationTxn]);
-		stdout.unmute();
+			stdout.mute();
+			const reconciliationResult = await actual.importTransactions(target.actualAccountId, [reconciliationTxn]);
+			stdout.unmute();
 
-		if (!reconciliationResult || _.isEmpty(reconciliationResult.added)) {
-			console.error('Reconciliation errors', reconciliationResult?.errors);
-		} else {
-			log('RECONCILIATION_ADDED', {from: currentBalance, to: accountBalance, diff: balanceDiff});
+			if (!reconciliationResult || _.isEmpty(reconciliationResult.added)) {
+				console.error('Reconciliation errors', reconciliationResult?.errors);
+			} else {
+				log('RECONCILIATION_ADDED', {
+					actualAccountId: target.actualAccountId,
+					from: currentBalance,
+					to: scraperBalance,
+					diff: balanceDiff,
+				});
+			}
 		}
 	} catch (error) {
 		console.error('Error', companyId, error);