israeli-banks-actual-budget-importer 1.3.1 → 1.5.0

package/CHANGELOG.md

@@ -1,3 +1,22 @@
+# [1.5.0](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/compare/v1.4.0...v1.5.0) (2025-05-24)
+
+
+### Bug Fixes
+
+* update scraper configuration for executable path and user data directory ([7227ac4](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/commit/7227ac40be5c50c2e8bf95e201af1f260478846c))
+
+
+### Features
+
+* improved the logs ([3a5f1ac](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/commit/3a5f1aca53be85413225c9e0e75df29cbd70cf91))
+
+# [1.4.0](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/compare/v1.3.1...v1.4.0) (2025-05-24)
+
+
+### Features
+
+* improved the logs ([8ca8c61](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/commit/8ca8c616a394501412262243e98e535a9bace6ad))
+
 ## [1.3.1](https://github.com/tomerh2001/israeli-banks-actual-budget-importer/compare/v1.3.0...v1.3.1) (2025-05-23)
 
 

package/README.md

@@ -28,6 +28,7 @@ https://hub.docker.com/r/tomerh2001/israeli-banks-actual-budget-importer
 services:
   importer:
     image: tomerh2001/israeli-banks-actual-budget-importer:latest
+    restart: always
     cap_add:
       - SYS_ADMIN
     environment:
@@ -39,27 +40,6 @@ services:
       - ./chrome-data:/app/chrome-data # Optional (Used to solve 2FA issues like with hapoalim)
 ```
 
-### Prerequisites
-
-- **Node.js:** Make sure you have Node.js installed (a version that supports ES modules is recommended).
-- **Yarn:** This project uses Yarn as its package manager. Ensure Yarn is installed.
-- **TypeScript:** Installed via dependencies.
-
-### Steps
-
-1. **Clone the Repository:**
-
-   ```bash
-   git clone https://github.com/tomerh2001/israeli-banks-actual-budget-importer.git
-   cd israeli-banks-actual-budget-importer
-   ```
-
-2. **Install Dependencies:**
-
-   ```bash
-   yarn install
-   ```
-
 ## Configuration
 
 The application configuration is defined using JSON and validated against a schema. The key configuration file is `config.json` and its schema is described in `config.schema.json`.
@@ -112,78 +92,6 @@ Example snippet:
 }
 ```
 
-## Usage
-
-The project comes with scripts configured in the `package.json`. To start the importer:
-
-```bash
-yarn start
-```
-
-This command runs the script defined as:
-
-```json
-"start": "tsx src/index.ts"
-```
-
-It initializes the connection to the Actual API, downloads the budget, scrapes bank transactions, imports them, and if enabled, performs reconciliation. After processing all tasks in the queue, the application shuts down cleanly.
-
-## Development
-
-### Overview
-
-The **Israeli Banks Actual Budget Importer** is a Node.js project written in TypeScript designed to:
-
-- **Scrape transactions** using specialized bank scrapers.
-- **Import transactions** into the Actual budgeting system.
-- **Perform account reconciliation** by computing and importing balance differences.
-- **Facilitate scheduled imports** by managing concurrent scraping tasks with a queue.
-
-The main logic is found in `src/index.ts`, which initializes the Actual API, triggers the scraping process, imports transactions, and then gracefully shuts down.
-
-
-### Project Structure
-
-- **src/**  
-  Contains the TypeScript source code.
-  - `index.ts`: Entry point.
-  - `utils.ts`: Utility functions for scraping and importing transactions.
-  - `config.d.ts`, `utils.d.ts`: Type definitions.
-
-- **config.schema.json:**  
-  JSON schema to validate your configuration file.
-
-- **package.json:**  
-  Lists the dependencies, scripts, and project metadata.
-
-- **.github, .vscode:**  
-  Contain CI/CD and editor-specific settings.
-
-### TypeScript & Linting
-
-The project uses [XO](https://github.com/xojs/xo) for linting. Ensure you follow coding styles as enforced by XO.
-
-## Testing & Linting
-
-Run tests and linting with the following command:
-
-```bash
-yarn test
-```
-
-Ensure that your changes pass linting rules and tests before submitting any pull requests.
-
-## Contributing
-
-Contributions are welcome! To contribute:
-
-1. **Fork the Repository:** Create your own fork and clone it locally.
-2. **Create a New Branch:** Use descriptive branch names (e.g., `feature/new-scraper-support`).
-3. **Commit Your Changes:** Follow the commit message guidelines, especially if you are using semantic release.
-4. **Submit a Pull Request:** Ensure your code passes all tests and linting checks.
-
-Please also review our [CHANGELOG.md](./CHANGELOG.md) and [SECURITY.md](./SECURITY.md) for more context on versioning and security practices.
-
 ## License
 
 This project is open-source. Please see the [LICENSE](./LICENSE) file for licensing details.

package/compose.yml

@@ -5,12 +5,13 @@ services:
     cap_add:
       - SYS_ADMIN
     platform: linux/amd64
+    restart: always
     build:
       context: .
       dockerfile: Dockerfile
     environment:
       - TZ=Asia/Jerusalem
-      - SCHEDULE=0 0 * * *
+      # - SCHEDULE=0 */6 * * *
     volumes:
       - ./config.json:/app/config.json
       - ./cache:/app/cache

package/package.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.3.1",
+  "version": "1.5.0",
   "name": "israeli-banks-actual-budget-importer",
   "module": "index.ts",
   "type": "module",
@@ -21,15 +21,16 @@
     "papaparse": "^5.5.2",
     "semantic-release": "^24.2.3",
     "typescript": "^5.8.3",
-    "xo": "^0.60.0"
+    "xo": "^1.0.0"
   },
   "packageManager": "yarn@4.9.1",
   "dependencies": {
     "@actual-app/api": "^25.5.0",
     "cronstrue": "^2.61.0",
-    "israeli-bank-scrapers": "^5.4.7",
+    "israeli-bank-scrapers": "^6.0.0",
     "lodash": "^4.17.21",
     "moment": "^2.30.1",
+    "mute-stdout": "^2.0.0",
     "node-cron": "^4.0.7",
     "p-queue": "^8.1.0",
     "tsx": "^4.19.4"

package/src/index.ts

@@ -1,3 +1,4 @@
+/* eslint-disable @typescript-eslint/no-unsafe-call */
 /* eslint-disable unicorn/no-process-exit */
 
 /* eslint-disable no-await-in-loop */
@@ -11,6 +12,7 @@ import Queue from 'p-queue';
 import moment from 'moment';
 import cron, {type ScheduledTask, validate} from 'node-cron';
 import cronstrue from 'cronstrue';
+import stdout from 'mute-stdout';
 import config from '../config.json' assert {type: 'json'};
 import type {ConfigBank} from './config.d.ts';
 import {scrapeAndImportTransactions} from './utils.ts';
@@ -25,15 +27,20 @@ async function run() {
 		intervalCap: 10,
 	});
 
+	stdout.mute();
 	await actual.init(config.actual.init);
 	await actual.downloadBudget(config.actual.budget.syncId, config.actual.budget);
+	stdout.unmute();
 
 	for (const [companyId, bank] of _.entries(config.banks) as Array<[CompanyTypes, ConfigBank]>) {
 		await queue.add(async () => scrapeAndImportTransactions({companyId, bank}));
 	}
 
 	await queue.onIdle();
+
+	stdout.mute();
 	await actual.shutdown();
+	stdout.unmute();
 
 	console.log('Done');
 }

package/src/utils.ts

@@ -1,5 +1,7 @@
-/* eslint-disable @typescript-eslint/naming-convention */
 /* eslint-disable @typescript-eslint/no-unsafe-assignment */
+/* eslint-disable @typescript-eslint/naming-convention */
+/* eslint-disable @typescript-eslint/no-unsafe-call */
+
 /* eslint-disable import/extensions */
 /* eslint-disable n/file-extension-in-import */
 
@@ -9,21 +11,26 @@ import _ from 'lodash';
 import moment from 'moment';
 import actual from '@actual-app/api';
 import {type PayeeEntity, type TransactionEntity} from '@actual-app/api/@types/loot-core/types/models';
+import stdout from 'mute-stdout';
 import {type ScrapeTransactionsContext} from './utils.d';
 
 export async function scrapeAndImportTransactions({companyId, bank}: ScrapeTransactionsContext) {
+	function log(status: any, other?: Record<string, unknown>) {
+		console.debug({bank: companyId, status, ...other});
+	}
+
 	try {
 		const scraper = createScraper({
 			companyId,
 			startDate: moment().subtract(6, 'month').toDate(),
-			// ExecutablePath: '/opt/homebrew/bin/chromium',
-			args: ['--user-data-dir=/app/chrome-data'],
+			executablePath: '/opt/homebrew/bin/chromium',
+			args: ['--user-data-dir=./chrome-data'],
 			additionalTransactionInformation: true,
 			verbose: process.env?.VERBOSE === 'true',
 			showBrowser: process.env?.SHOW_BROWSER === 'true',
 		});
-		scraper.onProgress((companyId, payload) => {
-			console.debug('Progress', companyId, payload);
+		scraper.onProgress((_companyId, payload) => {
+			log(payload.type);
 		});
 
 		const result = await scraper.scrape(bank as ScraperCredentials);
@@ -42,11 +49,7 @@ export async function scrapeAndImportTransactions({companyId, bank}: ScrapeTrans
 			}
 		}
 
-		const accounts = await actual.getAccounts() as TransactionEntity[];
-		const account = _.find(accounts, {id: bank.actualAccountId})!;
 		const accountBalance = result.accounts![0].balance!;
-		console.log('Account', account, 'Balance', accountBalance);
-
 		const payees: PayeeEntity[] = await actual.getPayees();
 		const mappedTransactions = transactions.map(async x => ({
 			date: moment(x.date).format('YYYY-MM-DD'),
@@ -57,12 +60,15 @@ export async function scrapeAndImportTransactions({companyId, bank}: ScrapeTrans
 			imported_id: `${x.identifier}-${moment(x.date).format('YYYY-MM-DD HH:mm:ss')}`,
 		}));
 
+		stdout.mute();
 		const importResult = await actual.importTransactions(bank.actualAccountId, await Promise.all(mappedTransactions), {defaultCleared: true});
+		stdout.unmute();
+
 		if (_.isEmpty(importResult)) {
 			console.error('Errors', importResult.errors);
 			throw new Error('Failed to import transactions');
 		} else {
-			console.log('Imported', importResult.added, 'transactions');
+			log('IMPORTED', {transactions: importResult.added.length});
 		}
 
 		if (!bank.reconcile) {
@@ -75,7 +81,9 @@ export async function scrapeAndImportTransactions({companyId, bank}: ScrapeTrans
 			return;
 		}
 
-		console.log('Balance diff', balanceDiff);
+		log('RECONCILIATION', {from: currentBalance, to: accountBalance, diff: balanceDiff});
+
+		stdout.mute();
 		const reconciliationResult = await actual.importTransactions(bank.actualAccountId, [{
 			date: moment().format('YYYY-MM-DD'),
 			amount: actual.utils.amountToInteger(balanceDiff),
@@ -84,13 +92,17 @@ export async function scrapeAndImportTransactions({companyId, bank}: ScrapeTrans
 			notes: `Reconciliation from ${currentBalance.toLocaleString()} to ${accountBalance.toLocaleString()}`,
 			imported_id: `reconciliation-${moment().format('YYYY-MM-DD HH:mm:ss')}`,
 		}]);
+		stdout.unmute();
+
 		if (_.isEmpty(reconciliationResult)) {
 			console.error('Reconciliation errors', reconciliationResult.errors);
 		} else {
-			console.info('Added a reconciliation transaction from', currentBalance, 'to', accountBalance);
+			log('RECONCILIATION_ADDED', {transactions: reconciliationResult.added.length});
 		}
 	} catch (error) {
 		console.error('Error', companyId, error);
+	} finally {
+		log('DONE');
 	}
 }