tiro-notes 0.30.1-dev → 0.30.4-2.15

package/README.md

@@ -1,63 +1,30 @@
 # Tiro Notes
 
-An Open-source, cross-platform and extensible web markdown editor 
+An open-source, cross-platform markdown editor designed for extensibility & flexibility
 
-![Tiro S2](https://user-images.githubusercontent.com/2981891/161093353-e9c2df15-ffca-4d2f-839d-ced9ced659b1.jpg)
+🌐 website| 📘 guides| ⬇️ downloads| 💬 forum | 📢 news| Version |
+-|-|-|-|-|-
+[tiro-notes.org](https://tiro-notes.org)|[guides](https://tiro-notes.org/#/guide-custom-tags)|[download](https://tiro-notes.org/#/downloads)|[forum](https://github.com/dotgreg/tiro-notes/discussions)|[twitter](https://twitter.com/NotesTiro)| 0.30 (beta)
 
-Twitter News & Updates : https://twitter.com/NotesTiro
 
-## WHY
-
-Tiro Notes is designed to be a viable open source alternative to proprietary software like Evernote, OneNote etc. 
+![Tiro 0.30](https://user-images.githubusercontent.com/2981891/171833721-2acdc59d-8c21-4280-bd62-a78023731c22.png)
 
 Tiro Notes focus points are : 
 
-- 👑 **Open source & Open Data**: Code is open source, free forever. No database, no hidden ties. Markdown files only. You can edit and sync your files with external software as you need.
-
-- ☁️ / 🖥️ /  💻 / 📱 **Cross-platform & Web** : Windows, macOS, Android, Linux, Web. Start an App or with CLI and access it anywhere on your local network. 
-
-- 🏋 **Extensible** : create custom markdown tag like [[calendar]], [[uml]] and other within seconds. Use [[latex]]. Create your own tag with [[script]] and the client api
-
-## MENU
-* [Why](#why)
-* [Usage](#usage)
-  + [Desktop Applications](#desktop-applications)
-  + [Command Line](#command-line)
-* [Functionalities List](#functionalities-list)
-* [Documentation](#documentation)
-    + [Available Custom Tags](#available-custom-tags)
-    + [Add scripts in your note](#add-scripts-in-your-note)
-    + [Create your own custom tags](#create-your-own-custom-tags)
-      - [1) How to create a new custom tag [[mail-address]]](#1--how-to-create-a-new-custom-tag--)
-      - [2) Using {{InnerTag}}](#2--using---innertag--)
-      - [3) Add script logic in your tag](#3--add-script-logic-in-your-tag--)
-    + [Client API](#client-api)
-* [Faq](#faq)
-  + [How can I sync my notes folder on other devices?](#how-can-i-sync-my-notes-folder-on-other-devices)
-  + [Can I edit my notes on another application?](#can-i-edit-my-notes-on-another-application)
-  + [How can I access to Tiro on my device](#how-can-i-access-to-tiro-on-my-device)
-  + [What is the current status of Tiro Notes?](#what-is-the-current-status-of-tiro-notes)
+- 🏋 **Extensibility** : Tiro Notes uses a system of custom tags to extend its capabilities :
+  - Add [```[[latex]]```](https://tiro-notes.org/#/custom-tags?id=latex), [```[[uml]]```](https://tiro-notes.org/#/custom-tags?id=mermaid) or [```[[charts]]```](https://tiro-notes.org/#/custom-tags?id=mermaid), preview your pdf, mp4 or youtube links using [```[[view]]```](https://tiro-notes.org/#/custom-tags?id=view). Add [```[[spreadsheet]]```](https://tiro-notes.org/#/custom-tags?id=spreadsheet) logic to your markdown tables.
+  - Use Tiro as an app platform with a [```[[rss]]```](https://tiro-notes.org/#/custom-tags?id=rss) reader or a [```[[calendar]]```](https://tiro-notes.org/#/custom-tags?id=calendar) in it.
+  - [Create your own custom tags](https://tiro-notes.org/#/guide-custom-tags) and use the [Client API](https://tiro-notes.org/#/guide-custom-tags?id=using-tiro-client-api).
 
-## USAGE
+- 🎛️ **Flexibility** : multi-window interface to create a custom workspace. Create multiple workspaces organized by tabs. Keep these workspaces synced accross your devices.
 
-### DESKTOP APPLICATIONS
+- 👑 **Open source & Open Data**: Code is open source and free. No database required. Markdown files only. You can edit and sync your files with external software as you need.
 
-Mac/Linux/Windows Applications can be downloaded [here](https://github.com/dotgreg/tiro-notes/releases/tag/production) 
-
-### COMMAND LINE
-- Requirements : [NodeJs](https://nodejs.org/en/download/) and [RipGrep](https://github.com/BurntSushi/ripgrep)
-
-```
-npx tiro-notes
-```
-
-<img src="https://user-images.githubusercontent.com/2981891/159723396-b5e81dcd-a4aa-4581-9b7f-e3b62bcdef65.gif" width="600"/>
+- ☁️ / 🖥️ /  💻 / 📱 **Cross-platform & Web** : Windows, macOS, Android, Linux, Web. Start an App or with CLI and access it anywhere on your local network. 
 
-- This works with any platform/device, including unrooted Android with [Termux](https://termux.com) and possibly iOS with iSH (untested)
-- ```npx tiro-notes --help``` for all available options
 
 
-## FUNCTIONALITIES LIST
+# Functionalities
   
 **Note Edition**
 - 🖊️ Mardown notes edition
@@ -66,16 +33,18 @@ npx tiro-notes
 - 🔑 End to end (E2E) note encryption (RSA 2048)
 - 🏛 Note changes history
 - 📄 Export to PDF
-- 🏋 Embed videos and other web content (with iframe)
-- 🏋 create javascript applications within your note with [[script]] 
-- 💱 display math formulas with [[latex]] 
-- 🧬 display UML and other diagrams with [[diagram]] (custom tags)
 - (🔧) 💬 Text to Speech (beta)
-- (🔧) 📡 Server Collaborative edition (planned...)
+
+**Custom Tags**
+- 💱 display math formulas with [[latex]] 
+- 🧬 display UML and other diagrams with [[diagram]] 
+- 🏋 Embed videos, pdf and web content with [[view]] 
+- 🏋 create javascript applications within your note with [[script]] 
+- 🧬 leverage the full [Client API](guide-custom-tags?id=using-tiro-client-api) in [[script]]
 
 **Search**
 - 🔍 Fast search (~2s for 30k notes)
-- 🔬 Search filters (intitle: etc...)
+- 🔬 Search filters (intitle: etc.)
 
 **Folders Tree & Notes List**
 - 🗂️ Folders management (create/rename/move/delete)
@@ -97,183 +66,4 @@ npx tiro-notes
 - 📟 Use it on any device with a browser (Mobile, tablet and Desktop interface)
 - (🔧)📱 Install on iOS with iSH and NPX (to be tested...)
 
-
-## DOCUMENTATION
-
-### Available Custom Tags
-  - [[calendar]] a fully functional calendar that stored data on a note : [install instructions & screenshot](https://github.com/dotgreg/tiro-notes/blob/master/documentation/custom-tag-calendar.md)
-  - [[mermaid]] Uml, Gantt and flowcharts : [install instructions & screenshot](https://github.com/dotgreg/tiro-notes/blob/master/documentation/custom-tag-mermaid.md)
-
-### Add scripts in your note
-
-<details>
-  <summary>Expand for details</summary>
-  You can add javascript logic in your note with the special tag [[script]]
-  
-  You need to use ```return``` to output it in the preview.
-  
-  ```
-  .... Note content ....
-  
-  [[script]]]
-  const stringToDisplay = `this is a hello world from javascript`
-  const randomNumber = Math.round(Math.random() * 1000)
-  return `${stringToDisplay} ${randomNumber} `
-  [[script]]
-  
-  .... Note content ....
-  ```
-  
-  will output something like 
-  
-  ```
-  .... Note content ....
-  
-  this is a hello world from javascript 102
-  
-  .... Note content ....
-  
-  ```
-</details>
-
-### Create your own custom tags
-
-<details>
-  <summary>Expand for details</summary>
-  
-  #### 1) How to create a new custom tag [[mail-address]]
-  In order to install a new tag, you need to create a new note in ```/.tiro/tags/``` folder. (create the /tags directory if doesn't exists).
-  
-  The content of that note will then be placed instead of the tag.
-  
-  So in order to create a custom tag [[mail-address]], you will need to create ```/.tiro/tags/mail-address.md```
-  
-  So with ```/.tiro/tags/mail-address.md``` content being :
-  ```
-  10 Downing Street
-  City of Westminster
-  London, SW1
-  ```
-  
-  And your note content being
-    
-  ```
-  .... Note content ....
-  
-  [[mail-address]] [[mail-address]]
-  
-  .... Note content ....
-  
-  ```
-  
-  The result will give
-  
-   ```
-  .... Note content ....
-  
-  10 Downing Street
-  City of Westminster
-  London, SW1
-  
-  .... Note content ....
-  
-  ```
-  
-  
-  #### 2) Using {{InnerTag}}
-  The content inside the custom tags can be fetched with the special tag ```{{innerTag}}```
-  
-  So with ```/.tiro/tags/mail-address.md``` content being :
-  ```
-  10 Downing Street
-  City of Westminster
-  London, SW1
-  {{innerTag}}
-  ```
-  
-  Calling ```[[mail-address]] John Foo [[mail-address]]``` in any note will be replaced by
-  
-  ```
-  10 Downing Street
-  City of Westminster
-  London, SW1
-  John Foo
-  ```
-  
-  ### 3) Add script logic in your tag
-  
-  You can add javascript to your custom tag with the special tag [[script]]
-  
-  If we have ```/.tiro/tags/square-number.md``` with the following content : 
-  ```
-  The multiplied result is : 
-  [[script]]]
-  const numberToSquare = {{innerTag}}
-  return `${numberToSquare * numberToSquare}`
-  [[script]]
-  ```
-  
-  Using ```[[square-number]] 4 [[square-number]]``` will return ```The multiplied result is :  16```
-  
-  
-</details>
-
-
-### Client API (Beta)
-<details>
-  <summary>Expand for details</summary>
-  
-  
-  Tiro Notes provides a Client API accessible using the developer console of your browser :
-  ![Screenshot 2022-03-31 at 18 39 22](https://user-images.githubusercontent.com/2981891/161106559-b27890d1-fca4-4e66-a6ff-0bdf38f679b3.jpg)
-
-  typing ``` window.tiroCli ``` will give you the updated list of available functions. Each function has a description ```window.tiroCli.FUNCTION.description``` and can be called like that ```window.tiroCli.FUNCTION.f()```
-  
-  The Tiro Cli allows you to :
-  - fetch a note content : ```window.tiroCli.clientApiGetFileContent```
-  - modify a note content : ```window.tiroCli.clientApiSetFileContent```
-  - Load an external javascript file : ```window.tiroCli.loadScripts(['https://', 'https://', ...], function () => {})```
-  - Show the current note content : ```window.tiroCli.fileContent```
-  - Trigger a search in the interface : ```window.tiroCli.triggerSearch```
-  - Set the Dual Editor view type temporarily : ```window.tiroCli.setTempViewType ("both", "editor", "preview")```
-  
-  /!\ this API and CLI structure is meant to change in term of structure, make sure to check window.tiroCli when upgrading to a new version /!\
-  
-</details>
-
-
-
-## FAQ
-
-### How can I sync my notes folder on other devices?
-You can use any sync solution like dropbox, google drive, resilio sync, syncthings etc...
-
-### Can I edit my notes on another application?
-Yes, all the datas Tiro uses is coming from markdown notes. So you can edit your notes in other applications without any problem.
-
-### Where are located the configuration file and the logs of Tiro?
-The configuration file is located at ```~/.tiro-config.json``` for linux, mac and termux android and under ```/Users/USER_NAME/.tiro-config.json```in windows.
-The logs are also located at the same path ```.tiro-logs.txt``` if using NPX or Node directly and ```.tiro-electron-log.txt``` if using Tiro Notes Desktop Apps.
-
-### How can I use Tiro in the Web?
-1) Device where tiro is installed : http://localhost:3023 
-2) If on same Wifi/local network: http://192.168.xx.xx:3023 where that ip is the ip of the device where tiro is installed
-3) Not on same network : Tiro works great using SSH tunneling, AutoSSH or similar (free ssh tunnels solutions exists like https://opentunnel.net/). Npx tiro-notes also includes a --tunnel option for easy tunneling.
-4) On the cloud: You can self-host/install Tiro Notes on your server and access it from anywhere.
-
-### What is the current status of Tiro Notes? 
-I have been developing it for over a year and using it as my daily note app for over 6 months now, its core functionalities seems stable enough for my use so far.
-I haven't been loosing any data so far, and there is always the history note functionality in that case.
-
-However, I consider that application still in alpha phase, it still needs a lot of testing to be considered robust.
-So use it at your own risk, and always with data you can afford to lose.
-
-### How can I contribute?
-
-Contributions and PR are welcome! You can contact me for more details on the ROADMAP and how to be involved.
-
-Right now, Tiro Notes needs testers to stabilize the current scope of functionalities and to give feedbacks on it.
-
-Also translation can be a nice thing to have in the future.
-
-Twitter News & Updates : https://twitter.com/NotesTiro
+>"Marcus Tullius Tiro, a learned freedman who was a member of Cicero’s household, invented the Tironian notes, the first Latin shorthand system. Tironian notes consist of letters of the alphabet, simplified and modified to achieve greater speed in their writing."

package/cli.js

@@ -3,6 +3,7 @@
 // const isDev = false
 const isDev = process.env.ISDEV
 const tHelpers = isDev ? require('../shared.helpers.js') : require(`./shared.helpers.build.js`);
+const pathServerJs = isDev ? `${__dirname}/node-build/server/tiro-server.js` : `${__dirname}/server/tiro-server.js`
 
 
 // CLI HELP MANUAL
@@ -22,7 +23,15 @@ ARGS:
 
 --https/-s : enable https ssl with self signed certificate (boolean, false by default)
 --port/-p : port to use (number, 3023 by default)
---tunnel/-t : uses autossh to "publish" the app on the web, requires a server you can access with ssh and autossh installed on that device. (ex:npx tiro-notes@latest -t REMOTE_USER@REMOTE_URL:REMOTE_PORT)
+--no-open/-no : do not open Tiro in browser when starting
+--verbose/-v : more verbose logs
+
+--tunnel/-t : [require autossh] uses autossh to "publish" the app on the web, requires a server you can access with ssh and autossh installed on that device. (ex:npx tiro-notes@latest -t REMOTE_USER@REMOTE_URL:REMOTE_PORT)
+
+--backup/-b : [require tar] will incrementally backup changes in archives like tiro.0.xz.tar, tiro.1.xz.tar... every day in a specific folder. You can then execute commands after that process in a post backup script (useful for syncing these archives to clouds, think rsync, rclone etc.) 
+--backup-folder : modify backup folder destination. (default: "your/path/to/tiro/data_folder"+_backup
+--backup-post-script : modify script to be executed after a backup finishes. Should be a ".txt" file with your OS commands. (default: "your/path/to/tiro/data_folder"+_backup/post_backup_script.txt
+
 --help/-h : help 
 
 EXAMPLES:
@@ -48,6 +57,13 @@ function getCliArgs () {
 				port: 3023,
 				https: false,
 				help: false,
+				open: true,
+				verbose: false,
+				backup: {
+						enabled: false,
+						location: "default", 
+						scriptLocation: "default"
+				},
 				tunnel: {
             enabled: false,
         },
@@ -60,6 +76,13 @@ function getCliArgs () {
 				if (argName === 'p' || argName === 'port') argsObj.port = parseInt(argVal);
 				if (argName === 's' || argName === 'https') argsObj.https = true
 				if (argName === 'h' || argName === 'help') argsObj.help = true
+				if (argName === 'no-open' || argName === 'no') argsObj.open = false
+				if (argName === 'v' || argName === 'verbose') argsObj.verbose = true
+
+				if (argName === 'b' || argName === 'backup') argsObj.backup.enabled = true
+				if (argName === 'backup-location') argsObj.backup.location = argVal
+				if (argName === 'backup-post-script') argsObj.backup.scriptLocation = argVal 
+
 				if (argName === 't' || argName === 'tunnel') {
             const argsArr = argVal.split(':')
             if (argsArr.length > 1) {
@@ -79,19 +102,90 @@ function startTiroServer (argsObj, cb) {
 		tHelpers.killPreviousInstances(() => {
 
 				// start tiro server, detect success message and get server params
-				tHelpers.execCmd('node', [`${__dirname}/server/tiro-server.js`], {
+				tHelpers.execCmd('node', [pathServerJs], {
+						showLog: argsObj.verbose,
 						env: {
 								TIRO_PORT: argsObj.port,
 								TIRO_HTTPS: argsObj.https
 						},  
 						logName: 'tiroServer',
 						onLog: str => {
+								// we get params like dataFolder from server.js directly
 								tHelpers.checkAndGetTiroConfig(str, {platform: 'cli', cb})
 						}
 				})
 		});
 }
 
+const startBackupScript = async (argsObj, dataFolder) => {
+		console.log ("[BACKUP] init1");
+		if (argsObj.backup.enabled) return;
+		console.log ("[BACKUP] init2");
+		if (!dataFolder) return console.warn ("[BACKUP] no dataFolder detected!");
+		console.log ("[BACKUP] init3");
+
+		// get BACKUP_FOLDER path
+		const defaultBackupFolder = dataFolder + "_backup/"
+		const backupFolder = argsObj.backup.location !== "default" ? argsObj.backup.location : defaultBackupFolder
+		// if does not exists, create it
+		if (!tHelpers.fileExists(backupFolder)) await tHelpers.createDir(backupFolder)
+		
+
+		// get POST_BACKUP_SCRIPT
+		const defaultPostBackupScript = backupFolder + "post_backup_script.txt"
+		const postBackupScriptFile = argsObj.backup.scriptLocation !== "default" ? argsObj.backup.scriptLocation : defaultPostBackupScript 
+		// if does not exists, create it, empty
+		if (!tHelpers.fileExists(postBackupScriptFile)) await tHelpers.saveFile(postBackupScriptFile, "")
+		let postBackupScript = await tHelpers.openFile(postBackupScriptFile) 
+		if (postBackupScript) postBackupScript += ";"
+
+
+		// get LAST_BACKUP_TIMESTAMP
+		const timestampFile = backupFolder + "last_backup_timestamp.txt"
+		const now = () => new Date().getTime()
+		// if does not exists, create it, then give a timestamp of 0
+		if (!tHelpers.fileExists(timestampFile)) await tHelpers.saveFile(timestampFile, "0")
+		let lastBackupTimestampRaw = await tHelpers.openFile(timestampFile);		
+		let lastBackupTimestamp = parseInt(lastBackupTimestampRaw) 
+		let replaceTimestampCli = () => `echo ${now()} > '${timestampFile}'`
+		
+		// START INTERVAL
+		const timeInterval = 1000 * 60 * 60 // one hour
+		const backupInterval = 1000 * 60 * 60 * 24 // one day
+
+		const tarExec = process.platform === "darwin" ? "gtar" : "tar"
+
+		let backupCli = `mkdir '${backupFolder}'; mkdir '${backupFolder}/backups'; cd '${backupFolder}'; ${tarExec} --xz --verbose --create --file="backups/tiro.$(ls backups/ | wc -l | sed 's/^ *//;s/ *$//').tar.xz" '${dataFolder}' --listed-incremental='${backupFolder}metadata.snar'; ${replaceTimestampCli()}; ${postBackupScript}` 
+
+		const debugObj = {backupFolder, postBackupScriptFile, postBackupScript, lastBackupTimestamp, timeInterval, backupCli}
+		console.log ("[BACKUP] starting backup logic!");
+		console.log (debugObj);
+
+		const debugBackupNow = isDev ? false : false
+		const processBackupEveryDay = () => {
+				// if > 1 day
+				const diff = backupInterval + lastBackupTimestamp - new Date().getTime()
+				const diffMin = Math.round(diff / (1000 * 60))
+				if (diff < 0 || debugBackupNow) {
+						console.log(`[BACKUP] time has come, BACKUP!`);
+						// append to backup CLI current timestamp to last_backup_timestamp
+						// execute cli
+						tHelpers.execCmdInFile(backupCli, backupFolder+"cli.sh", {
+								showLog: argsObj.verbose
+						})
+				} else {
+						console.log(`[BACKUP] time has no come... still waiting for ${diffMin} mins`);
+				}
+		}
+
+		// every hour
+		processBackupEveryDay();
+		let int = setInterval(() => {
+				processBackupEveryDay()
+		}, timeInterval)
+
+}
+
 const startSshTunnel = (argsObj) => {
 		if (argsObj.tunnel.enabled) {
 				// kill previous autossh processes
@@ -135,12 +229,16 @@ function main () {
 						const c = configServerObj
 						const protocol = c.https ? 'https' : 'http'
 						const port = c.port
+						const dataFolder = c.dataFolder
 
 						// open in browser
-						openInBrowser(`${protocol}://localhost:${port}`);
+						if (argsObj.open) openInBrowser(`${protocol}://localhost:${port}`);
 
 						// start tunnel with autossh if asked
 						startSshTunnel(argsObj);
+
+						// start backup script
+						startBackupScript(argsObj, dataFolder);
 						
 				})
 		}
@@ -148,7 +246,10 @@ function main () {
 
 const test = () => {
 		var argsObj = getCliArgs();
-		startSshTunnel(argsObj);
+		// startSshTunnel(argsObj);
+		console.log(argsObj);
+		// startBackupScript(argsObj, "/Users/gregoirethiebault/Desktop/your markdown notes")
+		startBackupScript(argsObj, "/Users/gregoirethiebault/Desktop/nodal_ex")
 }