nervous-notes 0.0.1__tar.gz

nervous_notes-0.0.1/.gitignore

@@ -0,0 +1,5 @@
+dist
+.idea
+.DS_Store
+venv3.10
+

nervous_notes-0.0.1/LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2018 The Python Packaging Authority
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

nervous_notes-0.0.1/PKG-INFO

@@ -0,0 +1,110 @@
+Metadata-Version: 2.5
+Name: nervous-notes
+Version: 0.0.1
+Summary: Personal knowledge management meets the actor model
+Project-URL: Homepage, https://github.com/micseydel/nervous_notes
+Project-URL: Issues, https://github.com/micseydel/nervous_notes/issues
+Author-email: Michael Seydel <contact@micseydel.me>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Requires-Dist: paho-mqtt==2.1.0
+Requires-Dist: pykka==4.4.2
+Requires-Dist: requests==2.34.2
+Requires-Dist: ruamel-yaml==0.19.1
+Requires-Dist: watchdog==6.0.0
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == 'test'
+Description-Content-Type: text/markdown
+
+> an ACTOR is an active agent which plays a role on cue according to a script
+
+([A Universal Modular ACTOR Formalism for Artificial Intelligence, 1973](https://www.ijcai.org/Proceedings/73/Papers/027B.pdf))
+
+Nervous Notes is a Python module for managing _actors_ that have an associated plaintext Markdown note for their memory. 
+Each Markdown note can have YAML at the start, gated with `---` on the lines before and after the YAML. 
+
+**There is no AI in this project**, the notes are "nervous" because the actor model allows them to _reflexively_ react to 
+events, like their note being modified by a human, or a message sent by another note. If someone wants the "reflex" to 
+be an AI invocation, they are free to do so.
+
+Useful for
+- hot-reloadable daemons with plaintext, human-friendly memory
+- [literate programming](https://en.wikipedia.org/wiki/Literate_programming) (Knuth, 1984) and "knowledge based programming" (same 1973 paper)
+- personal knowledge management (e.g. automating some of an Obsidian vault)
+
+## Quick Start
+
+```commandline
+pip install nervous-notes
+```
+
+`python -m` to `example_scripts` FIXME
+
+example note FIXME
+```
+---
+this_is: yaml
+---
+- (markdown)
+```
+
+## Glossary
+
+- FIXME
+- companion
+- embedded - for literate programming
+- actor / agent
+- note / nervous note
+
+## Standard event types
+
+- FIXME (placeholder)
+  - note updated
+  - timer up
+  - mqtt message
+  - "other" pykka message
+
+## Note API
+
+- FIXME (placeholder)
+
+## Polyglot programming
+
+This module currently includes mqtt support, which allows for sending messages between programs in different languages.
+
+## Knowledge Based Programming
+
+Again, from the actor model paper:
+
+> KNOWLEDGE BASED PROGRAMMING is programming in an environment which has a substantial knowledge base in the application area for which the programs are intended.
+
+## Limitations
+
+- Pykka's thread/GIL limits are inherited
+  - multiprocessing support is limited and experimental
+- globals are shared between hot-reloaded **embedded** nervous notes
+- if someone removes a function from a hot reloading script, they should replace it with `pass` in the body before removing it, or else the prior version will remain active
+- there are currently no built-in circuit breakers
+
+### Interfacing with Obsidian
+
+I found that updating a note _immediately_ after a note-modified event can cause Obsidian to ignore the update, so there
+are some sleep() calls to accommodate the behavior.
+
+## History
+
+I originally implemented this in Scala, where I called it "[woke notes](https://github.com/micseydel/tinker-casting/tree/main/scripts/woke_notes_package)" before spinning off this fully separate repo.
+
+## Future work
+
+- remove mqtt as a default dependency, and make it, kafka, and rabbitmq [optional dependencies](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies)
+  - maybe a "whisper" optional dependency for AI transcription (since this was one of my primary use-cases)
+- similar modules for other languages/platforms (e.g. JVM, Javascript, Go, etc.) to improve polyglot support
+- is Obsidian Canvas rich enough to wire actors together into flows?
+- improved isolation between companion notes
+- replace blocking I/O with non-blocking
+- replace Pykka with something more Akka-like
+  - this would likely be an _enormous_ undertaking

nervous_notes-0.0.1/README.md

@@ -0,0 +1,89 @@
+> an ACTOR is an active agent which plays a role on cue according to a script
+
+([A Universal Modular ACTOR Formalism for Artificial Intelligence, 1973](https://www.ijcai.org/Proceedings/73/Papers/027B.pdf))
+
+Nervous Notes is a Python module for managing _actors_ that have an associated plaintext Markdown note for their memory. 
+Each Markdown note can have YAML at the start, gated with `---` on the lines before and after the YAML. 
+
+**There is no AI in this project**, the notes are "nervous" because the actor model allows them to _reflexively_ react to 
+events, like their note being modified by a human, or a message sent by another note. If someone wants the "reflex" to 
+be an AI invocation, they are free to do so.
+
+Useful for
+- hot-reloadable daemons with plaintext, human-friendly memory
+- [literate programming](https://en.wikipedia.org/wiki/Literate_programming) (Knuth, 1984) and "knowledge based programming" (same 1973 paper)
+- personal knowledge management (e.g. automating some of an Obsidian vault)
+
+## Quick Start
+
+```commandline
+pip install nervous-notes
+```
+
+`python -m` to `example_scripts` FIXME
+
+example note FIXME
+```
+---
+this_is: yaml
+---
+- (markdown)
+```
+
+## Glossary
+
+- FIXME
+- companion
+- embedded - for literate programming
+- actor / agent
+- note / nervous note
+
+## Standard event types
+
+- FIXME (placeholder)
+  - note updated
+  - timer up
+  - mqtt message
+  - "other" pykka message
+
+## Note API
+
+- FIXME (placeholder)
+
+## Polyglot programming
+
+This module currently includes mqtt support, which allows for sending messages between programs in different languages.
+
+## Knowledge Based Programming
+
+Again, from the actor model paper:
+
+> KNOWLEDGE BASED PROGRAMMING is programming in an environment which has a substantial knowledge base in the application area for which the programs are intended.
+
+## Limitations
+
+- Pykka's thread/GIL limits are inherited
+  - multiprocessing support is limited and experimental
+- globals are shared between hot-reloaded **embedded** nervous notes
+- if someone removes a function from a hot reloading script, they should replace it with `pass` in the body before removing it, or else the prior version will remain active
+- there are currently no built-in circuit breakers
+
+### Interfacing with Obsidian
+
+I found that updating a note _immediately_ after a note-modified event can cause Obsidian to ignore the update, so there
+are some sleep() calls to accommodate the behavior.
+
+## History
+
+I originally implemented this in Scala, where I called it "[woke notes](https://github.com/micseydel/tinker-casting/tree/main/scripts/woke_notes_package)" before spinning off this fully separate repo.
+
+## Future work
+
+- remove mqtt as a default dependency, and make it, kafka, and rabbitmq [optional dependencies](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies)
+  - maybe a "whisper" optional dependency for AI transcription (since this was one of my primary use-cases)
+- similar modules for other languages/platforms (e.g. JVM, Javascript, Go, etc.) to improve polyglot support
+- is Obsidian Canvas rich enough to wire actors together into flows?
+- improved isolation between companion notes
+- replace blocking I/O with non-blocking
+- replace Pykka with something more Akka-like
+  - this would likely be an _enormous_ undertaking

nervous_notes-0.0.1/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["hatchling >= 1.26"]
+build-backend = "hatchling.build"
+
+[project]
+name = "nervous-notes"
+version = "0.0.1"
+authors = [
+  { name="Michael Seydel", email="contact@micseydel.me" },
+]
+description = "Personal knowledge management meets the actor model"
+readme = "README.md"
+requires-python = ">=3.10"
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Operating System :: OS Independent",
+]
+license = "MIT"
+license-files = ["LICEN[CS]E*"]
+dependencies = [
+	"pykka==4.4.2",
+	"ruamel.yaml==0.19.1",
+	"paho-mqtt==2.1.0",
+	"watchdog==6.0.0",
+	"requests==2.34.2",
+]
+
+[project.optional-dependencies]
+test = ["pytest>=7"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]
+
+[project.urls]
+Homepage = "https://github.com/micseydel/nervous_notes"
+Issues = "https://github.com/micseydel/nervous_notes/issues"

nervous_notes-0.0.1/src/nervous_notes/__init__.py

@@ -0,0 +1,10 @@
+from .wrappers.external_messages import get_config_from_env
+from .nervous_note import NervousNote
+from .note_embedded_script import NoteEmbeddedScript
+from .nervous_process import NervousProcess
+from multiprocessing.connection import Connection
+from . import dsl
+__all__ = [
+    "NervousNote", "NoteEmbeddedScript", "NervousProcess", "get_config_from_env", "dsl",
+    "Connection",  # for typing
+]

nervous_notes-0.0.1/src/nervous_notes/__main__.py

@@ -0,0 +1,44 @@
+import logging
+import argparse
+
+from .orchestration import Orchestrator
+from .nervous_note import NervousNote
+from .wrappers.external_messages import get_config_from_env
+
+
+logging.basicConfig(level=logging.INFO,
+                    format='[%(asctime)s %(pathname)s:%(lineno)s] - %(message)s',
+                    datefmt='%Y-%m-%d %H:%M:%S')
+
+parser = argparse.ArgumentParser()
+parser.add_argument("vault_path", help="Path to the vault directory", type=str)
+
+# FIXME: need to make sure this is properly optional, in the Orchestrator
+parser.add_argument("--scripts", help="The directory for scripts")
+
+args = parser.parse_args()
+
+mqtt_config = get_config_from_env()
+if mqtt_config:
+    logging.info(f"Using vault {args.vault_path}, mqtt broker {mqtt_config.broker}:{mqtt_config.port} for user {mqtt_config.username}, and scripts_dir {args.scripts}")
+else:
+    logging.info(f"Failed to get mqtt config, proceeding with None config...")
+
+NervousNote.start_background_actors(args.vault_path, mqtt_config)
+# FIXME: can I delete the password to let it get garbage collected?
+
+deploy_debug_notes = False
+# pykka.DeadLetterRouter.default_router().start() #?
+orchestrator = Orchestrator.wake("Nervous Notes Orchestrator", args.scripts, deploy_debug_notes)
+
+try:
+    while True:
+        input()
+except KeyboardInterrupt:
+    logging.info("KeyboardInterrupt detected, stopping...")
+    orchestrator.stop()
+    logging.info("Orchestrator stopped")
+    NervousNote.stop_background_actors()
+
+    # FIXME: what more is needed for a proper, clean shutdown?
+    # exit(0)

nervous_notes-0.0.1/src/nervous_notes/dsl.py

@@ -0,0 +1,63 @@
+"""
+used for typing, not at runtime
+
+runtime comes from scriping.CompiledScript.script_scope
+"""
+
+import logging
+import json
+import datetime
+from time import ctime, sleep
+
+import requests
+from pykka import ActorRef
+
+from .note_companion_script import TemplateScript
+from .util import TimeUtil, Clock
+
+from .nervous_note import MqttWrapper
+from .wrappers.note_api import datetimestamped_markdown_list_line, timestamped_markdown_list_line, NoteAPI
+
+
+my_note: NoteAPI
+note_name: str
+topic: str
+default_topic: str
+mqtt: MqttWrapper
+
+_clock = Clock()
+_timeutils = TimeUtil(_clock)
+next_occurrence = _timeutils.next_occurrence
+seconds_until = _timeutils.seconds_until
+today = _clock.today
+
+
+# may be expanded beyond template scripts
+def wake(notename: str, template_script_name: TemplateScript) -> ActorRef: pass
+
+
+def set_timer(delay_seconds: float | int, payload: bytes | None = None, key: str | None = None) -> None: pass
+
+
+def cancel_timer(key: str) -> None: pass
+
+
+def subscribe_internal(topic: str, subscriber) -> None: pass
+
+
+def publish_internal(topic: str, payload: bytes) -> None: pass
+
+
+def on_mqtt_message(topic: str, payload: bytes) -> None: pass
+
+
+def on_note_modified() -> None: pass
+
+
+def on_start() -> None: pass
+
+
+def on_timer(key: str, payload: object) -> None: pass
+
+
+def publish_to_ntfy(channel: str, message: str) -> None: pass

nervous_notes-0.0.1/src/nervous_notes/example_scripts/Example - Daily Responsibility.py

@@ -0,0 +1,75 @@
+if TYPE_CHECKING:  # IDE support
+    from ..dsl import *
+
+
+def on_start():
+    reset_timer_from_note()  # THERE's ALWAYS _a_ TIMER, for later today or tomorrow
+
+
+def on_note_modified():
+    note = my_note.note_if_markdown_starts_with_pressed_button()
+    if note:
+        _frontmatter, markdown = note
+        sleep(0.25)
+        if today().isoformat() not in markdown:
+            updated_markdown = markdown_list_prepender(markdown)
+            reset_timer_from_note(note)
+            my_note.set_markdown(updated_markdown)
+        else:
+            my_note.reset_button_at_start_of_markdown()
+
+
+def on_timer(_key, _payload):
+    note = frontmatter, _markdown = my_note.get_note()
+    channel = frontmatter.get("channel")
+    message = frontmatter.get("message")
+    logging.info(f"Timer called, publishing to {channel}: {message}")
+    publish_to_ntfy(channel, message)
+    reset_timer_from_note(note)
+
+
+# utils
+
+def markdown_list_prepender(markdown: str) -> str:
+    """
+    - reset the button
+    - prepend today's date (after the button)
+    """
+    lines = markdown.splitlines()
+    lines[0] = "- [ ] Mark as done"
+    line_to_prepend = f"- [[{my_note.note_name} ({today().isoformat()})]]"
+    lines.insert(1, line_to_prepend)
+    lines[-1] = lines[-1] + "\n"
+    return "\n".join(lines)
+
+
+def reset_timer_from_note(note=None):
+    """
+    resets the timer for later today or tomorrow
+    """
+    if note is None:
+        note = my_note.get_note()
+
+    if note is not None:
+        frontmatter, markdown = note
+
+        next_notification = next_occurrence(frontmatter.get("ifNotDoneBy"))
+        rightnows_today = today()
+
+        today_marked_as_done = rightnows_today.isoformat() in markdown
+        next_notification_is_for_today = ((rightnows_today.year, rightnows_today.month, rightnows_today.day) ==
+                                          (next_notification.year, next_notification.month, next_notification.day))
+
+        next_notification_already_done = today_marked_as_done and next_notification_is_for_today
+        if next_notification_already_done:
+            next_notification += datetime.timedelta(days=1)
+
+        delay = seconds_until(next_notification)
+        logging.info(f"Setting timer for {next_notification}, delay {delay}s")
+        set_timer(
+            delay,
+            None,  # no payload for this timer
+            key=f"{my_note.note_name}/TIMER"  # there's just one timer
+        )
+    else:
+        logging.warning("No note; need yaml for {channel, message, ifNotDoneBy} (e.g. ifNotDoneBy = 20:32:00-07:00)")

nervous_notes-0.0.1/src/nervous_notes/example_scripts/Example - Extrovert.py

@@ -0,0 +1,34 @@
+def on_start():
+    my_note.set_file_contents(f"""---
+in_topic: {default_topic}
+out_topic: {default_topic}/publish
+message_to_send: "Hello :)"
+---
+- [ ] Send message
+- Instructions:
+    - `out_topic` is updated live, but `in_topic` is currently provided but not read back
+- ---
+""")
+
+def on_mqtt_message(topic, message):
+    my_note.append_datetimestamped_markdown_list_line(f"Received on {topic} message: {message.decode()}")
+
+def on_note_modified():
+    if (maybe_note := my_note.note_if_markdown_starts_with_pressed_button()) is not None:
+        frontmatter, markdown = maybe_note
+    else:
+        return # not a note update we care about
+
+    if (out_topic := frontmatter.get("out_topic")) is None:
+        logging.warning("Button pressed but Frontmatter out_topic was empty")
+        return
+
+    if (message_to_send := frontmatter.get("message_to_send")) is None:
+        logging.warning("Button pressed but Frontmatter message_to_send was empty")
+        return
+
+    # introvert expects `(sender)> (message)` where sender is who they will reply to
+    mqtt.publish(out_topic, f"{topic}> {message_to_send}")
+
+    sleep(.25) # hack so Obsidian sees the update (otherwise it seems to miss it sometimes)
+    my_note.reset_button_at_start_of_markdown()

nervous_notes-0.0.1/src/nervous_notes/example_scripts/Example - Introvert.py

@@ -0,0 +1,54 @@
+def on_start():
+    my_note.set_file_contents(f"""---
+in_topic: {default_topic}
+reply_with: "👋"
+---
+- [ ] Pause replying
+- Instructions:
+    - Change `reply_with` to whatever you would like (`in_topic` does not read back updates currently)
+    - delay_seconds is an optional field, assumed to be 0 (seconds)
+- ---
+""")
+
+sending = True
+
+def on_mqtt_message(topic, message):
+    config = my_note.get_frontmatter()
+
+    try:
+        # expects:
+        # sending_topic> message
+        sender, what_they_sent = message.decode().split(">", 1)
+    except ValueError:
+        my_note.append_datetimestamped_markdown_list_line(f"Received a weirdly formatted message: {message}")
+        return
+
+    if not sending:
+        my_note.append_datetimestamped_markdown_list_line(f"{sender} sent {what_they_sent}, not replying (sending paused)")
+        return
+
+    delay_seconds = config.get("delay_seconds", 0)
+
+    if (reply_with := config.get("reply_with")) is not None:
+        if delay_seconds:
+            set_timer(delay_seconds, (sender, reply_with))
+            my_note.append_datetimestamped_markdown_list_line(f"Received {what_they_sent}, replying with {reply_with} after {delay_seconds}s")
+        else:
+            my_note.append_datetimestamped_markdown_list_line(f"Received {what_they_sent}, replying with {reply_with}")
+            mqtt.publish(sender, reply_with)
+    else:
+        my_note.append_datetimestamped_markdown_list_line(f"{sender} sent {what_they_sent}, not replying (no reply_with frontmatter)")
+
+def on_timer(key, payload):
+    sender, reply_with = payload
+    mqtt.publish(sender, reply_with)
+
+
+def on_note_modified():
+    global sending  # nonlocal doesn't work because sending is not enclosed in a function
+
+    maybe_note = my_note.note_if_markdown_starts_with_pressed_button()
+    if maybe_note is not None:
+        sending = False
+    else:
+        sending = True

nervous_notes-0.0.1/src/nervous_notes/example_scripts/Example - Recurring Note Generator.py

@@ -0,0 +1,61 @@
+from datetime import timedelta
+from typing import io
+
+if TYPE_CHECKING:  # IDE support
+    from dsl import *
+
+
+class Frontmatter:
+    pass
+
+
+class Document:
+    frontmatter: Frontmatter
+    markdown: str
+
+
+def on_start():
+    note = my_note.get_note()
+    if note is not None:
+        frontmatter, markdown = note
+
+        next_date = frontmatter.get("next_date")
+        default_interval_days = frontmatter.get("default_interval_days")
+
+        _button, current_next_note, *history = markdown.splitlines()
+
+        if current_next_note[:8] != "- Next: " or not current_next_note.endswith(")]]"):
+            logging.warning(f"Expected second line to start with `- Next: ` then a wikilink of the format [[Note (YYYY-MM-DD)]]")
+            return
+
+        current_next_date = current_next_note[-13:-3]
+        if next_date is None:
+            next_date = datetime.datetime.strptime(current_next_date, "%Y-%m-%d") + timedelta(days=default_interval_days)
+
+
+def on_note_modified():
+    note = my_note.note_if_markdown_starts_with_pressed_button()
+    if note:
+        frontmatter, markdown = note
+        # FIXME
+        stream = io.StringIO()
+        my_note.yaml.dump(data, stream)
+
+        # Extract the YAML string
+        yaml_string = stream.getvalue()
+
+
+# utils
+
+# FIXME: factor out?
+def markdown_list_prepender(markdown: str) -> str:
+    """
+    - reset the button
+    - prepend today's date (after the button)
+    """
+    lines = markdown.splitlines()
+    lines[0] = "- [ ] Mark as done"
+    line_to_prepend = f"- [[{my_note.note_name} ({datetime.date.today().isoformat()})]]"
+    lines.insert(1, line_to_prepend)
+    lines[-1] = lines[-1] + "\n"
+    return "\n".join(lines)

nervous_notes-0.0.1/src/nervous_notes/example_scripts/Literate Programming - Hello World.py

@@ -0,0 +1,53 @@
+def on_start():
+    my_note.set_file_contents(f"""---
+in_topic: {default_topic}
+out_topic: {default_topic}/publish
+message_to_send: "Hello :)"
+---
+- [ ] Send message
+- Instructions:
+    - press the button above to send `message_to_send` to `out_topic`
+    - (this can be observed via mosquitto_pub)
+    - send a message to `in_topic` and observe it listed below
+    - `out_topic` is read dynamically but `in_topic` is provided for informational purposes only
+- ---
+""")
+
+
+def on_mqtt_message(topic, message):
+    my_note.append_datetimestamped_markdown_list_line(f"received on `{topic}`: {message}")
+
+
+def on_note_modified():
+    if (maybe_note := my_note.note_if_markdown_starts_with_pressed_button()) is not None:
+        frontmatter, markdown = maybe_note
+    else:
+        logging.debug("Note modified but button not pressed")
+        return
+    
+    if (out_topic := frontmatter.get("out_topic")) is None:
+        logging.warning("Button pressed but Frontmatter out_topic was empty")
+        return
+
+    if (message_to_send := frontmatter.get("message_to_send")) is None:
+        logging.warning("Button pressed but Frontmatter message_to_send was empty")
+        return
+
+    logging.info(f"[{note_name}] publishing {message_to_send} to mqtt")
+    mqtt.publish(out_topic, message_to_send)
+
+    logging.info(f"[{note_name}] resetting markdown...")
+
+    sleep(.25) # hack to prevent a race condition with Obsidian's display
+    my_note.upsert_markdown(lambda markdown: upserter(out_topic, message_to_send, markdown))
+    logging.info(f"[{note_name}] Markdown reset")
+
+
+# util
+
+def upserter(out_topic, message_to_send, markdown):
+    lines = markdown.splitlines()
+    lines[0] = f"- [ ] Send (last sent ~{ctime()})"
+    lines.append(timestamped_markdown_list_line(f"sending on `{out_topic}`: {message_to_send}"))
+
+    return "\n".join(lines)