websocket-text-relay 1.0.0

package/.eslintrc

@@ -0,0 +1,29 @@
+{
+  "root": true,
+  "extends": [
+    "eslint:recommended"
+  ],
+  "rules": {
+    "semi": ["error", "never"],
+    "indent": ["error", 2],
+    "no-multiple-empty-lines": ["error", { "max": 1 }],
+    "no-unused-vars": 1,
+    "no-return-assign": 0,
+    "multiline-ternary": 0,
+    "object-curly-spacing": 0,
+    "object-property-newline": 0,
+    "object-curly-newline": 0,
+    "quotes": 0,
+    "quote-props": 0,
+    "import/no-absolute-path": 0
+  },
+  "env": {
+    "node": true,
+    "browser": true,
+    "es2024": true
+  },
+  "parserOptions": {
+    "sourceType": "module",
+    "ecmaVersion": "latest"
+  }
+}

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Niels Nielsen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,40 @@
+# websocket-text-relay (WTR)
+
+## Alpha
+
+Note: This tool is still in alpha state. Some features like the VsCode extension and vite plugin have not been published yet. Documentation is still a work in progress
+
+This application connects to your text editor using the Language Server Protocol (LSP) and then starts a websocket
+server that front end clients can connect to and subscribe to file change events. This allows users to see their
+front end changes evaluated immediately as they type, without having to first save the file to disk or reload the browser.
+It's a similar concept to sites like CodePen and JsFiddle, except you can develop locally using your own text editor with all
+of your personalized plugins instead of having to use an in browser code editor.
+
+## Usage
+
+### 1. Install the extension for your text editor
+
+WTR currently has plugins for Neovim and VSCode.
+    * Installation instructions for Neovim
+    * Installation instructions for VSCode
+
+If you wish to create a plugin for a new editor, the process is fairly straight forward if the editor has
+LSP support. See the [developer guide to creating a WTR text editor plugin](./docs/creating-text-editor-plugin.md)
+
+### 2. Connect to the websocket server from the front end application
+
+To use this on a professional level project that gives you the option to use modules, typescript, and react, I recommend using vite along with
+the plugin **vite-plugin-websocket-text-relay**. This plugin gives you all the power of vite when developing while also hooking
+the live text updates into Vite's hot module reload system.
+
+If you want to use this as a learning tool to play around with UI concepts using simple projects involving 1 html, css, and javascript file,
+then check out this **Simple WTR reference project**. This is a great setup for following along with any short and focused web development tutorials.
+
+And finally, the [status UI for this project](http://localhost:38378) was also created using live updates from websocket-text-relay.
+In addition to giving you live feedback on the status and activity of the application, it is also meant to serve as a
+reference UI project that is more complicated than a single javascript file, but still doesn't use any external dependencies.
+
+
+## Developing
+
+If you are a developer looking to run websocket-text-relay from source and make modifications, follow the [Developer Getting Started Guide](./docs/dev-getting-started.md)

package/docs/code-structure.md

@@ -0,0 +1,77 @@
+# Developer Guide: Code Structure
+
+## Core Logic: the start function
+_websocket-text-relay_ merges two asynchronous interfaces. The the LSP server <-> text editor interface and
+the websocket server <-> front end client interface.
+
+Everything starts with the `startLanguageServer` function exported by the index.js file in the root of the project. The start function
+is exported this way so it can be used as either a standalone command line application (the way the Neovim plugin interacts with the LSP server)
+or as an imported library (the way the VSCode extension uses websocket-text-relay).
+
+The function takes as parameters the input and output streams for the LSP server (they are optional and default to stdin and stdout if not included), and the port
+for the http/websocket server (optional and defaults to 38378).
+
+This function will connect the LSP server to the IO streams and spin up the websocket interface on the resolved port. Then it will wire up all
+of the LSP lifecycle events and websocket API events to their handlers.
+
+Every text editor instance will start up an instance of the websocket-text-relay app. Because it needs to host
+an http server on a specific port for front end clients to connect to, only one instance of the app may act as the
+websocket server. Every subsuquent instance must connect to that websocket server as a client to relay its text changes.
+This is all abstracted away behind the websocket-interface, which is explained in greater detail below.
+
+## Core Logic: The lsp and websocket events
+
+LSP: `initailze` / WS: `init`
+
+Whenever a new editor instance or front end client connects to the websocket, a websocket session is created for that client. These
+can be seen as the labeled blue wedges around the ring in the status UI. When the lsp client from the editor sends an
+initialize request to the language-server, it contains the editor name and editor processId. This data is then sent to the
+main websocket server in an `init` message, where it can be viewed in the status UI (used to determine which sessions are editors and label each wedge with the editor name)
+
+WS: `watch-file`
+
+The front end client will send `watch-file` events to the websocket server to let it know what file changes it wants to subscribe to.
+
+LSP: `wtr/update-open-files`
+
+This event has to be manually sent by the editor plugin. Any time a file is opened or closed in the editor, the editor
+will send a list of open files to the language-server.
+
+LSP: `wtr/update-active-files`
+
+The sessionManager will reconcile the files that the front end clients are watching with the files that the text
+editor has open and determine which files the LSP client should attach to. Once attached the client will start sending `textDocument/didChange` events.
+
+LSP: `textDocument/didChange`
+
+Whenever an active text document is updated, send the latest changes to all watching clients via the weboscket-interface
+
+
+## Language-Server
+
+The language-server directory contains the code that handles communicating with the text editor over stdio. The main export for
+interfacing with the LSP events is the JsonRpcInterface object in the JsonRpcInterface.js file. JsonRpc has 2
+message types, requests and notifications. Therefore the JsonRpcInterface has 4 functions to send and receive these message types.
+
+    * onRequest - handle incoming requests from editor. Can be asynchronous, return a promise, or return a response synchronously.
+    * onNotification - handle incoming notifications from editor. No return value
+    * sendRequest - send outgoing request to editor. Returns a promise that resolves with the editor's response
+    * sendNotification - send outgoing notification. No response or return value
+
+The code in LspReader and LspWriter handles all the lower level details of reading and writing JSON RPC messages. Test
+driven development (TDD) was used to develop the data stream handlers and verify that the resulting interface behaves
+as expected.
+
+## Websocket-Interface
+
+The websocket-interface directory contains the code for hosting and interacting with the websocket API. When the
+application first starts up, the websocket-interface will attempt to spin up the http server on the specified port. If
+successful, the application runs in server mode and all function calls made to the websocket-interface will be directed to itself.
+If there is already a server running on the specified port, then the interface will connect to that server as a client. All
+calls made through the websocket-interface will now be sent to the main websocket instance through the websocket client.
+
+If the main server ever goes offline, each instance will attempt to start up an http server on the port. The first instance able
+to start the http server will be the new main server, the rest of the instances will receive a port in use error from the OS and then
+proceed to connect to the new server in client mode again.
+
+All init messages are resent when a websocket client reconnects.

package/docs/creating-text-editor-plugin.md

@@ -0,0 +1,10 @@
+# Developers Guide: Creating Text Editor Plugin
+
+Placeholder guide for creating text editor plugin:
+    * Connecting the LSP client
+    * Seeing the session in the status UI
+    * Sending list of open files
+    * receiving list of active files (verify in status UI)
+    * Connecting the LSP client to the active text documents
+    * Subscribing to changes from a test UI
+    * Seeing updates in test UI and Status UI

package/docs/dev-getting-started.md

@@ -0,0 +1,8 @@
+# Developers Guide: Getting Started
+
+Placeholder for instructions on how to:
+    * check out repo
+    * run tests
+    * point your text editor plugin to use your checked out code
+    * run the chrome debugger and set breakpoints
+    * understanding the code structure

package/index.js

@@ -0,0 +1,87 @@
+import { JsonRpcInterface } from "./src/language-server/JsonRpcInterface.js"
+import { WebsocketInterface } from "./src/websocket-interface/WebsocketInterface.js"
+
+const DEFAULT_WS_PORT = 38378
+
+const resolvePort = (portParam) => {
+  if (portParam != null) { return portParam }
+  if (process.env["websocket_text_relay_port"] != null) { return process.env["websocket_text_relay_port"] }
+  return DEFAULT_WS_PORT
+}
+
+const resolveIoStreams = ({inputStreamParam, outputStreamParam}) => {
+  const inputStream = inputStreamParam || process.stdin
+  const outputStream = outputStreamParam || process.stdout
+  return {inputStream, outputStream}
+}
+
+const fileProtocolPrefix = "file://"
+
+const getNormalizedFileName = (uri) => {
+  if (uri.startsWith(fileProtocolPrefix)) { return uri.substring(fileProtocolPrefix.length) }
+  return uri
+}
+
+const lspInitializeResponse = {
+  serverInfo: {
+    name: "Websocket Text Relay LSP Server",
+    version: "1.0.0"
+  },
+  capabilities: {
+    textDocumentSync: 1 // full text sync
+  }
+}
+
+export const startLanguageServer = (options = {}) => {
+  const {inputStream: inputStreamParam, outputStream: outputStreamParam, port: portParam} = options
+  const jsonRpc = new JsonRpcInterface(resolveIoStreams({inputStreamParam, outputStreamParam}))
+
+  const wsInterface = new WebsocketInterface({port: resolvePort(portParam)})
+
+  wsInterface.emitter.on('message', (message) => {
+    if (message.method === "watch-editor-active-files") {
+      jsonRpc.sendNotification("wtr/update-active-files", {files: message.files})
+    }
+  })
+
+  jsonRpc.onRequest("initialize", (params) => {
+    const wsInitMessage = {
+      method: "init",
+      name: params.clientInfo?.name || "Unnamed Editor",
+      editorPid: params.processId,
+      lsPid: process.pid
+    }
+
+    wsInterface.sendInitMessage(wsInitMessage)
+
+    return lspInitializeResponse
+  })
+
+  // use this function to hook into initalized event
+  // jsonRpc.onNotification("initialized", () => {
+  // })
+
+  jsonRpc.onRequest("shutdown", () => {
+    return null
+  })
+
+  jsonRpc.onNotification("exit", () => {
+    process.exit(0)
+  })
+
+  jsonRpc.onNotification('wtr/update-open-files', ({files}) => {
+    wsInterface.sendOpenFileList(files)
+  })
+
+  jsonRpc.onNotification("textDocument/didOpen", (params) => {
+    const file = getNormalizedFileName(params.textDocument.uri)
+    const contents = params.textDocument.text
+    wsInterface.sendText({file, contents})
+  })
+
+  jsonRpc.onNotification("textDocument/didChange", (params) => {
+    const file = getNormalizedFileName(params.textDocument.uri)
+    const contents = params.contentChanges[0]?.text
+    wsInterface.sendText({file, contents})
+  })
+}

package/package.json

@@ -0,0 +1,45 @@
+{
+  "name": "websocket-text-relay",
+  "version": "1.0.0",
+  "description": "An LSP server for sending live file updates from your text editor to the front end via websockets.",
+  "bin": {
+    "websocket-text-relay": "./start.js"
+  },
+  "scripts": {
+    "lint": "eslint .",
+    "test": "vitest run --coverage",
+    "test-ui": "vitest --ui"
+  },
+  "type": "module",
+  "main": "index.js",
+  "directories": {
+    "doc": "docs"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+ssh://git@github.com/niels4/websocket-text-relay.git"
+  },
+  "keywords": [
+    "websocket",
+    "text",
+    "relay",
+    "live",
+    "coding",
+    "LSP"
+  ],
+  "author": "Niels Nielsen",
+  "license": "MIT",
+  "bugs": {
+    "url": "https://github.com/niels4/websocket-text-relay/issues"
+  },
+  "homepage": "https://github.com/niels4/websocket-text-relay#readme",
+  "devDependencies": {
+    "@vitest/coverage-v8": "^1.4.0",
+    "@vitest/ui": "^1.4.0",
+    "eslint": "^8.57.0",
+    "vitest": "^1.4.0"
+  },
+  "dependencies": {
+    "ws": "^8.16.0"
+  }
+}

package/src/language-server/JsonRpcInterface.js

@@ -0,0 +1,126 @@
+import { EventEmitter } from 'node:events'
+import { LspReader } from "./LspReader.js"
+import { writeNotification, writeRequest, writeResponse } from './LspWriter.js'
+import { invalidRequestErrorCode, invalidRequestErrorMessage, methodNotFoundErrorCode, methodNotFoundErrorMessage, unexpectedNotificationErrorCode,
+  unexpectedNotificationErrorMessage, unexpectedRequestErrorCode, unexpectedRequestErrorMessage } from './constants.js'
+
+export class JsonRpcInterface {
+  constructor ({inputStream, outputStream}) {
+    this.inputStream = inputStream
+    this.outputStream = outputStream
+
+    this.outstandingRequests = new Map()
+    this.events = new EventEmitter()
+    this.notificationHandlers = new Map()
+    this.requestHandlers = new Map()
+    this.lspInputStream = new LspReader()
+    this.inputStream.pipe(this.lspInputStream)
+
+    this.lspInputStream.on('data', this._handleIncomingLspMessage.bind(this))
+
+    this.lspInputStream.on('parse-error', (error) => {
+      writeResponse(this.outputStream, null, error)
+    })
+  }
+
+  onNotification (method, handler) {
+    if (this.notificationHandlers.has(method)) {
+      throw new Error("Can only register one notification handler at a time. Duplicate method handlers for " + method)
+    }
+    this.notificationHandlers.set(method, handler)
+  }
+
+  removeNotificationHandler (method) {
+    return this.notificationHandlers.delete(method)
+  }
+
+  onRequest (method, handler) {
+    if (this.requestHandlers.has(method)) {
+      throw new Error("Can only register one request handler at a time. Duplicate method handlers for " + method)
+    }
+    this.requestHandlers.set(method, handler)
+  }
+
+  removeRequestHandler (method) {
+    return this.requestHandlers.delete(method)
+  }
+
+  sendNotification (method, params) {
+    writeNotification(this.outputStream, method, params)
+  }
+
+  sendRequest (method, params) {
+    const responseHandler = {}
+    const promise = new Promise((resolve, reject) => {
+      responseHandler.resolve = resolve
+      responseHandler.reject = reject
+    })
+    const id = writeRequest(this.outputStream, method, params)
+    this.outstandingRequests.set(id, responseHandler)
+    return promise
+  }
+
+  _handleIncomingLspMessage (message) {
+    const {method, params, id, error, result} = message
+    if (id != null) {
+      if (method != null) {
+        this._handleIncomingRequest(method, params, id)
+      } else {
+        this._handleRequestResponse(id, error, result)
+      }
+      return
+    }
+
+    if (method != null) {
+      this._handleIncomingNotification(method, params)
+      return
+    }
+    const rpcError = {code: invalidRequestErrorCode, message: invalidRequestErrorMessage}
+    this.events.emit('rpc-error', rpcError)
+    writeResponse(this.outputStream, null, rpcError)
+  }
+
+  _handleIncomingNotification (method, params) {
+    const handler = this.notificationHandlers.get(method)
+    if (!handler) {
+      const errorObject = {code: methodNotFoundErrorCode, message: methodNotFoundErrorMessage, data: {method}}
+      this.events.emit("notification-error", errorObject)
+      return
+    }
+    try {
+      handler(params)
+    } catch (error) {
+      const errorObject = {code: unexpectedNotificationErrorCode, message: unexpectedNotificationErrorMessage, data: {method, params, error: error.message}}
+      this.events.emit("notification-error", errorObject)
+    }
+  }
+
+  async _handleIncomingRequest (method, params, id) {
+    const handler = this.requestHandlers.get(method)
+    if (!handler) {
+      const errorObject = {code: methodNotFoundErrorCode, message: methodNotFoundErrorMessage, data: {method}}
+      writeResponse(this.outputStream, id, errorObject)
+      this.events.emit("request-error", {id, error: errorObject})
+      return
+    }
+    try {
+      const result = await handler(params)
+      writeResponse(this.outputStream, id, null, result)
+    } catch (error) {
+      const errorObject = {code: unexpectedRequestErrorCode, message: unexpectedRequestErrorMessage, data: {method, params, error: error.message}}
+      this.events.emit("request-error", {id, error: errorObject})
+      writeResponse(this.outputStream, id, errorObject)
+    }
+  }
+
+  _handleRequestResponse (id, error, result) {
+    const responseHandler = this.outstandingRequests.get(id)
+    if (!responseHandler) { return }
+    this.outstandingRequests.delete(id)
+    if (error != null) {
+      responseHandler.reject(error)
+    } else {
+      responseHandler.resolve(result)
+    }
+  }
+}